rakeshkiriyath commited on
Commit
748ac9b
·
1 Parent(s): 6ed36fb

Created README.md

Browse files
Files changed (1) hide show
  1. README.md +54 -0
README.md ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ tags:
5
+ - text-to-sql
6
+ - gpt2
7
+ - gpt2-medium
8
+ - nlp-to-sql
9
+ - text2sql
10
+ - sql
11
+ ---
12
+ # Model Card for Model ID
13
+
14
+ <!-- The base model used for training is gpt2-medium. We finetuned it on the following dataset: b-mc2/sql-create-context -->
15
+
16
+ This is my first fine tuned LLM project.
17
+
18
+
19
+ ## Prompt
20
+
21
+ query = List the creation year, name and budget of each department
22
+
23
+ f"Translate the following English question to SQL: {query}
24
+
25
+ ## Output
26
+
27
+ SELECT zip_code FROM Visibility WHERE AVG(visibility) < 10
28
+
29
+ [More Information Needed]
30
+
31
+
32
+ #### Training Hyperparameters
33
+
34
+ num_train_epochs=1
35
+ per_device_train_batch_size=3
36
+ gradient_accumulation_steps=9
37
+ learning_rate=5e-5
38
+ weight_decay=0.01
39
+
40
+
41
+ ## Evaluation
42
+
43
+ Step Training Loss
44
+
45
+ 500 0.337800
46
+ 1000 0.262900
47
+ 1500 0.253200
48
+ 2000 0.246400
49
+
50
+ {'eval_loss': 0.23689331114292145, 'eval_runtime': 104.4102, 'eval_samples_per_second': 67.043, 'eval_steps_per_second': 8.38, 'epoch': 1.0}
51
+
52
+
53
+
54
+