Update README.md

README.md

@@ -1,7 +1,42 @@
 ---
 license: openrail
+datasets:
+- mrm8488/unnatural-instructions
+language:
+- en
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- codellama
+- llama2
+- llama
+- instruct
 ---
-for testing purposes only. qlora trained using peft on codellama/CodeLlama-7b-hf as base model. trained on  mrm8488/unnatural-instructions, config 'core' dataset.
+for testing purposes only. qlora trained using peft on codellama/CodeLlama-7b-hf as base model. trained on mrm8488/unnatural-instructions, config 'core' dataset.
+
+trained at 1000 steps with checkpoint every 50. training/validation loss below:
+
+```Step	Training Loss	Validation Loss
+50	1.480500	0.935647
+100	0.894800	0.867328
+150	0.835700	0.841386
+200	0.846100	0.823671
+250	0.804600	0.791546
+300	0.744000	0.799941
+350	0.721900	0.707534
+400	0.702700	0.697420
+450	0.698200	0.691702
+500	0.674600	0.687037
+550	0.666700	0.683634
+600	0.687200	0.680872
+650	0.679300	0.677384
+700	0.698900	0.675221
+750	0.652500	0.673152
+800	0.672200	0.671620
+850	0.668700	0.669980
+900	0.638100	0.669189
+950	0.663200	0.668443
+1000	0.668300	0.668069```
 
 training data transformed to the following structure for testing purposes:
 ```Example 1:
@@ -39,4 +74,4 @@ You are given a paragraph from an article. Your task is to replace all the third
 His team's plans for the day were quickly ruined when their bus got a flat tire on the way to their first event of the day.
 The tournament organizers were not very happy with his team when they showed up late to their match. [/INST]
 Output: Our team's plans for the day were quickly ruined when our bus got a flat tire on the way to our first event of the day.
-The tournament organizers were not very happy with us when we showed up late to our match.```
+The tournament organizers were not very happy with us when we showed up late to our match.```