kostasman1/phi3_adapter_finetuned

README.md

@@ -17,6 +17,8 @@ should probably proofread and complete it, then remove this comment. -->
 # results
 
 This model is a fine-tuned version of [microsoft/Phi-3-medium-128k-instruct](https://huggingface.co/microsoft/Phi-3-medium-128k-instruct) on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.3259
 
 ## Model description
 
@@ -44,6 +46,40 @@ The following hyperparameters were used during training:
 - num_epochs: 3
 - mixed_precision_training: Native AMP
 
+### Training results
+
+| Training Loss | Epoch  | Step | Validation Loss |
+|:-------------:|:------:|:----:|:---------------:|
+| 2.102         | 0.1065 | 100  | 2.1266          |
+| 2.0156        | 0.2130 | 200  | 1.9941          |
+| 1.8151        | 0.3195 | 300  | 1.8149          |
+| 1.6951        | 0.4260 | 400  | 1.5771          |
+| 1.2789        | 0.5325 | 500  | 1.3936          |
+| 1.0007        | 0.6390 | 600  | 1.1524          |
+| 0.7882        | 0.7455 | 700  | 0.9936          |
+| 0.9486        | 0.8520 | 800  | 0.8539          |
+| 0.7381        | 0.9585 | 900  | 0.7410          |
+| 0.6254        | 1.0650 | 1000 | 0.6283          |
+| 0.4915        | 1.1715 | 1100 | 0.5834          |
+| 0.3432        | 1.2780 | 1200 | 0.5034          |
+| 0.349         | 1.3845 | 1300 | 0.4476          |
+| 0.4378        | 1.4909 | 1400 | 0.4160          |
+| 0.4522        | 1.5974 | 1500 | 0.4061          |
+| 0.3183        | 1.7039 | 1600 | 0.3795          |
+| 0.3184        | 1.8104 | 1700 | 0.3707          |
+| 0.267         | 1.9169 | 1800 | 0.3601          |
+| 0.2966        | 2.0234 | 1900 | 0.3538          |
+| 0.2697        | 2.1299 | 2000 | 0.3492          |
+| 0.3662        | 2.2364 | 2100 | 0.3424          |
+| 0.3135        | 2.3429 | 2200 | 0.3407          |
+| 0.3339        | 2.4494 | 2300 | 0.3366          |
+| 0.1828        | 2.5559 | 2400 | 0.3340          |
+| 0.2824        | 2.6624 | 2500 | 0.3306          |
+| 0.3204        | 2.7689 | 2600 | 0.3289          |
+| 0.3062        | 2.8754 | 2700 | 0.3263          |
+| 0.313         | 2.9819 | 2800 | 0.3259          |
+
+
 ### Framework versions
 
 - PEFT 0.11.1

adapter_config.json

@@ -23,13 +23,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "up_proj",
-    "v_proj",
-    "gate_proj",
     "down_proj",
+    "o_proj",
     "k_proj",
+    "up_proj",
     "q_proj",
-    "o_proj"
+    "gate_proj",
+    "v_proj"
   ],
   "task_type": null,
   "use_dora": false,

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d4e776d871ce75a40ba766bc1672f80578bee80666614863944b37721ef75d46
+oid sha256:ab13ef7ad021a4eebe511d6770caff8b1db3bb6a090bb1ddda4765a7e7167a87
 size 85218160

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:578b0088c9ba2ffdf8e922e76e576b55e95a4430eff495c715c89f8b078930d9
+oid sha256:e4e477eb8ca005a9bac8b006c62f707a1c8e43efe765366f4604b723498fe34b
 size 5304