Model save

README.md

@@ -1,16 +1,10 @@
 ---
-license: apache-2.0
-base_model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
 tags:
-- alignment-handbook
-- trl
-- sft
-- generated_from_trainer
 - trl
 - sft
 - generated_from_trainer
 datasets:
-- HuggingFaceH4/ultrachat_200k
+- generator
 model-index:
 - name: zephyr-7b-sft-full
   results: []
@@ -21,9 +15,9 @@ should probably proofread and complete it, then remove this comment. -->
 
 # zephyr-7b-sft-full
 
-This model is a fine-tuned version of [TinyLlama/TinyLlama-1.1B-Chat-v1.0](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0) on the HuggingFaceH4/ultrachat_200k dataset.
+This model is a fine-tuned version of [](https://huggingface.co/) on the generator dataset.
 It achieves the following results on the evaluation set:
-- Loss: 2.1644
+- Loss: 2.2164
 
 ## Model description
 
@@ -46,7 +40,6 @@ The following hyperparameters were used during training:
 - train_batch_size: 16
 - eval_batch_size: 8
 - seed: 42
-- distributed_type: multi-GPU
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: constant
 - lr_scheduler_warmup_ratio: 0.1
@@ -56,12 +49,12 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
-| 2.2013        | 1.0   | 8969 | 2.1644          |
+| 2.249         | 1.0   | 8969 | 2.2164          |
 
 
 ### Framework versions
 
 - Transformers 4.39.0.dev0
-- Pytorch 2.1.1+cu121
+- Pytorch 2.1.0+cu118
 - Datasets 2.14.6
 - Tokenizers 0.15.2

all_results.json

@@ -1,13 +1,8 @@
 {
     "epoch": 1.0,
-    "eval_loss": 2.164351224899292,
-    "eval_runtime": 1703.872,
-    "eval_samples": 23109,
-    "eval_samples_per_second": 9.321,
-    "eval_steps_per_second": 1.166,
-    "train_loss": 2.789915239123786,
-    "train_runtime": 47169.5182,
+    "train_loss": 2.6744996346363004,
+    "train_runtime": 27779.5691,
     "train_samples": 207864,
-    "train_samples_per_second": 3.042,
-    "train_steps_per_second": 0.19
+    "train_samples_per_second": 5.165,
+    "train_steps_per_second": 0.323
 }

generation_config.json

@@ -1,8 +1,7 @@
 {
+  "_from_model_config": true,
   "bos_token_id": 1,
   "eos_token_id": 2,
-  "max_length": 2048,
-  "pad_token_id": 0,
   "transformers_version": "4.39.0.dev0",
   "use_cache": false
 }

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:08ce96c31744cae26d21b7888fceb9a10b68782aa8737f6cf8e35993f3351983
+oid sha256:ca8a01276ab9bd1c2b28525781848bd023395bc7e7214b7c973c9caf652d8f30
 size 485201912

train_results.json

@@ -1,8 +1,8 @@
 {
     "epoch": 1.0,
-    "train_loss": 2.789915239123786,
-    "train_runtime": 47169.5182,
+    "train_loss": 2.6744996346363004,
+    "train_runtime": 27779.5691,
     "train_samples": 207864,
-    "train_samples_per_second": 3.042,
-    "train_steps_per_second": 0.19
+    "train_samples_per_second": 5.165,
+    "train_steps_per_second": 0.323
 }