Commit
·
09b65e4
1
Parent(s):
bbe127c
Update README.md
Browse files
README.md
CHANGED
|
@@ -57,10 +57,10 @@ Advanced preprocessing into ChatML format.
|
|
| 57 |
### Training Hyperparameters and Fine-Tuning Details
|
| 58 |
- Training Regime: FP32
|
| 59 |
- Warmup Steps: 1
|
| 60 |
-
- Per Device Train Batch Size:
|
| 61 |
-
- Gradient Accumulation Steps:
|
| 62 |
- Max Steps: 1500
|
| 63 |
-
- Learning Rate: 0.
|
| 64 |
- Logging Steps: 1
|
| 65 |
- Save Steps: 1
|
| 66 |
- Lora Alpha: 32
|
|
|
|
| 57 |
### Training Hyperparameters and Fine-Tuning Details
|
| 58 |
- Training Regime: FP32
|
| 59 |
- Warmup Steps: 1
|
| 60 |
+
- Per Device Train Batch Size: 1
|
| 61 |
+
- Gradient Accumulation Steps: 1
|
| 62 |
- Max Steps: 1500
|
| 63 |
+
- Learning Rate: 0.0002
|
| 64 |
- Logging Steps: 1
|
| 65 |
- Save Steps: 1
|
| 66 |
- Lora Alpha: 32
|