AINovice2005
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -53,7 +53,7 @@ if __name__ == "__main__":
|
|
53 |
```
|
54 |
|
55 |
๐๐๐ฌ๐ฎ๐ฅ๐ญ๐ฌ:
|
56 |
-
|
57 |
with PEFT failed to restore convergence.Secondly, training only using PEFT helped to restore partial model convergence.
|
58 |
|
59 |
Lastly, future experiments to restore model convergence will require a systematic training and eval strategy.
|
|
|
53 |
```
|
54 |
|
55 |
๐๐๐ฌ๐ฎ๐ฅ๐ญ๐ฌ:
|
56 |
+
Firstly, a higher learning rate was required for the model to train the model on a dataset, training methods such as SFT and ORPO
|
57 |
with PEFT failed to restore convergence.Secondly, training only using PEFT helped to restore partial model convergence.
|
58 |
|
59 |
Lastly, future experiments to restore model convergence will require a systematic training and eval strategy.
|