Update README.md
Browse files
README.md
CHANGED
@@ -36,8 +36,60 @@ Text generation is currently able to support a range of creative writing tasks i
|
|
36 |
|
37 |
Pleias-360m-Preview has been successfully adapted for continuous pretraining and full-fine-tuning on document processing tasks such as RAG, translation or OCR correction. Given the small size of the model we do not recommend fine-tuning methods based on LORA.
|
38 |
|
39 |
-
##
|
|
|
40 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
41 |
|
42 |
## Training
|
43 |
Pleias-360m-Preview was fully pretrained at Jean Zay on 64 h100s for 46 hours with Nanotron, the pretraining library from HuggingFace. We provide the complete settings as a yaml file as part of our release.
|
|
|
36 |
|
37 |
Pleias-360m-Preview has been successfully adapted for continuous pretraining and full-fine-tuning on document processing tasks such as RAG, translation or OCR correction. Given the small size of the model we do not recommend fine-tuning methods based on LORA.
|
38 |
|
39 |
+
## Examples
|
40 |
+
With only 360 million parameters, Pleias-360m-Preview is able to generate literary texts in multiple languages:
|
41 |
|
42 |
+
English poetry:
|
43 |
+
|
44 |
+
<blockquote>
|
45 |
+
<b>Saturn is fallen, am I too to fall?
|
46 |
+
Am I to leave this haven of my rest,
|
47 |
+
This cradle of my glory, this soft clime,
|
48 |
+
This calm luxuriance of blissful light,</b>
|
49 |
+
And the sweetness that was not in me?
|
50 |
+
I have been a slave and an enemy.
|
51 |
+
The world has made me its prey ; it's wrong;
|
52 |
+
It's cruelty! It's madness! — but you know
|
53 |
+
That all things are good for one man or another :
|
54 |
+
You see how he can be happy without them:
|
55 |
+
He cannot be miserable unless they come
|
56 |
+
To his heart with their own poisonous breaths.
|
57 |
+
But if there were no other way left us,
|
58 |
+
We should never find our happiness again.
|
59 |
+
If we could only live on earth alone,
|
60 |
+
With nothing else about us than ourselves,
|
61 |
+
Our very being would become more real.
|
62 |
+
There must be some other place where we may
|
63 |
+
Live happily as before, and enjoy life
|
64 |
+
As well as now. But what shall we do here?
|
65 |
+
What will happen when we die? What will hap-
|
66 |
+
[pen then?
|
67 |
+
When death comes, let him who knows the end
|
68 |
+
</blockquote>
|
69 |
+
|
70 |
+
French classic play with the 17th century writing style (including the long s)
|
71 |
+
|
72 |
+
<blockquote>
|
73 |
+
<b>Helas! pour mon malheur ie l’ay trop écouté.
|
74 |
+
Ie n’ay point du ſilence affecté le myſtère.
|
75 |
+
Ie croyois ſans péril pouuoir eſtre ſincere.
|
76 |
+
Et ſans armer mes yeux d’vn moment de rigueur,</b>
|
77 |
+
Je ne voy pas que ce foit vn crime à la nature:
|
78 |
+
Mais je vous prie, qu'on me donne quelque recompenſe :
|
79 |
+
Pourquoy donc auec tant de gloire &c en vne heure?
|
80 |
+
Cela fait voir bien clairement que ma defpenfe
|
81 |
+
Eft plus grande que celle des autres hommes ;
|
82 |
+
Car il eft certain que les Princes font moins heureux;
|
83 |
+
Que ceux qui ont eu leur vie par leurs mains confacrée.
|
84 |
+
Il y a deux fortes de perfonnes dans le monde :
|
85 |
+
Les uns fe nomment gens de lettres ou de prouefleurz,
|
86 |
+
Dont on dit que c'eft eux-mesme qui nous donnent tousjours
|
87 |
+
Leur efprit & leur cœur aux affaires publiques ;
|
88 |
+
Ainfi ils s'appellent gens de cour, comme fi cela
|
89 |
+
N'eſtoit autre chofe que ces grands Seigneurs qui gouuernenc
|
90 |
+
L'Etat,&qui ſe conferuent toujours avec honneur.
|
91 |
+
Ils font fort peu de choſes fans ambition ;
|
92 |
+
</blockquote>
|
93 |
|
94 |
## Training
|
95 |
Pleias-360m-Preview was fully pretrained at Jean Zay on 64 h100s for 46 hours with Nanotron, the pretraining library from HuggingFace. We provide the complete settings as a yaml file as part of our release.
|