Safetensors
llama
Pclanglais commited on
Commit
06937ff
·
verified ·
1 Parent(s): 0e97287

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +53 -1
README.md CHANGED
@@ -36,8 +36,60 @@ Text generation is currently able to support a range of creative writing tasks i
36
 
37
  Pleias-360m-Preview has been successfully adapted for continuous pretraining and full-fine-tuning on document processing tasks such as RAG, translation or OCR correction. Given the small size of the model we do not recommend fine-tuning methods based on LORA.
38
 
39
- ## Inference example:
 
40
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
41
 
42
  ## Training
43
  Pleias-360m-Preview was fully pretrained at Jean Zay on 64 h100s for 46 hours with Nanotron, the pretraining library from HuggingFace. We provide the complete settings as a yaml file as part of our release.
 
36
 
37
  Pleias-360m-Preview has been successfully adapted for continuous pretraining and full-fine-tuning on document processing tasks such as RAG, translation or OCR correction. Given the small size of the model we do not recommend fine-tuning methods based on LORA.
38
 
39
+ ## Examples
40
+ With only 360 million parameters, Pleias-360m-Preview is able to generate literary texts in multiple languages:
41
 
42
+ English poetry:
43
+
44
+ <blockquote>
45
+ <b>Saturn is fallen, am I too to fall?
46
+ Am I to leave this haven of my rest,
47
+ This cradle of my glory, this soft clime,
48
+ This calm luxuriance of blissful light,</b>
49
+ And the sweetness that was not in me?
50
+ I have been a slave and an enemy.
51
+ The world has made me its prey ; it's wrong;
52
+ It's cruelty! It's madness! — but you know
53
+ That all things are good for one man or another :
54
+ You see how he can be happy without them:
55
+ He cannot be miserable unless they come
56
+ To his heart with their own poisonous breaths.
57
+ But if there were no other way left us,
58
+ We should never find our happiness again.
59
+ If we could only live on earth alone,
60
+ With nothing else about us than ourselves,
61
+ Our very being would become more real.
62
+ There must be some other place where we may
63
+ Live happily as before, and enjoy life
64
+ As well as now. But what shall we do here?
65
+ What will happen when we die? What will hap-
66
+ [pen then?
67
+ When death comes, let him who knows the end
68
+ </blockquote>
69
+
70
+ French classic play with the 17th century writing style (including the long s)
71
+
72
+ <blockquote>
73
+ <b>Helas! pour mon malheur ie l’ay trop écouté.
74
+ Ie n’ay point du ſilence affecté le myſtère.
75
+ Ie croyois ſans péril pouuoir eſtre ſincere.
76
+ Et ſans armer mes yeux d’vn moment de rigueur,</b>
77
+ Je ne voy pas que ce foit vn crime à la nature:
78
+ Mais je vous prie, qu'on me donne quelque recompenſe :
79
+ Pourquoy donc auec tant de gloire &c en vne heure?
80
+ Cela fait voir bien clairement que ma defpenfe
81
+ Eft plus grande que celle des autres hommes ;
82
+ Car il eft certain que les Princes font moins heureux;
83
+ Que ceux qui ont eu leur vie par leurs mains confacrée.
84
+ Il y a deux fortes de perfonnes dans le monde :
85
+ Les uns fe nomment gens de lettres ou de prouefleurz,
86
+ Dont on dit que c'eft eux-mesme qui nous donnent tousjours
87
+ Leur efprit & leur cœur aux affaires publiques ;
88
+ Ainfi ils s'appellent gens de cour, comme fi cela
89
+ N'eſtoit autre chofe que ces grands Seigneurs qui gouuernenc
90
+ L'Etat,&qui ſe conferuent toujours avec honneur.
91
+ Ils font fort peu de choſes fans ambition ;
92
+ </blockquote>
93
 
94
  ## Training
95
  Pleias-360m-Preview was fully pretrained at Jean Zay on 64 h100s for 46 hours with Nanotron, the pretraining library from HuggingFace. We provide the complete settings as a yaml file as part of our release.