DavidAU commited on
Commit
d8ec365
·
verified ·
1 Parent(s): 404f2cb

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +44 -0
README.md CHANGED
@@ -45,6 +45,50 @@ Recommended Rep Pen of 1.05 or higher, temp range 0-5.
45
 
46
  Example outputs below.
47
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
48
  <B>Other Versions of "Gemma The Writer": </B>
49
 
50
  The second version of this model is "Deadline" at 10B parameters. It is a specially modified version that changes
 
45
 
46
  Example outputs below.
47
 
48
+ <B>Settings, Quants and Critical Operations Notes:</b>
49
+
50
+ This model has been modified ("Brainstorm") to alter prose output, and generally outputs longer text than average.
51
+
52
+ Change in temp (ie, .4, .8, 1.5, 2, 3 ) will drastically alter output.
53
+
54
+ Rep pen settings will also alter output too.
55
+
56
+ This model needs "rep pen" of 1.02 or higher.
57
+
58
+ For role play: Rep pen of 1.05 to 1.08 is suggested.
59
+
60
+ Raise/lower rep pen SLOWLY ie: 1.011, 1.012 ...
61
+
62
+ Rep pen will alter prose, word choice (lower rep pen=small words / more small word - sometimes) and creativity.
63
+
64
+ To really push the model:
65
+
66
+ Rep pen 1.05 or lower / Temp 3+ ... be ready to stop the output because it may go and go at these strong settings.
67
+
68
+ You can also set a "hard stop" - maximum tokens generation - too to address lower rep pen settings / high creativity settings.
69
+
70
+ Longer prompts vastly increase the quality of the model's output.
71
+
72
+ QUANT CHOICE(S):
73
+
74
+ Higher quants will have more detail, nuance and in some cases stronger "emotional" levels. Characters will also be
75
+ more "fleshed out" too. Sense of "there" will also increase.
76
+
77
+ Q4KM/Q4KS are good, strong quants however if you can run Q5, Q6 or Q8 - go for the highest quant you can.
78
+
79
+ This repo also has 3 "ARM" quants for computers that support this quant. If you use these on a "non arm" machine token per second will be very low.
80
+
81
+ IQ4XS: Due to the unusual nature of this quant (mixture/processing), generations from it will be different then other quants.
82
+
83
+ You may want to try it / compare it to other quant(s) output.
84
+
85
+ Special note on Q2k/Q3 quants:
86
+
87
+ You may need to use temp 2 or lower with these quants (1 or lower for q2k). Just too much compression at this level, damaging the model. I will see if Imatrix versions
88
+ of these quants will function better.
89
+
90
+ Rep pen adjustments may also be required to get the most out of this model at this/these quant level(s).
91
+
92
  <B>Other Versions of "Gemma The Writer": </B>
93
 
94
  The second version of this model is "Deadline" at 10B parameters. It is a specially modified version that changes