Update README.md

README.md

@@ -1,3 +1,56 @@
----
-license: llama3.1
----
+---
+license: llama3
+license_name: llama3
+license_link: LICENSE
+library_name: transformers
+tags:
+- not-for-all-audiences
+datasets:
+- crestf411/LimaRP-DS
+- Gryphe/Sonnet3.5-Charcard-Roleplay
+- anthracite-org/c2_logs_32k_mistral-v3_v1.2_no_system
+- anthracite-org/kalo-opus-instruct-22k-no-refusal-no-system
+- anthracite-org/kalo-opus-instruct-3k-filtered-no-system
+- anthracite-org/nopm_claude_writing_fixed
+base_model:
+- nvidia/Llama-3.1-Nemotron-70B-Instruct-HF
+---
+
+Sunfall (2024-10-28) v0.7.0 trained directly against, and merged with [Nemotron 70B Instruct](nvidia/Llama-3.1-Nemotron-70B-Instruct-HF).
+
+It also contains samples from [Antracite.Org](https://huggingface.co/anthracite-org) datasets. See bottom for details.
+
+Significant revamping of the dataset metadata generation process, resulting in higher quality dataset overall. *The "Diamond Law" experiment has been removed as it didn't seem to affect the model output enough to warrant set up complexity.*
+
+Recommended starting point:
+
+* Temperature: **1**
+* MinP: **0.05~0.1**
+* DRY: **0.8 1.75 2 0**
+
+At early context, I recommend keeping XTC disabled. Once you hit higher context sizes (10k+), enabling XTC at 0.1 / 0.5 seems to significantly improve the output, but YMMV. If the output drones on and is uninspiring, XTC can be extremely effective.
+
+General heuristic:
+
+* Lots of slop? Temperature is too low. Raise it, or enable XTC. For early context, temp bump is probably preferred.
+* Is the model making mistakes about subtle or obvious details in the scene? Temperature is too high, OR XTC is enabled and/or XTC settings are too high. Lower temp and/or disable XTC.
+
+*Mergers/fine-tuners: [there is a LoRA of this model](https://huggingface.co/crestf411/sunfall-peft/tree/main/l3.1-nemotron-70B). Consider merging that instead of merging this model.*
+
+This model has been trained on context that mimics that of Silly Tavern's "Llama 3 Instruct" preset, with character names added.
+
+The model has also been trained to do interactive storywriting. You may steer the model towards specific content by "responding" to the model like so:
+
+```
+Continue writing adhering to the following scenario: (things you want to happen next)
+```
+
+Additional inclusions (random sampled sub-set, cursorily quality-checked) from:
+
+- [Gryphe/Sonnet3.5-Charcard-Roleplay](https://huggingface.co/datasets/Gryphe/Sonnet3.5-Charcard-Roleplay)
+- [anthracite-org/c2_logs_32k_mistral-v3_v1.2_no_system](https://huggingface.co/datasets/anthracite-org/c2_logs_32k_mistral-v3_v1.2_no_system)
+- [anthracite-org/kalo-opus-instruct-22k-no-refusal-no-system](https://huggingface.co/datasets/anthracite-org/kalo-opus-instruct-22k-no-refusal-no-system)
+- [anthracite-org/kalo-opus-instruct-3k-filtered-no-system](https://huggingface.co/datasets/anthracite-org/kalo-opus-instruct-3k-filtered-no-system)
+- [anthracite-org/nopm_claude_writing_fixed](https://huggingface.co/datasets/anthracite-org/nopm_claude_writing_fixed)
+
+As such, the dataset is not 100% slop free, but this addition likely helps the model be a better roleplayer. At some point, I intend to clean up and release the samples, deslopped.