Merge branch 'main' of https://huggingface.co/stabilityai/stablelm-2-12b into main

README.md

@@ -24,6 +24,8 @@ datasets:
 
 `Stable LM 2 12B` is a 12.1 billion parameter decoder-only language model pre-trained on 2 trillion tokens of diverse multilingual and code datasets for two epochs.
 
+Please note: For commercial use, please refer to https://stability.ai/membership.
+
 ## Usage
 
 Get started generating text with `Stable LM 2 12B` by using the following code snippet:
@@ -81,7 +83,8 @@ print(tokenizer.decode(tokens[0], skip_special_tokens=True))
 * **Language(s)**: English
 * **Paper**: [Stable LM 2 Technical Report](https://arxiv.org/abs/2402.17834)
 * **Library**: [GPT-NeoX](https://github.com/EleutherAI/gpt-neox)
-* **License**: [Stability AI Non-Commercial Research Community License](https://huggingface.co/stabilityai/stablelm-2-12b/blob/main/LICENSE). If you'd like to use this model for commercial products or purposes, please contact us [here](https://stability.ai/membership) to learn more.
+* **License**: [Stability AI Non-Commercial Research Community License](https://huggingface.co/stabilityai/stablelm-2-12b/blob/main/LICENSE).
+* **Commercial License**: to use this model commercially, please refer to https://stability.ai/membership
 * **Contact**: For questions and comments about the model, please email `[email protected]`
 
 ### Model Architecture
@@ -120,7 +123,7 @@ The model is pre-trained on the aforementioned datasets in `bfloat16` precision,
 
 ### Intended Use
 
-The model is intended to be used as a foundational base model for application-specific fine-tuning. Developers must evaluate and fine-tune the model for safe performance in downstream applications.
+The model is intended to be used as a foundational base model for application-specific fine-tuning. Developers must evaluate and fine-tune the model for safe performance in downstream applications. For commercial use, please refer to https://stability.ai/membership.
 
 ### Limitations and Bias
 ​
@@ -128,7 +131,7 @@ As a base model, this model may exhibit unreliable, unsafe, or other undesirable
 
 ## How to Cite
 
-```
+```bibtex
 @article{bellagente2024stable,
   title={Stable LM 2 1.6 B Technical Report},
   author={Bellagente, Marco and Tow, Jonathan and Mahan, Dakota and Phung, Duy and Zhuravinskyi, Maksym and Adithyan, Reshinth and Baicoianu, James and Brooks, Ben and Cooper, Nathan and Datta, Ashish and others},

tokenizer_config.json

@@ -306,6 +306,6 @@
   "eos_token": "<|endoftext|>",
   "model_max_length": 1000000000000000019884624838656,
   "pad_token": "<|endoftext|>",
-  "tokenizer_class": "GPT2Tokenizer",
+  "tokenizer_class": "GPT2TokenizerFast",
   "unk_token": "<|endoftext|>"
 }