AchyuthGamer
commited on
Commit
·
03ae4e7
1
Parent(s):
77b181b
Update README.md
Browse files
README.md
CHANGED
@@ -1,9 +1,9 @@
|
|
1 |
---
|
2 |
-
base_model:
|
3 |
inference: true
|
4 |
license: apache-2.0
|
5 |
model_creator: Achyuth Gamer
|
6 |
-
model_name: OpenGPT 7b
|
7 |
model_type: opengpt
|
8 |
pipeline_tag: text-generation
|
9 |
prompt_template: '<s>[INST] {prompt} [/INST]'
|
@@ -12,14 +12,14 @@ tags:
|
|
12 |
- finetuned
|
13 |
---
|
14 |
|
15 |
-
#
|
16 |
- Model creator: [Achyuth Gamer](https://huggingface.co/AchyuthGamer)
|
17 |
- Original model: [OpenGPT](https://huggingface.co/AchyuthGamer/OpenGPT)
|
18 |
|
19 |
<!-- description start -->
|
20 |
## Description
|
21 |
|
22 |
-
This repo contains GPTQ model files for [Achyuth AI's OpenGPT 7B
|
23 |
|
24 |
Multiple GPTQ parameter permutations are provided; see Provided Files below for details of the options provided, their parameters, and the software used to create them.
|
25 |
|
@@ -27,7 +27,7 @@ Multiple GPTQ parameter permutations are provided; see Provided Files below for
|
|
27 |
|
28 |
These models are confirmed to work with ExLlama v1.
|
29 |
|
30 |
-
At the time of writing (September 28th), AutoGPTQ has not yet added support for the new
|
31 |
|
32 |
These GPTQs were made directly from Transformers, and so can be loaded via the Transformers interface. They can't be loaded directly from AutoGPTQ.
|
33 |
|
@@ -42,11 +42,11 @@ pip3 install git+https://github.com/huggingface/transformers.git@72958fcd3c98a7a
|
|
42 |
|
43 |
* [AWQ model(s) for GPU inference.](https://huggingface.co/AchyuthGamer/OpenGPT-7b-0.1)
|
44 |
* [GPTQ models for GPU inference, with multiple quantisation parameter options.](https://huggingface.co/AchyuthGamer/OpenGPT-7b-0.1)
|
45 |
-
* [
|
46 |
<!-- repositories-available end -->
|
47 |
|
48 |
<!-- prompt-template start -->
|
49 |
-
## Prompt template:
|
50 |
|
51 |
```
|
52 |
<s>[INST] {prompt} [/INST]
|
@@ -104,7 +104,7 @@ I recommend using the `huggingface-hub` Python library:
|
|
104 |
pip3 install huggingface-hub
|
105 |
```
|
106 |
|
107 |
-
To download the `main` branch to a folder called `
|
108 |
|
109 |
```shell
|
110 |
mkdir OpenGPT-7b-0.1
|
@@ -114,7 +114,7 @@ huggingface-cli download AchyuthGamer/OpenGPT-7b-0.1 --local-dir OpenGPT-7b-0.1
|
|
114 |
To download from a different branch, add the `--revision` parameter:
|
115 |
|
116 |
```shell
|
117 |
-
mkdir
|
118 |
huggingface-cli download AchyuthGamer/OpenGPT-7b-0.1 --revision gptq-4bit-32g-actorder_True --local-dir OpenGPT-7b-0.1 --local-dir-use-symlinks False
|
119 |
```
|
120 |
|
@@ -166,13 +166,13 @@ Please make sure you're using the latest version of [text-generation-webui](http
|
|
166 |
It is strongly recommended to use the text-generation-webui one-click-installers unless you're sure you know how to make a manual install.
|
167 |
|
168 |
1. Click the **Model tab**.
|
169 |
-
2. Under **Download custom model or LoRA**, enter `
|
170 |
-
- To download from a specific branch, enter for example `
|
171 |
- see Provided Files above for the list of branches for each option.
|
172 |
3. Click **Download**.
|
173 |
4. The model will start downloading. Once it's finished it will say "Done".
|
174 |
5. In the top left, click the refresh icon next to **Model**.
|
175 |
-
6. In the **Model** dropdown, choose the model you just downloaded: `
|
176 |
7. The model will automatically load, and is now ready for use!
|
177 |
8. If you want any custom settings, set them and then click **Save settings for this model** followed by **Reload the Model** in the top right.
|
178 |
* Note that you do not need to and should not set manual GPTQ parameters any more. These are set automatically from the file `quantize_config.json`.
|
@@ -180,7 +180,7 @@ It is strongly recommended to use the text-generation-webui one-click-installers
|
|
180 |
<!-- README_GPTQ.md-text-generation-webui end -->
|
181 |
|
182 |
<!-- README_GPTQ.md-use-from-python start -->
|
183 |
-
## How to use this
|
184 |
|
185 |
### Install the necessary packages
|
186 |
|
@@ -207,7 +207,7 @@ pip3 install .
|
|
207 |
```python
|
208 |
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
|
209 |
|
210 |
-
model_name_or_path = "
|
211 |
# To use a different branch, change revision
|
212 |
# For example: revision="gptq-4bit-32g-actorder_True"
|
213 |
model = AutoModelForCausalLM.from_pretrained(model_name_or_path,
|
@@ -259,7 +259,7 @@ The files provided are only tested to work with ExLlama v1, and Transformers 4.3
|
|
259 |
|
260 |
For further support, and discussions on these models and AI in general, join us at:
|
261 |
|
262 |
-
[
|
263 |
|
264 |
## Thanks, and how to contribute
|
265 |
|
@@ -273,28 +273,16 @@ If you're able and willing to contribute it will be most gratefully received and
|
|
273 |
|
274 |
Donaters will get priority support on any and all AI/LLM/model questions and requests, access to a private Discord room, plus other benefits.
|
275 |
|
276 |
-
* Patreon: https://patreon.com/TheBlokeAI
|
277 |
-
* Ko-Fi: https://ko-fi.com/TheBlokeAI
|
278 |
-
|
279 |
-
**Special thanks to**: Aemon Algiz.
|
280 |
-
|
281 |
-
**Patreon special mentions**: Pierre Kircher, Stanislav Ovsiannikov, Michael Levine, Eugene Pentland, Andrey, 준교 김, Randy H, Fred von Graf, Artur Olbinski, Caitlyn Gatomon, terasurfer, Jeff Scroggin, James Bentley, Vadim, Gabriel Puliatti, Harry Royden McLaughlin, Sean Connelly, Dan Guido, Edmond Seymore, Alicia Loh, subjectnull, AzureBlack, Manuel Alberto Morcote, Thomas Belote, Lone Striker, Chris Smitley, Vitor Caleffi, Johann-Peter Hartmann, Clay Pascal, biorpg, Brandon Frisco, sidney chen, transmissions 11, Pedro Madruga, jinyuan sun, Ajan Kanaga, Emad Mostaque, Trenton Dambrowitz, Jonathan Leane, Iucharbius, usrbinkat, vamX, George Stoitzev, Luke Pendergrass, theTransient, Olakabola, Swaroop Kallakuri, Cap'n Zoog, Brandon Phillips, Michael Dempsey, Nikolai Manek, danny, Matthew Berman, Gabriel Tamborski, alfie_i, Raymond Fosdick, Tom X Nguyen, Raven Klaugh, LangChain4j, Magnesian, Illia Dulskyi, David Ziegler, Mano Prime, Luis Javier Navarrete Lozano, Erik Bjäreholt, 阿明, Nathan Dryer, Alex, Rainer Wilmers, zynix, TL, Joseph William Delisle, John Villwock, Nathan LeClaire, Willem Michiel, Joguhyik, GodLy, OG, Alps Aficionado, Jeffrey Morgan, ReadyPlayerEmma, Tiffany J. Kim, Sebastain Graf, Spencer Kim, Michael Davis, webtim, Talal Aujan, knownsqashed, John Detwiler, Imad Khwaja, Deo Leter, Jerry Meng, Elijah Stavena, Rooh Singh, Pieter, SuperWojo, Alexandros Triantafyllidis, Stephen Murray, Ai Maven, ya boyyy, Enrico Ros, Ken Nordquist, Deep Realms, Nicholas, Spiking Neurons AB, Elle, Will Dee, Jack West, RoA, Luke @flexchar, Viktor Bowallius, Derek Yates, Subspace Studios, jjj, Toran Billups, Asp the Wyvern, Fen Risland, Ilya, NimbleBox.ai, Chadd, Nitin Borwankar, Emre, Mandus, Leonard Tan, Kalila, K, Trailburnt, S_X, Cory Kujawski
|
282 |
-
|
283 |
-
|
284 |
-
Thank you to all my generous patrons and donaters!
|
285 |
-
|
286 |
-
And thank you again to a16z for their generous grant.
|
287 |
-
|
288 |
<!-- footer end -->
|
289 |
|
290 |
-
# Original model card:
|
291 |
|
292 |
|
293 |
-
# Model Card for
|
294 |
|
295 |
-
The
|
296 |
|
297 |
-
For full details of this model please read our [release blog post](https://
|
298 |
|
299 |
## Instruction format
|
300 |
|
@@ -314,8 +302,8 @@ from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
314 |
|
315 |
device = "cuda" # the device to load the model onto
|
316 |
|
317 |
-
model = AutoModelForCausalLM.from_pretrained("
|
318 |
-
tokenizer = AutoTokenizer.from_pretrained("
|
319 |
|
320 |
messages = [
|
321 |
{"role": "user", "content": "What is your favourite condiment?"},
|
@@ -334,7 +322,7 @@ print(decoded[0])
|
|
334 |
```
|
335 |
|
336 |
## Model Architecture
|
337 |
-
This instruction model is based on
|
338 |
- Grouped-Query Attention
|
339 |
- Sliding-Window Attention
|
340 |
- Byte-fallback BPE tokenizer
|
@@ -350,7 +338,7 @@ File "/transformers/models/auto/configuration_auto.py", line 1022, in from_pretr
|
|
350 |
config_class = CONFIG_MAPPING[config_dict["model_type"]]
|
351 |
File "/transformers/models/auto/configuration_auto.py", line 723, in getitem
|
352 |
raise KeyError(key)
|
353 |
-
KeyError: '
|
354 |
```
|
355 |
|
356 |
Installing transformers from source should solve the issue
|
@@ -360,10 +348,10 @@ This should not be required after transformers-v4.33.4.
|
|
360 |
|
361 |
## Limitations
|
362 |
|
363 |
-
The
|
364 |
It does not have any moderation mechanisms. We're looking forward to engaging with the community on ways to
|
365 |
make the model finely respect guardrails, allowing for deployment in environments requiring moderated outputs.
|
366 |
|
367 |
-
## The
|
368 |
|
369 |
-
|
|
|
1 |
---
|
2 |
+
base_model: OpenGPTai/OpenGPT-7B-Instruct-v1.0
|
3 |
inference: true
|
4 |
license: apache-2.0
|
5 |
model_creator: Achyuth Gamer
|
6 |
+
model_name: OpenGPT 7b v1.0
|
7 |
model_type: opengpt
|
8 |
pipeline_tag: text-generation
|
9 |
prompt_template: '<s>[INST] {prompt} [/INST]'
|
|
|
12 |
- finetuned
|
13 |
---
|
14 |
|
15 |
+
# OpenGPT 7B Instruct v1.0
|
16 |
- Model creator: [Achyuth Gamer](https://huggingface.co/AchyuthGamer)
|
17 |
- Original model: [OpenGPT](https://huggingface.co/AchyuthGamer/OpenGPT)
|
18 |
|
19 |
<!-- description start -->
|
20 |
## Description
|
21 |
|
22 |
+
This repo contains GPTQ model files for [Achyuth AI's OpenGPT 7B v1.0](https://huggingface.co/AchyuthGamer/OpenGPT-7b-1.0).
|
23 |
|
24 |
Multiple GPTQ parameter permutations are provided; see Provided Files below for details of the options provided, their parameters, and the software used to create them.
|
25 |
|
|
|
27 |
|
28 |
These models are confirmed to work with ExLlama v1.
|
29 |
|
30 |
+
At the time of writing (September 28th), AutoGPTQ has not yet added support for the new OpenGPT models.
|
31 |
|
32 |
These GPTQs were made directly from Transformers, and so can be loaded via the Transformers interface. They can't be loaded directly from AutoGPTQ.
|
33 |
|
|
|
42 |
|
43 |
* [AWQ model(s) for GPU inference.](https://huggingface.co/AchyuthGamer/OpenGPT-7b-0.1)
|
44 |
* [GPTQ models for GPU inference, with multiple quantisation parameter options.](https://huggingface.co/AchyuthGamer/OpenGPT-7b-0.1)
|
45 |
+
* [OpenGPT AI's original unquantised fp16 model in pytorch format, for GPU inference and for further conversions](https://huggingface.co/AchyuthGamer/OpenGPT-7b-0.1)
|
46 |
<!-- repositories-available end -->
|
47 |
|
48 |
<!-- prompt-template start -->
|
49 |
+
## Prompt template: OpenGPT
|
50 |
|
51 |
```
|
52 |
<s>[INST] {prompt} [/INST]
|
|
|
104 |
pip3 install huggingface-hub
|
105 |
```
|
106 |
|
107 |
+
To download the `main` branch to a folder called `OpenGPT-7B-Instruct-v1.0-GPTQ`:
|
108 |
|
109 |
```shell
|
110 |
mkdir OpenGPT-7b-0.1
|
|
|
114 |
To download from a different branch, add the `--revision` parameter:
|
115 |
|
116 |
```shell
|
117 |
+
mkdir OpenGPT-7B-Instruct-v1.0-GPTQ
|
118 |
huggingface-cli download AchyuthGamer/OpenGPT-7b-0.1 --revision gptq-4bit-32g-actorder_True --local-dir OpenGPT-7b-0.1 --local-dir-use-symlinks False
|
119 |
```
|
120 |
|
|
|
166 |
It is strongly recommended to use the text-generation-webui one-click-installers unless you're sure you know how to make a manual install.
|
167 |
|
168 |
1. Click the **Model tab**.
|
169 |
+
2. Under **Download custom model or LoRA**, enter `AchyuthGamer/OpenGPT-7B-Instruct-v1.0-GPTQ`.
|
170 |
+
- To download from a specific branch, enter for example `AchyuthGamer/OpenGPT-7B-Instruct-v1.0-GPTQ:gptq-4bit-32g-actorder_True`
|
171 |
- see Provided Files above for the list of branches for each option.
|
172 |
3. Click **Download**.
|
173 |
4. The model will start downloading. Once it's finished it will say "Done".
|
174 |
5. In the top left, click the refresh icon next to **Model**.
|
175 |
+
6. In the **Model** dropdown, choose the model you just downloaded: `OpenGPT-7B-Instruct-v1.0-GPTQ`
|
176 |
7. The model will automatically load, and is now ready for use!
|
177 |
8. If you want any custom settings, set them and then click **Save settings for this model** followed by **Reload the Model** in the top right.
|
178 |
* Note that you do not need to and should not set manual GPTQ parameters any more. These are set automatically from the file `quantize_config.json`.
|
|
|
180 |
<!-- README_GPTQ.md-text-generation-webui end -->
|
181 |
|
182 |
<!-- README_GPTQ.md-use-from-python start -->
|
183 |
+
## How to use this OpenGPT model from Python code
|
184 |
|
185 |
### Install the necessary packages
|
186 |
|
|
|
207 |
```python
|
208 |
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
|
209 |
|
210 |
+
model_name_or_path = "AchyuthGamer/OpenGPT-7B-v1.0"
|
211 |
# To use a different branch, change revision
|
212 |
# For example: revision="gptq-4bit-32g-actorder_True"
|
213 |
model = AutoModelForCausalLM.from_pretrained(model_name_or_path,
|
|
|
259 |
|
260 |
For further support, and discussions on these models and AI in general, join us at:
|
261 |
|
262 |
+
[Our AI's Discord server](https://discord.gg/accspard)
|
263 |
|
264 |
## Thanks, and how to contribute
|
265 |
|
|
|
273 |
|
274 |
Donaters will get priority support on any and all AI/LLM/model questions and requests, access to a private Discord room, plus other benefits.
|
275 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
276 |
<!-- footer end -->
|
277 |
|
278 |
+
# Original model card: OpenGPT AI's OpenGPT 7B Instruct v1.0
|
279 |
|
280 |
|
281 |
+
# Model Card for OpenGPT-7B-v1.0
|
282 |
|
283 |
+
The OpenGPT-7B-Instruct-v1.0 Large Language Model (LLM) is a instruct fine-tuned version of the [OpenGPT-7B-v1.0](https://huggingface.co/OpenGPTai/OpenGPT-7B-v1.0) generative text model using a variety of publicly available conversation datasets.
|
284 |
|
285 |
+
For full details of this model please read our [release blog post](https://OpenGPT.ai/news/announcing-OpenGPT-7b/)
|
286 |
|
287 |
## Instruction format
|
288 |
|
|
|
302 |
|
303 |
device = "cuda" # the device to load the model onto
|
304 |
|
305 |
+
model = AutoModelForCausalLM.from_pretrained("OpenGPTai/OpenGPT-7B-Instruct-v1.0")
|
306 |
+
tokenizer = AutoTokenizer.from_pretrained("OpenGPTai/OpenGPT-7B-Instruct-v1.0")
|
307 |
|
308 |
messages = [
|
309 |
{"role": "user", "content": "What is your favourite condiment?"},
|
|
|
322 |
```
|
323 |
|
324 |
## Model Architecture
|
325 |
+
This instruction model is based on OpenGPT-7B-v1.0, a transformer model with the following architecture choices:
|
326 |
- Grouped-Query Attention
|
327 |
- Sliding-Window Attention
|
328 |
- Byte-fallback BPE tokenizer
|
|
|
338 |
config_class = CONFIG_MAPPING[config_dict["model_type"]]
|
339 |
File "/transformers/models/auto/configuration_auto.py", line 723, in getitem
|
340 |
raise KeyError(key)
|
341 |
+
KeyError: 'OpenGPT'
|
342 |
```
|
343 |
|
344 |
Installing transformers from source should solve the issue
|
|
|
348 |
|
349 |
## Limitations
|
350 |
|
351 |
+
The OpenGPT 7B Instruct model is a quick demonstration that the base model can be easily fine-tuned to achieve compelling performance.
|
352 |
It does not have any moderation mechanisms. We're looking forward to engaging with the community on ways to
|
353 |
make the model finely respect guardrails, allowing for deployment in environments requiring moderated outputs.
|
354 |
|
355 |
+
## The OpenGPT Team
|
356 |
|
357 |
+
Achyuth, Ayush
|