File size: 2,390 Bytes
06f65a8 4e2d4f0 06f65a8 525a0f1 6d43162 525a0f1 6d43162 525a0f1 1de3164 525a0f1 04972db d34e613 3a40793 d34e613 3a40793 d34e613 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 |
---
library_name: peft
---
# AlpaGo: GPT-NeoX-20B Model Trained with Qlora Technique
AlpaGo is an adapter model trained using the Qlora technique on top of the GPT-NeoX-20B model. This repository contains the code and resources for AlpaGo, which can be used for natural language processing tasks. AlpaGo is built on the [GPT-NeoX-20B](https://huggingface.co/EleutherAI/gpt-neox-20b) architecture and developed by Math And AI Institute.
## Features
- AlpaGo adapter model trained with the Qlora technique
- Based on the GPT-NeoX-20B model, providing high-quality natural language processing capabilities on Engilish Language
## Installation
1. Clone the AlpaGo repository:
```python
!git clone https://github.com/exampleuser/alphago.git
```
2. Install the latest version of Python 3 if you haven't already.
3. Install the required dependencies:
```python
!pip install -r requirements.txt
```
## Usage
You can utilize AlpaGo to perform natural language processing tasks. Here's an example of how to use it:
```python
from peft import PeftModel
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig, GenerationConfig
model_id = "EleutherAI/gpt-neox-20b"
tokenizer = AutoTokenizer.from_pretrained(model_id)
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_use_double_quant=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.bfloat16
)
model = AutoModelForCausalLM.from_pretrained(model_id, quantization_config=bnb_config, device_map={"":0})
model = PeftModel.from_pretrained(model, "myzens/AlpaGo")
#You can change Here.
PROMPT = """Below is an instruction that describes a task. Write a response that appropriately completes the request.
### Instruction:
Write a short story about a lost key that unlocks a mysterious door.
### Response:"""
inputs = tokenizer(PROMPT, return_tensors="pt")
input_ids = inputs["input_ids"].cuda()
generation_config = GenerationConfig(
temperature=0.6,
top_p=0.95,
repetition_penalty=1.15,
)
print("Generating...")
generation_output = model.generate(
input_ids=input_ids,
generation_config=generation_config,
return_dict_in_generate=True,
output_scores=True,
max_new_tokens=256,
eos_token_id=tokenizer.eos_token_id,
pad_token_id=tokenizer.pad_token_id,
)
for s in generation_output.sequences:
print(tokenizer.decode(s))
```
|