File size: 1,318 Bytes
7abc2a5 5f426c0 7abc2a5 5f426c0 9c53fb0 540469a 5f426c0 c526ddd 5f426c0 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 |
---
library_name: transformers
tags: []
---
## Model Details
### Model Description
<!-- Provide a longer summary of what this model is. -->
This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
- **Developed by:** FrankL
- **Language(s) (NLP):** English
### Direct Use
```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained('FrankL/storytellerLM-v0.1', trust_remote_code=True, torch_dtype=torch.float16)
model = model.to(device='cuda')
tokenizer = AutoTokenizer.from_pretrained('FrankL/storytellerLM-v0.1', trust_remote_code=True)
def inference(
model: AutoModelForCausalLM,
tokenizer: AutoTokenizer,
input_text: str = "Once upon a time, ",
max_new_tokens: int = 16
):
inputs = tokenizer(input_text, return_tensors="pt").to(device)
outputs = model.generate(
**inputs,
pad_token_id=tokenizer.eos_token_id,
max_new_tokens=max_new_tokens,
do_sample=True,
top_k=40,
top_p=0.95,
temperature=0.8
)
generated_text = tokenizer.decode(
outputs[0],
skip_special_tokens=True
)
# print(outputs)
print(generated_text)
inference(model, tokenizer)
``` |