Added cuda config on Sample Code
Browse filesI tried to replicate the `sample code` in the free versions of `kaggle`, `Colab` and `SageMaker Studio Lab`.
When running on GPU:
``` bash
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!
```
And While running on CPU:
```
RuntimeError: "LayerNormKernelImpl" not implemented for 'Half'
```
| Changes I made:
```diff
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("microsoft/phi-1_5", trust_remote_code=True, torch_dtype="auto")
tokenizer = AutoTokenizer.from_pretrained("microsoft/phi-1_5", trust_remote_code=True, torch_dtype="auto")
+ device = torch.device("cuda:0")
+ model.cuda()
inputs = tokenizer('''```python
def print_prime(n):
"""
Print all primes between 1 and n
- """''', return_tensors="pt", return_attention_mask=False)
+ """''', return_tensors="pt", return_attention_mask=False).to('cuda')
outputs = model.generate(**inputs, max_length=200)
text = tokenizer.batch_decode(outputs)[0]
print(text)
```
I could resolve the runtime issue on `GPU` by adding cuda settiings. I beleive this would help the co-developers to try out phi-1_5. Thanks!!!
@@ -104,11 +104,15 @@ from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
104 |
|
105 |
model = AutoModelForCausalLM.from_pretrained("microsoft/phi-1_5", trust_remote_code=True, torch_dtype="auto")
|
106 |
tokenizer = AutoTokenizer.from_pretrained("microsoft/phi-1_5", trust_remote_code=True, torch_dtype="auto")
|
|
|
|
|
|
|
|
|
107 |
inputs = tokenizer('''```python
|
108 |
def print_prime(n):
|
109 |
"""
|
110 |
Print all primes between 1 and n
|
111 |
-
"""''', return_tensors="pt", return_attention_mask=False)
|
112 |
|
113 |
outputs = model.generate(**inputs, max_length=200)
|
114 |
text = tokenizer.batch_decode(outputs)[0]
|
|
|
104 |
|
105 |
model = AutoModelForCausalLM.from_pretrained("microsoft/phi-1_5", trust_remote_code=True, torch_dtype="auto")
|
106 |
tokenizer = AutoTokenizer.from_pretrained("microsoft/phi-1_5", trust_remote_code=True, torch_dtype="auto")
|
107 |
+
|
108 |
+
device = torch.device("cuda:0")
|
109 |
+
model.cuda()
|
110 |
+
|
111 |
inputs = tokenizer('''```python
|
112 |
def print_prime(n):
|
113 |
"""
|
114 |
Print all primes between 1 and n
|
115 |
+
"""''', return_tensors="pt", return_attention_mask=False).to('cuda')
|
116 |
|
117 |
outputs = model.generate(**inputs, max_length=200)
|
118 |
text = tokenizer.batch_decode(outputs)[0]
|