mahimairaja commited on
Commit
f7c1b27
·
1 Parent(s): ea95720

Added cuda config on Sample Code

Browse files

I tried to replicate the `sample code` in the free versions of `kaggle`, `Colab` and `SageMaker Studio Lab`.
When running on GPU:
``` bash
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!
```

And While running on CPU:
```
RuntimeError: "LayerNormKernelImpl" not implemented for 'Half'
```

| Changes I made:

```diff
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("microsoft/phi-1_5", trust_remote_code=True, torch_dtype="auto")
tokenizer = AutoTokenizer.from_pretrained("microsoft/phi-1_5", trust_remote_code=True, torch_dtype="auto")

+ device = torch.device("cuda:0")
+ model.cuda()

inputs = tokenizer('''```python
def print_prime(n):
"""
Print all primes between 1 and n
- """''', return_tensors="pt", return_attention_mask=False)
+ """''', return_tensors="pt", return_attention_mask=False).to('cuda')

outputs = model.generate(**inputs, max_length=200)
text = tokenizer.batch_decode(outputs)[0]
print(text)
```


I could resolve the runtime issue on `GPU` by adding cuda settiings. I beleive this would help the co-developers to try out phi-1_5. Thanks!!!

Files changed (1) hide show
  1. README.md +5 -1
README.md CHANGED
@@ -104,11 +104,15 @@ from transformers import AutoModelForCausalLM, AutoTokenizer
104
 
105
  model = AutoModelForCausalLM.from_pretrained("microsoft/phi-1_5", trust_remote_code=True, torch_dtype="auto")
106
  tokenizer = AutoTokenizer.from_pretrained("microsoft/phi-1_5", trust_remote_code=True, torch_dtype="auto")
 
 
 
 
107
  inputs = tokenizer('''```python
108
  def print_prime(n):
109
  """
110
  Print all primes between 1 and n
111
- """''', return_tensors="pt", return_attention_mask=False)
112
 
113
  outputs = model.generate(**inputs, max_length=200)
114
  text = tokenizer.batch_decode(outputs)[0]
 
104
 
105
  model = AutoModelForCausalLM.from_pretrained("microsoft/phi-1_5", trust_remote_code=True, torch_dtype="auto")
106
  tokenizer = AutoTokenizer.from_pretrained("microsoft/phi-1_5", trust_remote_code=True, torch_dtype="auto")
107
+
108
+ device = torch.device("cuda:0")
109
+ model.cuda()
110
+
111
  inputs = tokenizer('''```python
112
  def print_prime(n):
113
  """
114
  Print all primes between 1 and n
115
+ """''', return_tensors="pt", return_attention_mask=False).to('cuda')
116
 
117
  outputs = model.generate(**inputs, max_length=200)
118
  text = tokenizer.batch_decode(outputs)[0]