Update README.md
Browse files
README.md
CHANGED
@@ -20,7 +20,7 @@ Dzarashield is a fine-tuned model based on [DzaraBert](https://huggingface.co/Si
|
|
20 |
It's important to note that this model has been fine-tuned solely on Arabic characters, which means that tokens from other languages have been pruned.
|
21 |
|
22 |
# How to use
|
23 |
-
|
24 |
```
|
25 |
!git lfs install
|
26 |
!git clone https://huggingface.co/Sifal/dzarashield
|
@@ -36,7 +36,22 @@ dzarashield.load_state_dict(torch.load(PATH))
|
|
36 |
tokenizer = PreTrainedTokenizerFast(tokenizer_file="tokenizer.json")
|
37 |
|
38 |
```
|
|
|
39 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
40 |
## Acknowledgments
|
41 |
|
42 |
Dzarashield is built upon the foundations of [Dziribert](https://huggingface.co/alger-ia/dziribert), and I am grateful for their work in making this project possible.
|
|
|
20 |
It's important to note that this model has been fine-tuned solely on Arabic characters, which means that tokens from other languages have been pruned.
|
21 |
|
22 |
# How to use
|
23 |
+
## Setup:
|
24 |
```
|
25 |
!git lfs install
|
26 |
!git clone https://huggingface.co/Sifal/dzarashield
|
|
|
36 |
tokenizer = PreTrainedTokenizerFast(tokenizer_file="tokenizer.json")
|
37 |
|
38 |
```
|
39 |
+
## Example:
|
40 |
|
41 |
+
```
|
42 |
+
idx_to_label = {0: 'non-hate', 1: 'hate'}
|
43 |
+
sentences = ['يا وحد الشموتي، تكول دجاج آآآه', 'واش خويا راك غايا؟']
|
44 |
+
|
45 |
+
def predict_label(sentence):
|
46 |
+
tokenized = tokenizer(sentence, return_tensors='pt')
|
47 |
+
with torch.no_grad():
|
48 |
+
outputs = dzarashield(**tokenized)
|
49 |
+
return idx_to_label[outputs.logits.argmax().item()]
|
50 |
+
|
51 |
+
for sentence in sentences:
|
52 |
+
label = predict_label(sentence)
|
53 |
+
print(f'sentence: {sentence} label: {label}')
|
54 |
+
```
|
55 |
## Acknowledgments
|
56 |
|
57 |
Dzarashield is built upon the foundations of [Dziribert](https://huggingface.co/alger-ia/dziribert), and I am grateful for their work in making this project possible.
|