Commit
·
76e7935
1
Parent(s):
2f91817
Update README.md
Browse files
README.md
CHANGED
|
@@ -60,6 +60,18 @@ You can use this model in your scripts as follows:
|
|
| 60 |
0.14446501433849335]}
|
| 61 |
```
|
| 62 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 63 |
|
| 64 |
## Training procedure
|
| 65 |
|
|
|
|
| 60 |
0.14446501433849335]}
|
| 61 |
```
|
| 62 |
|
| 63 |
+
## Performance
|
| 64 |
+
|
| 65 |
+
As Danish is, as far as we are aware, the only Scandinavian language with a gold standard NLI dataset, namely the [DanFEVER dataset](https://aclanthology.org/2021.nodalida-main.pdf#page=439), we report evaluation scores on the test split of that dataset.
|
| 66 |
+
|
| 67 |
+
We report Matthew's Correlation Coefficient (MCC), macro-average F1-score as well as accuracy.
|
| 68 |
+
|
| 69 |
+
| **Model** | **MCC** | **Macro-F1** | **Accuracy** | **Number of Parameters** |
|
| 70 |
+
| :-------- | :------------ | :--------- | :----------- | :----------- |
|
| 71 |
+
| [`alexandrainst/nb-bert-large-nli-scandi`](https://huggingface.co/alexandrainst/nb-bert-large-nli-scandi) | **73.80%** | **58.41%** | **86.98%** | 354M |
|
| 72 |
+
| [`alexandrainst/nb-bert-base-nli-scandi`](https://huggingface.co/alexandrainst/nb-bert-base-nli-scandi) | 62.44% | 55.00% | 80.42% | 178M |
|
| 73 |
+
| `alexandrainst/electra-small-nordic-nli-scandi` (this) | 47.28% | 48.88% | 73.46% | **22M** |
|
| 74 |
+
|
| 75 |
|
| 76 |
## Training procedure
|
| 77 |
|