Intel
/

SOLAR-10.7B-Instruct-v1.0-int4-inc

Model card Files Files and versions Community

yintongl commited on May 21, 2024

Commit

97b169f

·

verified ·

1 Parent(s): f96032e

Update README.md

Files changed (1) hide show

README.md +7 -7

README.md CHANGED Viewed

@@ -53,19 +53,19 @@ lm_eval --model hf --model_args pretrained="Intel/SOLAR-10.7B-Instruct-v1.0-int4
-| Metric         | FP16   | INT4   |
 | -------------- | ------ | ------ |
-| Avg.           | 0.6887 | 0.6849 |
-| mmlu           | 0.6328 | 0.6243 |
-| lambada_openai | 0.7279 | 0.7336 |
 | hellaswag      | 0.6868 | 0.6829 |
-| winogrande     | 0.7664 | 0.7695 |
 | piqa           | 0.8069 | 0.8074 |
 | truthfulqa_mc1 | 0.5777 | 0.5606 |
 | openbookqa     | 0.3640 | 0.3560 |
 | boolq          | 0.8853 | 0.8829 |
-| arc_easy       | 0.8325 | 0.8266 |
-| arc_challenge  | 0.6067 | 0.6049 |

+| Metric         | BF16   | INT4   |
 | -------------- | ------ | ------ |
+| Avg.           | 0.6891 | 0.6849 |
+| mmlu           | 0.6358 | 0.6243 |
+| lambada_openai | 0.7277 | 0.7336 |
 | hellaswag      | 0.6868 | 0.6829 |
+| winogrande     | 0.7672 | 0.7695 |
 | piqa           | 0.8069 | 0.8074 |
 | truthfulqa_mc1 | 0.5777 | 0.5606 |
 | openbookqa     | 0.3640 | 0.3560 |
 | boolq          | 0.8853 | 0.8829 |
+| arc_easy       | 0.8321 | 0.8266 |
+| arc_challenge  | 0.6075 | 0.6049 |