English
yintongl commited on
Commit
0858111
·
verified ·
1 Parent(s): f5d2d04

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +11 -12
README.md CHANGED
@@ -28,18 +28,17 @@ lm_eval --model hf --model_args pretrained="Intel/gpt-j-6b-int4-inc",autogptq=Tr
28
 
29
  | Metric | FP16 | INT4 |
30
  | -------------- | ------ | ------ |
31
- | Avg. | 0.5039 | 0.5034 |
32
- | mmlu | 0.2694 | 0.2793 |
33
- | lambada_openai | 0.6831 | 0.6790 |
34
- | hellaswag | 0.4953 | 0.4902 |
35
- | winogrande | 0.6409 | 0.6401 |
36
- | piqa | 0.7541 | 0.7465 |
37
- | truthfulqa_mc1 | 0.2020 | 0.2179 |
38
- | openbookqa | 0.2900 | 0.2900 |
39
- | boolq | 0.6544 | 0.6554 |
40
- | rte | 0.5451 | 0.5271 |
41
- | arc_easy | 0.6692 | 0.6734 |
42
- | arc_challenge | 0.3396 | 0.3387 |
43
 
44
 
45
  ### Reproduce the model
 
28
 
29
  | Metric | FP16 | INT4 |
30
  | -------------- | ------ | ------ |
31
+ | Avg. | 0.4998 | 0.4993 |
32
+ | mmlu | 0.2694 | 0.2750 |
33
+ | lambada_openai | 0.6831 | 0.6845 |
34
+ | hellaswag | 0.4953 | 0.4897 |
35
+ | winogrande | 0.6409 | 0.6425 |
36
+ | piqa | 0.7541 | 0.7448 |
37
+ | truthfulqa_mc1 | 0.2020 | 0.2093 |
38
+ | openbookqa | 0.2900 | 0.2820 |
39
+ | boolq | 0.6544 | 0.6511 |
40
+ | arc_easy | 0.6692 | 0.6709 |
41
+ | arc_challenge | 0.3396 | 0.3430 |
 
42
 
43
 
44
  ### Reproduce the model