| Wandb Run: https://wandb.ai/eleutherai/pythia-rlhf/runs/gy2g8jj1 | |
| Model Evals: | |
| | Tasks |Version|Filter| Metric |Value | |Stderr| | |
| |--------------|-------|------|----------|-----:|---|-----:| | |
| |arc_challenge |Yaml |none |acc |0.2253|± |0.0122| | |
| | | |none |acc_norm |0.2278|± |0.0123| | |
| |arc_easy |Yaml |none |acc |0.2551|± |0.0089| | |
| | | |none |acc_norm |0.2567|± |0.0090| | |
| |lambada_openai|Yaml |none |perplexity| NaN|± | NaN| | |
| | | |none |acc |0.0016|± |0.0005| | |
| |logiqa |Yaml |none |acc |0.2028|± |0.0158| | |
| | | |none |acc_norm |0.2028|± |0.0158| | |
| |piqa |Yaml |none |acc |0.4946|± |0.0117| | |
| | | |none |acc_norm |0.4924|± |0.0117| | |
| |sciq |Yaml |none |acc |0.0140|± |0.0037| | |
| | | |none |acc_norm |0.0140|± |0.0037| | |
| |winogrande |Yaml |none |acc |0.5036|± |0.0141| | |
| |wsc |Yaml |none |acc |0.6346|± |0.0474| |