--- license: apache-2.0 datasets: - abacusai/MetaMathFewshot model-index: - name: MM-Orc-Vic-bagel-34b-c1000 results: - task: type: text-generation name: Text Generation dataset: name: AI2 Reasoning Challenge (25-Shot) type: ai2_arc config: ARC-Challenge split: test args: num_few_shot: 25 metrics: - type: acc_norm value: 67.32 name: normalized accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=abacusai/MM-Orc-Vic-bagel-34b-c1000 name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: HellaSwag (10-Shot) type: hellaswag split: validation args: num_few_shot: 10 metrics: - type: acc_norm value: 83.52 name: normalized accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=abacusai/MM-Orc-Vic-bagel-34b-c1000 name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: MMLU (5-Shot) type: cais/mmlu config: all split: test args: num_few_shot: 5 metrics: - type: acc value: 76.09 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=abacusai/MM-Orc-Vic-bagel-34b-c1000 name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: TruthfulQA (0-shot) type: truthful_qa config: multiple_choice split: validation args: num_few_shot: 0 metrics: - type: mc2 value: 60.57 source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=abacusai/MM-Orc-Vic-bagel-34b-c1000 name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: Winogrande (5-shot) type: winogrande config: winogrande_xl split: validation args: num_few_shot: 5 metrics: - type: acc value: 82.32 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=abacusai/MM-Orc-Vic-bagel-34b-c1000 name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: GSM8k (5-shot) type: gsm8k config: main split: test args: num_few_shot: 5 metrics: - type: acc value: 72.25 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=abacusai/MM-Orc-Vic-bagel-34b-c1000 name: Open LLM Leaderboard --- Finetune of the DPO Bagel model (https://huggingface.co/jondurbin/nontoxic-bagel-34b-v0.2) on the MetamathFewshot (https://huggingface.co/datasets/abacusai/MetaMathFewshot) dataset ### Evaluation Results | Average | ARC | HellaSwag | MMLU | TruthfulQA | Winogrande | GSM8K | | --- | --- | --- | --- | --- | --- | --- | | | | | | | | | For comparison the GSM8K score for the original `nontoxic-bagel-34b-v0.2` model was 58.45 and average score was 74.69 # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard) Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_abacusai__MM-Orc-Vic-bagel-34b-c1000) | Metric |Value| |---------------------------------|----:| |Avg. |73.68| |AI2 Reasoning Challenge (25-Shot)|67.32| |HellaSwag (10-Shot) |83.52| |MMLU (5-Shot) |76.09| |TruthfulQA (0-shot) |60.57| |Winogrande (5-shot) |82.32| |GSM8k (5-shot) |72.25|