metadata

license: llama2

Kaori-34B-v1 Model Card

This Model Finetuned By Kaeri and Jenti.

Datasets

Open-Platypus
Dolphin
OpenOrca

We did not use GSM8k samples when generating data. Also we were careful of data contamination by similarity filtering the training data if the data fell into any of the following lists.

filtering_task_list = [
    'task228_arc_answer_generation_easy',
    'ai2_arc/ARC-Challenge:1.0.0',
    'ai2_arc/ARC-Easy:1.0.0',
    'task229_arc_answer_generation_hard',
    'hellaswag:1.1.0', 
    'task1389_hellaswag_completion',
    'cot_gsm8k',
    'cot_gsm8k_ii',
    'drop:2.0.0',
    'winogrande:1.1.0'
]

Framework:

https://github.com/hiyouga/LLaMA-Factory

Parameters:

Finetune_Type : LoRA
GPUs : A100x4(80GB)
Epochs : 3
Batchsize : 8