gemma-merge / README.md
PostScriptum's picture
Upload folder using huggingface_hub
7bbe968 verified
metadata
base_model:
  - lemon07r/Gemma-2-Ataraxy-9B
  - rtzr/ko-gemma-2-9b-it
  - kdt3/1101_3_tab_v2.2.split.2_512
  - wzhouad/gemma-2-9b-it-WPO-HB
  - rtzr/ko-gemma-2-9b-it
  - ghost613/gemma9_on_korean_summary_events
library_name: transformers
tags:
  - mergekit
  - merge

output-model

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the breadcrumbs_ties merge method using lemon07r/Gemma-2-Ataraxy-9B as a base.

Models Merged

The following models were included in the merge:

Configuration

The following YAML configuration was used to produce this model:

slices:
  - sources:
      - model: lemon07r/Gemma-2-Ataraxy-9B
        layer_range: [0, 42]
        parameters:
            weight: 1
            density: 0.7
            gamma: 0.03
      - model: wzhouad/gemma-2-9b-it-WPO-HB
        layer_range: [0, 42]
        parameters:
            weight: 1
            density: 0.42
            gamma: 0.03
      - model: rtzr/ko-gemma-2-9b-it+kdt3/1101_3_tab_v2.2.split.2_512
        layer_range: [0, 42]
        parameters:
            weight: 1
            density: 0.42
            gamma: 0.03
      - model: rtzr/ko-gemma-2-9b-it+ghost613/gemma9_on_korean_summary_events # lora model loading
        layer_range: [0, 42]
        parameters:
            weight: 1
            density: 0.42
            gamma: 0.03
merge_method: breadcrumbs_ties
base_model: lemon07r/Gemma-2-Ataraxy-9B
dtype: bfloat16