---
license: apache-2.0
language:
- en
- fi
base_model:
- LumiOpen/Poro-34B
datasets:
- sablo/oasst2_curated
- LumiOpen/instruction-collection-fin
---

This is an SFT-tuned model of [Poro-34B](https://huggingface.co/LumiOpen/Poro-34B) with English and Finnish data. We trained this model as part of our experiments on the impact of multilingual instruction-tuning on Poro-34B. For a better chat experience, we recommend using [Poro-34B-chat](https://huggingface.co/LumiOpen/Poro-34B-chat) instead.

## Datasets

### SFT

We use a curated subset of Open Assistant 2 and translated the dataset into Finnish using Poro-34B.

- **English**: [oasst2_curated](https://huggingface.co/datasets/sablo/oasst2_curated)

- **Finnish**: [instruction-collection-fin](https://huggingface.co/datasets/LumiOpen/instruction-collection-fin) (oasst2 subset)

### DPO

We use the HelpSteer2 preference binarized into chosen-rejected pairs using the helpfulness score as recommended in the [HelpSteer2](https://arxiv.org/abs/2406.08673) paper. We translated the dataset into Finnish using Poro.

- **English**: [HelpSteer2](https://huggingface.co/datasets/nvidia/HelpSteer2)

- **Finnish**: TBA

## Recipes

For finetuning, we used 4 nodes (8 x AMD MI250X) to obtain a global batch size of 128 for SFT and 64 for DPO. We used the [Alignment Handbook](https://github.com/huggingface/alignment-handbook/) codebase.

**SFT**

```
bf16: true
do_eval: true
evaluation_strategy: epoch
gradient_accumulation_steps: 2
gradient_checkpointing: true
gradient_checkpointing_kwargs:
  use_reentrant: False
learning_rate: 2.0e-05
log_level: info
logging_steps: 50  
logging_strategy: steps
lr_scheduler_type: cosine
max_seq_length: 2048
max_steps: -1
num_train_epochs: 3
output_dir: data/poro-sft-oasst2
overwrite_output_dir: true
per_device_eval_batch_size: 4
per_device_train_batch_size: 2
remove_unused_columns: true
save_strategy: "epoch"
save_total_limit: 1
seed: 42
warmup_ratio: 0.1
```

**DPO**
```
bf16: true
beta: 0.05
do_eval: true
evaluation_strategy: epoch
gradient_accumulation_steps: 1
gradient_checkpointing: true
gradient_checkpointing_kwargs:
  use_reentrant: False
learning_rate: 5.0e-7
log_level: info
logging_steps: 20
lr_scheduler_type: cosine
max_length: 1024
max_prompt_length: 512
num_train_epochs: 5
optim: adamw_torch
output_dir: data/poro-dpo-helpsteer2
per_device_train_batch_size: 2
per_device_eval_batch_size: 4
save_strategy: "epoch"
save_total_limit: 1
seed: 42
warmup_ratio: 0.1
```


## Evaluation

We use [IFEval](https://huggingface.co/datasets/google/IFEval) to evaluate the performance of the model in English. For Finnish, we translated the IFEval prompts to [Finnish](https://huggingface.co/datasets/LumiOpen/ifeval_mt) with DeepL. We report the instruction-level strict accuracy:

- **English**: 0.3997
- **Finnish**: 0.3448

## Citation

We discuss our experimental setup and results in our NoDaLiDa 2025 paper.

```
@inproceedings{
zosa2024got,
title={Got Compute, but No Data: Lessons From Post-training a Finnish {LLM}},
author={Elaine Zosa and Ville Komulainen and Sampo Pyysalo},
booktitle={The Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Language Technologies},
year={2024},
url={https://openreview.net/forum?id=8wWlu1stNK}
}
```