README.md

metadata

tags:
  - generated_from_keras_callback
model-index:
  - name: madatnlp/kor-math-roberta-finetune
    results: []

madatnlp/kor-math-roberta-finetune

This model is a fine-tuned version of klue/roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.7393
• Validation Loss: 0.7801
• Epoch: 45

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'AdamWeightDecay', 'learning_rate': 1e-05, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight_decay_rate': 0.01}
• training_precision: float32

Training results

Train Loss	Validation Loss	Epoch
3.6637	2.2056	0
2.0866	1.6338	1
1.7565	1.4797	2
1.5604	1.4183	3
1.4554	1.2678	4
1.3784	1.2015	5
1.2908	0.9909	6
1.2265	1.1029	7
1.2524	1.0225	8
1.1722	1.1167	9
1.1396	0.9434	10
1.1002	1.0683	11
1.0693	1.0637	12
1.0896	0.9698	13
1.0590	0.9038	14
1.0357	0.9302	15
1.0557	0.8600	16
1.0036	0.7892	17
1.0283	0.9425	18
0.9883	0.7521	19
0.9797	0.7950	20
0.9511	0.8072	21
0.9023	0.8780	22
0.9074	0.8745	23
0.9324	0.7436	24
0.8921	0.9032	25
0.9098	0.8011	26
0.8843	0.8527	27
0.8756	0.7803	28
0.8759	0.8922	29
0.8472	0.8286	30
0.8156	0.7801	31
0.8401	0.7904	32
0.8400	0.9007	33
0.8368	0.6959	34
0.8429	0.6646	35
0.8496	0.7386	36
0.8168	0.7544	37
0.7927	0.8467	38
0.8025	0.7375	39
0.7893	0.7091	40
0.7762	0.6758	41
0.7516	0.8641	42
0.7645	0.7587	43
0.7790	0.7386	44
0.7393	0.7801	45

Framework versions

• Transformers 4.18.0
• TensorFlow 2.8.0
• Datasets 2.1.0
• Tokenizers 0.12.1