Update model from 2024-08-29

README.md

@@ -49,8 +49,8 @@ print(tokenizer.detokenize(translated[0][0]['tokens']))
 
 | testset                               | BLEU  |
 |---------------------------------------|-------|
-| test dataset (from train/dev/test)   	| 39.7	|
-| Flores200 dataset  					| 26.5	|
+| test dataset (from train/dev/test)   	| 41.0	|
+| Flores200 dataset  					| 27.0	|
 
 ## Additional information
 * https://github.com/Softcatala/nmt-models

config.json

@@ -1,2 +1,9 @@
 {
-}
+  "add_source_bos": false,
+  "add_source_eos": false,
+  "bos_token": "<s>",
+  "decoder_start_token": "<s>",
+  "eos_token": "</s>",
+  "layer_norm_epsilon": null,
+  "unk_token": "<unk>"
+}

model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:7f1d1be796eb583fedfb95181fec421329c85d4af0bcab74fdfde775a309b67c
-size 70727761
+oid sha256:a4c6e264fb0000a9685c5c457c28ea59f32473d2dddaa8aabbeb7f2b86e1e43b
+size 70727741

model_description.txt

@@ -1,9 +1,9 @@
 Model description: ita-cat
-Date: 2022-11-11
-TF version 2.10.0, OpenNMT version 2.29.1, CTranslate2 version 2.24.0
+Date: 2024-08-29
+TF version 2.10.0, OpenNMT version 2.29.1, CTranslate2 version 3.22.0
 Test data set
-  BLEU|nrefs:1|case:mixed|eff:no|tok:13a|smooth:exp|version:2.1.0 = 39.7 67.2/46.1/34.3/26.0 (BP = 0.973 ratio = 0.974 hyp_len = 147508 ref_len = 151502)
-chrF2|nrefs:1|case:mixed|eff:yes|nc:6|nw:0|space:no|version:2.1.0 = 63.2
+  BLEU|nrefs:1|bs:1000|seed:12345|case:mixed|eff:no|tok:13a|smooth:exp|version:2.1.0 = 41.0 (μ = 41.0 ± 0.6) 68.3/47.6/35.7/27.2 (BP = 0.972 ratio = 0.972 hyp_len = 181416 ref_len = 186644)
+chrF2|nrefs:1|bs:1000|seed:12345|case:mixed|eff:yes|nc:6|nw:0|space:no|version:2.1.0 = 64.2 (μ = 64.2 ± 0.5)
 Flores data set
-  BLEU|nrefs:1|case:mixed|eff:no|tok:13a|smooth:exp|version:2.1.0 = 26.5 58.1/32.6/20.2/12.8 (BP = 1.000 ratio = 1.028 hyp_len = 28075 ref_len = 27304)
-chrF2|nrefs:1|case:mixed|eff:yes|nc:6|nw:0|space:no|version:2.1.0 = 56.1
+  BLEU|nrefs:1|bs:1000|seed:12345|case:mixed|eff:no|tok:13a|smooth:exp|version:2.1.0 = 27.0 (μ = 26.9 ± 0.9) 58.3/33.1/20.7/13.3 (BP = 1.000 ratio = 1.034 hyp_len = 28237 ref_len = 27304)
+chrF2|nrefs:1|bs:1000|seed:12345|case:mixed|eff:yes|nc:6|nw:0|space:no|version:2.1.0 = 56.6 (μ = 56.6 ± 0.6)

sp_m.model

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:17adbdc0d9f2593177a7274e1d928828fff2ebb17d488355b328d4e707a47d47
-size 1167983
+oid sha256:02ceeec9cb66dcd10292dfcf80c2dbedf25fe427722b84845a6b2a58032254da
+size 1161765