Training in progress, epoch 1, checkpoint

last-checkpoint/README.md

@@ -0,0 +1,202 @@
+---
+base_model: openai/whisper-large-v3-turbo
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.2

last-checkpoint/adapter_config.json

@@ -0,0 +1,36 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": {
+    "base_model_class": "WhisperForConditionalGeneration",
+    "parent_library": "transformers.models.whisper.modeling_whisper"
+  },
+  "base_model_name_or_path": "openai/whisper-large-v3-turbo",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 8,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "out_proj",
+    "fc2",
+    "v_proj",
+    "fc1",
+    "q_proj",
+    "k_proj"
+  ],
+  "task_type": null,
+  "use_dora": false,
+  "use_rslora": true
+}

last-checkpoint/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f364ca1a30c381e9fa0c84c446a8c8e4a4116f7267a43852b45515d83f84aba1
+size 111475752

last-checkpoint/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0411321624ef553ea120e5017fd533a9a7d6fc23824d496de0e235081abb595b
+size 223212738

last-checkpoint/preprocessor_config.json

@@ -0,0 +1,14 @@
+{
+  "chunk_length": 30,
+  "feature_extractor_type": "WhisperFeatureExtractor",
+  "feature_size": 128,
+  "hop_length": 160,
+  "n_fft": 400,
+  "n_samples": 480000,
+  "nb_max_frames": 3000,
+  "padding_side": "right",
+  "padding_value": 0.0,
+  "processor_class": "WhisperProcessor",
+  "return_attention_mask": false,
+  "sampling_rate": 16000
+}

last-checkpoint/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:226d2e467ed97a5ca4d38c2f4f7e622036e0e9261aef2f866b8182d185d4441e
+size 14244

last-checkpoint/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b269ced0d4e640636cd4c1b3665b29b1138d024d7e88887a303e726f90e7352b
+size 1064

last-checkpoint/trainer_state.json

@@ -0,0 +1,170 @@
+{
+  "best_metric": 37.251356238698015,
+  "best_model_checkpoint": "./exp/whisper-large-v3-turbo-score-5-rebalanced-2/checkpoint-18",
+  "epoch": 1.0,
+  "eval_steps": 500,
+  "global_step": 18,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.05555555555555555,
+      "grad_norm": 10.286238670349121,
+      "learning_rate": 0.0004986111111111111,
+      "loss": 0.4972,
+      "step": 1
+    },
+    {
+      "epoch": 0.1111111111111111,
+      "grad_norm": 2.8972043991088867,
+      "learning_rate": 0.0004972222222222222,
+      "loss": 1.1874,
+      "step": 2
+    },
+    {
+      "epoch": 0.16666666666666666,
+      "grad_norm": 2.798466205596924,
+      "learning_rate": 0.0004958333333333334,
+      "loss": 0.4278,
+      "step": 3
+    },
+    {
+      "epoch": 0.2222222222222222,
+      "grad_norm": 6.3090996742248535,
+      "learning_rate": 0.0004944444444444445,
+      "loss": 0.4353,
+      "step": 4
+    },
+    {
+      "epoch": 0.2777777777777778,
+      "grad_norm": 0.9731603860855103,
+      "learning_rate": 0.0004930555555555556,
+      "loss": 0.2181,
+      "step": 5
+    },
+    {
+      "epoch": 0.3333333333333333,
+      "grad_norm": 2.3366434574127197,
+      "learning_rate": 0.0004916666666666666,
+      "loss": 0.3393,
+      "step": 6
+    },
+    {
+      "epoch": 0.3888888888888889,
+      "grad_norm": 2.3805484771728516,
+      "learning_rate": 0.0004902777777777777,
+      "loss": 0.1776,
+      "step": 7
+    },
+    {
+      "epoch": 0.4444444444444444,
+      "grad_norm": 1.2242686748504639,
+      "learning_rate": 0.0004888888888888889,
+      "loss": 0.1185,
+      "step": 8
+    },
+    {
+      "epoch": 0.5,
+      "grad_norm": 0.8841593265533447,
+      "learning_rate": 0.0004875,
+      "loss": 0.0741,
+      "step": 9
+    },
+    {
+      "epoch": 0.5555555555555556,
+      "grad_norm": 0.8210709691047668,
+      "learning_rate": 0.0004861111111111111,
+      "loss": 0.0974,
+      "step": 10
+    },
+    {
+      "epoch": 0.6111111111111112,
+      "grad_norm": 0.9800593256950378,
+      "learning_rate": 0.0004847222222222222,
+      "loss": 0.0574,
+      "step": 11
+    },
+    {
+      "epoch": 0.6666666666666666,
+      "grad_norm": 1.2354100942611694,
+      "learning_rate": 0.00048333333333333334,
+      "loss": 0.0228,
+      "step": 12
+    },
+    {
+      "epoch": 0.7222222222222222,
+      "grad_norm": 0.8006910681724548,
+      "learning_rate": 0.00048194444444444446,
+      "loss": 0.0318,
+      "step": 13
+    },
+    {
+      "epoch": 0.7777777777777778,
+      "grad_norm": 1.1510701179504395,
+      "learning_rate": 0.0004805555555555556,
+      "loss": 0.018,
+      "step": 14
+    },
+    {
+      "epoch": 0.8333333333333334,
+      "grad_norm": 1.0643190145492554,
+      "learning_rate": 0.0004791666666666667,
+      "loss": 0.1162,
+      "step": 15
+    },
+    {
+      "epoch": 0.8888888888888888,
+      "grad_norm": 1.034947395324707,
+      "learning_rate": 0.0004777777777777778,
+      "loss": 0.1181,
+      "step": 16
+    },
+    {
+      "epoch": 0.9444444444444444,
+      "grad_norm": 0.5520635843276978,
+      "learning_rate": 0.0004763888888888889,
+      "loss": 0.0733,
+      "step": 17
+    },
+    {
+      "epoch": 1.0,
+      "grad_norm": 1.193587303161621,
+      "learning_rate": 0.000475,
+      "loss": 0.0489,
+      "step": 18
+    },
+    {
+      "epoch": 1.0,
+      "eval_cer": 23.450413223140494,
+      "eval_loss": 3.250870943069458,
+      "eval_pred": "| i | Label | Prediction |\n| --- | --- | --- |\n| 0 | The people in the picture are playing soccer. I\u2019ve played soccer twice before in physical education class and I liked it. Well, mostly because I have really strong muscles in my legs from running, so I have a lot of advantages in soccer. If I was a parent, {I would} I would agree for my kid to play soccer. Mostly because playing a sport helps you stay healthy and fit and that\u2019s what ??? society thinks you should do. Stay fit and healthy. | The people in the picture are playing soccer. I've\ufffdve played soccer twice before in physical education class, I liked it, Well, mostly because I have really strong muscles in my legs from running, so I have a lot of advantages in soccer. If I was a parent, I I would agree for would agree for my kid to play soccer, Mostly because playing a sport helps you stay healthy and fit, that's\ufffds what societysoci thinks you should do: Stay fit and healthy. |\n| 1 | And it\u2019s also good for your health too. You can have {lower} a lower risk of getting any diseases from body fat. Also, the people in this picture mostly are wearing jerseys and shorts. Some of them are wearing knee-high socks. And all of them are wearing sneakers. And the details in here. There are a lot of trees, which I really like. And really beautiful grass. And there are also two buildings in the background. | And it's\ufffds also good for your health,. You can have aa risk of lower risk of getting any diseases from body fat. Also, the people in this picture mostly are wearing jerseys and shorts. Some of them are wearing knee-high socks, And all of them are wearing sneakers. And the details in here: There are a lot of trees which which I really like. And the beautiful grass. And there are also two buildings in the background. |\n| 2 | Also a bridge. There is a <silver> <car> silver car and there are multiple soccer balls which (um) means that they are <probably> probably practicing and not playing against each other. The [people] (um) Also there are benches in the background which also indicates that they ??? <might*> might be in a park instead of a soccer field. And there are | alsoso, bridge. There is a silversilver car andand, and car, there are multiple soccer balls, meansme means means that they are probablyprobably practicinging practicing and not playing against each other. The people\ufffd also Alsoum also Also, are benches in the background which also indicates that they might mightmight be might might be in the park instead of a soccer field. And there are |\n| 3 | some\u2026 | some...some |\n| 4 | (um) I think people in the picture is playing soccer, if I\u2019m not wrong. Yes, (eN) they are playing soccer. And (um) did I? I did. When I was in elementary school, <we> <we> <had> <we> <had> <a> <class> we had a PE class and (um) the teacher taught us how to play soccer before. But, honestly, I\u2019m very poor <at> <at> (um) at sports, so I\u2019m not really enjoy it. But I did see some people, like {my} some of my classmate, really do know how to play soccer. | II,I think people in the picture is playing soccer, if I'm\ufffdm not wrong. They, they,,d They are playing soccer, And dide, Did I? I did. When I was in elementary school, we we hadedwe a,we toedweall,we toedwe,awe,a do a class class and thee,t teacher taught us how to play soccer before. But honestly honestly, I'm\ufffdm very poor at,,, so, soa so so sports, so I'm\ufffdm not really enjoy it. But I did see some people, like some,,, of my classmatesmate really really do know how to play soccer. |\n| 5 | <I> I was like, oh my god, this is a very good, very, very (um) phenomenal cause (um) {it's like} {it\u2019s a} it\u2019s very hard <to> <see> <some> for some the students in Taiwan to play soccer, so as I think it\u2019s <quite> quite cool. And if I am parents, well, (um) (um) because I\u2019m not really interested in this, so, (um) if they want to, of course, I would encourage them, but (um) if they doesn\u2019t like that, <I> I won\u2019t force them to do it cause I think it\u2019s not really (um), it\u2019s alright. {It doesn\u2019t like} it\u2019s not | andand wasent was like, oh my god, this is very very good, very, very phenomenalinaleral because because itum, It it's like, toit's\ufffd It like lot to's\ufffds a hard to because seetoit someto so itto some the students in Taiwan to play soccer so so I I think it's\ufffds quite,l,, cool. And if I'm parents, well, because,, becauseum, because I'm\ufffdm not really interested in this, so if ifum if if they want to, of course, I encourage encourage them but but if if if but they doesn't\ufffdt like that, I I won to won't\ufffdt force them to do it because I think it's\ufffds not really, it it it's\ufffds all, It it's't\ufffdt it it it's\ufffds it, |\n| 6 | necessary and just like if they want, I will. And (um) if I have time, (um) so people and it\u2019s (um) only boys. Why? [ish] In the picture, they should have girls, but (um) in the picture, (um) the only boys in the pictures and (uN) they {separate} divided into two groups, is it? And all they\u2019re wearing this long socks is quite cool and (um) it\u2019s quite a beautiful place. It\u2019s a really beautiful place and (um) I think they enjoyed very | necessary, just like if they want I I will and And if if, If I have time. soum, So people and only's\ufffds onlyum, only boys in But? InIn? In the picture, the only have girls, but inum, the the picture, theum, and only boys in the pictures and theyum,ame they're into it into into to into two groups, is it? And oh,'re\ufffdre wearing this long socks, quite cool and itum it it's\ufffds quite a beautiful place. It's\ufffds a really beautiful place and Ium I I think they enjoyed very |\n| 7 | much. It's quite\u2026 | much. It's quite |\n| 8 | (Um) I think the picture is taken (um) at a park* and it\u2019s a very bright sunny day. And (um) there are some people are in the park and they are painting. <And> and there is a woman (uh) on the right of the picture. She\u2019s sitting on the chair and {she is} she has short hair and {wearing*} some, wearing dress. And she | II,I think the picture is taken,at,at a park, and it's\ufffds a very bright sunny day and And there there, There are some people are in the park and they are pant. And And um there there is a woman onum on on the right of the picture. She's\ufffds sitting on the chair and she she has she and has short hair and wearing some some some some and wearing wearing dress and And she |\n| 9 | is painting some trees and I really like the picture. It\u2019s beautiful. And there are a bags beside the woman. I think there\u2019s (uh) {the wore} the pants or some color in the bags that she wants <to> to draw. And there are also (um) lots of people {near} nearby the {the girl} the woman | She pant some trees, I really like the picture. It's\ufffds beautiful. And there are the backs beside the woman. I think there's\ufffds the the, The the pants the the girl or some color in the bags that she wants to to toto draw. And there are also lotsuh, lots of people nearby thear by the the girl, girl,, woman. |\n| 10 | are <painting> painting and there are two people which has a bags and others are besides her and they are discussing* (um) how to draw the picture. And I think (uh) the advantage to join a park is that you can really near <the> the picture you want to draw and it\u2019s in | are pantpantingting,ing, there are two people which has a bag and others are besides her and they are disgusting how howhow how how to draw the picture and And I think thea thethe advantage to drawing the park is that you can't near thethe picture the picture you want to draw and it's\ufffds in |\n| 11 | nature scenery\u2026 | Nature scenery, |\n| 12 | Well, I see at least nine people in the picture, and I can see that {all} they are all young men, and they are probably professional soccer players, or a soccer team at school, since they are all wearing sports wears that look quite professional. And, the man at the back of the picture {is} has a funky look, while he has spiky haircut. | Well, I see at least nine people in the picture, and I can see that they they, and're all young men, and they are probably professional soccer players or or a soccer team at school, since they're all wearing sportswear that look quite professional. And the the man at the back of the picture has has has has a funky look, while he has spiky hair, |\n| 13 | And he\u2019s wearing a white short sleeved t-shirt and like he\u2019s wearing blue shirts and long socks which soccer players usually wear, and he\u2019s also wearing blue sneakers. He\u2019s trying to chase the yellow soccer ball. (um) I see many people wearing long soccer socks, which really impressed me, and they\u2019re all wearing like (um) red | and he's\ufffds wearing a white,-ved t-shirt and like he\u2019s wearing blue shirts and long socks which soccer players usually wear. and he\u2019s also wearing blue sneakers. He\u2019s trying to chase the yellow soccer balls. I I, I see many people wearing long soccer socks which which really impressed me, and they're\ufffdre all wearing like redum, red |\n| 14 | shorts or (eh) like green shirts with numbers on it. So they might be quite professional. And the weather looks good. At the back of the picture, I could see many trees too. And I could even see the MRT. (eh) Also, I see two buildings. And I\u2019m not really good at soccer, but I like watching soccer games. So I hope maybe | shirts or likeaiveI green shirts with numbers on it, So they might be quite professional. And the weather looks good. At the back of the picture, I could see many trees too. And I could even see the MRT. Alsoa also Also, I see two buildings. And I'm\ufffdm not really good at soccer, but I like watching soccer games. So I hope maybe |\n| 15 | I can\u2026 | I can't |\n| 16 | I think this might be a room {up a} {up in a} up in the top building in the city because {the windows} [out] outside the windows there\u2019s a lot of colorful [buil] buildings and it\u2019s also really high up on the ground. The woman in the middle is playing her violin <to> <the> to the guests <and> and lots of people are taking pictures of her. I think this is a good place to have a celebration because <it> <is> [really] {it is really} | I think this might be a room upa in-athe in the top in in the top building in the city because theupir,the, oft the windows,'s\ufffds a lot of colorful buildingsbuildildings buildings and it's\ufffds also really high up the the ground. The woman in the middle is playing her violin to to thetoto toto the guests andand lots lots lots of people are taking pictures of her. I think this is a good place to have a celebration because it it is itit it itit it it it is it it it |\n| 17 | (um) it looks very comfortable <and> <the> <food> and the food must taste really good. I think (um) <the> <woman> <on> <the> the woman on the left is wearing a {red skirt} red dress and is wearing black heels. She\u2019s looking happily at the woman playing the violin while filming her on the phone. And then there is a waiter <on> <the> on the top right corner serving food to the guest <that> that lives beside him. There is also a man | ItIt,It looks really comfortable,and the thethe foodestthemest the food tastes taste really good. I think the the, thethe womanthethe thethethe thethethe ising woman on the bed is wearing a rede dress and and dress and is wearing black shoes. She's\ufffds, happily at the woman playing the violin while filming her on the phone. And then there is a waiter on on the on on on on the top right corner, food to the guests that that that that leads it him. Bes is also a man |\n| 18 | {with a} {with a white} [T sh] with a white shirt smiling happily. And behind the man, <there> <is> there is one guy on his iPad <while> <another> while another guy looks at him. {There is four} [pers] There's four people in total that\u2019s {looking at the violin} looking at the woman playing the violin, and they seem very satisfied with it. (Um) On the table, there are wines and different drinks for them. I don\u2019t see any | withand a white shirt and a white shirt.iit a white shirt, happily, And behind the men, there, is,there oneis is one guy on his iPad,while another looksthereother guy looks another guy looks at him. Therethere is four people,,, There's four people in total that's\ufffds looking looking at the women,, at the woman, the violin and and they seem very satisfied with it. On on, On the table, there are wines and different drinks for them. I don't\ufffdt see any |\n| 19 | food yet, so maybe\u2026 | ifies, so maybe |\n",
+      "eval_runtime": 3.2175,
+      "eval_samples_per_second": 6.216,
+      "eval_steps_per_second": 0.932,
+      "eval_wer": 37.251356238698015,
+      "step": 18
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 360,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 20,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.238386802688e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

last-checkpoint/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f78a42a7a26fd014d616e7eeca3cd0b8ee1648f006098cc708a876603fce1e37
+size 5624