Updates model card

Signed-off-by: Giovani <[email protected]>

README.md

@@ -14,9 +14,9 @@ tags:
 
 <!-- Provide a quick summary of what the model is/does. -->
 
-This is a **[XLM-RoBERTa-base](https://huggingface.co/xlm-roberta-base) fine-tuned model** on 6.5K (premise, hypothesis) sentence pairs from
-the **ASSIN2 (Avaliação de Similaridade Semântica e Inferência Textual)** corpus. The original references are:
-[Unsupervised Cross-Lingual Representation Learning At Scale](https://arxiv.org/pdf/1911.02116), [ASSIN2: Avaliação de Similaridade Semântica e Inferência Textual](https://huggingface.co/datasets/assin), respectivelly. This model is suitable for Brazilian Portuguese.
+This is a **[BERTimbau-base](https://huggingface.co/neuralmind/bert-large-portuguese-cased) fine-tuned model** on 5K (premise, hypothesis) sentence pairs from
+the **ASSIN (Avaliação de Similaridade Semântica e Inferência textual)** corpus. The original reference papers are:
+[BERTimbau: Pretrained BERT Models for Brazilian Portuguese](https://www.researchgate.net/publication/345395208_BERTimbau_Pretrained_BERT_Models_for_Brazilian_Portuguese), [ASSIN: Avaliação de Similaridade Semântica e Inferência Textual](https://huggingface.co/datasets/assin), respectivelly. This model is suitable for Portuguese (from Brazil or Portugal).
 
 ## Model Details
 
@@ -44,7 +44,7 @@ the **ASSIN2 (Avaliação de Similaridade Semântica e Inferência Textual)** co
 
 ### Direct Use
 
-This fine-tuned version of [XLM-RoBERTa-base](https://huggingface.co/xlm-roberta-base) performs Natural
+This fine-tuned version of [BERTimbau-base](https://huggingface.co/neuralmind/bert-large-portuguese-cased) performs Natural
 Language Inference (NLI), which is a text classification task.
 
 <!-- <div id="assin_function">
@@ -64,7 +64,7 @@ $$
 
 The *(premise, hypothesis)* entailment definition used is the same as the one found in Salvatore's paper [1].
 
-Therefore, this fine-tuned version of [XLM-RoBERTa-base](https://huggingface.co/xlm-roberta-base) classifies pairs of sentences in the form *(premise, hypothesis)* into the classes *ENTAILMENT* or *NONE*.
+Therefore, this fine-tuned version of [BERTimbau-base](https://huggingface.co/neuralmind/bert-large-portuguese-cased) classifies pairs of sentences in the form *(premise, hypothesis)* into the classes *ENTAILMENT*, *NONE* and *PARAPHRASE*.
  
 <!-- ## Bias, Risks, and Limitations
 
@@ -77,7 +77,7 @@ Therefore, this fine-tuned version of [XLM-RoBERTa-base](https://huggingface.co/
 from transformers import AutoModelForSequenceClassification, AutoTokenizer
 import torch
 
-model_path = "giotvr/xlm_roberta_base_assin2_fine_tuned"
+model_path = "giotvr/bertimbau_large_assin_fine_tuned"
 premise = "As mudanças climáticas são uma ameaça séria para a biodiversidade do planeta."
 hypothesis ="A biodiversidade do planeta é seriamente ameaçada pelas mudanças climáticas."
 tokenizer = XLMRobertaTokenizer.from_pretrained(model_path, use_auth_token=True)
@@ -112,7 +112,7 @@ Use the code below to get started with the model.
 
 - **Train Dataset**: [ASSIN](https://huggingface.co/datasets/assin) <br>
 
-- **Evaluation Dataset used for Hyperparameter Tuning:** [ASSIN2](https://huggingface.co/datasets/assin2)'s validation split
+- **Evaluation Dataset used for Hyperparameter Tuning:** [PLUE/MNLI](https://huggingface.co/datasets/dlb/plue)'s validation split
 
 - **Test Datasets:**
     - [ASSIN](https://huggingface.co/datasets/assin)'s test split
@@ -121,16 +121,15 @@ Use the code below to get started with the model.
 
 
 ---
-This is a fine tuned version of [XLM-RoBERTa-base](https://huggingface.co/xlm-roberta-base) using the [ASSIN2 (Avaliação de Similaridade Semântica e Inferência textual)](https://huggingface.co/datasets/assin2) dataset. [ASSIN2](https://huggingface.co/datasets/assin2) is a corpus annotated with hypothesis/premise Portuguese sentence pairs suitable for detecting textual entailment or neutral 
-relationship between the members of such pairs. Such corpus is balanced with 7k *ptbr* (Brazilian Portuguese) sentence pairs.
+This is a fine tuned version of  [BERTimbau-base](https://huggingface.co/neuralmind/bert-large-portuguese-cased) using the [ASSIN](https://huggingface.co/datasets/assin) dataset. [ASSIN](https://huggingface.co/datasets/assin) is a corpus annotated with hypothesis/premise Portuguese sentence pairs suitable for detecting textual entailment, none or paraphrase relationships between the members of such pairs. Such corpus is balanced among the three classes.
 
 ### Fine-Tuning Procedure 
 
 <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
 The model's fine-tuning procedure can be summarized in three major subsequent tasks:
     <ol type="i">
-        <li>**Data Processing:**</li> [ASSIN2](https://huggingface.co/datasets/assin2)'s *validation* and *train* splits were loaded from the **Hugging Face Hub** and processed afterwards; 
-        <li>**Hyperparameter Tuning:**</li> [XLM-RoBERTa-base](https://huggingface.co/xlm-roberta-base)'s hyperparameters were chosen with the help of the [Weights & Biases] API to track the results and upload the fine-tuned models;
+        <li>**Data Processing:**</li> [ASSIN](https://huggingface.co/datasets/assin)'s *validation* and *train* splits were loaded from the **Hugging Face Hub** and processed afterwards; 
+        <li>**Hyperparameter Tuning:**</li>[BERTimbau-base](https://huggingface.co/neuralmind/bert-large-portuguese-cased)'s hyperparameters were chosen with the help of the [Weights & Biases] API to track the results and upload the fine-tuned models;
         <li>**Final Model Loading and Testing:**</li>
         using the *cross-tests* approach described in the [this section](#evaluation), the models' performance were measured using different datasets and metrics. 
     </ol>
@@ -154,37 +153,39 @@ $$
 
 The following hyperparameters were tested in order to maximize the evaluation accuracy.
 
-- **Number of Training Epochs:** $(1,2,3)$
+- **Number of Training Epochs:** $(2,3,4)$
 - **Per Device Train Batch Size:** $(8,16,32)$
-- **Learning Rate:** $(1e−5, 2e−5, 5e−5)$
+- **Learning Rate:** $(3e−5, 2e−5, 3e−5)$
 
 
-The hyperaparemeter tuning experiments were run and tracked using the [Weights & Biases' API](https://docs.wandb.ai/ref/python/public-api/api) and can be found at this [link](https://wandb.ai/gio_projs/assin_xlm_roberta_v5?workspace=user-giogvn).
+The hyperparemeter tuning experiments were run and tracked using the [Weights & Biases' API](https://docs.wandb.ai/ref/python/public-api/api) and can be found at this [link](https://wandb.ai/gio_projs/assin_xlm_roberta_v5?workspace=user-giogvn).
 
 
 #### Training Hyperparameters
 
 The [hyperparameter tuning](#hyperparameter-tuning) performed yelded the following values:
 
-- **Number of Training Epochs:** $3$
-- **Per Device Train Batch Size:** $8$
+- **Number of Training Epochs:** $2$
+- **Per Device Train Batch Size:** $16$
 - **Learning Rate:** $5e-5$
 
 ## Evaluation
 
 ### ASSIN
 
-Testing this model in ASSIN's test split was straightforward because this model was fine tuned using ASSIN2's training set which contains the same labels as ASSIN. Hence, it can predict the same labels as the ones found in ASSIN's test set.
+
+Testing this model in ASSIN's test set was straightforward as it was fine-tuned in its training set.
+
 
 ### ASSIN2
 
-Testing this model in ASSIN2's test split is straightforward because this model was fine tuned using ASSIN2's training set and therefore can predict the same labels as the ones found in its test set.
+Testing this model in ASSIN2's test set was straightforward as ASSIN2 contains the same classes as ASSIN. 
 
 ### PLUE/MNLI
 
-Testing this model in PLUE/MNLI was only possible by considering PLUE/MNLI's *contradiction* and *neutral* labels as *NONE* and PLUE/MNLI's *entailment* label as equivalent to the *ENTAILMENT* predicted by the model.
 
-More information on how such mapping is performed can be found in [Modelos para Inferência em Linguagem Natural que entendem a Língua Portuguesa](https://linux.ime.usp.br/~giovani/).
+Testing this model in PLUE/MNLI's test split required some translation of the *neutral* and *contradictions* classes found in it, because such classes are not present in ASSIN. Both were considered equivalent to ASSIN's *NONE* class. More details on such translation can be found in [Modelos para Inferência em Linguagem Natural que entendem a Língua Portuguesa](https://linux.ime.usp.br/~giovani/).  
+
 
 ### Metrics
 
@@ -195,9 +196,9 @@ The model's performance metrics for each test dataset are presented separately.
 
 | test set | accuracy | f1 score | precision | recall |
 |----------|----------|----------|-----------|--------|
-| assin    |0.78      |0.78      |0.80       |0.80    |
-| assin2   |0.86      |0.86      |0.87       |0.86    |
-| plue/mnli|0.71      |0.67      |0.71       |0.71    |
+| assin    |0.92      |0.92      |0.92       |0.92    |
+| assin2   |0.73      |0.72      |0.77       |0.73    |
+| plue/mnli|0.49      |0.40      |0.35       |0.49    |
 
 ## Model Examination
 
@@ -238,3 +239,4 @@ Carbon emissions can be estimated using the [Machine Learning Impact calculator]
 <!--[2][Andrade, G. T. (2023)  Modelos para Inferência em Linguagem Natural que entendem a Língua Portuguesa  (train_assin_xlmr_base_results PAGES GO HERE)](https://linux.ime.usp.br/~giovani/)
 
 [3][Andrade, G. T. (2023)  Modelos para Inferência em Linguagem Natural que entendem a Língua Portuguesa (train_assin_xlmr_base_conclusions PAGES GO HERE)](https://linux.ime.usp.br/~giovani/) -->
+