End of training

README.md

@@ -0,0 +1,88 @@
+---
+library_name: transformers
+license: agpl-3.0
+base_model: vinai/phobert-base-v2
+tags:
+- generated_from_trainer
+metrics:
+- accuracy
+- f1
+- precision
+- recall
+model-index:
+- name: PhishLang_PhoBERTCNN_15k
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# PhishLang_PhoBERTCNN_15k
+
+This model is a fine-tuned version of [vinai/phobert-base-v2](https://huggingface.co/vinai/phobert-base-v2) on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.3386
+- Accuracy: 0.923
+- F1: 0.9068
+- Precision: 0.9180
+- Recall: 0.8974
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 1e-05
+- train_batch_size: 32
+- eval_batch_size: 32
+- seed: 42
+- gradient_accumulation_steps: 2
+- total_train_batch_size: 64
+- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
+- lr_scheduler_type: cosine
+- lr_scheduler_warmup_ratio: 0.1
+- num_epochs: 10
+- label_smoothing_factor: 0.1
+
+### Training results
+
+| Training Loss | Epoch  | Step | Validation Loss | Accuracy | F1     | Precision | Recall |
+|:-------------:|:------:|:----:|:---------------:|:--------:|:------:|:---------:|:------:|
+| 0.6996        | 0.5333 | 100  | 0.4548          | 0.8867   | 0.8615 | 0.8761    | 0.8503 |
+| 0.6996        | 1.064  | 200  | 0.3524          | 0.9187   | 0.8998 | 0.9207    | 0.8844 |
+| 0.6996        | 1.5973 | 300  | 0.3441          | 0.9217   | 0.9039 | 0.9223    | 0.8900 |
+| 0.4376        | 2.128  | 400  | 0.3457          | 0.9247   | 0.9070 | 0.9296    | 0.8906 |
+| 0.4376        | 2.6613 | 500  | 0.3423          | 0.928    | 0.9106 | 0.9366    | 0.8924 |
+| 0.4376        | 3.192  | 600  | 0.3370          | 0.9253   | 0.9089 | 0.9245    | 0.8966 |
+| 0.4376        | 3.7253 | 700  | 0.3378          | 0.9227   | 0.9074 | 0.9132    | 0.9021 |
+| 0.3287        | 4.256  | 800  | 0.3392          | 0.921    | 0.9051 | 0.9126    | 0.8985 |
+| 0.3287        | 4.7893 | 900  | 0.3374          | 0.9237   | 0.9078 | 0.9180    | 0.8992 |
+| 0.3287        | 5.32   | 1000 | 0.3377          | 0.9223   | 0.9067 | 0.9142    | 0.9000 |
+| 0.3287        | 5.8533 | 1100 | 0.3386          | 0.9273   | 0.9118 | 0.9248    | 0.9012 |
+| 0.3037        | 6.384  | 1200 | 0.3454          | 0.919    | 0.9037 | 0.9064    | 0.9011 |
+| 0.3037        | 6.9173 | 1300 | 0.3367          | 0.9233   | 0.9070 | 0.9192    | 0.8971 |
+| 0.3037        | 7.448  | 1400 | 0.3373          | 0.9237   | 0.9073 | 0.9200    | 0.8970 |
+| 0.291         | 7.9813 | 1500 | 0.3373          | 0.9223   | 0.9055 | 0.9196    | 0.8942 |
+| 0.291         | 8.512  | 1600 | 0.3394          | 0.9233   | 0.9071 | 0.9189    | 0.8974 |
+| 0.291         | 9.0427 | 1700 | 0.3379          | 0.9237   | 0.9075 | 0.9194    | 0.8976 |
+| 0.291         | 9.576  | 1800 | 0.3386          | 0.923    | 0.9068 | 0.9180    | 0.8974 |
+
+
+### Framework versions
+
+- Transformers 4.48.0
+- Pytorch 2.5.1+cu121
+- Datasets 3.2.0
+- Tokenizers 0.21.0

config.json

@@ -0,0 +1,28 @@
+{
+  "_name_or_path": "vinai/phobert-base-v2",
+  "architectures": [
+    "PhoBERTWithCNN"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 0,
+  "classifier_dropout": null,
+  "eos_token_id": 2,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 258,
+  "model_type": "roberta",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 1,
+  "position_embedding_type": "absolute",
+  "tokenizer_class": "PhobertTokenizer",
+  "torch_dtype": "float32",
+  "transformers_version": "4.48.0",
+  "type_vocab_size": 1,
+  "use_cache": true,
+  "vocab_size": 64001
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:90b6edf214d7a04cd95dc7962459888133253b3e140fb061cea16da72893c4f8
+size 554972400

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4fccb6ea39bfe9aa377b007bce2fd0d8e6b2518d7d867d4b682190b7591d4b9d
+size 5240