---
library_name: peft
license: apache-2.0
base_model: AdaptLLM/biomed-Qwen2-VL-2B-Instruct
tags:
- llama-factory
- lora
- generated_from_trainer
model-index:
- name: qwenvl-2B-cadica-direction-scale4
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# qwenvl-2B-cadica-direction-scale4

This model is a fine-tuned version of [AdaptLLM/biomed-Qwen2-VL-2B-Instruct](https://huggingface.co/AdaptLLM/biomed-Qwen2-VL-2B-Instruct) on the CADICA血管分支方向題scale4(TRAIN) dataset.
It achieves the following results on the evaluation set:
- Loss: 0.0000
- Num Input Tokens Seen: 11990784

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 1
- eval_batch_size: 1
- seed: 42
- distributed_type: multi-GPU
- num_devices: 4
- gradient_accumulation_steps: 6
- total_train_batch_size: 24
- total_eval_batch_size: 4
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.05
- training_steps: 1200

### Training results

| Training Loss | Epoch  | Step | Validation Loss | Input Tokens Seen |
|:-------------:|:------:|:----:|:---------------:|:-----------------:|
| 0.3441        | 0.0258 | 50   | 0.3383          | 499200            |
| 0.2274        | 0.0515 | 100  | 0.1866          | 998400            |
| 0.0667        | 0.0773 | 150  | 0.0967          | 1497600           |
| 0.0459        | 0.1030 | 200  | 0.0996          | 1996800           |
| 0.0805        | 0.1288 | 250  | 0.0559          | 2496000           |
| 0.0381        | 0.1545 | 300  | 0.0309          | 2995200           |
| 0.1761        | 0.1803 | 350  | 0.0439          | 3494400           |
| 0.0146        | 0.2060 | 400  | 0.0244          | 3993600           |
| 0.0157        | 0.2318 | 450  | 0.0067          | 4492800           |
| 0.0122        | 0.2575 | 500  | 0.0080          | 4992000           |
| 0.0339        | 0.2833 | 550  | 0.0034          | 5491200           |
| 0.0217        | 0.3090 | 600  | 0.0133          | 5990400           |
| 0.0327        | 0.3348 | 650  | 0.0210          | 6489600           |
| 0.0267        | 0.3605 | 700  | 0.0053          | 6988800           |
| 0.014         | 0.3863 | 750  | 0.0053          | 7488000           |
| 0.0065        | 0.4121 | 800  | 0.0068          | 7987200           |
| 0.0306        | 0.4378 | 850  | 0.0072          | 8486400           |
| 0.0063        | 0.4636 | 900  | 0.0107          | 8985600           |
| 0.0415        | 0.4893 | 950  | 0.0072          | 9484800           |
| 0.0547        | 0.5151 | 1000 | 0.0007          | 9984000           |
| 0.0007        | 0.5408 | 1050 | 0.0568          | 10483200          |
| 0.0056        | 0.5666 | 1100 | 0.0004          | 10982400          |
| 0.0127        | 0.5923 | 1150 | 0.0000          | 11481600          |
| 0.0038        | 0.6181 | 1200 | 0.0022          | 11980800          |


### Framework versions

- PEFT 0.12.0
- Transformers 4.47.0.dev0
- Pytorch 2.5.1+cu121
- Datasets 3.1.0
- Tokenizers 0.20.3