swiftformer-xs-DMAE

This model is a fine-tuned version of MBZUAI/swiftformer-xs on the imagefolder dataset. It achieves the following results on the evaluation set:

  • Loss: 0.7654
  • Accuracy: 0.7609

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.005
  • train_batch_size: 16
  • eval_batch_size: 16
  • seed: 42
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 64
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_ratio: 0.1
  • num_epochs: 40

Training results

Training Loss Epoch Step Validation Loss Accuracy
No log 0.86 3 1.3706 0.5435
No log 2.0 7 1.1672 0.6304
1.2937 2.86 10 1.2248 0.6087
1.2937 4.0 14 2.6986 0.3478
1.2937 4.86 17 2.6008 0.4565
1.2355 6.0 21 1.8900 0.4130
1.2355 6.86 24 1.5958 0.5
1.2355 8.0 28 1.3847 0.5652
1.1548 8.86 31 0.8762 0.7174
1.1548 10.0 35 1.0871 0.5435
1.1548 10.86 38 0.9779 0.5435
1.0561 12.0 42 1.2454 0.5435
1.0561 12.86 45 1.2277 0.5652
1.0561 14.0 49 0.9527 0.6087
1.0546 14.86 52 0.8797 0.6304
1.0546 16.0 56 0.9479 0.6304
1.0546 16.86 59 0.8696 0.6739
0.9493 18.0 63 0.9348 0.6522
0.9493 18.86 66 0.9890 0.5435
0.9354 20.0 70 0.9073 0.5870
0.9354 20.86 73 0.8763 0.6304
0.9354 22.0 77 0.9592 0.6522
0.8791 22.86 80 0.8940 0.6522
0.8791 24.0 84 0.8165 0.6957
0.8791 24.86 87 0.8249 0.6957
0.8017 26.0 91 0.8946 0.6739
0.8017 26.86 94 0.8210 0.6957
0.8017 28.0 98 0.7654 0.7609
0.8532 28.86 101 0.7492 0.7391
0.8532 30.0 105 0.7725 0.6957
0.8532 30.86 108 0.7932 0.7174
0.8205 32.0 112 0.8129 0.7391
0.8205 32.86 115 0.8089 0.6957
0.8205 34.0 119 0.8151 0.6957
0.8112 34.29 120 0.8074 0.6957

Framework versions

  • Transformers 4.36.2
  • Pytorch 2.1.2+cu118
  • Datasets 2.16.1
  • Tokenizers 0.15.0
Downloads last month
23
Safetensors
Model size
3.04M params
Tensor type
F32
·
Inference API
Unable to determine this model's library. Check the docs .

Model tree for Augusto777/swiftformer-xs-DMAE

Finetuned
(17)
this model

Evaluation results