---
tags:
- generated_from_trainer
model-index:
- name: mistral-7b-drug-prots_pair
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# mistral-7b-drug-prots_pair

This model was trained from scratch on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.0454

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- distributed_type: multi-GPU
- num_devices: 4
- gradient_accumulation_steps: 4
- total_train_batch_size: 64
- total_eval_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 30
- training_steps: 5000

### Training results

| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| 0.1497        | 0.01  | 50   | 0.0787          |
| 0.0734        | 0.02  | 100  | 0.0740          |
| 0.0694        | 0.03  | 150  | 0.0688          |
| 0.0709        | 0.04  | 200  | 0.0679          |
| 0.0668        | 0.05  | 250  | 0.0662          |
| 0.0651        | 0.06  | 300  | 0.0643          |
| 0.0619        | 0.07  | 350  | 0.0627          |
| 0.0614        | 0.08  | 400  | 0.0627          |
| 0.0607        | 0.09  | 450  | 0.0612          |
| 0.0609        | 0.1   | 500  | 0.0601          |
| 0.0586        | 0.11  | 550  | 0.0596          |
| 0.0578        | 0.12  | 600  | 0.0590          |
| 0.0577        | 0.13  | 650  | 0.0584          |
| 0.0569        | 0.14  | 700  | 0.0580          |
| 0.0564        | 0.15  | 750  | 0.0575          |
| 0.0566        | 0.16  | 800  | 0.0569          |
| 0.0558        | 0.17  | 850  | 0.0562          |
| 0.0556        | 0.18  | 900  | 0.0556          |
| 0.0552        | 0.19  | 950  | 0.0551          |
| 0.0546        | 0.2   | 1000 | 0.0547          |
| 0.0544        | 0.21  | 1050 | 0.0543          |
| 0.0541        | 0.22  | 1100 | 0.0545          |
| 0.0532        | 0.23  | 1150 | 0.0536          |
| 0.0527        | 0.24  | 1200 | 0.0534          |
| 0.0526        | 0.25  | 1250 | 0.0525          |
| 0.0524        | 0.26  | 1300 | 0.0523          |
| 0.0518        | 0.27  | 1350 | 0.0517          |
| 0.0514        | 0.28  | 1400 | 0.0518          |
| 0.0507        | 0.29  | 1450 | 0.0516          |
| 0.0501        | 0.3   | 1500 | 0.0512          |
| 0.0493        | 0.31  | 1550 | 0.0508          |
| 0.0495        | 0.32  | 1600 | 0.0507          |
| 0.0484        | 0.33  | 1650 | 0.0506          |
| 0.0482        | 0.34  | 1700 | 0.0504          |
| 0.0477        | 0.35  | 1750 | 0.0501          |
| 0.0474        | 0.36  | 1800 | 0.0503          |
| 0.0467        | 0.37  | 1850 | 0.0501          |
| 0.0469        | 0.38  | 1900 | 0.0494          |
| 0.0462        | 0.39  | 1950 | 0.0492          |
| 0.0455        | 0.4   | 2000 | 0.0490          |
| 0.0456        | 0.41  | 2050 | 0.0486          |
| 0.0454        | 0.42  | 2100 | 0.0488          |
| 0.0452        | 0.43  | 2150 | 0.0487          |
| 0.0456        | 0.44  | 2200 | 0.0485          |
| 0.0444        | 0.45  | 2250 | 0.0484          |
| 0.0441        | 0.46  | 2300 | 0.0477          |
| 0.0439        | 0.47  | 2350 | 0.0476          |
| 0.0441        | 0.48  | 2400 | 0.0471          |
| 0.0437        | 0.49  | 2450 | 0.0475          |
| 0.0432        | 0.5   | 2500 | 0.0472          |
| 0.0428        | 0.51  | 2550 | 0.0471          |
| 0.0431        | 0.52  | 2600 | 0.0472          |
| 0.0425        | 0.53  | 2650 | 0.0467          |
| 0.0429        | 0.54  | 2700 | 0.0464          |
| 0.0421        | 0.55  | 2750 | 0.0464          |
| 0.0422        | 0.56  | 2800 | 0.0460          |
| 0.0417        | 0.57  | 2850 | 0.0464          |
| 0.0419        | 0.58  | 2900 | 0.0462          |
| 0.0407        | 0.59  | 2950 | 0.0461          |
| 0.0409        | 0.6   | 3000 | 0.0462          |
| 0.0408        | 0.61  | 3050 | 0.0459          |
| 0.04          | 0.62  | 3100 | 0.0458          |
| 0.0401        | 0.63  | 3150 | 0.0453          |
| 0.0398        | 0.64  | 3200 | 0.0454          |
| 0.0395        | 0.65  | 3250 | 0.0451          |
| 0.0395        | 0.66  | 3300 | 0.0452          |
| 0.0387        | 0.67  | 3350 | 0.0453          |
| 0.0384        | 0.68  | 3400 | 0.0454          |
| 0.0386        | 0.69  | 3450 | 0.0451          |
| 0.0385        | 0.7   | 3500 | 0.0451          |
| 0.0383        | 0.71  | 3550 | 0.0450          |
| 0.0385        | 0.72  | 3600 | 0.0448          |
| 0.0378        | 0.73  | 3650 | 0.0449          |
| 0.0378        | 0.74  | 3700 | 0.0448          |
| 0.0376        | 0.75  | 3750 | 0.0453          |
| 0.0376        | 0.76  | 3800 | 0.0453          |
| 0.0374        | 0.77  | 3850 | 0.0454          |
| 0.0376        | 0.78  | 3900 | 0.0454          |
| 0.0369        | 0.79  | 3950 | 0.0454          |
| 0.037         | 0.8   | 4000 | 0.0448          |
| 0.0367        | 0.81  | 4050 | 0.0451          |
| 0.0369        | 0.82  | 4100 | 0.0450          |
| 0.0366        | 0.83  | 4150 | 0.0449          |
| 0.0368        | 0.84  | 4200 | 0.0449          |
| 0.0361        | 0.85  | 4250 | 0.0453          |
| 0.037         | 0.86  | 4300 | 0.0454          |
| 0.0364        | 0.87  | 4350 | 0.0453          |
| 0.0356        | 0.88  | 4400 | 0.0453          |
| 0.0359        | 0.89  | 4450 | 0.0453          |
| 0.0357        | 0.9   | 4500 | 0.0455          |
| 0.0357        | 0.91  | 4550 | 0.0454          |
| 0.0351        | 0.92  | 4600 | 0.0464          |
| 0.0356        | 0.93  | 4650 | 0.0458          |
| 0.0353        | 0.94  | 4700 | 0.0460          |
| 0.0351        | 0.95  | 4750 | 0.0456          |
| 0.0348        | 0.96  | 4800 | 0.0456          |
| 0.0355        | 0.97  | 4850 | 0.0455          |
| 0.0354        | 0.98  | 4900 | 0.0454          |
| 0.0357        | 0.99  | 4950 | 0.0453          |
| 0.0356        | 1.0   | 5000 | 0.0454          |


### Framework versions

- Transformers 4.44.0.dev0
- Pytorch 2.1.0+cu121
- Datasets 2.20.0
- Tokenizers 0.19.1