---
language: en
tags:
- fill-mask
---

## Environmental Impact (CODE CARBON DEFAULT)

| Metric                   | Value                           |
|--------------------------|---------------------------------|
| Duration (in seconds)    | [More Information Needed]  |
| Emissions (Co2eq in kg)  | [More Information Needed] |
| CPU power (W)            | [NO CPU]  |
| GPU power (W)            | [No GPU]  |
| RAM power (W)            | [More Information Needed]  |
| CPU energy (kWh)         | [No CPU]  |
| GPU energy (kWh)         | [No GPU]  |
| RAM energy (kWh)         | [More Information Needed]  |
| Consumed energy (kWh)    | [More Information Needed]  |
| Country name             | [More Information Needed]  |
| Cloud provider           | [No Cloud]  |
| Cloud region             | [No Cloud]  |
| CPU count                | [No CPU]  |
| CPU model                | [No CPU]  |
| GPU count                | [No GPU]  |
| GPU model                | [No GPU]  |

## Environmental Impact (for one core)

| Metric                   | Value                           |
|--------------------------|---------------------------------|
| CPU energy (kWh)         | [No CPU]  |
| Emissions (Co2eq in kg)  | [More Information Needed] |

## Note

30 April 2024

## My Config

| Config                   | Value           |
|--------------------------|-----------------|
| checkpoint               | albert-base-v2  |
| model_name               | BERTrand_bs32_lr6 |
| sequence_length          | 400  |
| num_epoch                | 12  |
| learning_rate            | 5e-06  |
| batch_size               | 32  |
| weight_decay             | 0.0  |
| warm_up_prop             | 0  |
| drop_out_prob            | 0.1 |
| packing_length           | 100 |
| train_test_split         | 0.2 |
| num_steps                | 6318 |

## Training and Testing steps






 
Epoch | Train Loss | Test Loss
---|---|---
| 0.0 | 15.603048 | 15.109937 |
| 0.5 | 8.715844 | 8.071290 |
| 1.0 | 7.608879 | 8.114126 |
| 1.5 | 7.407612 | 7.914163 |
| 2.0 | 7.323461 | 7.774658 |
| 2.5 | 7.248362 | 7.696718 |
| 3.0 | 7.101276 | 7.856242 |
| 3.5 | 7.134161 | 7.617901 |
| 4.0 | 7.105548 | 7.837306 |
| 4.5 | 7.221799 | 7.653854 |
| 5.0 | 7.047156 | 7.659136 |
| 5.5 | 7.080983 | 7.554190 |
| 6.0 | 7.083629 | 7.670907 |
| 6.5 | 7.180606 | 7.623875 |
| 7.0 | 7.036574 | 7.571451 |
| 7.5 | 7.037596 | 7.550659 |
| 8.0 | 7.082738 | 7.634689 |
| 8.5 | 7.136363 | 7.576325 |
| 9.0 | 7.046428 | 7.594891 |
| 9.5 | 7.022868 | 7.588534 |
| 10.0 | 7.075124 | 7.532026 |
| 10.5 | 7.078401 | 7.519065 |
| 11.0 | 7.109886 | 7.550544 |
| 11.5 | 7.002302 | 7.586807 |
| 12.0 | 7.054321 | 7.544724 |