|
--- |
|
license: apache-2.0 |
|
language: |
|
- fr |
|
library_name: transformers |
|
inference: false |
|
--- |
|
# CamemBERT-L8 |
|
|
|
This model is a pruned version of the pre-trained [CamemBERT](https://huggingface.co/camembert-base) checkpoint, obtained by [dropping the top-layers](https://arxiv.org/abs/2004.03844) from the original model. |
|
|
|
 |
|
|
|
## Usage |
|
|
|
```python |
|
from transformers import AutoTokenizer, AutoModel |
|
|
|
tokenizer = AutoTokenizer.from_pretrained('antoinelouis/camembert-L8') |
|
model = AutoModel.from_pretrained('antoinelouis/camembert-L8') |
|
``` |
|
|
|
## Comparison |
|
|
|
| Model | #Params | Size | Pruning | |
|
|--------------------------------------------------------------------|:-------:|:-----:|:-------:| |
|
| [CamemBERT](https://huggingface.co/camembert-base) | 110.6M | 445MB | - | |
|
| | | | | |
|
| [CamemBERT-L10](https://huggingface.co/antoinelouis/camembert-L10) | 96.4M | 386MB | -13% | |
|
| **CamemBERT-L8** | 82.3M | 329MB | -26% | |
|
| [CamemBERT-L6](https://huggingface.co/antoinelouis/camembert-L6) | 68.1M | 272MB | -38% | |
|
| [CamemBERT-L4](https://huggingface.co/antoinelouis/camembert-L4) | 53.9M | 216MB | -51% | |
|
| [CamemBERT-L2](https://huggingface.co/antoinelouis/camembert-L2) | 39.7M | 159MB | -64% | |
|
|
|
## Citation |
|
|
|
```bibtex |
|
@online{louis2023, |
|
author = 'Antoine Louis', |
|
title = 'CamemBERT-L8: A Pruned Version of CamemBERT', |
|
publisher = 'Hugging Face', |
|
month = 'october', |
|
year = '2023', |
|
url = 'https://huggingface.co/antoinelouis/camembert-L8', |
|
} |
|
``` |
