Hugging Face's logo

antoinelouis
/
camembert-L8

Feature Extraction
Transformers
PyTorch
Safetensors
French
camembert
text-embeddings-inference
Model card Files Community
1
camembert-L8 / README.md
antoinelouis's picture
antoinelouis
Update README.md
be3604a
preview code
|
raw
history blame
1.74 kB
metadata
license: apache-2.0
language:
  - fr
library_name: transformers
inference: false

CamemBERT-L8

This model is a pruned version of the pre-trained CamemBERT checkpoint, obtained by dropping the top-layers from the original model.

Usage 

from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained('antoinelouis/camembert-L8')
model = AutoModel.from_pretrained('antoinelouis/camembert-L8')

Comparison

Model #Params Size Pruning
CamemBERT 110.6M 445MB -
CamemBERT-L10 96.4M 386MB -13%
CamemBERT-L8 82.3M 329MB -26%
CamemBERT-L6 68.1M 272MB -38%
CamemBERT-L4 53.9M 216MB -51%
CamemBERT-L2 39.7M 159MB -64%

Citation 

@online{louis2023,
   author    = 'Antoine Louis',
   title     = 'CamemBERT-L8: A Pruned Version of CamemBERT',
   publisher = 'Hugging Face',
   month     = 'october',
   year      = '2023',
   url       = 'https://huggingface.co/antoinelouis/camembert-L8',
}