Bingus-v0.1-60M-Base
A not-so-state-of-the-art 60M parameter transformer model.
Uses the olmo default architecture.
Specs
Heads: 8
Layers: 8
Dimension model: 512
Dimension mlp: 4096
eval/v3-small-c4_en-validation/Perplexity: 40.33
Training Data
Pretraining:
- 5B Tokens C4 (preprocessed, from olmo-data.org)
- Downloads last month
- 94
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.