MuseMistral-v3-35k
This model is a fine-tuned version of mistralai/Mistral-7B-v0.1
Model description
This model generates classical music in ABC notation with only 7B parameters and internet data related to ABC notation.
Training procedure
Trained on free GPUs provided by Kaggle using Axolotl.
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0002
- train_batch_size: 1
- eval_batch_size: 8
- seed: 3407
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 10
- training_steps: 1000
- mixed_precision_training: Native AMP
Framework versions
- PEFT 0.7.1
- Transformers 4.36.0
- Pytorch 2.1.2
- Datasets 2.16.0
- Tokenizers 0.15.0
- Downloads last month
- 3
Inference Providers
NEW
This model is not currently available via any of the supported third-party Inference Providers, and
HF Inference API was unable to determine this model’s pipeline type.
Model tree for oof-baroomf/MuseMistral-v3-35k
Base model
mistralai/Mistral-7B-v0.1