---
license: cc-by-nc-4.0
language:
- en
---

# MistralPirate-7b-v0.3

## Model Card

### Description
MistralPirate-7b-v0.3 is a sophisticated language model fine-tuned for generating intricate and authentic pirate-themed content. This version, correcting our version control from v2 to v0.3, builds upon MistralPirate-7b-v2 and leverages advancements from Mistral Instruct v0.2. It shows improved performance in pirate dialect accuracy and perplexity scores.

- **Developed by**: phanerozoic
- **License**: cc-by-nc-4.0
- **Finetuned from**: Mistral Instruct v0.2

### Version Control Correction
Correcting version control to v0.3 to reflect developmental progression and enhancements over the previous version.

### Comparative Analysis with Previous Model
MistralPirate-7b-v0.3 demonstrates notable improvements over its predecessor in several key areas:
- **Pirate Dialect**: The new model uses richer and more immersive pirate vernacular, enhancing the thematic experience.
- **Technical Accuracy**: It shows a deeper understanding of complex sailing scenarios, providing detailed and practical advice in response to intricate questions.
- **Language Coherence**: The model maintains a consistent tone and style, effectively blending pirate jargon with technical expertise.

### Direct Use
Ideal for interactive storytelling, gaming, advanced educational content, and conversational AI in pirate-themed settings.

### Downstream Use
Suitable for tasks requiring detailed language generation and domain-specific knowledge, like advanced thematic content creation or immersive language learning.

### Out-of-Scope Use
Not intended for general-purpose language modeling or non-pirate-themed contexts. Usage outside its specialization may result in suboptimal performance.

### Bias, Risks, and Limitations
Limited by its training data, may inherit biases. Best used where pirate-themed language is appropriate, not for serious or sensitive communication.

### Recommendations
Recommended for thematic contexts, with an understanding of its specialized focus. Not for accurate information outside pirate dialect specialization.

### Custom Stopping Strings Usage
Custom stopping strings employed for output quality:

- "},"
- "User:"
- "You:"
- "\nUser"
- "\nUser:"

### Training Data
Trained on a vast dataset in ChatML format, ensuring diverse and rich inputs.

### Preprocessing
Advanced preprocessing into ChatML format.

### Training Hyperparameters and Fine-Tuning Details
- Training Regime: FP32
- Warmup Steps: 1
- Per Device Train Batch Size: 2
- Gradient Accumulation Steps: 64
- Max Steps: 1500
- Learning Rate: 0.00015
- Logging Steps: 1
- Save Steps: 1
- Lora Alpha: 32
- Dimension Count: 16
- Specific Lora Fine-Tuning Run:
  - Step: 26
  - Loss: 1.4906
  - Learning Rate: 0.00019814951887490748
  - Epoch: 0.01

### Speeds, Sizes, Times
Approximately 12 minutes training time on RTX 6000 Ada GPU.

### Testing Data
Achieved a perplexity score of 5.17 against the Wikitext database.

### Factors
Focus on language coherence, pirate dialect adherence, and technical accuracy.

### Metrics
Primary metric: Perplexity. Qualitative assessments of dialect authenticity and technical content.

### Results
Marked improvement in sophisticated output with authentic pirate tone. Lower perplexity score demonstrates enhanced language modeling.

### Summary
Represents a significant advancement in domain-specific language modeling, excelling in complex, authentic pirate-themed content.

### Model Architecture and Objective
Based on Mistral Instruct v0.2, fine-tuned for high coherence and technical accuracy in pirate-themed content.

### Compute Infrastructure
Trained on RTX 6000 Ada GPU for efficient training and improved perplexity scores.

### Hardware
- Type: RTX 6000 Ada
- Utilization: Approx. 12 minutes for training.

### Acknowledgments
Gratitude to Mistral and Mistral Instruct v0.2 teams. Appreciation to the language modeling community for support in domain-specific model enhancement.