Jamba: A Hybrid Transformer-Mamba Language Model
Paper
•
2403.19887
•
Published
•
104
LLM
-
-
Aurora-M: The First Open Source Multilingual Language Model Red-teamed according to the U.S. Executive Order
Paper • 2404.00399 • Published • 41 -
Mixture-of-Depths: Dynamically allocating compute in transformer-based language models
Paper • 2404.02258 • Published • 104 -
Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length
Paper • 2404.08801 • Published • 64 -
Multi-Head Mixture-of-Experts
Paper • 2404.15045 • Published • 59 -
Octopus v4: Graph of language models
Paper • 2404.19296 • Published • 116 -
Layerwise Recurrent Router for Mixture-of-Experts
Paper • 2408.06793 • Published • 31 -
Amuro & Char: Analyzing the Relationship between Pre-Training and Fine-Tuning of Large Language Models
Paper • 2408.06663 • Published • 15