Collections

1

Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLM

Paper • 2403.07816 • Published Mar 12, 2024 • 40
OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models

Paper • 2402.01739 • Published Jan 29, 2024 • 27
MoE-LLaVA: Mixture of Experts for Large Vision-Language Models

Paper • 2401.15947 • Published Jan 29, 2024 • 50
Mixture-of-LoRAs: An Efficient Multitask Tuning for Large Language Models

Paper • 2403.03432 • Published Mar 6, 2024 • 1

-

Turn Waste into Worth: Rectifying Top-k Router of MoE

Paper • 2402.12399 • Published Feb 17, 2024 • 2
CompeteSMoE -- Effective Training of Sparse Mixture of Experts via Competition

Paper • 2402.02526 • Published Feb 4, 2024 • 3
Buffer Overflow in Mixture of Experts

Paper • 2402.05526 • Published Feb 8, 2024 • 8
OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models

Paper • 2402.01739 • Published Jan 29, 2024 • 27

Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLM

OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models

MoE-LLaVA: Mixture of Experts for Large Vision-Language Models

Mixture-of-LoRAs: An Efficient Multitask Tuning for Large Language Models

Turn Waste into Worth: Rectifying Top-k Router of MoE

CompeteSMoE -- Effective Training of Sparse Mixture of Experts via Competition

Buffer Overflow in Mixture of Experts

OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models

Non-asymptotic oracle inequalities for the Lasso in high-dimensional mixture of experts

MoE-LLaVA: Mixture of Experts for Large Vision-Language Models

MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts

MoE-Infinity: Activation-Aware Expert Offloading for Efficient MoE Serving

Mixtral of Experts

MoE-LLaVA: Mixture of Experts for Large Vision-Language Models

MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts

EdgeMoE: Fast On-Device Inference of MoE-based Large Language Models

LoRA+: Efficient Low Rank Adaptation of Large Models

The FinBen: An Holistic Financial Benchmark for Large Language Models

TofuEval: Evaluating Hallucinations of LLMs on Topic-Focused Dialogue Summarization

TrustLLM: Trustworthiness in Large Language Models

OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models

Rethinking Interpretability in the Era of Large Language Models

Self-Discover: Large Language Models Self-Compose Reasoning Structures

Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model

BlackMamba: Mixture of Experts for State-Space Models

OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models

MoE-LLaVA: Mixture of Experts for Large Vision-Language Models

DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models

BiLLM: Pushing the Limit of Post-Training Quantization for LLMs

Self-Discover: Large Language Models Self-Compose Reasoning Structures

Can Mamba Learn How to Learn? A Comparative Study on In-Context Learning Tasks

Scaling Laws for Downstream Task Performance of Large Language Models

Rethinking Interpretability in the Era of Large Language Models

OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models

BitNet: Scaling 1-bit Transformers for Large Language Models

Stealing Part of a Production Language Model

OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models

DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models

Rethinking Interpretability in the Era of Large Language Models