Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models
Paper
•
2501.11873
•
Published
•
63
Bryan Teo
bteo98
·
AI & ML interests
None yet
Recent Activity
upvoted
a
paper
12 days ago
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via
Reinforcement Learning
upvoted
a
paper
12 days ago
VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video
Understanding
Organizations
None yet
bteo98's activity
upvoted
a
paper
3 days ago
upvoted
2
papers
12 days ago
upvoted
4
papers
about 2 months ago
Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference
Paper
•
2412.13663
•
Published
•
125
No More Adam: Learning Rate Scaling at Initialization is All You Need
Paper
•
2412.11768
•
Published
•
41
Qwen2.5 Technical Report
Paper
•
2412.15115
•
Published
•
345
Byte Latent Transformer: Patches Scale Better Than Tokens
Paper
•
2412.09871
•
Published
•
89
upvoted
a
paper
5 months ago
upvoted
a
paper
6 months ago
upvoted
a
paper
7 months ago
upvoted
a
paper
8 months ago
upvoted
4
papers
9 months ago
Chameleon: Mixed-Modal Early-Fusion Foundation Models
Paper
•
2405.09818
•
Published
•
130
Kangaroo: Lossless Self-Speculative Decoding via Double Early Exiting
Paper
•
2404.18911
•
Published
•
30
OpenELM: An Efficient Language Model Family with Open-source Training and Inference Framework
Paper
•
2404.14619
•
Published
•
127
LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding
Paper
•
2404.16710
•
Published
•
77
upvoted
a
paper
10 months ago
upvoted
a
paper
11 months ago
upvoted
a
paper
about 1 year ago