Read-ME: Refactorizing LLMs as Router-Decoupled Mixture of Experts with System Co-Design
Paper
•
2410.19123
•
Published
•
15
Peihao Wang
peihaowang
AI & ML interests
Geometric Learning, Generative Models, Neural Rendering, Large Language Models
Recent Activity
commented
a paper
3 days ago
Understanding and Mitigating Bottlenecks of State Space Models through
the Lens of Recency and Over-smoothing
updated
a model
about 2 months ago
peihaowang/llama-410m-nh8-dclm-7.5b
updated
a model
about 2 months ago
peihaowang/llama-410m-nh16-dclm-7.5b
Organizations
