Tag: mixture of experts
Browse our exclusive articles!
SpecMoE: Fast, Efficient Mixture-of-Experts Inference
Discover SpecMoE, a memory-efficient MoE inference system boosting throughput by 4.3x with self-assisted speculative decoding for scalable AI models.
Understanding Expert Specialization in MoEs: Geometry Over Domain
Explore why expert specialization in Mixture of Experts (MoEs) stems from hidden state geometry, not domain expertise, revealing new AI insights.
Scaling Laws and Generalization in MoE Transformers
Explore key scaling laws and generalization bounds for Mixture-of-Experts Transformers to optimize model size, data, and compute trade-offs.
ROAM: Optimal Transport Routing for WSI Classification
Discover ROAM, a novel region-graph optimal transport method enhancing Mixture-of-Experts routing for accurate whole-slide image classification in patholog...
MoBiE: Fast, Efficient Mixture of Binary Experts Inference
Discover MoBiE, a breakthrough in efficient inference for Mixture-of-Experts models using post-training quantization to boost speed and reduce memory.