Tag: mixture of experts
Browse our exclusive articles!
Routing-Free Mixture-of-Experts: Scalable AI Model
Discover Routing-Free Mixture-of-Experts, a scalable AI model offering decentralized activation, continuous gradients, and adaptive load balancing.
Scalable Pretraining of Large MoE Language Models on Aurora
Discover scalable pretraining of large Mixture of Experts language models using the Aurora supercomputer with high GPU efficiency and advanced optimization...
Self-Routing: Parameter-Free Expert Routing for MoE Models
Discover Self-Routing, a parameter-free expert routing method that boosts MoE model efficiency by using hidden states without a learned router.
TrafficMoE: Advanced Encrypted Traffic Classification Method
Discover TrafficMoE, a heterogeneity-aware framework using Mixture of Experts for accurate encrypted traffic classification and improved network security.
DS-MoE: LLM-Driven Dynamic Sparse Expert Activation
Discover DS-MoE, a dynamic sparse expert activation model driven by large language models for superior visual recognition and real-time inference.