Tag: mixture of experts
Browse our exclusive articles!
Efficient Mixed-Precision Quantization for Mixture-of-Experts
Discover a novel mixed-precision quantization method for Mixture-of-Experts models that boosts accuracy and reduces inference costs with theoretical guaran...
TalkLoRA: Efficient Communication in Low-Rank Adaptation for LLMs
Discover TalkLoRA, a communication-aware low-rank adaptation method that boosts efficiency and stability in large language model fine-tuning.
JoyAI-LLM Flash: Efficient Mid-Scale LLM with Token Savings
Discover JoyAI-LLM Flash, a mid-scale language model optimizing token efficiency and performance with novel RL and sparsity techniques.
DR-LoRA: Adaptive Fine-Tuning for Mixture-of-Experts Models
Discover DR-LoRA, a dynamic rank LoRA method that optimizes fine-tuning of Mixture-of-Experts models for improved AI performance and efficiency.
StanceMoE: Advanced Mixture-of-Experts for Stance Detection
Discover StanceMoE, a Mixture-of-Experts model with BERT for high-accuracy actor-level stance detection in geopolitical texts.