Tag: knowledge distillation
Browse our exclusive articles!
Trace-Based Knowledge Distillation: Accuracy vs Interpretability
Explore the impact of trace correctness on accuracy and interpretability in trace-based knowledge distillation for large language models.
Prevent Unauthorized Distillation of Language Models
Learn how trace rewriting protects large language models from unauthorized knowledge distillation while embedding verifiable watermarks.
Token Importance in On-Policy Distillation Explained
Discover how token importance boosts on-policy distillation efficiency by focusing on student entropy and teacher-student divergence signals.
WAND: Efficient Windowed Attention for Text-to-Speech
Discover WAND, a novel method using windowed attention and knowledge distillation to boost efficiency in autoregressive text-to-speech models.
KD-MARL: Efficient Knowledge Distillation for Multi-Agent RL
KD-MARL enables resource-aware knowledge distillation in multi-agent reinforcement learning, reducing compute costs while preserving expert coordination an...