Tag: CPU LLM serving
Browse our exclusive articles!
Cost-Effective Network Topologies for MoE LLM Serving
Discover cost-effective network topologies that optimize Mixture-of-Experts LLM serving, improving performance while reducing infrastructure expenses.
AMMA: Low-Latency Memory-Centric Architecture for 1M Context
Discover AMMA, a multi-chiplet memory-centric architecture delivering 15.5X lower latency and 6.9X energy savings for 1M context attention in LLM serving.
HFX: Optimized Multi-SLO Serving & Fast Scaling for LLMs
Discover HFX, the advanced system for multi-SLO serving and rapid scaling in LLMs, boosting performance and cutting costs effectively.
Sandwich: Efficient CPU Serving for Large Language Models
Discover Sandwich, a CPU LLM serving system that boosts performance with phase switching, hardware-aware allocation, and fast dynamic tensor tuning.