Tag: LLM inference
Browse our exclusive articles!
LLM Inference: Nvidia vs Apple Silicon Performance & Efficiency
Explore Nvidia and Apple Silicon's trade-offs in consumer-grade LLM inference, focusing on performance, efficiency, and ecosystem challenges.
DUAL-BLADE: Optimized NVMe KV-Cache for Edge LLM Inference
Discover DUAL-BLADE's dual-path NVMe KV-cache offloading that boosts edge LLM inference by reducing latency and improving memory efficiency.
DepthKV: Layer-Wise KV Cache Pruning for Efficient LLMs
Optimize long-context LLM inference with DepthKV's layer-dependent KV cache pruning for improved memory efficiency and model performance.
Hybrid JIT-CUDA Graph for Fast LLM Inference
Boost LLM inference speed with hybrid JIT-CUDA graph optimization, reducing latency and improving responsiveness for AI applications.
2D Early Exit Optimization for Faster LLM Inference
Boost large language model efficiency with 2D early exit optimization, combining layer-wise and sentence-wise exits for faster inference.