Tag: AI inference
Browse our exclusive articles!
Boost Generative AI Inference with Amazon SageMaker G7e
Accelerate generative AI on Amazon SageMaker using powerful G7e instances with NVIDIA RTX PRO 6000 GPUs for unmatched performance and cost efficiency.
Ragged Paged Attention: Fast LLM Inference Kernel for TPU
Discover Ragged Paged Attention, a high-performance LLM inference kernel optimized for TPU, boosting efficiency and reducing costs in large language model...
Cost-Effective Custom Text-to-SQL with Amazon Nova Micro
Learn how to build cost-efficient custom text-to-SQL solutions using Amazon Nova Micro and Bedrock's on-demand inference for scalable SQL generation.
SpecBound: Boost LLM Speed with Adaptive Speculation
Discover SpecBound's adaptive self-speculation and layer-wise confidence calibration to accelerate large language model decoding by up to 2.33x.
StreamServe: Low-Latency LLM Serving with Adaptive Flows
StreamServe boosts LLM serving efficiency with adaptive speculative decoding and metric-aware routing, cutting latency by up to 18x on multi-GPU setups.