Amazon SageMaker Inference introduces prefix-aware routing to optimize large language model (LLM) processing. It reduces time-to-first-token (TTFT) by up to 77% and increases throughput by up to 16%, improving KV cache hit rates from 25% to over 80%.
Agent Evaluation Metric (AEM) reveals root cause of failure in multi-turn conversations. Existing metrics fail to track quality breakdown turn by turn.
AvioBook, a Thales Group Company, helps airlines save money by optimizing turnaround times with their software AvioBook Connect. Their Connected Analytics feature uses Amazon Bedrock AgentCore to provide real-time answers to operational questions, improving efficiency and reducing delays.
Heurist uses Amazon Bedrock AgentCore to create AI-powered financial intelligence for retail investors in a unified chat experience. By leveraging AgentCore payments, Heurist provides access to premium data per query, making complex financial tools accessible to all.
Alibaba's Qwen team released Qwen3.8-2.4T-A95B, the first open-weight Qwen-Max-class model with 2.4 trillion total parameters for demanding agentic and reasoning tasks. Deploying on Amazon SageMaker HyperPod enables customized inference behavior and native Multi-Token Prediction for speculative decoding.
MIT hosted AI Educators Pilot workshop to expand AI education, empowering instructors to teach foundational concepts across disciplines. Collaborative effort involved faculty from various universities adapting MIT's Modeling with Machine Learning course materials for their classrooms.
Automating custom permissions in Amazon Quick ensures fine-grained access control for users. Four architectural patterns automate custom permissions at key stages of the user lifecycle.
TorchServe is no longer maintained, leaving teams to handle security patches and compatibility issues. AWS introduces Ray Serve DLC for efficient model inference deployment on AWS platforms, streamlining the process with pre-built Docker images.
NVIDIA AI for Media enhances broadcast and production workflows with real-time intelligence at IBC 2026. Dalet, TwelveLabs, and Wowza integrate NVIDIA's Synthetic Video Detector for content verification and compliance in media and entertainment industries.
Automating model registration between MLflow and the SageMaker AI Model Registry streamlines governance and lifecycle management for data scientists and governance officers. The richer sync now includes training metrics, evaluation results, lineage, and lifecycle stage promotion, simplifying the process of moving models from staging to production seamlessly.
Choosing the right GPU instance for large language model inference is crucial for deploying generative AI at scale. Benchmarking shows how the new G7 instances deliver gains in throughput, latency, and cost-per-token for enterprise coding and reasoning tasks.
HPE Zerto teams up with AWS to create an AI-powered agentic troubleshooting system for hybrid and multi-cloud infrastructures. The system provides natural language support for faster recovery decisions, reducing data loss and downtime.
Pathway's brain-inspired BDH offers a new approach to AI reasoning, avoiding traditional sequential token generation. Amazon SageMaker HyperPod helps scale out training for models like BDH, addressing limitations of transformer architecture.
DiDi and AWS collaborated to develop a transparent AI-driven contact center QA system on Amazon Bedrock, enhancing accuracy and efficiency in intent verification, compliance evaluation, and VOC analysis. DiDi IBG's CX department overcame challenges of traceability, combinatorial complexity, QA standard changes, and trend detection to improve service quality for millions of users across ride-hai...
Amazon SageMaker Feature Store introduces UpdateRecord API, allowing feature-level writes to enhance ML model training efficiency. This eliminates the need for full-record writes, reducing latency and costs associated with unnecessary read capacity units.