Streaming Kokoro TTS for AI agents on SageMaker AI
AWS demonstrates sending each completed agent sentence to Kokoro-82M on a SageMaker AI endpoint, so speech synthesis can begin before the full reply is ready.
LLM Inference Infrastructure
15 posts
AWS demonstrates measured autoscaling for SageMaker AI TTS
AWS deploys Kokoro-82M in a custom container on a SageMaker AI real-time endpoint and derives autoscaling settings from measurements on that endpoint.
NVIDIA Demonstrates Eight-Node DGX Spark Clusters
Eight DGX Spark nodes combine for 1 TB of unified memory, enough to load the full GLM 5.3 753B checkpoint locally.
OpenAI Python SDK 3.36.1 fixes WebSocket and browser-call behavior
OpenAI Python SDK 3.36.1 now applies WebSocket query options supplied for an individual API call.
Strands Agents demo for routing work between local and cloud models
AAIF uses a local llama3.2:1b model as a request classifier in a Strands Agents setup, directing work to local tools or Amazon Bedrock instead of sending every request to one cloud model.
Ollama v0.40.8 adds Recursive’s Flint model
Ollama v0.40.8 makes Recursive’s Flint coding model available to run through Ollama.
OpenAI Python SDK 3.36.0 adds Vertex AI support
Version 3.36.0 lets Python SDK users call the Responses API with Vertex AI.
NVIDIA Introduces Training-Free Relational Predictions
NVIDIA’s Kumo Relational runs predictive queries directly on relational databases, without task-specific training or manual feature engineering.
VocabX Cuts Inference Cost with Domain-Specific Vocabularies
Dharma AI tested VocabX on a Portuguese essay grader, keeping Qwen3-4B, the training data, and the training recipe while replacing its general-purpose vocabulary.
LiteLLM v1.103.7 updates usage fixes and image verification
The v1.103.7 Docker image can be checked with cosign using a public key linked to either a fixed commit or the release tag.
NVIDIA outlines an agent-assisted workflow for AI factory digital twins
NVIDIA’s Omniverse DSX Blueprint provides reference assets and workflows for building AI factory digital twins on OpenUSD and Omniverse libraries.
Anthropic Python SDK v1.17.0 adds API endpoints and header fixes
The release adds organization webhook endpoints and user profile enforcement events to the API.
i14 Trains and Serves Laplace 3B on AWS Trainium2
i14’s Laplace 3B uses Gated Linear Attention, keeping its recurrent state at 29.5 MB per sequence regardless of context length.
LiteLLM v1.106.1 fixes proxy and vector store issues
LiteLLM fixed multi-turn Anthropic requests dropping signed thinking blocks and `thinking.block_binding`.
Amazon Bedrock Adds Cross-Region Access to OpenAI GPT Models from Canada
Organizations in the Canada (Central) and Canada West (Calgary) Regions can access OpenAI GPT-5.6, GPT-6, and GPT-6.1 models through Amazon Bedrock cross-Region inference.