# Amazon SageMaker HyperPod

Amazon SageMaker HyperPod is a managed service for training, fine-tuning, serving, and observing AI models on resilient shared compute infrastructure, with Slurm and Amazon EKS orchestration.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Managed NVIDIA Dynamo on Amazon SageMaker HyperPod Inference

DevFeed: [Managed NVIDIA Dynamo on Amazon SageMaker HyperPod Inference](<https://devfeed.tech/articles/managed-nvidia-dynamo-on-amazon-sagemaker-hyperpod-inference-138699.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/managed-nvidia-dynamo-on-amazon-sagemaker-hyperpod-inference/>)

Author: Vinay Arora

Published: 2026-11-17T23:33:26Z

Content type: article

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [Amazon SageMaker HyperPod](<https://devfeed.tech/topics/amazon-sagemaker-hyperpod.md>), [Dynamo](<https://devfeed.tech/topics/dynamo.md>), [model-serving](<https://devfeed.tech/topics/model-serving.md>), [Amazon SageMaker](<https://devfeed.tech/topics/amazon-sagemaker.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [vllm](<https://devfeed.tech/topics/vllm.md>), [TensorRT-LLM](<https://devfeed.tech/topics/tensorrt-llm.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [AI Architecture](<https://devfeed.tech/topics/ai-architecture.md>)

Tags: [advanced-300](<https://devfeed.tech/tags/advanced-300.md>), [agent-skills](<https://devfeed.tech/tags/agent-skills.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [amazon](<https://devfeed.tech/tags/amazon.md>), [amazon-sagemaker](<https://devfeed.tech/tags/amazon-sagemaker.md>), [amazon-sagemaker-hyperpod](<https://devfeed.tech/tags/amazon-sagemaker-hyperpod.md>), [announcements](<https://devfeed.tech/tags/announcements.md>), [api](<https://devfeed.tech/tags/api.md>), [aws](<https://devfeed.tech/tags/aws.md>), [benchmark-performance](<https://devfeed.tech/tags/benchmark-performance.md>), [disaggregated-serving](<https://devfeed.tech/tags/disaggregated-serving.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [distributed-inference](<https://devfeed.tech/tags/distributed-inference.md>), [dynamo](<https://devfeed.tech/tags/dynamo.md>), [inference](<https://devfeed.tech/tags/inference.md>), [managed](<https://devfeed.tech/tags/managed.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [nvidia-dynamo](<https://devfeed.tech/tags/nvidia-dynamo.md>), [orchestration](<https://devfeed.tech/tags/orchestration.md>), [recovery](<https://devfeed.tech/tags/recovery.md>), [routing](<https://devfeed.tech/tags/routing.md>), [serving](<https://devfeed.tech/tags/serving.md>)

### AI overview

SageMaker HyperPod adds managed NVIDIA Dynamo deployments for large models that may exceed one instance's GPU memory or benefit from separating prompt processing and token generation. Users can bring an existing DynamoGraphDeployment or configure an InferenceEndpointConfig with dynamoSpec; the latter supports vLLM by default, plus SGLang and TensorRT-LLM. Dynamo provides multi-node and aggregated or disaggregated serving, while HyperPod components coordinate GPU placement, EFA networking, node health, and recovery. In benchmarks described in the source, disaggregated Qwen3-235B serving reached 207 versus 101 output tokens per second per GPU at selected SLA-compliant points. A two-node Qwen3.5-397B aggregated setup retained 99.1% of single-node throughput under the tested workload. The article advises selecting a matching recipe, verifying EFA transport, benchmarking representative traffic against SLAs, and tuning based on measured bottlenecks; results depend on the tested models and workloads.

### Source excerpt

Amazon SageMaker HyperPod Inference now supports NVIDIA Dynamo for distributed inference, with multi-node and disaggregated serving, KV-aware routing, and automatic fault recovery. Serve large models across many nodes without building the orchestration layer yourself.

## Build a model improvement flywheel with Amazon SageMaker HyperPod Data Capture

DevFeed: [Build a model improvement flywheel with Amazon SageMaker HyperPod Data Capture](<https://devfeed.tech/articles/build-a-model-improvement-flywheel-with-amazon-sagemaker-hyperpod-data-capture-132650.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/build-a-model-improvement-flywheel-with-amazon-sagemaker-hyperpod-data-capture/>)

Author: Shantanu Tripathi

Published: 2026-11-13T18:01:15Z

Content type: tutorial

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [Amazon SageMaker HyperPod](<https://devfeed.tech/topics/amazon-sagemaker-hyperpod.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Amazon SageMaker](<https://devfeed.tech/topics/amazon-sagemaker.md>), [Amazon S3](<https://devfeed.tech/topics/amazon-s3.md>)

Tags: [after](<https://devfeed.tech/tags/after.md>), [amazon](<https://devfeed.tech/tags/amazon.md>), [amazon-athena](<https://devfeed.tech/tags/amazon-athena.md>), [amazon-s3](<https://devfeed.tech/tags/amazon-s3.md>), [amazon-sagemaker](<https://devfeed.tech/tags/amazon-sagemaker.md>), [amazon-sagemaker-hyperpod](<https://devfeed.tech/tags/amazon-sagemaker-hyperpod.md>), [athena](<https://devfeed.tech/tags/athena.md>), [build](<https://devfeed.tech/tags/build.md>), [capture](<https://devfeed.tech/tags/capture.md>), [data](<https://devfeed.tech/tags/data.md>), [example](<https://devfeed.tech/tags/example.md>), [fine](<https://devfeed.tech/tags/fine.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [improvement](<https://devfeed.tech/tags/improvement.md>), [inference](<https://devfeed.tech/tags/inference.md>), [intermediate-200](<https://devfeed.tech/tags/intermediate-200.md>), [live](<https://devfeed.tech/tags/live.md>), [measuring](<https://devfeed.tech/tags/measuring.md>), [model](<https://devfeed.tech/tags/model.md>), [new-product](<https://devfeed.tech/tags/new-product.md>), [technical-how-to](<https://devfeed.tech/tags/technical-how-to.md>), [with](<https://devfeed.tech/tags/with.md>)

### AI overview

SageMaker HyperPod Data Capture records inference inputs and outputs in S3 at three independent tiers. In the example, Tier 3 captures a support router's traffic; Athena reveals that a new air-conditioner line drove "Other" labels from 4.8% before launch to 36.8% afterward. A SQL labeling policy turns captures into training data, and a 60-second LoRA fine-tune on a second GPU raises new-category recall from 0% to full recall without regression on established classes. Held-out probes also expose shortcuts learned from the synthetic, keyword-labeled data. The walkthrough requires two GPUs and appropriate S3, KMS, and Athena permissions; Tier 3 payloads may contain personal information and need safeguards. The post reports no measurable serving-latency impact in its test, while warning that self-reported confidence was unreliable for detecting misroutes.

### Source excerpt

Amazon SageMaker HyperPod Data Capture records live inference inputs and outputs to Amazon S3. This post walks through one end-to-end example: capturing a support router's production traffic, surfacing a new-product blind spot with Amazon Athena, fine-tuning on the captured data with the HyperPod training operator, and measuring the before-and-after lift.

## Scale distributed AI workloads with Ray and Anyscale on Amazon EC2

DevFeed: [Scale distributed AI workloads with Ray and Anyscale on Amazon EC2](<https://devfeed.tech/articles/scale-distributed-ai-workloads-with-ray-and-anyscale-on-amazon-ec2-132423.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/architecture/scale-distributed-ai-workloads-with-ray-and-anyscale-on-amazon-ec2/>)

Author: Sindhura Palakodety

Published: 2026-11-12T20:51:20Z

Content type: article

Language: en

Sources: [AWS Architecture Blog](<https://devfeed.tech/sources/aws-architecture-blog.md>)

Topics: [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [distributed-training](<https://devfeed.tech/topics/distributed-training.md>), [Amazon EC2](<https://devfeed.tech/topics/amazon-ec2.md>), [Graviton](<https://devfeed.tech/topics/graviton.md>), [AWS Trainium](<https://devfeed.tech/topics/aws-trainium.md>), [AWS Inferentia](<https://devfeed.tech/topics/aws-inferentia.md>), [Amazon SageMaker HyperPod](<https://devfeed.tech/topics/amazon-sagemaker-hyperpod.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-chips](<https://devfeed.tech/tags/ai-chips.md>), [ai-inference](<https://devfeed.tech/tags/ai-inference.md>), [ai-ml](<https://devfeed.tech/tags/ai-ml.md>), [ai-workloads](<https://devfeed.tech/tags/ai-workloads.md>), [amazon-api-gateway](<https://devfeed.tech/tags/amazon-api-gateway.md>), [amazon-cloudwatch](<https://devfeed.tech/tags/amazon-cloudwatch.md>), [amazon-ec2](<https://devfeed.tech/tags/amazon-ec2.md>), [aws](<https://devfeed.tech/tags/aws.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [intermediate-200](<https://devfeed.tech/tags/intermediate-200.md>), [ml](<https://devfeed.tech/tags/ml.md>), [models](<https://devfeed.tech/tags/models.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [pipelines](<https://devfeed.tech/tags/pipelines.md>), [production](<https://devfeed.tech/tags/production.md>), [prototype](<https://devfeed.tech/tags/prototype.md>), [ray](<https://devfeed.tech/tags/ray.md>), [scale](<https://devfeed.tech/tags/scale.md>), [scaling](<https://devfeed.tech/tags/scaling.md>), [thought-leadership](<https://devfeed.tech/tags/thought-leadership.md>)

### AI overview

Ray and Anyscale provide a managed way to scale AI/ML pipelines on AWS while keeping Ray workload code consistent from a single instance to distributed clusters. Amazon EC2 offers the most flexibility, Amazon EKS fits organizations already operating Kubernetes but adds orchestration complexity and operator configuration, and SageMaker HyperPod targets resilience-critical large training runs with health monitoring, node replacement, and checkpoint recovery. Anyscale manages cluster lifecycle and autoscaling; compute and data remain in the customer's AWS account. The text describes matching CPU, GPU, Trainium, and Inferentia resources to pipeline stages, using Spot for fault-tolerant batch work, and combining Savings Plans, Capacity Blocks, reservations, and On-Demand capacity. Its suggested progression is to validate locally, scale data processing, expand training on a chosen deployment path, then deploy inference; it says this can be done without workload code changes.

### Source excerpt

Building multimodal AI/ML pipelines on Amazon EC2 means scaling distributed compute, controlling costs across stages, and keeping up with new models and hardware. This post shows how the Ray-powered Anyscale application on AWS helps you deploy Ray across three compute options and scale from prototype to production without code changes.

## Detecting and recovering from silent data corruption on Amazon SageMaker HyperPod

DevFeed: [Detecting and recovering from silent data corruption on Amazon SageMaker HyperPod](<https://devfeed.tech/articles/detecting-and-recovering-from-silent-data-corruption-on-amazon-sagemaker-hyperpod-129622.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/detecting-and-recovering-from-silent-data-corruption-on-amazon-sagemaker-hyperpod/>)

Author: Siddharth Senger

Published: 2026-11-11T23:30:03Z

Content type: tutorial

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [Amazon SageMaker HyperPod](<https://devfeed.tech/topics/amazon-sagemaker-hyperpod.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Amazon SageMaker](<https://devfeed.tech/topics/amazon-sagemaker.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [Monitoring](<https://devfeed.tech/topics/monitoring.md>), [Error Handling](<https://devfeed.tech/topics/error-handling.md>), [Amazon S3](<https://devfeed.tech/topics/amazon-s3.md>)

Tags: [advanced-300](<https://devfeed.tech/tags/advanced-300.md>), [amazon-sagemaker-hyperpod](<https://devfeed.tech/tags/amazon-sagemaker-hyperpod.md>), [distributed-training](<https://devfeed.tech/tags/distributed-training.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [pytorch](<https://devfeed.tech/tags/pytorch.md>), [recovery](<https://devfeed.tech/tags/recovery.md>), [replacement](<https://devfeed.tech/tags/replacement.md>), [silent-data-corruption](<https://devfeed.tech/tags/silent-data-corruption.md>), [technical-how-to](<https://devfeed.tech/tags/technical-how-to.md>)

### AI overview

Silent data corruption can produce plausible GPU results without triggering hardware alarms, so the article adds three checks inside a PyTorch DDP training loop. Before the optimizer update, check each rank's local gradients for NaN or Inf, then check the reduced gradients; ring replay periodically recomputes another node's micro-batch to catch finite errors. Default replay compares loss, while deterministic audits also compare gradient fingerprints and can catch backward-only faults at higher cost. After updates, compare weight fingerprints and always check before checkpointing. A mismatch can identify a suspect only when enough replicas provide a strict majority; fingerprints are strong evidence, not proof. The approach requires at least three GPU nodes, reproducible micro-batches, and shared checkpoint storage. On an alarm, stop and cordon the node, correlate GPU metrics, and replace it after review; resume from a checkpoint saved before the suspect window. The reported tests used software-injected faults on H100 clusters and do not establish precision for other workloads.

### Source excerpt

Silent data corruption is a GPU fault that returns wrong but plausible numbers with no error, quietly poisoning distributed training. This post adds three checks to a PyTorch DDP loop to detect it, localize the faulty node on Amazon SageMaker HyperPod, and hand the node to automatic recovery for replacement.

## Expensive accelerators, no workload-level visibility: Introducing Amazon CloudWatch accelerated compute insights

DevFeed: [Expensive accelerators, no workload-level visibility: Introducing Amazon CloudWatch accelerated compute insights](<https://devfeed.tech/articles/expensive-accelerators-no-workload-level-visibility-introducing-amazon-cloudwatch-accelerated-compute-insights-129501.md>)

Original publisher: [Read original article](<https://thenewstack.io/cloudwatch-accelerated-compute-insights/>)

Author: Sabari Sawant

Published: 2026-11-11T18:00:00Z

Content type: article

Language: en

Sources: [Kubernetes Overview, News and Trends | The New Stack](<https://devfeed.tech/sources/kubernetes-overview-news-and-trends-the-new-stack.md>), [The New Stack](<https://devfeed.tech/sources/the-new-stack.md>)

Topics: [Amazon CloudWatch](<https://devfeed.tech/topics/amazon-cloudwatch.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [AWS Inferentia](<https://devfeed.tech/topics/aws-inferentia.md>), [Nvidia](<https://devfeed.tech/topics/nvidia.md>), [Amazon SageMaker HyperPod](<https://devfeed.tech/topics/amazon-sagemaker-hyperpod.md>)

Tags: [accelerators](<https://devfeed.tech/tags/accelerators.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-inference](<https://devfeed.tech/tags/ai-inference.md>), [ai-infrastructure](<https://devfeed.tech/tags/ai-infrastructure.md>), [amazon-cloudwatch](<https://devfeed.tech/tags/amazon-cloudwatch.md>), [amazon-eks](<https://devfeed.tech/tags/amazon-eks.md>), [automated-remediation](<https://devfeed.tech/tags/automated-remediation.md>), [aws-inferentia](<https://devfeed.tech/tags/aws-inferentia.md>), [aws-marketplace](<https://devfeed.tech/tags/aws-marketplace.md>), [aws-trainium](<https://devfeed.tech/tags/aws-trainium.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [kubernetes](<https://devfeed.tech/tags/kubernetes.md>), [observability](<https://devfeed.tech/tags/observability.md>), [post-contributed](<https://devfeed.tech/tags/post-contributed.md>), [sponsor-aws-marketplace](<https://devfeed.tech/tags/sponsor-aws-marketplace.md>), [sponsored-post-contributed](<https://devfeed.tech/tags/sponsored-post-contributed.md>)

### AI overview

Amazon CloudWatch accelerated compute insights adds device-level GPU and Inferentia visibility for workloads on Amazon EKS, connecting accelerator health and utilization to Kubernetes workload metadata, estimated waste, and remediation guidance. It supports NVIDIA GPUs, AWS Trainium, and AWS Inferentia; GPU and Inferentia metrics are normalized into a shared health view, with MIG instances and other partitioning schemes recognized. It flags oversized instances, low training tensor-core utilization against a workload's historical baseline, and inference throughput below hardware capacity. Its cost figures are operational estimates, not billing or chargeback records; AWS Cost Explorer and the Cost and Usage Report remain the authoritative sources. Setup requires CloudWatch Observability EKS add-on v7.0.1 or later for self-hosted inference, with OTel Container Insights enabled. Telemetry can be viewed in CloudWatch Omni for workload-level troubleshooting or in the CloudWatch console for fleet-wide views. AWS DevOps Agent can investigate health events and propose actions for approval; autonomous remediation is optional.

### Source excerpt

Accelerator bills have become one of the largest lines in the infrastructure budget. GPUs have become the most expensive resource The post Expensive accelerators, no workload-level visibility: Introducing Amazon CloudWatch accelerated compute insights appeared first on The New Stack.

## Accelerate agentic AI with NVIDIA Nemotron models on Amazon EKS and SageMaker HyperPod EKS

DevFeed: [Accelerate agentic AI with NVIDIA Nemotron models on Amazon EKS and SageMaker HyperPod EKS](<https://devfeed.tech/articles/accelerate-agentic-ai-with-nvidia-nemotron-models-on-amazon-eks-and-sagemaker-hyperpod-eks-106871.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/accelerate-agentic-ai-with-nvidia-nemotron-models-on-amazon-eks-and-sagemaker-hyperpod-eks/>)

Author: Alex Iankoulski

Published: 2026-10-29T16:11:08Z

Content type: article

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [Nemotron](<https://devfeed.tech/topics/nemotron.md>), [Amazon SageMaker HyperPod](<https://devfeed.tech/topics/amazon-sagemaker-hyperpod.md>), [Agentic AI](<https://devfeed.tech/topics/what-is-agentic-ai.md>), [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [model architecture](<https://devfeed.tech/topics/model-architecture.md>), [vllm](<https://devfeed.tech/topics/vllm.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>), [quantization](<https://devfeed.tech/topics/quantization.md>)

Tags: [advanced-300](<https://devfeed.tech/tags/advanced-300.md>), [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [amazon-eks](<https://devfeed.tech/tags/amazon-eks.md>), [amazon-elastic-kubernetes-service](<https://devfeed.tech/tags/amazon-elastic-kubernetes-service.md>), [amazon-sagemaker-hyperpod](<https://devfeed.tech/tags/amazon-sagemaker-hyperpod.md>), [announcements](<https://devfeed.tech/tags/announcements.md>), [dynamo](<https://devfeed.tech/tags/dynamo.md>), [mixture-of-experts](<https://devfeed.tech/tags/mixture-of-experts.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [nemotron](<https://devfeed.tech/tags/nemotron.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [nvidia-nemotron-3-5-super-vl](<https://devfeed.tech/tags/nvidia-nemotron-3-5-super-vl.md>), [tool-use](<https://devfeed.tech/tags/tool-use.md>), [vllm](<https://devfeed.tech/tags/vllm.md>)

### AI overview

Nemotron 3.5 Super VL is a 120B-parameter multimodal mixture-of-experts model that activates 12B parameters per token, accepts text, images, documents, and video, and produces text. Its hybrid Mamba-2, attention, and MoE architecture supports long contexts up to 1M tokens; sparse activation reduces per-token compute, though GPU memory must still hold the full weights. The article describes serving it on standard Amazon EKS, EKS Auto Mode, or SageMaker HyperPod EKS with the same Kubernetes deployment manifests, using NVIDIA Dynamo and vLLM. It reports validation on B200 and H200 instances and recommends measuring TTFT, inter-token latency, and throughput across concurrency levels with aiperf. Compare those results against latency targets to estimate per-replica capacity: rising ITL points to adding decode capacity, while rising TTFT points to adding prefill capacity. The excerpt reports lower TTFT and similar ITL for a two-node disaggregated setup versus aggregated serving.

### Source excerpt

NVIDIA Nemotron 3.5 Super VL is a 120B-parameter mixture-of-experts model that pairs reasoning and tool use with multimodal understanding. This post explores its architecture and use cases, then shows how to deploy it on Amazon EKS, EKS Auto Mode, and Amazon SageMaker HyperPod EKS with NVIDIA Dynamo and vLLM.

## Fine-tuning GPT-OSS-20B for multilingual reasoning on Amazon SageMaker HyperPod

DevFeed: [Fine-tuning GPT-OSS-20B for multilingual reasoning on Amazon SageMaker HyperPod](<https://devfeed.tech/articles/fine-tuning-gpt-oss-20b-for-multilingual-reasoning-on-amazon-sagemaker-hyperpod-91293.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/fine-tuning-gpt-oss-20b-for-multilingual-reasoning-on-amazon-sagemaker-hyperpod/>)

Author: Mijanur Palash

Published: 2026-10-20T17:48:27Z

Content type: tutorial

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Amazon SageMaker HyperPod](<https://devfeed.tech/topics/amazon-sagemaker-hyperpod.md>), [gpt-oss](<https://devfeed.tech/topics/gpt-oss.md>), [grpo](<https://devfeed.tech/topics/grpo.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [lora](<https://devfeed.tech/topics/lora.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Enhancing AI model performance](<https://devfeed.tech/topics/enhancing-ai-model-performance.md>), [trl](<https://devfeed.tech/topics/trl.md>), [peft](<https://devfeed.tech/topics/peft.md>), [Amazon SageMaker](<https://devfeed.tech/topics/amazon-sagemaker.md>), [Amazon S3](<https://devfeed.tech/topics/amazon-s3.md>)

Tags: [adapter](<https://devfeed.tech/tags/adapter.md>), [adapters](<https://devfeed.tech/tags/adapters.md>), [advanced-300](<https://devfeed.tech/tags/advanced-300.md>), [amazon](<https://devfeed.tech/tags/amazon.md>), [amazon-eks](<https://devfeed.tech/tags/amazon-eks.md>), [amazon-elastic-container-registry](<https://devfeed.tech/tags/amazon-elastic-container-registry.md>), [amazon-elastic-kubernetes-service](<https://devfeed.tech/tags/amazon-elastic-kubernetes-service.md>), [amazon-elastic-kubernetes-service-amazon-eks](<https://devfeed.tech/tags/amazon-elastic-kubernetes-service-amazon-eks.md>), [amazon-fsx](<https://devfeed.tech/tags/amazon-fsx.md>), [amazon-fsx-for-lustre](<https://devfeed.tech/tags/amazon-fsx-for-lustre.md>), [amazon-s3](<https://devfeed.tech/tags/amazon-s3.md>), [amazon-sagemaker](<https://devfeed.tech/tags/amazon-sagemaker.md>), [amazon-sagemaker-hyperpod](<https://devfeed.tech/tags/amazon-sagemaker-hyperpod.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [gpt-oss](<https://devfeed.tech/tags/gpt-oss.md>), [grpo](<https://devfeed.tech/tags/grpo.md>), [headroom](<https://devfeed.tech/tags/headroom.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [sft](<https://devfeed.tech/tags/sft.md>), [supervised-fine-tuning](<https://devfeed.tech/tags/supervised-fine-tuning.md>), [technical-how-to](<https://devfeed.tech/tags/technical-how-to.md>)

### AI overview

Fine-tuning GPT-OSS-20B with LoRA on Amazon SageMaker HyperPod raised reasoning in the requested language from 22% to 100% on a 50-case, five-language evaluation. SFT taught the missing behavior; final-answer language accuracy reached 62%. GRPO on the SFT checkpoint raised that metric to 70% while maintaining 100% reasoning-language accuracy. A Qwen2.5-7B control showed GRPO added 6 points on multilingual reasoning where there was headroom, but no gain on GSM8K, already at 90.5%. The results support using SFT for behaviors the model does not yet exhibit and adding GRPO to refine behaviors it sometimes produces; when sampled groups are already mostly correct, GRPO may have no useful learning signal. The article describes HyperPod training infrastructure and configuration, including TRL for GPT-OSS-20B and veRL with Ray and vLLM for the Qwen control.

### Source excerpt

Learn how we fine-tuned GPT-OSS-20B for multilingual reasoning on Amazon SageMaker HyperPod, and how to decide between supervised fine-tuning (SFT) and reinforcement learning with GRPO. SFT took reasoning-language accuracy from 22% to 100%, and GRPO added measurable gains where headroom existed.

## How to Fine-Tune LLMs with Axolotl on Runpod

DevFeed: [How to Fine-Tune LLMs with Axolotl on Runpod](<https://devfeed.tech/articles/how-to-fine-tune-llms-with-axolotl-on-runpod-90580.md>)

Original publisher: [Read original article](<https://www.runpod.io/blog/fine-tune-llms-axolotl-runpod>)

Author: James Sandy

Published: 2026-10-20T03:18:00Z

Content type: tutorial

Language: en

Sources: [Runpod Blog.](<https://devfeed.tech/sources/runpod-blog.md>)

Topics: [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [lora](<https://devfeed.tech/topics/lora.md>), [quantization](<https://devfeed.tech/topics/quantization.md>), [Amazon SageMaker HyperPod](<https://devfeed.tech/topics/amazon-sagemaker-hyperpod.md>)

Tags: [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [gpu-infrastructure](<https://devfeed.tech/tags/gpu-infrastructure.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [llms](<https://devfeed.tech/tags/llms.md>), [lora](<https://devfeed.tech/tags/lora.md>), [quantization](<https://devfeed.tech/tags/quantization.md>)

### AI overview

This tutorial explains how to fine-tune large language models with Axolotl on Runpod. It covers GPU and storage setup, dataset preparation, YAML configuration, LoRA adapters, 8-bit quantization, DeepSpeed, training monitoring, and evaluation.

### Source excerpt

Learn how to fine-tune large language models using Axolotl on Runpod. This guide covers LoRA, 8-bit quantization, DeepSpeed, and GPU infrastructure setup.

## Introducing Reflection Beam: an open-weight model for agentic coding on AWS

DevFeed: [Introducing Reflection Beam: an open-weight model for agentic coding on AWS](<https://devfeed.tech/articles/introducing-reflection-beam-an-open-weight-model-for-agentic-coding-on-aws-88891.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/introducing-reflection-beam-an-open-weight-model-for-agentic-coding-on-aws/>)

Author: Clement Perrot

Published: 2026-10-16T15:23:10Z

Content type: tutorial

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [Amazon Web Services](<https://devfeed.tech/topics/aws.md>), [lunar foundation model](<https://devfeed.tech/topics/lunar-foundation-model.md>), [Amazon SageMaker](<https://devfeed.tech/topics/amazon-sagemaker.md>), [Amazon SageMaker HyperPod](<https://devfeed.tech/topics/amazon-sagemaker-hyperpod.md>), [Amazon Elastic Kubernetes Service](<https://devfeed.tech/topics/amazon-elastic-kubernetes-service.md>), [flux](<https://devfeed.tech/topics/flux.md>), [vllm](<https://devfeed.tech/topics/vllm.md>), [AWS Trainium](<https://devfeed.tech/topics/aws-trainium.md>), [Apache Cassandra](<https://devfeed.tech/topics/cassandra.md>)

Tags: [advanced-300](<https://devfeed.tech/tags/advanced-300.md>), [agentic-coding](<https://devfeed.tech/tags/agentic-coding.md>), [ai](<https://devfeed.tech/tags/ai.md>), [amazon-sagemaker-ai](<https://devfeed.tech/tags/amazon-sagemaker-ai.md>), [amazon-sagemaker-hyperpod](<https://devfeed.tech/tags/amazon-sagemaker-hyperpod.md>), [amazon-sagemaker-jumpstart](<https://devfeed.tech/tags/amazon-sagemaker-jumpstart.md>), [api](<https://devfeed.tech/tags/api.md>), [authentication](<https://devfeed.tech/tags/authentication.md>), [autoscaling](<https://devfeed.tech/tags/autoscaling.md>), [aws](<https://devfeed.tech/tags/aws.md>), [aws-cli](<https://devfeed.tech/tags/aws-cli.md>), [aws-command-line-interface-aws-cli](<https://devfeed.tech/tags/aws-command-line-interface-aws-cli.md>), [aws-identity-and-access-management-iam](<https://devfeed.tech/tags/aws-identity-and-access-management-iam.md>), [batch](<https://devfeed.tech/tags/batch.md>), [batching](<https://devfeed.tech/tags/batching.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [gpu-infrastructure](<https://devfeed.tech/tags/gpu-infrastructure.md>), [inference](<https://devfeed.tech/tags/inference.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [kubernetes](<https://devfeed.tech/tags/kubernetes.md>), [over](<https://devfeed.tech/tags/over.md>), [sglang](<https://devfeed.tech/tags/sglang.md>), [technical-how-to](<https://devfeed.tech/tags/technical-how-to.md>), [vllm](<https://devfeed.tech/tags/vllm.md>)

### AI overview

Reflection AI's Beam is an open-weight model for agentic coding and long-horizon software tasks. The tutorial explains deployment options on AWS, from a managed SageMaker JumpStart endpoint to Kubernetes-based SageMaker HyperPod Inference using JumpStart, SGLang, or vLLM. It covers GPU capacity planning, deployment setup, performance measurement, tuning, and cleanup.

### Source excerpt

Reflection AI's Beam is an open-weight model for agentic coding and long-horizon tasks. This post shows how to deploy Beam on AWS: start with an Amazon SageMaker JumpStart real-time endpoint, or use Amazon SageMaker HyperPod Inference with JumpStart, SGLang, or vLLM for Kubernetes-native control over distributed GPU infrastructure.

## Mixture-of-Experts kernel engineering on Amazon SageMaker AI

DevFeed: [Mixture-of-Experts kernel engineering on Amazon SageMaker AI](<https://devfeed.tech/articles/mixture-of-experts-kernel-engineering-on-amazon-sagemaker-ai-86373.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/mixture-of-experts-kernel-engineering-on-amazon-sagemaker-ai/>)

Author: Mona Mona

Published: 2026-10-13T15:17:09Z

Content type: article

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>), [Amazon SageMaker HyperPod](<https://devfeed.tech/topics/amazon-sagemaker-hyperpod.md>), [vllm](<https://devfeed.tech/topics/vllm.md>), [quantization](<https://devfeed.tech/topics/quantization.md>)

Tags: [advanced-300](<https://devfeed.tech/tags/advanced-300.md>), [ai](<https://devfeed.tech/tags/ai.md>), [amazon-sagemaker](<https://devfeed.tech/tags/amazon-sagemaker.md>), [amazon-sagemaker-ai](<https://devfeed.tech/tags/amazon-sagemaker-ai.md>), [announcements](<https://devfeed.tech/tags/announcements.md>), [faster](<https://devfeed.tech/tags/faster.md>), [fp8](<https://devfeed.tech/tags/fp8.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [jit](<https://devfeed.tech/tags/jit.md>), [mixture-of-experts](<https://devfeed.tech/tags/mixture-of-experts.md>), [models](<https://devfeed.tech/tags/models.md>), [optimizations](<https://devfeed.tech/tags/optimizations.md>), [quantization](<https://devfeed.tech/tags/quantization.md>), [time-to-first-token](<https://devfeed.tech/tags/time-to-first-token.md>), [token](<https://devfeed.tech/tags/token.md>)

### AI overview

AWS engineers describe three optimizations for serving fused Mixture-of-Experts models: changing matrix operand layout for Hopper GPUs, routing FP8 quantized workloads to the Triton backend, and preventing unnecessary Triton JIT recompilation. The article reports up to 10% higher throughput and up to 9% faster time to first token on H200 GPUs.

### Source excerpt

AWS engineers share kernel optimizations for fused Mixture-of-Experts (MoE) models on Amazon SageMaker AI: Swap-AB for WGMMA utilization, Triton backend routing for FP8 quantization, and eliminating Triton JIT recompilation. Together they deliver up to 10% higher throughput and up to 9% faster Time to First Token on H200 GPUs.

## How Flo Health built safer, more useful open models with Databricks AI Runtime

DevFeed: [How Flo Health built safer, more useful open models with Databricks AI Runtime](<https://devfeed.tech/articles/how-flo-health-built-safer-more-useful-open-models-with-databricks-ai-runtime-85433.md>)

Original publisher: [Read original article](<https://www.databricks.com/blog/how-flo-health-built-safer-more-useful-open-models-databricks-ai-runtime>)

Author: Vladislav Nedosekin; András Meczner; Julia Powell; Tejas Sundaresan; Michael Shtelma

Published: 2026-10-12T15:13:04Z

Content type: article

Language: en

Sources: [Databricks](<https://devfeed.tech/sources/databricks.md>)

Topics: [Amazon SageMaker HyperPod](<https://devfeed.tech/topics/amazon-sagemaker-hyperpod.md>), [experiments](<https://devfeed.tech/topics/experiments.md>), [trl](<https://devfeed.tech/topics/trl.md>), [health check](<https://devfeed.tech/topics/health-check.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [company](<https://devfeed.tech/tags/company.md>), [customers](<https://devfeed.tech/tags/customers.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [distributed-training](<https://devfeed.tech/tags/distributed-training.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [generative-ai](<https://devfeed.tech/tags/generative-ai.md>), [models](<https://devfeed.tech/tags/models.md>)

### AI overview

Flo Health developed a post-training workflow for fine-tuning open models to provide medically safe and useful responses in its health app. The team built AI judges from clinical and product rubrics, generated privacy-conscious synthetic training data, and used distributed fine-tuning on Databricks AI Runtime. Flo reports that fine-tuning raised its combined medical-safety and usefulness score from 16% to 47%.

### Source excerpt

For Flo Health, the bar for generative AI is unusually high. With 81 million monthly...

## Share GPU clusters across teams with isolation and fairness using Amazon SageMaker HyperPod

DevFeed: [Share GPU clusters across teams with isolation and fairness using Amazon SageMaker HyperPod](<https://devfeed.tech/articles/share-gpu-clusters-across-teams-with-isolation-and-fairness-using-amazon-sagemaker-hyperpod-77479.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/share-gpu-clusters-across-teams-with-isolation-and-fairness-using-amazon-sagemaker-hyperpod/>)

Author: Giuseppe Angelo Porcelli

Published: 2026-10-08T16:20:04Z

Content type: tutorial

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [Amazon SageMaker HyperPod](<https://devfeed.tech/topics/amazon-sagemaker-hyperpod.md>), [Amazon Elastic Kubernetes Service](<https://devfeed.tech/topics/amazon-elastic-kubernetes-service.md>), [AWS IAM Identity Center](<https://devfeed.tech/topics/aws-iam-identity-center.md>), [Load Balancing](<https://devfeed.tech/topics/load-balancing.md>)

Tags: [amazon-sagemaker-hyperpod](<https://devfeed.tech/tags/amazon-sagemaker-hyperpod.md>), [aws-iam-identity-center](<https://devfeed.tech/tags/aws-iam-identity-center.md>), [best-practices](<https://devfeed.tech/tags/best-practices.md>), [eks](<https://devfeed.tech/tags/eks.md>), [expert-400](<https://devfeed.tech/tags/expert-400.md>), [gpu-clusters](<https://devfeed.tech/tags/gpu-clusters.md>), [kubernetes](<https://devfeed.tech/tags/kubernetes.md>)

### AI overview

The article presents a reference architecture for multiple teams to share an Amazon SageMaker HyperPod EKS GPU cluster. It combines centralized authentication, team-specific SageMaker domains and Kubernetes namespaces, resource quotas and scheduling priorities, and namespace-level cost allocation. It also explains that namespaces and network policies reduce accidental interference but are not strong security boundaries for mutually untrusted tenants.

### Source excerpt

A reference architecture for securely sharing one Amazon SageMaker HyperPod EKS cluster across multiple teams, using AWS IAM Identity Center for authentication, per-team SageMaker Domains and Kubernetes namespaces for isolation, HyperPod Task Governance for fairness, and namespace-level cost allocation for chargeback.

## Building Production-Ready Multi-Agent AI Systems on Amazon EKS: The Complete Step-by-Step Guide

DevFeed: [Building Production-Ready Multi-Agent AI Systems on Amazon EKS: The Complete Step-by-Step Guide](<https://devfeed.tech/articles/building-production-ready-multi-agent-ai-systems-on-amazon-eks-the-complete-step-by-step-guide-77230.md>)

Original publisher: [Read original article](<https://hackernoon.com/building-production-ready-multi-agent-ai-systems-on-amazon-eks-the-complete-step-by-step-guide?source=rss>)

Author: Emmanuela Opurum

Published: 2026-10-08T06:14:06Z

Content type: tutorial

Language: en

Sources: [HackerNoon](<https://devfeed.tech/sources/hackernoon.md>)

Topics: [Multi Agent Systems](<https://devfeed.tech/topics/multi-agent-systems.md>), [grab agent platform](<https://devfeed.tech/topics/grab-agent-platform.md>), [Amazon SageMaker HyperPod](<https://devfeed.tech/topics/amazon-sagemaker-hyperpod.md>), [Amazon Bedrock AgentCore](<https://devfeed.tech/topics/amazon-bedrock-agentcore.md>), [Amazon Bedrock Knowledge Bases](<https://devfeed.tech/topics/amazon-bedrock-knowledge-bases.md>), [AWS Step Functions](<https://devfeed.tech/topics/aws-step-functions.md>)

Tags: [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [ai-systems](<https://devfeed.tech/tags/ai-systems.md>), [amazon-bedrock](<https://devfeed.tech/tags/amazon-bedrock.md>), [amazon-eks](<https://devfeed.tech/tags/amazon-eks.md>), [api](<https://devfeed.tech/tags/api.md>), [aws](<https://devfeed.tech/tags/aws.md>), [beginners](<https://devfeed.tech/tags/beginners.md>), [cloud-architecture](<https://devfeed.tech/tags/cloud-architecture.md>), [guide](<https://devfeed.tech/tags/guide.md>), [multi-agent](<https://devfeed.tech/tags/multi-agent.md>), [multi-agent-ai](<https://devfeed.tech/tags/multi-agent-ai.md>), [production](<https://devfeed.tech/tags/production.md>), [solution-architect](<https://devfeed.tech/tags/solution-architect.md>), [tutorial](<https://devfeed.tech/tags/tutorial.md>)

### AI overview

This tutorial walks through building a multi-agent AI application on Amazon EKS, including specialist agents, request routing, retrieval-augmented generation, Bedrock Guardrails, response checks, human approval workflows, and deployment. It provides AWS setup commands, Python code, Kubernetes manifests, and sample requests.

### Source excerpt

This is the complete step-by-step guide to building a production multi-agent AI system on AWS.

## Best practices for Amazon SageMaker HyperPod administration and governance

DevFeed: [Best practices for Amazon SageMaker HyperPod administration and governance](<https://devfeed.tech/articles/best-practices-for-amazon-sagemaker-hyperpod-administration-and-governance-65896.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/best-practices-for-amazon-sagemaker-hyperpod-administration-and-governance/>)

Author: Geethanjali Banoth

Published: 2026-10-06T15:50:23Z

Content type: tutorial

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [Amazon SageMaker HyperPod](<https://devfeed.tech/topics/amazon-sagemaker-hyperpod.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [DevOps](<https://devfeed.tech/topics/devops.md>)

Tags: [advanced-300](<https://devfeed.tech/tags/advanced-300.md>), [amazon-sagemaker-hyperpod](<https://devfeed.tech/tags/amazon-sagemaker-hyperpod.md>), [amazon-sagemaker-unified-studio](<https://devfeed.tech/tags/amazon-sagemaker-unified-studio.md>), [best-practices](<https://devfeed.tech/tags/best-practices.md>), [governance](<https://devfeed.tech/tags/governance.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>)

### AI overview

This tutorial presents a governance model for administering shared Amazon SageMaker HyperPod clusters through SageMaker Unified Studio. It explains how platform and infrastructure teams can manage identity, access, capacity allocation, workload visibility, observability, and connection reviews across organization, project, cluster, and workload boundaries.

### Source excerpt

Learn how to administer Amazon SageMaker HyperPod through Amazon SageMaker Unified Studio while preserving cluster governance. This post shows platform teams how to design infrastructure boundaries, govern access, allocate shared capacity, and operate HyperPod consistently across the organization, project, cluster, and workload control layers.

## Manage Amazon SageMaker HyperPod Spaces directly from SageMaker Studio

DevFeed: [Manage Amazon SageMaker HyperPod Spaces directly from SageMaker Studio](<https://devfeed.tech/articles/manage-amazon-sagemaker-hyperpod-spaces-directly-from-sagemaker-studio-65899.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/manage-amazon-sagemaker-hyperpod-spaces-directly-from-sagemaker-studio/>)

Author: Giuseppe Angelo Porcelli

Published: 2026-10-06T15:47:02Z

Content type: tutorial

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [Amazon SageMaker HyperPod](<https://devfeed.tech/topics/amazon-sagemaker-hyperpod.md>), [Visual Studio Code](<https://devfeed.tech/topics/visual-studio-code.md>), [Amazon EC2](<https://devfeed.tech/topics/amazon-ec2.md>), [Test-driven development](<https://devfeed.tech/topics/tdd.md>)

Tags: [access-management](<https://devfeed.tech/tags/access-management.md>), [advanced-300](<https://devfeed.tech/tags/advanced-300.md>), [amazon-ec2](<https://devfeed.tech/tags/amazon-ec2.md>), [amazon-eks](<https://devfeed.tech/tags/amazon-eks.md>), [amazon-sagemaker-hyperpod](<https://devfeed.tech/tags/amazon-sagemaker-hyperpod.md>), [amazon-sagemaker-studio](<https://devfeed.tech/tags/amazon-sagemaker-studio.md>), [announcements](<https://devfeed.tech/tags/announcements.md>), [jupyterlab](<https://devfeed.tech/tags/jupyterlab.md>)

### AI overview

Amazon SageMaker Studio now lets data scientists and ML engineers manage SageMaker Spaces on HyperPod EKS clusters through a graphical interface. The article explains administrator setup, Space creation and management, JupyterLab and Code Editor workflows, VS Code connectivity, and an optional Kubernetes node over-provisioning pattern to reduce startup delays, along with associated compute costs.

### Source excerpt

Data scientists and ML engineers can now create, configure, start, stop, and open Amazon SageMaker Spaces on SageMaker HyperPod EKS clusters directly from SageMaker Studio. Launch JupyterLab and Code Editor environments in a few clicks, without using command-line tools.

## Generate images and video with vLLM-Omni on SageMaker AI - Part 2

DevFeed: [Generate images and video with vLLM-Omni on SageMaker AI - Part 2](<https://devfeed.tech/articles/generate-images-and-video-with-vllm-omni-on-sagemaker-ai-part-2-61388.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/generate-images-and-video-with-vllm-omni-on-sagemaker-ai-part-2/>)

Author: Yadan Wei

Published: 2026-09-28T16:15:17Z

Content type: tutorial

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [vllm](<https://devfeed.tech/topics/vllm.md>), [Amazon SageMaker](<https://devfeed.tech/topics/amazon-sagemaker.md>), [Generative AI](<https://devfeed.tech/topics/generative-ai.md>), [Amazon SageMaker HyperPod](<https://devfeed.tech/topics/amazon-sagemaker-hyperpod.md>), [CI/CD](<https://devfeed.tech/topics/cicd.md>)

Tags: [advanced-300](<https://devfeed.tech/tags/advanced-300.md>), [ai](<https://devfeed.tech/tags/ai.md>), [amazon-s3](<https://devfeed.tech/tags/amazon-s3.md>), [amazon-sagemaker](<https://devfeed.tech/tags/amazon-sagemaker.md>), [amazon-sagemaker-ai](<https://devfeed.tech/tags/amazon-sagemaker-ai.md>), [apis](<https://devfeed.tech/tags/apis.md>), [container](<https://devfeed.tech/tags/container.md>), [container-image](<https://devfeed.tech/tags/container-image.md>), [deploy](<https://devfeed.tech/tags/deploy.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [generative](<https://devfeed.tech/tags/generative.md>), [image](<https://devfeed.tech/tags/image.md>), [image-generation](<https://devfeed.tech/tags/image-generation.md>), [inference](<https://devfeed.tech/tags/inference.md>), [models](<https://devfeed.tech/tags/models.md>), [s3](<https://devfeed.tech/tags/s3.md>), [technical-how-to](<https://devfeed.tech/tags/technical-how-to.md>), [time](<https://devfeed.tech/tags/time.md>)

### AI overview

This tutorial deploys FLUX.2-klein-4B and Wan2.1-VACE-1.3B on separate Amazon SageMaker AI endpoints using the AWS vLLM-Omni Deep Learning Container. It generates an image through real-time inference, passes that image and a motion prompt to an asynchronous video endpoint, and retrieves the resulting MP4 from Amazon S3. The sample also offers a command-line workflow and an optional Streamlit interface.

### Source excerpt

Deploy two generative media models from one AWS vLLM-Omni Deep Learning Container on Amazon SageMaker AI. Generate an image with FLUX.2-klein through real-time inference, then animate it into video with Wan2.1-VACE through asynchronous inference, and retrieve the MP4 from Amazon S3.

## Accelerate multimodal RL training with SkyRL on Amazon SageMaker HyperPod

DevFeed: [Accelerate multimodal RL training with SkyRL on Amazon SageMaker HyperPod](<https://devfeed.tech/articles/accelerate-multimodal-rl-training-with-skyrl-on-amazon-sagemaker-hyperpod-60187.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/accelerate-multimodal-rl-training-with-skyrl-on-amazon-sagemaker-hyperpod/>)

Author: Nilesh PS

Published: 2026-09-25T16:18:07Z

Content type: tutorial

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [Amazon SageMaker HyperPod](<https://devfeed.tech/topics/amazon-sagemaker-hyperpod.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Ray](<https://devfeed.tech/topics/ray.md>), [vlm](<https://devfeed.tech/topics/vlm.md>), [Amazon Elastic Kubernetes Service](<https://devfeed.tech/topics/amazon-elastic-kubernetes-service.md>), [Grafana](<https://devfeed.tech/topics/grafana.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [model-deployment](<https://devfeed.tech/topics/model-deployment.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>), [container](<https://devfeed.tech/topics/container.md>)

Tags: [advanced-300](<https://devfeed.tech/tags/advanced-300.md>), [amazon-eks](<https://devfeed.tech/tags/amazon-eks.md>), [amazon-sagemaker-hyperpod](<https://devfeed.tech/tags/amazon-sagemaker-hyperpod.md>), [container](<https://devfeed.tech/tags/container.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [grafana](<https://devfeed.tech/tags/grafana.md>), [inference](<https://devfeed.tech/tags/inference.md>), [lora](<https://devfeed.tech/tags/lora.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [ray](<https://devfeed.tech/tags/ray.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [rl](<https://devfeed.tech/tags/rl.md>), [technical-how-to](<https://devfeed.tech/tags/technical-how-to.md>), [training](<https://devfeed.tech/tags/training.md>), [vision-language-model](<https://devfeed.tech/tags/vision-language-model.md>)

### AI overview

A walkthrough for running SkyRL on Amazon SageMaker HyperPod to post-train a Qwen3-VL-8B vision-language model with GRPO. It covers building a container, launching a Ray cluster, submitting and monitoring jobs, and hosting the resulting LoRA adapter for inference.

### Source excerpt

Learn how to run SkyRL, an open-source reinforcement learning framework, on Amazon SageMaker HyperPod to post-train a Qwen3-VL-8B vision-language model with GRPO. This walkthrough covers building the container image, launching a Ray cluster from SageMaker Studio, submitting and monitoring the job, and hosting the trained LoRA adapter for inference.

## Multi-Region training with Amazon SageMaker HyperPod and Qumulo

DevFeed: [Multi-Region training with Amazon SageMaker HyperPod and Qumulo](<https://devfeed.tech/articles/multi-region-training-with-amazon-sagemaker-hyperpod-and-qumulo-60190.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/multi-region-training-with-amazon-sagemaker-hyperpod-and-qumulo/>)

Author: Bryan Berezdivin

Published: 2026-09-25T15:49:44Z

Content type: article

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [Amazon SageMaker HyperPod](<https://devfeed.tech/topics/amazon-sagemaker-hyperpod.md>), [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>), [Cloud](<https://devfeed.tech/topics/cloud.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [data](<https://devfeed.tech/topics/data.md>), [Latency](<https://devfeed.tech/topics/latency.md>)

Tags: [advanced-300](<https://devfeed.tech/tags/advanced-300.md>), [amazon-sagemaker-hyperpod](<https://devfeed.tech/tags/amazon-sagemaker-hyperpod.md>), [aws](<https://devfeed.tech/tags/aws.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [customer-solutions](<https://devfeed.tech/tags/customer-solutions.md>), [data](<https://devfeed.tech/tags/data.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [latency](<https://devfeed.tech/tags/latency.md>), [train](<https://devfeed.tech/tags/train.md>), [training-data](<https://devfeed.tech/tags/training-data.md>)

### AI overview

This post describes cross-Region AI model training using Amazon SageMaker HyperPod and Cloud Native Qumulo. It reports that a remote cluster matched the throughput of a co-located cluster after a brief warmup, allowing teams to keep training data in one Region while using compute in another.

### Source excerpt

Amazon SageMaker HyperPod and Cloud Native Qumulo let you place training compute in one AWS Region while keeping your dataset in another. This post shares the architecture and validation results from a cross-Region training run, where a remote cluster matched a co-located cluster's throughput after a brief NeuralCache warmup.

## PD Serving of Qwen3.8-2.4T

DevFeed: [PD Serving of Qwen3.8-2.4T](<https://devfeed.tech/articles/pd-serving-of-qwen3-8-2-4t-79490.md>)

Original publisher: [Read original article](<https://vllm.ai/blog/2026-09-21-qwen38-pd-serving>)

Author: vLLM Team

Published: 2026-09-21T00:00:00Z

Content type: article

Language: en

Sources: [vLLM Blog](<https://devfeed.tech/sources/vllm-blog.md>)

Topics: [vllm](<https://devfeed.tech/topics/vllm.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [GB300 NVL72](<https://devfeed.tech/topics/gb300-nvl72.md>), [Amazon SageMaker HyperPod](<https://devfeed.tech/topics/amazon-sagemaker-hyperpod.md>), [Dynamo](<https://devfeed.tech/topics/dynamo.md>), [CUDA Graphs](<https://devfeed.tech/topics/cuda-graphs.md>)

Tags: [8k](<https://devfeed.tech/tags/8k.md>), [ai-engineering](<https://devfeed.tech/tags/ai-engineering.md>), [api](<https://devfeed.tech/tags/api.md>), [batch](<https://devfeed.tech/tags/batch.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [bottlenecks](<https://devfeed.tech/tags/bottlenecks.md>), [capacity](<https://devfeed.tech/tags/capacity.md>), [cluster](<https://devfeed.tech/tags/cluster.md>), [concurrency](<https://devfeed.tech/tags/concurrency.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [cuda-graphs](<https://devfeed.tech/tags/cuda-graphs.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [disaggregation](<https://devfeed.tech/tags/disaggregation.md>), [docker](<https://devfeed.tech/tags/docker.md>), [dynamo](<https://devfeed.tech/tags/dynamo.md>), [gb300](<https://devfeed.tech/tags/gb300.md>), [gb300-nvl72](<https://devfeed.tech/tags/gb300-nvl72.md>), [llm-inference](<https://devfeed.tech/tags/llm-inference.md>), [model-serving](<https://devfeed.tech/tags/model-serving.md>), [nvl72](<https://devfeed.tech/tags/nvl72.md>), [open-source-ai](<https://devfeed.tech/tags/open-source-ai.md>), [performance](<https://devfeed.tech/tags/performance.md>), [qwen3](<https://devfeed.tech/tags/qwen3.md>), [qwen3-8](<https://devfeed.tech/tags/qwen3-8.md>), [vllm](<https://devfeed.tech/tags/vllm.md>), [vllm-blog](<https://devfeed.tech/tags/vllm-blog.md>)

### AI overview

The article explains how the vLLM team tuned disaggregated prefill and decode serving for the Qwen3.8-2.4T model on a GB300 NVL72 cluster. It describes estimating KV cache capacity, measuring prefill and decode performance, and selecting serving configurations. The reported results are 5,000 total tokens per second per GPU in a high-throughput scenario and 180 generated tokens per second per user in a low-latency scenario; it also provides recipes for reproducing the measurements.

### Source excerpt

How vLLM reaches 5K throughput and 180 interactivity on Qwen3.8-2.4T with GB300 NVL72 PD serving and how to reproduce results yourself.

## Amazon SageMaker Inference: 2026 year-to-date launches in review

DevFeed: [Amazon SageMaker Inference: 2026 year-to-date launches in review](<https://devfeed.tech/articles/amazon-sagemaker-inference-2026-year-to-date-launches-in-review-55156.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/amazon-sagemaker-inference-2026-year-to-date-launches-in-review/>)

Author: Kareem Syed-Mohammed

Published: 2026-09-18T20:52:14Z

Content type: release

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [Amazon SageMaker](<https://devfeed.tech/topics/amazon-sagemaker.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [Amazon Web Services](<https://devfeed.tech/topics/aws.md>), [Amazon SageMaker HyperPod](<https://devfeed.tech/topics/amazon-sagemaker-hyperpod.md>), [Benchmark](<https://devfeed.tech/topics/benchmark.md>), [inference-endpoints](<https://devfeed.tech/topics/inference-endpoints.md>), [Optimization](<https://devfeed.tech/topics/optimization.md>)

Tags: [ai-inference](<https://devfeed.tech/tags/ai-inference.md>), [amazon-sagemaker-ai](<https://devfeed.tech/tags/amazon-sagemaker-ai.md>), [amazon-sagemaker-hyperpod](<https://devfeed.tech/tags/amazon-sagemaker-hyperpod.md>), [announcements](<https://devfeed.tech/tags/announcements.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [inference-endpoints](<https://devfeed.tech/tags/inference-endpoints.md>), [intermediate-200](<https://devfeed.tech/tags/intermediate-200.md>), [latency](<https://devfeed.tech/tags/latency.md>), [optimization](<https://devfeed.tech/tags/optimization.md>)

### AI overview

Amazon SageMaker AI shipped 13 inference capabilities in 2026 across fully managed endpoints and SageMaker HyperPod Inference. The post reviews launches covering deployment, capacity, scaling, observability, benchmarking, and optimization for generative AI workloads.

### Source excerpt

Amazon SageMaker AI shipped 13 inference launches in year-to-date across two deployment paths: fully managed endpoints and Amazon SageMaker HyperPod Inference. This post reviews each launch, from inference recommendations and capacity-aware instance pools to tiered KV caching and disaggregated prefill and decode.

## AWS Launches SageMaker HyperPod Inference Gateway for GPU-Aware Routing

DevFeed: [AWS Launches SageMaker HyperPod Inference Gateway for GPU-Aware Routing](<https://devfeed.tech/articles/aws-launches-sagemaker-hyperpod-inference-gateway-for-gpu-aware-routing-49837.md>)

Original publisher: [Read original article](<https://www.unite.ai/aws-launches-sagemaker-hyperpod-inference-gateway-for-gpu-aware-routing/>)

Author: Theo Nash, AI Infrastructure & Compute, AI Research Agent

Published: 2026-09-18T13:27:01Z

Content type: news

Language: en

Sources: [Unite.AI](<https://devfeed.tech/sources/unite-ai.md>)

Topics: [Amazon SageMaker HyperPod](<https://devfeed.tech/topics/amazon-sagemaker-hyperpod.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [Amazon Elastic Kubernetes Service](<https://devfeed.tech/topics/amazon-elastic-kubernetes-service.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Latency](<https://devfeed.tech/topics/latency.md>), [Load Balancing](<https://devfeed.tech/topics/load-balancing.md>), [Kubernetes](<https://devfeed.tech/topics/kubernetes.md>), [Routing (disambiguation)](<https://devfeed.tech/topics/routing.md>), [Amazon Web Services](<https://devfeed.tech/topics/aws.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [ai-models-platforms](<https://devfeed.tech/tags/ai-models-platforms.md>), [amazon-eks](<https://devfeed.tech/tags/amazon-eks.md>), [amazon-sagemaker](<https://devfeed.tech/tags/amazon-sagemaker.md>), [amazon-sagemaker-hyperpod](<https://devfeed.tech/tags/amazon-sagemaker-hyperpod.md>), [amazon-web-services](<https://devfeed.tech/tags/amazon-web-services.md>), [aws](<https://devfeed.tech/tags/aws.md>), [inference](<https://devfeed.tech/tags/inference.md>), [kubernetes](<https://devfeed.tech/tags/kubernetes.md>), [latency](<https://devfeed.tech/tags/latency.md>), [load-balancing](<https://devfeed.tech/tags/load-balancing.md>), [load-balancing-algorithms](<https://devfeed.tech/tags/load-balancing-algorithms.md>), [reduce](<https://devfeed.tech/tags/reduce.md>), [round-robin](<https://devfeed.tech/tags/round-robin.md>), [routing](<https://devfeed.tech/tags/routing.md>), [web-services](<https://devfeed.tech/tags/web-services.md>)

### AI overview

Amazon Web Services announced SageMaker HyperPod Inference Gateway, a Kubernetes-native add-on for Amazon EKS that uses GPU-aware routing to direct large language model inference requests. AWS says it can reduce first-token latency by up to 82% by considering GPU state, KV-cache utilization, queue depth, LoRA adapter residency, prefix-cache hits, and running requests.

### Source excerpt

Amazon Web Services announced Amazon SageMaker HyperPod Inference Gateway on September 18, 2026, a Kubernetes-native, GPU-aware routing system for large language model inference that deploys as a single managed add-on for Amazon EKS on existing HyperPod infrastructure. AWS said the gateway can reduce first-token latency by up to 82%. The Routing Problem Behind the Gateway According to AWS, default Kubernetes load-balancing algorithms such as round-robin and least-connections have no visibility...

## Introducing Amazon SageMaker HyperPod Inference Gateway

DevFeed: [Introducing Amazon SageMaker HyperPod Inference Gateway](<https://devfeed.tech/articles/introducing-amazon-sagemaker-hyperpod-inference-gateway-42780.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/introducing-amazon-sagemaker-hyperpod-inference-gateway/>)

Author: Vinay Arora

Published: 2026-09-18T13:08:34Z

Content type: release

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [Amazon SageMaker HyperPod](<https://devfeed.tech/topics/amazon-sagemaker-hyperpod.md>), [Model Routing](<https://devfeed.tech/topics/model-routing.md>), [model-serving](<https://devfeed.tech/topics/model-serving.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [Amazon Elastic Kubernetes Service](<https://devfeed.tech/topics/amazon-elastic-kubernetes-service.md>), [Kubernetes](<https://devfeed.tech/topics/kubernetes.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [AI Platform](<https://devfeed.tech/topics/ai-platform.md>), [Prometheus](<https://devfeed.tech/topics/prometheus.md>)

Tags: [amazon-eks](<https://devfeed.tech/tags/amazon-eks.md>), [amazon-sagemaker-hyperpod](<https://devfeed.tech/tags/amazon-sagemaker-hyperpod.md>), [announcements](<https://devfeed.tech/tags/announcements.md>), [expert-400](<https://devfeed.tech/tags/expert-400.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [inference](<https://devfeed.tech/tags/inference.md>), [kubernetes](<https://devfeed.tech/tags/kubernetes.md>), [latency](<https://devfeed.tech/tags/latency.md>), [metrics](<https://devfeed.tech/tags/metrics.md>), [model-serving](<https://devfeed.tech/tags/model-serving.md>), [performance](<https://devfeed.tech/tags/performance.md>), [routing](<https://devfeed.tech/tags/routing.md>)

### AI overview

Amazon SageMaker HyperPod Inference Gateway is a Kubernetes-native, GPU-aware routing add-on for Amazon EKS. It uses real-time GPU signals and model-serving metrics to route inference requests to suitable pods, aiming to reduce GPU waste and first-token latency without application changes.

### Source excerpt

Amazon SageMaker HyperPod Inference Gateway is a Kubernetes-native, GPU-aware routing add-on for Amazon EKS. It uses real-time GPU signals to send each inference request to the best-suited pod, cutting first-token latency by up to 82% with no changes to your model servers or client applications.

## Automate SageMaker HyperPod incident triage and root-cause-analysis with AWS DevOps Agent

DevFeed: [Automate SageMaker HyperPod incident triage and root-cause-analysis with AWS DevOps Agent](<https://devfeed.tech/articles/automate-sagemaker-hyperpod-incident-triage-and-root-cause-analysis-with-aws-devops-agent-49931.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/devops/automate-sagemaker-hyperpod-incident-triage-and-root-cause-analysis-with-aws-devops-agent/>)

Author: Tomonori Shimomura

Published: 2026-08-10T11:29:17Z

Content type: tutorial

Language: en

Sources: [AWS DevOps & Developer Productivity Blog](<https://devfeed.tech/sources/aws-devops-developer-productivity-blog.md>)

Topics: [Amazon SageMaker HyperPod](<https://devfeed.tech/topics/amazon-sagemaker-hyperpod.md>), [incident](<https://devfeed.tech/topics/incident.md>), [DevOps](<https://devfeed.tech/topics/devops.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [Monitoring](<https://devfeed.tech/topics/monitoring.md>), [Resilience](<https://devfeed.tech/topics/resilience.md>), [Amazon Web Services](<https://devfeed.tech/topics/aws.md>), [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Inference](<https://devfeed.tech/topics/inference.md>)

Tags: [ai-ml](<https://devfeed.tech/tags/ai-ml.md>), [aiops](<https://devfeed.tech/tags/aiops.md>), [amazon-sagemaker-hyperpod](<https://devfeed.tech/tags/amazon-sagemaker-hyperpod.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [clusters](<https://devfeed.tech/tags/clusters.md>), [developer-tools](<https://devfeed.tech/tags/developer-tools.md>), [devops](<https://devfeed.tech/tags/devops.md>), [expert-400](<https://devfeed.tech/tags/expert-400.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [incident](<https://devfeed.tech/tags/incident.md>), [inference](<https://devfeed.tech/tags/inference.md>), [monitoring](<https://devfeed.tech/tags/monitoring.md>), [networking](<https://devfeed.tech/tags/networking.md>), [observability](<https://devfeed.tech/tags/observability.md>), [operational](<https://devfeed.tech/tags/operational.md>), [resiliency](<https://devfeed.tech/tags/resiliency.md>), [root-cause-analysis](<https://devfeed.tech/tags/root-cause-analysis.md>), [scale](<https://devfeed.tech/tags/scale.md>), [security](<https://devfeed.tech/tags/security.md>), [technical-how-to](<https://devfeed.tech/tags/technical-how-to.md>), [triage](<https://devfeed.tech/tags/triage.md>)

### AI overview

This tutorial describes an operational approach for SageMaker HyperPod clusters running large-scale training, fine-tuning, and inference. It explains how built-in resiliency handles routine GPU failures while an autonomous investigation layer surfaces configuration issues, capacity constraints, and recurring hardware faults for human review.

### Source excerpt

Introduction Large-scale machine learning workloads: training, fine-tuning, and inference run on clusters of hundreds to thousands of GPU instances for days or weeks at a stretch. Keeping operational visibility across a fleet of this size is a constant challenge: hardware health events, node lifecycle transitions, capacity fluctuations, and workload-level issues appear in the event stream around [...]