# Deploying Qwen3.8-2.4T-A95B on Amazon SageMaker HyperPod with vLLM

DevFeed: [Deploying Qwen3.8-2.4T-A95B on Amazon SageMaker HyperPod with vLLM](<https://devfeed.tech/articles/deploying-qwen3-8-2-4t-a95b-on-amazon-sagemaker-hyperpod-with-vllm-4731.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/deploying-qwen3-8-2-4t-a95b-on-amazon-sagemaker-hyperpod-with-vllm/>)

Author: Dmitry Soldatkin

Published: 2026-09-09T22:26:29Z

Content type: tutorial

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [Deployment](<https://devfeed.tech/topics/deployment.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Language models](<https://devfeed.tech/topics/language-models.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [long-context](<https://devfeed.tech/topics/long-context.md>), [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>)

Tags: [advanced-300](<https://devfeed.tech/tags/advanced-300.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [amazon-sagemaker](<https://devfeed.tech/tags/amazon-sagemaker.md>), [amazon-sagemaker-hyperpod](<https://devfeed.tech/tags/amazon-sagemaker-hyperpod.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [how-to](<https://devfeed.tech/tags/how-to.md>), [inference](<https://devfeed.tech/tags/inference.md>), [long-context](<https://devfeed.tech/tags/long-context.md>), [moe](<https://devfeed.tech/tags/moe.md>), [nvfp4](<https://devfeed.tech/tags/nvfp4.md>), [quantization](<https://devfeed.tech/tags/quantization.md>), [qwen](<https://devfeed.tech/tags/qwen.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [technical-how-to](<https://devfeed.tech/tags/technical-how-to.md>), [tool](<https://devfeed.tech/tags/tool.md>), [vllm](<https://devfeed.tech/tags/vllm.md>)

## AI overview

A deployment walkthrough for serving the open-weight Qwen3.8-2.4T-A95B language model on Amazon SageMaker HyperPod with vLLM and NVIDIA B300 GPUs. It covers provisioning, NVFP4 quantization, an OpenAI-compatible endpoint, reasoning, tool calling, and MTP speculative decoding.

## Source excerpt

Learn how to deploy Qwen3.8-2.4T-A95B, a 2.4-trillion-parameter open-weight model, on Amazon SageMaker HyperPod with vLLM. This walkthrough covers cluster provisioning, NVFP4 quantization, and an OpenAI-compatible endpoint with built-in reasoning, tool calling, and native MTP speculative decoding.