# Achieving near-zero-downtime scaling in Elastic Expert Parallelism

DevFeed: [Achieving near-zero-downtime scaling in Elastic Expert Parallelism](<https://devfeed.tech/articles/achieving-near-zero-downtime-scaling-in-elastic-expert-parallelism-90281.md>)

Original publisher: [Read original article](<https://vllm.ai/blog/2026-10-19-elastic-ep-near-zero-downtime>)

Author: Itay Alroy (NVIDIA)

Published: 2026-10-19T00:00:00Z

Content type: article

Language: en

Sources: [vLLM Blog](<https://devfeed.tech/sources/vllm-blog.md>)

Topics: [Deployment & Scaling](<https://devfeed.tech/topics/deployment-scaling.md>), [vllm](<https://devfeed.tech/topics/vllm.md>), [CUDA Graphs](<https://devfeed.tech/topics/cuda-graphs.md>), [Heroku AI](<https://devfeed.tech/topics/heroku-ai.md>)

Tags: [ai-engineering](<https://devfeed.tech/tags/ai-engineering.md>), [asynchronous](<https://devfeed.tech/tags/asynchronous.md>), [autoscalers](<https://devfeed.tech/tags/autoscalers.md>), [batch](<https://devfeed.tech/tags/batch.md>), [compilation](<https://devfeed.tech/tags/compilation.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [cuda-graphs](<https://devfeed.tech/tags/cuda-graphs.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [deployments](<https://devfeed.tech/tags/deployments.md>), [downtime](<https://devfeed.tech/tags/downtime.md>), [elastic-ep](<https://devfeed.tech/tags/elastic-ep.md>), [expert-parallelism](<https://devfeed.tech/tags/expert-parallelism.md>), [large-scale-serving](<https://devfeed.tech/tags/large-scale-serving.md>), [llm-inference](<https://devfeed.tech/tags/llm-inference.md>), [model-serving](<https://devfeed.tech/tags/model-serving.md>), [moe](<https://devfeed.tech/tags/moe.md>), [nixl](<https://devfeed.tech/tags/nixl.md>), [open-source-ai](<https://devfeed.tech/tags/open-source-ai.md>), [scaling](<https://devfeed.tech/tags/scaling.md>), [vllm](<https://devfeed.tech/tags/vllm.md>), [vllm-blog](<https://devfeed.tech/tags/vllm-blog.md>)

## AI overview

The article describes how vLLM uses asynchronous preparation and NIXL EP to scale Elastic Expert Parallelism while keeping existing workers serving requests. In a DeepSeek V4 Pro benchmark, this approach reduced serving interruptions to 0.82 seconds when scaling from 16 to 20 GPUs and 1.96 seconds when scaling back. It explains how NIXL EP reuses communication resources and CUDA graphs, and how joining workers prepare independently. In a load test, the deployment completed all 11,391 requests while remaining below the stated p95 time-to-first-token objective.

## Source excerpt

How asynchronous preparation and NIXL EP enabled near-zero-downtime Elastic EP scaling in vLLM.