# Push-Based KV Transfer to Improve TTFT and TPOT for Disaggregated Inference in vLLM

DevFeed: [Push-Based KV Transfer to Improve TTFT and TPOT for Disaggregated Inference in vLLM](<https://devfeed.tech/articles/push-based-kv-transfer-to-improve-ttft-and-tpot-for-disaggregated-inference-in-vllm-87122.md>)

Original publisher: [Read original article](<https://vllm.ai/blog/2026-10-14-nixl-push-connector>)

Author: Sunita Nadampalli (AWS), Nicolò Lucchesi (Mistral)

Published: 2026-10-14T00:00:00Z

Content type: article

Language: en

Sources: [vLLM Blog](<https://devfeed.tech/sources/vllm-blog.md>)

Topics: [vllm](<https://devfeed.tech/topics/vllm.md>), [rdma](<https://devfeed.tech/topics/rdma.md>), [gpudirect rdma](<https://devfeed.tech/topics/gpudirect-rdma.md>), [AWS Trainium](<https://devfeed.tech/topics/aws-trainium.md>), [AWS Inferentia](<https://devfeed.tech/topics/aws-inferentia.md>), [quantization](<https://devfeed.tech/topics/quantization.md>)

Tags: [ai-engineering](<https://devfeed.tech/tags/ai-engineering.md>), [amazon-ec2](<https://devfeed.tech/tags/amazon-ec2.md>), [aws-ec2](<https://devfeed.tech/tags/aws-ec2.md>), [aws-trainium](<https://devfeed.tech/tags/aws-trainium.md>), [disaggregation](<https://devfeed.tech/tags/disaggregation.md>), [inference](<https://devfeed.tech/tags/inference.md>), [kv-cache](<https://devfeed.tech/tags/kv-cache.md>), [llm-inference](<https://devfeed.tech/tags/llm-inference.md>), [model-serving](<https://devfeed.tech/tags/model-serving.md>), [nixl](<https://devfeed.tech/tags/nixl.md>), [open-source-ai](<https://devfeed.tech/tags/open-source-ai.md>), [performance](<https://devfeed.tech/tags/performance.md>), [rdma](<https://devfeed.tech/tags/rdma.md>), [vllm](<https://devfeed.tech/tags/vllm.md>), [vllm-blog](<https://devfeed.tech/tags/vllm-blog.md>)

## AI overview

The article introduces a push-based NIXL connector for vLLM disaggregated inference. It overlaps KV-cache transfer setup with prefill, moving work off the decode instance's latency-critical path. In a Qwen3-32B benchmark on a one-prefill, one-decode deployment using Amazon EC2 P5en instances with EFA, push mode achieved up to 3x better time to first token and up to 30% better time per output token than pull mode; results vary by workload and configuration.

## Source excerpt

A push-based NIXL connector for vLLM disaggregated serving: the prefill instance writes KV directly into decode memory over RDMA, taking the router relay, decode-side allocation, and transfer setup off the TTFT critical path.