# Simplifying Model Serving Across Multiple GPUs with NVIDIA TensorRT Multi-Device Integration in NVIDIA Dynamo-Triton

DevFeed: [Simplifying Model Serving Across Multiple GPUs with NVIDIA TensorRT Multi-Device Integration in NVIDIA Dynamo-Triton](<https://devfeed.tech/articles/simplifying-model-serving-across-multiple-gpus-with-nvidia-tensorrt-multi-device-integration-in-nvidia-dynamo-triton-57601.md>)

Original publisher: [Read original article](<https://developer.nvidia.com/blog/simplifying-model-serving-across-multiple-gpus-with-nvidia-tensorrt-multi-device-integration-in-nvidia-dynamo-triton/>)

Author: Tanya Lenz

Published: 2026-09-21T21:51:06Z

Content type: tutorial

Language: en

Sources: [NVIDIA Developer](<https://devfeed.tech/sources/nvidia-developer.md>), [NVIDIA Technical Blog](<https://devfeed.tech/sources/nvidia-technical-blog.md>)

Topics: [TensorRT](<https://devfeed.tech/topics/tensorrt.md>), [Dynamo-Triton](<https://devfeed.tech/topics/dynamo-triton.md>), [model-serving](<https://devfeed.tech/topics/model-serving.md>), [Multi-GPU](<https://devfeed.tech/topics/multi-gpu.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>), [CUDA](<https://devfeed.tech/topics/cuda.md>), [gRPC](<https://devfeed.tech/topics/grpc.md>), [Parallelism](<https://devfeed.tech/topics/parallelism.md>), [Latency](<https://devfeed.tech/topics/latency.md>), [Deployment](<https://devfeed.tech/topics/deployment.md>)

Tags: [agentic-ai-generative-ai](<https://devfeed.tech/tags/agentic-ai-generative-ai.md>), [ai-inference](<https://devfeed.tech/tags/ai-inference.md>), [backend](<https://devfeed.tech/tags/backend.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [developer-tools-techniques](<https://devfeed.tech/tags/developer-tools-techniques.md>), [dynamo-triton](<https://devfeed.tech/tags/dynamo-triton.md>), [featured](<https://devfeed.tech/tags/featured.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [grpc](<https://devfeed.tech/tags/grpc.md>), [inference](<https://devfeed.tech/tags/inference.md>), [inference-performance](<https://devfeed.tech/tags/inference-performance.md>), [latency](<https://devfeed.tech/tags/latency.md>), [model-serving](<https://devfeed.tech/tags/model-serving.md>), [multi-gpu](<https://devfeed.tech/tags/multi-gpu.md>), [nccl](<https://devfeed.tech/tags/nccl.md>), [networking-communications](<https://devfeed.tech/tags/networking-communications.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [parallelism](<https://devfeed.tech/tags/parallelism.md>), [scaling](<https://devfeed.tech/tags/scaling.md>), [tensorrt](<https://devfeed.tech/tags/tensorrt.md>)

## AI overview

This tutorial explains how TensorRT multi-device inference runs a single network across multiple GPUs using NCCL-backed collectives, while Dynamo-Triton exposes one gRPC model endpoint. It demonstrates the integration with NVIDIA Cosmos 3 Nano video generation and describes distributing the denoising transformer across up to eight GPUs.

## Source excerpt

The compute and memory demands of generative AI increasingly exceed what a single GPU can provide. NVIDIA TensorRT multi-device inference is a new capability...