# Five Ways NVIDIA Co-Design Maximizes the Value of Every Byte in the AI Factory

DevFeed: [Five Ways NVIDIA Co-Design Maximizes the Value of Every Byte in the AI Factory](<https://devfeed.tech/articles/five-ways-nvidia-co-design-maximizes-the-value-of-every-byte-in-the-ai-factory-119477.md>)

Original publisher: [Read original article](<https://developer.nvidia.com/blog/five-ways-nvidia-co-design-maximizes-the-value-of-every-byte-in-the-ai-factory/>)

Author: Shara Tibken

Published: 2026-11-05T16:00:00Z

Content type: article

Language: en

Sources: [NVIDIA Developer](<https://devfeed.tech/sources/nvidia-developer.md>), [NVIDIA Technical Blog](<https://devfeed.tech/sources/nvidia-technical-blog.md>)

Topics: [AI Factory](<https://devfeed.tech/topics/ai-factory.md>), [AI Architecture](<https://devfeed.tech/topics/ai-architecture.md>), [NVFP4](<https://devfeed.tech/topics/nvfp4.md>), [NVLink](<https://devfeed.tech/topics/nvlink.md>), [Multi-GPU](<https://devfeed.tech/topics/multi-gpu.md>), [vllm](<https://devfeed.tech/topics/vllm.md>)

Tags: [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-ai-generative-ai](<https://devfeed.tech/tags/agentic-ai-generative-ai.md>), [agentic-workloads](<https://devfeed.tech/tags/agentic-workloads.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai-factory](<https://devfeed.tech/tags/ai-factory.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [arithmetic](<https://devfeed.tech/tags/arithmetic.md>), [attention](<https://devfeed.tech/tags/attention.md>), [batch](<https://devfeed.tech/tags/batch.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [better](<https://devfeed.tech/tags/better.md>), [bits](<https://devfeed.tech/tags/bits.md>), [block](<https://devfeed.tech/tags/block.md>), [bluefield-dpu](<https://devfeed.tech/tags/bluefield-dpu.md>), [cache](<https://devfeed.tech/tags/cache.md>), [cached-context](<https://devfeed.tech/tags/cached-context.md>), [capacity](<https://devfeed.tech/tags/capacity.md>), [co-design](<https://devfeed.tech/tags/co-design.md>), [data-center-cloud](<https://devfeed.tech/tags/data-center-cloud.md>), [doca](<https://devfeed.tech/tags/doca.md>), [dynamo](<https://devfeed.tech/tags/dynamo.md>), [ethernet](<https://devfeed.tech/tags/ethernet.md>), [featured](<https://devfeed.tech/tags/featured.md>), [gb200](<https://devfeed.tech/tags/gb200.md>), [gb300](<https://devfeed.tech/tags/gb300.md>), [gb300-nvl72](<https://devfeed.tech/tags/gb300-nvl72.md>), [gpudirect](<https://devfeed.tech/tags/gpudirect.md>), [h100](<https://devfeed.tech/tags/h100.md>), [hbm](<https://devfeed.tech/tags/hbm.md>), [high-bandwidth-memory](<https://devfeed.tech/tags/high-bandwidth-memory.md>), [memory](<https://devfeed.tech/tags/memory.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [nvlink](<https://devfeed.tech/tags/nvlink.md>), [rubin-gpu](<https://devfeed.tech/tags/rubin-gpu.md>), [spectrum-x](<https://devfeed.tech/tags/spectrum-x.md>), [tensorrt-llm](<https://devfeed.tech/tags/tensorrt-llm.md>), [vera-cpu](<https://devfeed.tech/tags/vera-cpu.md>), [vera-rubin](<https://devfeed.tech/tags/vera-rubin.md>), [vera-rubin-nvl72](<https://devfeed.tech/tags/vera-rubin-nvl72.md>)

## AI overview

The excerpt describes five ways NVIDIA's AI factory design aims to increase useful work per memory byte: NVFP4 shrinks model weights and KV cache; TensorRT-LLM Wide-EP spreads MoE experts across GPUs to free per-GPU memory and improve weight reuse; Dynamo routes requests to cached prefixes and manages KV cache across GPU, CPU, and storage; Vera CPU memory provides a place to stage context during tool-call gaps; and CMX offers pod-level shared flash for context reuse across nodes. The article reports benchmark-specific gains, including up to 4x throughput over an FP8 baseline for NVFP4 and up to 30x throughput per megawatt for Vera Rubin NVL72 versus GB300 NVL72 at the same 288 GB per GPU. It also notes that KV offloading helps when retrieval is faster than recomputation; storage performance examples are vendor validation tests.

## Source excerpt

In an AI factory, high bandwidth memory (HBM)is the fastest, most expensive, and most contested memory tier. Every byte spent on a redundant weight copy, an...