# How DigitalOcean's Agentic Inference Cloud powered by NVIDIA GPUs Achieved 67% Lower Inference Costs for Workato

DevFeed: [How DigitalOcean's Agentic Inference Cloud powered by NVIDIA GPUs Achieved 67% Lower Inference Costs for Workato](<https://devfeed.tech/articles/how-digitalocean-s-agentic-inference-cloud-powered-by-nvidia-gpus-achieved-67-lower-inference-costs-for-workato-19953.md>)

Original publisher: [Read original article](<https://www.digitalocean.com/blog/workato-nvidia-technical-deep-dive-agentic-inference-cloud>)

Author: Tim Kim

Published: 2026-03-03T04:55:00Z

Content type: article

Language: en

Sources: [DigitalOcean](<https://devfeed.tech/sources/digitalocean.md>)

Topics: [Digital Ocean](<https://devfeed.tech/topics/digital-ocean.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [Inference Performance](<https://devfeed.tech/topics/inference-performance.md>), [model-deployment](<https://devfeed.tech/topics/model-deployment.md>), [vllm](<https://devfeed.tech/topics/vllm.md>), [Nvidia](<https://devfeed.tech/topics/nvidia.md>), [Kubernetes](<https://devfeed.tech/topics/kubernetes.md>)

Tags: [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [compute](<https://devfeed.tech/tags/compute.md>), [deep-dive](<https://devfeed.tech/tags/deep-dive.md>), [digitalocean](<https://devfeed.tech/tags/digitalocean.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [inference](<https://devfeed.tech/tags/inference.md>), [kubernetes](<https://devfeed.tech/tags/kubernetes.md>), [latency](<https://devfeed.tech/tags/latency.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [performance](<https://devfeed.tech/tags/performance.md>), [technical](<https://devfeed.tech/tags/technical.md>), [unit-economics](<https://devfeed.tech/tags/unit-economics.md>), [vllm](<https://devfeed.tech/tags/vllm.md>)

## AI overview

This technical deep dive describes how DigitalOcean and Workato's AI Research Lab tuned an agentic inference deployment using NVIDIA Dynamo with vLLM on DigitalOcean Kubernetes Service. The article reports higher throughput, lower latency, and reduced hardware and model costs in the tested configurations, including 67% lower inference costs.

## Source excerpt

Workato's AI Research Lab is focused on helping customers extend their production automation with agentic AI capabilities, systems that can reason, act, and orchestrate work across the business. At Workato's scale, processing 1 trillion automated workloads, LLM inference efficiency is a hard requirement: every millisecond of latency and every wasted GPU cycle directly impacts cost, throughput, and reliability. To make agentic workloads production-ready, the team needed an inference stack built for production scale - delivering predictable performance and unit economics at scale, not just raw compute. DigitalOcean partnered with Workato's AI Research Lab team to design and tune this deployment on its Agentic Inference Cloud, using NVIDIA Dynamo with vLLM on DigitalOcean Kubernetes Service (DOKS). To support 100K-token context lengths without degrading performance, NVIDIA H200 GPUs were selected for their 141GB HBM3e memory capacity. The memory footprint of the workload was around 125 GB (comprising the model weights, key value cache, and activation buffer), so a single NVIDIA H200 GPU is able to fit the whole footprint. However, the team used 8-way tensor parallelism per node to maximize sustained throughput and latency stability under a concurrent load. DigitalOcean tested across two different configurations for Workato, and afterwards, the results for NVIDIA Dynamo + vLLM on DOKS showed: Best in class queries-per-second across all tested configurations 67% higher throughput per GPU with 79% lower end-to-end latency and 77% time-to-first-token compared to different configurations on identical hardware 33% lower hardware cost using a NVIDIA H200 GPU vs. a NVIDIA A100 GPU for equivalent performance 67% lower model cost while using half the GPUs The key here was to introduce key/value (KV)-aware routing in order to reduce redundancies and capture maximum value across performance and cost for the inference stack. How LLMs Process Requests and Why It Gets Expensive at Sc