# DigitalOcean's Inference Optimized Image improves Llama 3.3 70B inference efficiency on GPU Droplets

DevFeed: [DigitalOcean's Inference Optimized Image improves Llama 3.3 70B inference efficiency on GPU Droplets](<https://devfeed.tech/articles/digitalocean-gradienttm-ai-gpu-droplets-optimized-for-inference-increasing-throughput-at-lower-the-cost-19889.md>)

Original publisher: [Read original article](<https://www.digitalocean.com/blog/inference-optimized-image-droplet>)

Author: Hemasumanth Rasineni

Published: 2026-02-19T14:42:18Z

Content type: article

Language: en

Sources: [DigitalOcean](<https://devfeed.tech/sources/digitalocean.md>)

Topics: [Inference](<https://devfeed.tech/topics/inference.md>), [Digital Ocean](<https://devfeed.tech/topics/digital-ocean.md>), [Optimization](<https://devfeed.tech/topics/optimization.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [benchmarking](<https://devfeed.tech/topics/benchmarking.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [llama](<https://devfeed.tech/topics/llama.md>), [model-deployment](<https://devfeed.tech/topics/model-deployment.md>), [quantization](<https://devfeed.tech/topics/quantization.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [cost](<https://devfeed.tech/tags/cost.md>), [digitalocean](<https://devfeed.tech/tags/digitalocean.md>), [droplets](<https://devfeed.tech/tags/droplets.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [inference](<https://devfeed.tech/tags/inference.md>), [llama](<https://devfeed.tech/tags/llama.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [quantization](<https://devfeed.tech/tags/quantization.md>), [running](<https://devfeed.tech/tags/running.md>), [tokens](<https://devfeed.tech/tags/tokens.md>)

## AI overview

This article describes DigitalOcean's Inference Optimized Image, a pre-configured OS image for GPU Droplets that combines several inference optimizations. In the reported test, Llama 3.3 70B achieved higher throughput, lower time to first token, and lower cost per million tokens while running on two H100 GPUs instead of four.

## Source excerpt

Production-grade LLM inference demands more than just access to GPUs; it requires deep optimization across the entire serving stack, from quantization and attention kernels to memory management and parallelism strategies. Most teams deploying models like Llama 3.3 70B on vanilla configurations are leaving the majority of their hardware's capability on the table: underutilized FLOPs, wasted memory bandwidth, and GPU hours spent waiting instead of computing. To solve this, we built the Inference Optimized Image a fully pre-configured OS image available on DigitalOcean's GPU Droplets -- that layers speculative decoding, FP8 quantization, FlashAttention-3, paged attention, concurrent optimization, and prompt caching into a single deployable image. The result of our particular test: 143% higher throughput (2,000 vs. 823 tokens/second), 40.7% lower TTFT (187.9ms vs. 316.83ms), and a 75% reduction in cost per million tokens ($1.472 vs. $5.80) -- all while running Llama 3.3 70B on 2 H100 GPUs instead of 4. In this post, we walk through the optimization stack, the engineering reasoning behind each layer, and the benchmark methodology and our test results showing these gains. Prefill, Decode, and Why Optimization is Multiplicative As we covered in our LLM Inference Benchmarking post, inference works in two distinct phases with fundamentally different computation characteristics. The prefill phase processes the entire input prompt through the model's forward pass self-attention, layer norms, feed-forward networks and is compute-bound, with high arithmetic intensity (FLOPs per byte transferred). The decode phase generates tokens one at a time, loading the full weight matrix and KV cache from HBM for each token, making it strictly memory-bandwidth-bound. This distinction matters because each optimization in our stack targets a specific bottleneck. Speculative decoding attacks the sequential nature of decode. FP8 quantization reduces memory footprint and accelerates compute via hig