# Fully Automated AI Inference on AWS, Azure, and Google Cloud with Pulumi

DevFeed: [Fully Automated AI Inference on AWS, Azure, and Google Cloud with Pulumi](<https://devfeed.tech/articles/fully-automated-ai-inference-on-aws-azure-and-google-cloud-with-pulumi-19002.md>)

Original publisher: [Read original article](<https://www.pulumi.com/blog/fully-automated-ai-inference-aws-azure-gcp-pulumi/>)

Author: Engin Diri

Published: 2026-06-30T00:00:00Z

Content type: tutorial

Language: en

Sources: [Pulumi](<https://devfeed.tech/sources/pulumi.md>)

Topics: [Ollama](<https://devfeed.tech/topics/ollama.md>), [Cloud](<https://devfeed.tech/topics/cloud.md>), [Infrastructure as code](<https://devfeed.tech/topics/infrastructure-as-code.md>), [model-serving](<https://devfeed.tech/topics/model-serving.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [OpenID connect (OIDC)](<https://devfeed.tech/topics/oidc.md>), [Ubuntu](<https://devfeed.tech/topics/ubuntu.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [aws](<https://devfeed.tech/tags/aws.md>), [azure](<https://devfeed.tech/tags/azure.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [gcp](<https://devfeed.tech/tags/gcp.md>), [google-cloud](<https://devfeed.tech/tags/google-cloud.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [inference](<https://devfeed.tech/tags/inference.md>), [infrastructure-as-code](<https://devfeed.tech/tags/infrastructure-as-code.md>), [model-serving](<https://devfeed.tech/tags/model-serving.md>), [oidc](<https://devfeed.tech/tags/oidc.md>), [ollama](<https://devfeed.tech/tags/ollama.md>), [pulumi-esc](<https://devfeed.tech/tags/pulumi-esc.md>), [tutorials](<https://devfeed.tech/tags/tutorials.md>), [typescript](<https://devfeed.tech/tags/typescript.md>), [ubuntu](<https://devfeed.tech/tags/ubuntu.md>)

## AI overview

This tutorial shows how to use Pulumi to provision GPU virtual machines on AWS, Azure, and Google Cloud, install drivers, run Ollama, and pull an open-source model without manual setup. It also uses Pulumi ESC with OIDC instead of a long-lived access key and avoids treating model-download completion as an infrastructure resource.

## Source excerpt

Putting Ollama on a cloud GPU is something I keep coming back to. A while ago I wrote up running open-source LLMs on an AWS EC2 box with Ollama and Pulumi, and the shape never really changes: a GPU instance, a model server, and a firewall rule in front. Infrastructure as code earned its place by making that kind of setup predictable and repeatable, and AI infrastructure is no exception. A GPU box serving a model is still a VM, a disk, and a firewall rule, and it should be declared like one. Thorsten Hans made exactly that case in his Akamai post, Fully Automated AI Infrastructures with Terraform and Akamai Cloud, which stands up a single GPU instance on Linode, installs the drivers, runs Ollama, and pulls a model, with no manual steps after terraform apply. I liked the shape of it, so this post ports the same idea to Pulumi and runs it across AWS, Azure, and Google Cloud instead of one. The result is one program shape per cloud: a single pulumi up brings up a GPU box that installs its own driver, runs Ollama, and pulls a model with no manual steps, and a single pulumi destroy takes it back down. Along the way it drops the two imperative bits the Terraform version leans on: a static access token sitting in an environment variable, and a null_resource running a shell loop to wait for the model. The first becomes an OIDC login from a Pulumi ESC environment, so no long-lived key lives anywhere. The second turns out not to be a resource at all. What you are building Strip away the per-cloud naming and every version of this is the same three things: a GPU virtual machine, a firewall in front of it, and a cloud-init script that turns a bare Ubuntu box into a running inference server. The model serving runs on Ollama, which exposes an HTTP API on port 11434 and keeps the model resident in GPU memory between requests. flowchart LR Dev([Your machine / curl]) -->|"HTTP :11434"| FW["Firewall / security group
(allow 11434, optional 22)"] FW --> VM["GPU VM (Ubuntu 24.04)