# Use Your Mac for AI Agents: Self-Host Gemma 4 12 B with Pulumi and Tailscale

DevFeed: [Use Your Mac for AI Agents: Self-Host Gemma 4 12 B with Pulumi and Tailscale](<https://devfeed.tech/articles/use-your-mac-for-ai-agents-self-host-gemma-4-12-b-with-pulumi-and-tailscale-19026.md>)

Original publisher: [Read original article](<https://www.pulumi.com/blog/self-host-gemma4-llama-cpp-k8s-tailscale-pulumi/>)

Author: Pablo Seibelt

Published: 2026-06-04T00:00:00Z

Content type: tutorial

Language: en

Sources: [Pulumi](<https://devfeed.tech/sources/pulumi.md>)

Topics: [gemma4](<https://devfeed.tech/topics/gemma4.md>), [llama.cpp](<https://devfeed.tech/topics/llama-cpp.md>), [Local AI](<https://devfeed.tech/topics/local-ai.md>), [Infrastructure as code](<https://devfeed.tech/topics/infrastructure-as-code.md>), [macOS](<https://devfeed.tech/topics/macos.md>), [On-device AI](<https://devfeed.tech/topics/on-device-ai.md>), [multimodal](<https://devfeed.tech/topics/multimodal.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [download](<https://devfeed.tech/tags/download.md>), [gemma-4](<https://devfeed.tech/tags/gemma-4.md>), [gemma4](<https://devfeed.tech/tags/gemma4.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [infrastructure-as-code](<https://devfeed.tech/tags/infrastructure-as-code.md>), [kubernetes](<https://devfeed.tech/tags/kubernetes.md>), [llama-cpp](<https://devfeed.tech/tags/llama-cpp.md>), [macos](<https://devfeed.tech/tags/macos.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [python](<https://devfeed.tech/tags/python.md>), [tailscale](<https://devfeed.tech/tags/tailscale.md>), [tutorials](<https://devfeed.tech/tags/tutorials.md>)

## AI overview

A tutorial for self-hosting Gemma 4 12 B on a modern Mac using llama.cpp with Apple Metal acceleration. It combines host-native inference with a local Kubernetes cluster, Pulumi infrastructure as code, and Tailscale for secure access, and reports validation results on a MacBook Pro with an Apple M3 Max and 36 GB RAM.

## Source excerpt

If you run AI tools and agents, you've probably accepted three tradeoffs: your data leaves your network, you can't work offline, and your bill scales with usage. Open-weight models now run well on consumer hardware. Once the model is on your machine, your data stays local, inference works offline, and tokens cost nothing. If you own a modern Mac, you can run a high-quality model yourself. Gemma 4 is an open-weights model family from Google. This post focuses on Gemma 4 12 B, released in June 2026, using Unsloth's Q8_0 GGUF. The 12 B model fits comfortably on a modern Mac while leaving enough headroom for local llama.cpp and a chat UI. We'll use llama.cpp for host-native inference, k3d for a local Kubernetes cluster, Pulumi for infrastructure as code, and Tailscale for secure access. Prerequisites This setup was validated on the following hardware: macOS 26 Tahoe, version 26.5 MacBook Pro with Apple M3 Max 36 GB RAM On this machine, llama.cpp reported about 20 output tokens per second for a 160-token validation response with unsloth/gemma-4-12b-it-GGUF, gemma-4-12b-it-Q8_0.gguf, and a 131,072-token context. Sustained throughput varies by prompt length, thermal state, and llama.cpp settings. You'll need brew, docker, pulumi, and tailscale installed. We'll also install k3d during the process. Run Gemma 4 with host-native llama.cpp We use llama.cpp directly on macOS to leverage Apple Metal acceleration. Running the LLM on the host is more efficient than trying to pass GPU access into a local Kubernetes VM. Install the build tools: brew install cmake git Then build llama.cpp from source and download the multimodal projector. In validation, Homebrew llama.cpp 9430 could run text inference, but it could not load the new Gemma 4 12 B projector and failed with unknown projector type: gemma4uv. Building current llama.cpp from source fixed that. llm_home="$HOME/pulumi-gemma4-llm" mkdir -p "$llm_home/models" "$llm_home/logs" if [ ! -d "$llm_home/llama.cpp/.git" ]; then git clon