# AI gateways: why and how

DevFeed: [AI gateways: why and how](<https://devfeed.tech/articles/ai-gateways-why-and-how-18916.md>)

Original publisher: [Read original article](<https://blog.frankel.ch/ai-gateways/>)

Author: Nicolas Fränkel

Published: 2026-05-31T00:00:00Z

Content type: article

Language: en

Sources: [Nicolas Fränkel](<https://devfeed.tech/sources/nicolas-frankel.md>)

Topics: [Amazon API Gateway](<https://devfeed.tech/topics/amazon-api-gateway.md>), [LLMs](<https://devfeed.tech/topics/llms.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Routing (disambiguation)](<https://devfeed.tech/topics/routing.md>), [Claude](<https://devfeed.tech/topics/claude.md>), [Security](<https://devfeed.tech/topics/security.md>), [litellm](<https://devfeed.tech/topics/litellm.md>), [Caching](<https://devfeed.tech/topics/caching.md>), [Scalability](<https://devfeed.tech/topics/scalability.md>), [Load Balancing](<https://devfeed.tech/topics/load-balancing.md>), [observability](<https://devfeed.tech/topics/observability.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-gateway](<https://devfeed.tech/tags/ai-gateway.md>), [anthropic](<https://devfeed.tech/tags/anthropic.md>), [api-gateway](<https://devfeed.tech/tags/api-gateway.md>), [backend](<https://devfeed.tech/tags/backend.md>), [caching](<https://devfeed.tech/tags/caching.md>), [claude](<https://devfeed.tech/tags/claude.md>), [claude-code](<https://devfeed.tech/tags/claude-code.md>), [coding-assistant](<https://devfeed.tech/tags/coding-assistant.md>), [decoupling](<https://devfeed.tech/tags/decoupling.md>), [devstral](<https://devfeed.tech/tags/devstral.md>), [gateway](<https://devfeed.tech/tags/gateway.md>), [litellm](<https://devfeed.tech/tags/litellm.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llms](<https://devfeed.tech/tags/llms.md>), [load-balancing](<https://devfeed.tech/tags/load-balancing.md>), [mistral](<https://devfeed.tech/tags/mistral.md>), [observability](<https://devfeed.tech/tags/observability.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [routing](<https://devfeed.tech/tags/routing.md>), [scalability](<https://devfeed.tech/tags/scalability.md>), [security](<https://devfeed.tech/tags/security.md>), [technical](<https://devfeed.tech/tags/technical.md>)

## AI overview

This article explains how AI gateways apply the API gateway pattern to AI clients and LLM backends. It discusses using a gateway to centralize model and provider management, governance, cost control, routing, caching, failover, load balancing, and observability, and considers routing Claude Code requests to another provider.

## Source excerpt

Before working for 2 years on the Apache APISIX API gateway, I was mainly oblivious to API gateways. It's only by working with them that I understood their value. Decoupling the client and the server unlocks a lot of options: moving authentication to the API Gateway, securing APIs, deduplicating API requests, etc. In this post, I want to describe how the same pattern applies to AI. AI gateways AI gateways work in a similar way.