# Build realtime voice agents on AI Gateway

DevFeed: [Build realtime voice agents on AI Gateway](<https://devfeed.tech/articles/build-realtime-voice-agents-on-ai-gateway-772.md>)

Original publisher: [Read original article](<https://vercel.com/blog/realtime-voice-agents-on-ai-gateway>)

Author: Kevin Dawkins

Published: 2026-06-29T07:00:00Z

Content type: tutorial

Language: en

Sources: [Vercel News](<https://devfeed.tech/sources/vercel-news.md>)

Topics: [vercel ai sdk](<https://devfeed.tech/topics/vercel-ai-sdk.md>), [Language models](<https://devfeed.tech/topics/language-models.md>), [SDKs](<https://devfeed.tech/topics/sdks.md>)

Tags: [agents](<https://devfeed.tech/tags/agents.md>), [ai-gateway](<https://devfeed.tech/tags/ai-gateway.md>), [audio](<https://devfeed.tech/tags/audio.md>), [observability](<https://devfeed.tech/tags/observability.md>), [openai](<https://devfeed.tech/tags/openai.md>), [routing](<https://devfeed.tech/tags/routing.md>), [speech](<https://devfeed.tech/tags/speech.md>), [streaming](<https://devfeed.tech/tags/streaming.md>), [text-to-speech](<https://devfeed.tech/tags/text-to-speech.md>), [tools](<https://devfeed.tech/tags/tools.md>), [transcription](<https://devfeed.tech/tags/transcription.md>), [voice](<https://devfeed.tech/tags/voice.md>)

## AI overview

Vercel AI Gateway adds beta support for realtime voice, text-to-speech, and speech-to-text in AI SDK 7. The article explains browser-based realtime sessions, token-based authentication, interruption handling, tool calls, and direct audio input/output models.

## Source excerpt

AI Gateway now supports audio/voice. You can add realtime voice, text to speech, and speech to text with the same calls you already use for text, image, and video, routed through AI Gateway alongside every other modality. Audio launches with models from OpenAI and xAI. Each call gets the same provider routing, observability, spend controls, and bring-your-own-key support you already use for your other models. These capabilities are in beta and available in AI SDK 7. Capability How it works Use it for Realtime voice Live audio in and out, for streaming, low-latency session Two-way voice agents and live conversation Text to speech Text in, audio file out, single request Voiceovers, spoken responses, audio versions of written content Speech to text Recorded audio in, text out, single request Transcribing voice notes, call recordings Getting started Realtime, speech, and transcription model are supported on AI SDK 7. Realtime voice agents Realtime turns your app into something a user can hold a conversation with. When they speak, the model responds right away. Because it replies in the moment instead of waiting for a full turn, users can interrupt and talk over it the way they would with a person. It fits voice assistants, customer support agents, hands-free tools, and anywhere a user would rather talk than type. What sets it apart from chaining models together is that a single realtime model hears audio and produces audio directly, instead of running a speech-to-text, then language model, then text-to-speech pipeline. In the browser, the useRealtime hook manages the WebSocket connection, microphone capture, and audio playback. The connection is authenticated with your AI Gateway credential, so you mint a short-lived token on the server and hand the browser only that token. Your API key never reaches the client. Add a route that mints the token: Then connect from a client component: The hook captures the microphone, streams the audio to the model through AI Gateway, and