# Open-weight models are fast on Neon AI Gateway. Here's why

DevFeed: [Open-weight models are fast on Neon AI Gateway. Here's why](<https://devfeed.tech/articles/open-weight-models-are-fast-on-neon-ai-gateway-here-s-why-5701.md>)

Original publisher: [Read original article](<https://neon.com/blog/open-weight-models-are-fast-on-neon-ai-gateway>)

Author: Carlota Soto

Published: 2026-08-20T12:00:00Z

Content type: article

Language: en

Sources: [Blog -- Neon Docs](<https://devfeed.tech/sources/blog-neon-docs.md>)

Topics: [databricks](<https://devfeed.tech/topics/databricks.md>), [Inference Performance](<https://devfeed.tech/topics/inference-performance.md>), [model-deployment](<https://devfeed.tech/topics/model-deployment.md>), [Low-Latency Inference](<https://devfeed.tech/topics/low-latency-inference.md>), [Caching](<https://devfeed.tech/topics/caching.md>), [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [gpt-oss](<https://devfeed.tech/topics/gpt-oss.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [caching](<https://devfeed.tech/tags/caching.md>), [databricks](<https://devfeed.tech/tags/databricks.md>), [gpt-oss](<https://devfeed.tech/tags/gpt-oss.md>), [inference](<https://devfeed.tech/tags/inference.md>), [mixture-of-experts-moe](<https://devfeed.tech/tags/mixture-of-experts-moe.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [performance](<https://devfeed.tech/tags/performance.md>), [product](<https://devfeed.tech/tags/product.md>), [speed](<https://devfeed.tech/tags/speed.md>)

## AI overview

The article explains why open-weight models can deliver low latency on Neon AI Gateway when paired with Databricks Foundation Model APIs and an optimized serving stack. It highlights continuous batching, KV-cache paging, custom kernels, quantization, multi-GPU layouts, prompt caching, and Mixture-of-Experts architectures.

## Source excerpt

Open-weight models are built for performance, but how they're served determines whether end users actually feel that speed. Neon AI Gateway calls Databricks Foundation Model APIs for Databricks-hosted open-weight models, a stack that has years of inference engineering behind it.