# long-context A large-language-model capability for processing unusually large token contexts in a single request. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Optimize your team's price-performance with hosted open weight models DevFeed: [Optimize your team's price-performance with hosted open weight models]() Original publisher: [Read original article]() Author: Brittany Lutz Published: 2026-09-17T00:00:00Z Content type: release Language: en Sources: [GitLab]() Topics: [GitLab](), [Artificial Intelligence](), [Agentic development](), [AI Development](), [AI-assisted coding](), [Latency](), [long-context](), [Security]() Tags: [agentic-ai](), [ai](), [coding-agents](), [cost](), [features](), [gitlab](), [latency](), [models](), [open](), [optimize](), [product]() ### AI overview GitLab announces three hosted open weight models for GitLab Duo Agent Platform: Kimi K3, GLM 5.3, and MiniMax M3. The release gives software teams more model choices for balancing quality, latency, and cost across different development workloads. ### Source excerpt There's no single best model for every software development task. Implementing a new feature, diagnosing a failed pipeline, and resolving security vulnerabilities all place different demands on the model handling them. GitLab Duo Agent Platform is expanding GitLab-managed model choice with three hosted open weight models: Kimi K3, GLM 5.3, and MiniMax M3. Together with the frontier models already available in GitLab, your team now has more control over how you optimize for quality, latency, and cost, tuned to the needs of each workload. Point a hard task at Kimi K3 or GLM 5.3, which outperformed comparable frontier models in internal testing and costs less per call, or hand routine, high-volume work to MiniMax M3. Either way, your team gets up to 4x more calls per GitLab Credit than some comparable frontier models, with more AI model options to match cost to task complexity. GitLab Transcend returns in October Coding agents are increasing your speed of development, but your reviews, security policies, and release cycles still have to keep pace. Our Transcend event on October 6 will demonstrate how GitLab is helping teams close that gap and explore what it takes to carry the speed of agentic AI across the software lifecycle. Register for the livestream today! Why one model has been the easy way out Different tasks in agentic development need different things from an AI model. A long-running refactor needs a large context window and deeper reasoning, while a routine, high-volume task is often better served by a faster, more cost-efficient model. Optimizing for one task type means giving up ground on the other. As teams work through the technical tradeoff, whether they can actually access a new model emerges as an additional governance challenge. In regulated environments, every model and the infrastructure it runs on has to clear security, compliance, and internal review before the team can use it. Reviews and approvals move slowly enough that many teams standardize o ## How LLMs Handle Memory Through Context and Surrounding Applications DevFeed: [How LLMs Handle Memory Through Context and Surrounding Applications]() Original publisher: [Read original article]() Author: ByteByteGo Published: 2026-09-15T15:31:12Z Content type: article Language: en Sources: [ByteByteGo]() Topics: [Large Language Model](), [context window](), [App](), [long-context]() Tags: [article](), [context](), [context-window](), [cost](), [latency](), [llms](), [memory]() ### AI overview LLMs do not usually retain personal or persistent memory between interactions. Surrounding applications create the appearance of memory by storing messages, maintaining summaries, retrieving relevant information, and supplying it to the model. As conversations grow, this processing increases cost and latency, while context-window limits require older information to be removed, summarized, or stored elsewhere. ### Source excerpt In this article, we will learn how LLMs handle memory so that they are useful to end users in performing complex tasks that require conversation and holding context. ## Enterprise-Grade Precision for Long-Context Multimodal Embedding Inference on Cloud TPU DevFeed: [Enterprise-Grade Precision for Long-Context Multimodal Embedding Inference on Cloud TPU]() Original publisher: [Read original article]() Author: Anthony Su; Injae Kwak Published: 2026-09-12T11:04:33.891311Z Content type: article Language: en Sources: [Google Developers Blog]() Topics: [Embeddings](), [vllm](), [Inference](), [long-context](), [autoscaling](), [Kubernetes](), [Cloud](), [Google](), [Artificial Intelligence](), [model-deployment](), [Open Source]() Tags: [autoscaling](), [embedding](), [github](), [gpu](), [inference](), [kubernetes](), [long-context](), [multimodal](), [open-source](), [retrieval](), [tpu](), [vllm]() ### AI overview This article explains how Google Cloud integrated native TPU support into vLLM to serve long-context, multimodal embedding models at production scale. It describes elastic TPU provisioning with Google Kubernetes Engine, autoscaling across accelerator types, and TPU-specific optimizations for long sequences and chunked prefill. The resulting setup is designed to preserve numerical parity with GPU reference baselines while supporting high-throughput semantic retrieval applications. ### Source excerpt Google Cloud has natively integrated TPU support into the vLLM serving engine, allowing developers to elastically scale high-demand embedding pipelines using Google Kubernetes Engine (GKE). To handle massive 15K+ token contexts for models like Qwen3-Embedding-8B, the engineering team implemented TPU-specific optimizations such as hardware-safe tensor alignment, JAX/XLA compilation pre-warming, and a hybrid StepPool architecture for chunked prefill management. These enhancements achieve near-perfect numerical parity with reference GPU baselines, and developers can immediately leverage the open-sourced setup recipes on the AI-Hypercomputer GitHub to build their own high-throughput semantic retrieval applications. ## OpenAI's researchers burned $7,000 a day on AI agents -- now it's opening the floodgates DevFeed: [OpenAI's researchers burned $7,000 a day on AI agents -- now it's opening the floodgates]() Original publisher: [Read original article]() Author: Amanda Caswell Published: 2026-09-11T21:27:42Z Content type: news Language: en Sources: [The New Stack]() Topics: [Inference](), [long-context]() Tags: [agents](), [ai](), [ai-agents](), [ai-infrastructure](), [api](), [cloud-services](), [codex](), [compute](), [developers](), [inference](), [openai](), [research](), [sandbox](), [tools]() ### AI overview OpenAI's public-beta Agents API lets developers run long-lived agents with managed job state, context compression, optional tools, parallel subagents, and execution in OpenAI's sandbox or developer-controlled infrastructure. The article highlights the resulting inference and compute costs, citing internal research-agent usage figures. ### Source excerpt OpenAI rolled out its Agents API in public beta Thursday, opening the backend behind Codex to developers looking to run The post OpenAI's researchers burned $7,000 a day on AI agents -- now it's opening the floodgates appeared first on The New Stack. ## Deploying Qwen3.8-2.4T-A95B on Amazon SageMaker HyperPod with vLLM DevFeed: [Deploying Qwen3.8-2.4T-A95B on Amazon SageMaker HyperPod with vLLM]() Original publisher: [Read original article]() Author: Dmitry Soldatkin Published: 2026-09-09T22:26:29Z Content type: tutorial Language: en Sources: [Artificial Intelligence]() Topics: [Deployment](), [Inference](), [Machine learning](), [Language models](), [GPU](), [long-context](), [Mixture of Experts (MoE)]() Tags: [advanced-300](), [agentic](), [amazon-sagemaker](), [amazon-sagemaker-hyperpod](), [deployment](), [gpu](), [how-to](), [inference](), [long-context](), [moe](), [nvfp4](), [quantization](), [qwen](), [reasoning](), [technical-how-to](), [tool](), [vllm]() ### AI overview A deployment walkthrough for serving the open-weight Qwen3.8-2.4T-A95B language model on Amazon SageMaker HyperPod with vLLM and NVIDIA B300 GPUs. It covers provisioning, NVFP4 quantization, an OpenAI-compatible endpoint, reasoning, tool calling, and MTP speculative decoding. ### Source excerpt Learn how to deploy Qwen3.8-2.4T-A95B, a 2.4-trillion-parameter open-weight model, on Amazon SageMaker HyperPod with vLLM. This walkthrough covers cluster provisioning, NVFP4 quantization, and an OpenAI-compatible endpoint with built-in reasoning, tool calling, and native MTP speculative decoding. ## Why your AI bill tripled while token prices fell 75% DevFeed: [Why your AI bill tripled while token prices fell 75%]() Original publisher: [Read original article]() Author: Sunile Manjee Published: 2026-09-04T00:00:00Z Content type: article Language: en Sources: [Elastic Blog - Elasticsearch, Kibana, and ELK Stack]() Topics: [long-context](), [Inference](), [Chat Bot]() Tags: [agent](), [agentic-ai](), [ai](), [cost](), [inference](), [reasoning]() ### AI overview The article explains why spending on agentic AI can rise despite lower token prices: multistep, multiturn, and multihop work repeats and expands context, increasing billed token use. ### Source excerpt Token prices dropped 75%, so why did your agentic AI bill triple? Here's the real math behind agentic AI cost and how IT leaders can measure it. ## GLM-5.3 is 50% off through DigitalOcean on AI Gateway DevFeed: [GLM-5.3 is 50% off through DigitalOcean on AI Gateway]() Original publisher: [Read original article]() Author: Jerilyn Zheng Published: 2026-09-02T00:00:00Z Content type: release Language: en Sources: [Vercel News]() Topics: [Vercel](), [Digital Ocean](), [Language models](), [long-context](), [AI-assisted coding](), [Claude Code](), [codex](), [cursor]() Tags: [ai-gateway](), [claude-code](), [codex](), [coding-agents](), [context-window](), [cursor](), [digitalocean](), [language-models](), [model](), [tokens](), [vercel]() ### AI overview Vercel announces a 50% discount on GLM-5.3 through DigitalOcean on AI Gateway until September 8. The article explains the temporary promo model name, standard provider routing, model limits, spend tracking, and setup for coding agents. ### Source excerpt GLM-5.3 is 50% off on AI Gateway through Tuesday, September 8, in partnership with DigitalOcean. How to use the model during the offer period Using the promo name (zai/glm-5.3-promo-50) gets the discounted rate. It routes only to DigitalOcean, with no fallback to another provider, and it stops serving when the offer ends. Using the standard name (i.e., zai/glm-5.3) with provider options to sort DigitalOcean as the preferred provider keeps working after September 8 and routes across every provider that serves the model, at their usual rates. Because the promo name goes away when the offer ends, treat it as something you switch on for the window rather than hardcode. To keep the standard name in your code instead, pin the provider with order: ['digitalocean'] under providerOptions.gateway, which prefers DigitalOcean and falls back to the others if it cannot serve the request. GLM-5.3 takes text input, with a 1M token context window and a maximum output of 128K tokens. Discounted requests appear in your spend dashboard and carry a trace like any other request. Try GLM-5.3 in the model playground. To use it in a coding agent, see the coding agents guide, then run vercel ai-gateway coding-agents setup to connect agents like Claude Code, Codex, OpenCode, Cursor, Pi, and more and select zai/glm-5.3-promo-50 inside the agent. You can view all language models available on AI Gateway. Read more ## Using Six Scoped Subagents to Manage Context Windows DevFeed: [Using Six Scoped Subagents to Manage Context Windows]() Original publisher: [Read original article]() Author: Paul Iusztin Published: 2026-09-01T05:00:29Z Content type: tutorial Language: en Sources: [Decoding ML]() Topics: [context window](), [long-context](), [Artificial Intelligence](), [Code](), [coding](), [Claude](), [Claude Code](), [codex](), [LangChain](), [Python](), [Cloud]() Tags: [agent](), [agents](), [ai](), [claude](), [claude-code](), [cloud](), [codex](), [coding](), [coding-agents](), [context-engineering](), [context-window](), [langchain](), [python]() ### AI overview This tutorial explains how splitting research work among six parallel subagents can reduce context-window noise and limit the state returned to an orchestrator agent. It also covers designing subagent protocols, maintaining an agent registry, and running parallel agent workflows. ### Source excerpt The harness determines what your orchestrator agent never sees, not the model. ## Experiment with Qwen3.8-Flash-Next on NVIDIA GB300 NVL72 for Agentic Coding DevFeed: [Experiment with Qwen3.8-Flash-Next on NVIDIA GB300 NVL72 for Agentic Coding]() Original publisher: [Read original article]() Author: Michelle Horton Published: 2026-08-26T17:07:12Z Content type: article Language: en Sources: [NVIDIA Developer](), [NVIDIA Technical Blog]() Topics: [qwen](), [long-context](), [Mixture of Experts (MoE)](), [agentic-coding](), [Inference](), [GPU](), [multimodal](), [NeMo](), [sglang](), [TensorRT-LLM](), [vllm](), [rust-ai]() Tags: [agentic-ai-generative-ai](), [agentic-coding](), [benchmarks](), [context-window](), [developer-tools-techniques](), [gb300-nvl72](), [inference](), [mixture-of-experts-moe](), [multimodal](), [nemo](), [nvlink](), [qwen](), [sglang](), [tensorrt-llm](), [top-stories](), [vllm]() ### AI overview This NVIDIA developer article introduces Qwen3.8-Flash-Next, a multimodal mixture-of-experts model released by Alibaba for experimentation and evaluation. It explains the model's long-context hybrid architecture, including Gated DeltaNet and Qwen Sparse Attention, and discusses reported efficiency improvements for million-token workloads. The article also covers inference support through SGLang, vLLM, TensorRT-LLM, and NVIDIA NeMo, plus performance on the NVIDIA GB300 NVL72 platform. ### Source excerpt Alibaba released the model weights for Qwen3.8-Flash-Next as a preview of the upcoming Qwen4 architecture for developers to experiment with and evaluate. It's... ## Qwen 3.8 Flash now available on AI Gateway DevFeed: [Qwen 3.8 Flash now available on AI Gateway]() Original publisher: [Read original article]() Author: Jerilyn Zheng Published: 2026-08-26T00:00:00Z Content type: release Language: en Sources: [Vercel News]() Topics: [qwen](), [Artificial Intelligence](), [SDKs](), [AI-assisted coding](), [long-context](), [Claude Code](), [codex](), [cursor]() Tags: [ai](), [ai-gateway](), [api-keys](), [claude-code](), [codex](), [coding-agents](), [context-window](), [cost](), [cursor](), [opencode](), [pricing](), [qwen](), [routing](), [sdk](), [tokens](), [vercel]() ### AI overview Qwen 3.8 Flash from Alibaba is now available through Vercel AI Gateway. The model accepts text and images, supports a 1-million-token context window, and can generate responses of up to 65,000 tokens. It is recommended for coding, tool use, and multi-step agent workflows, and can be used through the AI SDK, coding agents, and the model playground. AI Gateway offers unified model access, usage and cost tracking, reliability features, reporting, retention controls, API key budgets, routing, and provider pricing without markup or inference platform fees. ### Source excerpt Qwen 3.8 Flash from Alibaba is now available on AI Gateway. It takes text and images as input, serves a context window of 1 million tokens, and can return up to 65k tokens in a response. Alibaba recommends it for coding, tool use, and multi-step agent workflows. To use Qwen3.8-Flash, set model to alibaba/qwen3.8-flash in the AI SDK: To use it in a coding agent, see the coding agents guide, then run vercel ai-gateway coding-agents setup to connect agents like Claude Code, Codex, OpenCode, Cursor, Pi, and more and select alibaba/qwen3.8-flash inside the agent. Try Qwen3.8-Flash in the model playground. AI Gateway provides a unified API for calling models, tracking usage and cost, and configuring retries, failover, and performance optimizations for higher-than-provider uptime. It includes built-in custom reporting, Zero Data Retention support, budgets for API keys, routing rules, and more. AI Gateway reflects provider pricing with no markup and does not charge a platform fee on inference, including on Bring Your Own Key (BYOK) requests. Read more ## What makes a good AI coworker? With OpenAI's Codex product lead DevFeed: [What makes a good AI coworker? With OpenAI's Codex product lead]() Original publisher: [Read original article]() Author: info@1password.com (Chris Fowler) Published: 2026-08-25T00:00:00Z Content type: article Language: en Sources: [Blog on 1Password Blog]() Topics: [Artificial Intelligence](), [codex](), [OpenAI](), [long-context](), [Filesystems](), [coding](), [cursor]() Tags: [agent](), [agentic](), [agents](), [ai](), [ai-coding](), [autonomous](), [codex](), [context-window](), [cursor](), [dev](), [developers](), [engineering](), [filesystems](), [memory](), [openai](), [podcast](), [podcasts]() ### AI overview An episode featuring OpenAI Codex product lead Rohan Varma explores what makes AI agents effective coworkers. It focuses on shared context, durable memory and state outside the context window, filesystem-based task continuity, and steerability so people can understand and guide agent output. ### Source excerpt Zero-Shot Learning is a podcast about how AI is built, secured, and deployed. Hosted by Nancy Wang, 1Password CTO, and Dev Tagare, Senior Director of Engineering at Google Gemini, it offers a builder's view of the architecture and complex decisions involved in shipping AI. In this episode, Rohan Varma, Product Lead for Codex at OpenAI, described what happens when teams move from using agents for one-off tasks to enabling autonomous coworkers. Having worked on AI coding products at Cursor and OpenAI, Ro understands what people need to work effectively with agents and what agents need to work effectively with people. What makes a good (agentic) coworker? With any coworker, collaboration works best when everyone is working from the same context, toward a shared goal. Human coworkers are accustomed to working toward shared goals. With proper context and resources, they can divide work without losing sight of how their contribution affects the team. The difference between teams of people and agents is that people don't need to be told how to remember things. Everything they do carries historical context. When a team works together, their shared knowledge expands exponentially. Agents work within context windows, a temporary working memory that fills as a task continues. When the window is full, the system has to summarize the work without losing decisions and constraints that could cause the agent to miss crucial directives. State is a fundamental building block for making an agent feel more like a coworker. Without memory, every time you prompt an agent, it's kind of like its first day on planet Earth." -Rohan Varma, Codex Product Lead, OpenAI To be a useful long-term collaborator, an agent's memory has to exist outside its context window. The system has to preserve the work durably to understand which files were changed, which decisions were made, which results were gathered, and which tasks are yet to be completed. That shared state lets one agent resume a task, anoth ## Preloading Knowledge Into a Model Instead of Retrieving It DevFeed: [Preloading Knowledge Into a Model Instead of Retrieving It]() Original publisher: [Read original article]() Author: Avi Chawla Published: 2026-08-24T17:13:39Z Content type: tutorial Language: en Sources: [Daily Dose of Data Science]() Topics: [Retrieval Augmented Generation (RAG)](), [Caching](), [long-context](), [Inference](), [Self-hosted](), [model-deployment](), [GPU]() Tags: [caching](), [compute](), [context-window](), [deployment](), [inference](), [rag](), [retrieval](), [self-hosted](), [storage]() ### AI overview This article introduces a three-part RAG Systems course about preloading a knowledge corpus into a model's stored KV cache so subsequent queries can skip retrieval, chunking, and embedding. It covers naive, compressed, modular, and trained preloading, along with context limits, provider and self-hosted economics, cache compression constraints, and production deployment. ### Source excerpt How to process your corpus once, skip retrieval entirely, and serve every query from a stored cache. Three parts covering the full spectrum. ## With Groq 3 LPX in Full Production, NVIDIA Extends Vera Rubin Inference for Agents DevFeed: [With Groq 3 LPX in Full Production, NVIDIA Extends Vera Rubin Inference for Agents]() Original publisher: [Read original article]() Author: NVIDIA Writers Published: 2026-08-24T15:00:41Z Content type: news Language: en Sources: [NVIDIA Blog]() Topics: [Inference Performance](), [long-context](), [Low-Latency Inference](), [networking](), [gemma4]() Tags: [agentic](), [agents](), [ai-factory](), [ai-inference](), [ai-infrastructure](), [benchmark](), [bluefi](), [cloud](), [cpx](), [groq](), [groq-3-lpx](), [inference](), [infrastructure](), [latency](), [long-context](), [low-latency-inference](), [lpx](), [networking](), [nvidia](), [nvidia-spectrum-x-ethernet](), [nvidia-vera](), [nvidia-vera-rubin](), [nvl72](), [nvlink](), [vera-cpu](), [vera-rubin]() ### AI overview NVIDIA announces Groq 3 LPX production availability alongside Vera Rubin NVL72, positioning the integrated compute, networking, and inference stack for long-context, agentic AI workloads. The article highlights a benchmark result of 3,400 output tokens per second on Gemma 4 31B and describes partner adoption of Vera Rubin, Spectrum-X Multiplane, and Groq 3 LPX. ### Source excerpt The next era of AI inference won't be defined by a single breakthrough chip, network or system. It'll be defined by how every layer of the AI factory works together. That's why NVIDIA is extending Vera Rubin NVL72 with fast token generation for agentic systems. Announced today, the NVIDIA Vera Rubin rack-scale system NVIDIA Groq [...] ## Up to 30x More Work Per Watt: NVIDIA Vera Rubin NVL72 Sets a New Efficiency Standard for AI Agents DevFeed: [Up to 30x More Work Per Watt: NVIDIA Vera Rubin NVL72 Sets a New Efficiency Standard for AI Agents]() Original publisher: [Read original article]() Author: Shruti Koparkar Published: 2026-08-24T15:00:19Z Content type: article Language: en Sources: [NVIDIA Blog]() Topics: [AI Agent](), [NVIDIA Vera Rubin](), [Vera Rubin NVL72](), [Inference Performance](), [long-context](), [GB300 NVL72](), [Inference](), [benchmarking](), [Nvidia]() Tags: [agentic-ai](), [agents](), [ai](), [ai-infrastructure](), [gb300-nvl72](), [hardware](), [inference](), [infrastructure](), [long-context](), [networking](), [nvidia-blackwell](), [nvidia-vera-rubin](), [performance](), [software](), [think-smart]() ### AI overview NVIDIA reports that Vera Rubin NVL72 systems deliver up to 30x higher throughput per megawatt than GB300 NVL72 on agentic workloads measured with the SemiAnalysis AgentX workload. The article attributes the efficiency challenge to long, variable agent workflows involving tool calls, accumulated context, and sub-agents. ### Source excerpt According to OpenRouter data, agentic AI workloads consume 15x more tokens than a simple chat request. Why? Consider what happens when an AI agent researches a company for an investment decision. The agent queries financial databases, searches news and filings, invokes a sub-agent to run peer comparisons and model valuations, then synthesizes everything into a [...] ## How NVIDIA Groq 3 LPX Unlocks Ultrafast Interactivity at Long Context on NVIDIA Vera Rubin DevFeed: [How NVIDIA Groq 3 LPX Unlocks Ultrafast Interactivity at Long Context on NVIDIA Vera Rubin]() Original publisher: [Read original article]() Author: Tanya Lenz Published: 2026-08-24T15:00:00Z Content type: article Language: en Sources: [NVIDIA Developer](), [NVIDIA Technical Blog]() Topics: [d-matrix](), [Vera Rubin](), [Inference Performance](), [long-context](), [Benchmark](), [Artificial Intelligence](), [gemma4](), [systems](), [Cache]() Tags: [agent](), [agentic](), [agentic-ai-generative-ai](), [ai](), [ai-factory](), [ai-inference](), [benchmark](), [cache](), [collective](), [data-center-cloud](), [developer-tools-techniques](), [groq](), [groq-3-lpx](), [inference-performance](), [long-context](), [low-latency-inference](), [lpx](), [nvidia](), [nvidia-vera](), [performance](), [rubin-gpu](), [speed](), [systems](), [tokens](), [training-ai-models](), [vera-rubin](), [vera-rubin-nvl72]() ### AI overview NVIDIA Groq 3 LPX, paired with Vera Rubin NVL72, delivers high-interactivity AI inference for long-context workloads. A reported benchmark measured 3,431 output tokens per second on Gemma 4 31B with a 100K context. ### Source excerpt NVIDIA Groq 3 LPX is the interactive AI inference accelerator for the NVIDIA Vera Rubin platform. At the core of the platform is NVIDIA Vera Rubin NVL72, the... ## How Much Memory Does Your Agent Actually Need? DevFeed: [How Much Memory Does Your Agent Actually Need?]() Original publisher: [Read original article]() Author: Vatche Isahagian; Gaodan Fang; Jayaram Radhakrishnan; Punleuk Oum; Ashwath Vaithinathan Aravindan; Evelyn Duesterwald; G Thomas; Vinod Muthusamy; Merve Unuvar; Ayhan Sebin Published: 2026-08-18T18:09:38Z Content type: article Language: en Sources: [Hugging Face - Blog]() Topics: [Benchmark](), [Inference](), [long-context](), [gpt-oss](), [deepseek](), [Mixture of Experts (MoE)]() Tags: [agent](), [agentic](), [caching](), [deepseek](), [evaluation](), [gpt-oss](), [inference](), [memory](), [model](), [models](), [moe](), [performance](), [production](), [tokens]() ### AI overview ALTK-Evolve equips agents with agentic memory by distilling reusable guidelines from past trajectories and injecting them at inference time without weight updates or human annotation. An evaluation across eight models found that the effective amount of memory varies by model: stronger models benefited from full guideline sets, weaker models from compact cores plus task-specific retrieval, and saturated models showed no measurable improvement. For gpt-oss-120b, selective retrieval improved task completion by 16.1 percentage points with only a 5% token increase. ### Source excerpt Equipping an agent with agentic memory sounds simple: distill lessons from its past work, put them back in context, and more experience should mean better performance. It doesn't always work that way. When we scaled the evaluation to eight models -- from a 30B dense model to frontier proprietary systems -- one finding stood out: Agentic memory is not a feature you switch on. It's a dose you calibrate to the model. ## GPT-5.6 Sol is 50% off on AI Gateway for the next month DevFeed: [GPT-5.6 Sol is 50% off on AI Gateway for the next month]() Original publisher: [Read original article]() Author: Jerilyn Zheng Published: 2026-08-17T00:00:00Z Content type: release Language: en Sources: [Vercel News]() Topics: [OpenAI](), [API](), [API keys](), [Claude Code](), [codex](), [long-context](), [AI-assisted coding](), [browser](), [pdf]() Tags: [ai-gateway](), [api](), [browser](), [cache](), [claude-code](), [codex](), [coding-agents](), [gpt](), [long-context](), [openai](), [pdf](), [pricing](), [tokens]() ### AI overview OpenAI's GPT-5.6 Sol is available at a 50% discount through September 18 for requests billed directly through AI Gateway. The discount covers all listed service tiers, token types, regions, modes, cached tokens, cache writes, and long-context requests, while BYOK requests remain subject to the user's provider pricing. Existing requests use the discounted rate without a model ID change. ### Source excerpt GPT-5.6 Sol, the flagship of OpenAI's GPT-5.6 series, is 50% off on AI Gateway through September 18. The discount applies on the OpenAI provider to all token types, tiers, regions, and modes, and it is available only on requests running directly through AI Gateway (not BYOK). Pricing: 50% off Service tier New price per M tokens (input / output) Original price per M tokens (input / output) Default $2.50 / $15.00 $5.00 / $30.00 Flex $1.25 / $7.50 $2.50 / $15.00 Priority (fast mode) $5.00 / $30.00 $10.00 / $60.00 The discount applies on the same terms everywhere else the model is priced: across every service tier, including fast mode, and to cached tokens, cache writes, long-context requests, and different regions. It covers requests billed through AI Gateway on the OpenAI provider. BYOK requests run on your own provider accounts and bill at whatever rate you have with them. The model ID is unchanged, so requests you already send pick up the discounted rate with no code change: Sol takes a reasoning effort up to max for the hardest problems, accepts text, image, and PDF input, and carries a long context window. To use it in a coding agent, run vercel ai-gateway coding-agents setup to connect Claude Code, Codex, OpenCode, or Pi, then select openai/gpt-5.6-sol inside the agent. The 50% discount will apply there. Get started Create an API key in the AI Gateway section of your dashboard, or try the model in the browser first from its playground page. Read more ## Configuring compaction thresholds and context windows for coding agents DevFeed: [Configuring compaction thresholds and context windows for coding agents]() Original publisher: [Read original article]() Author: WorkOS Published: 2026-08-14T19:33:33Z Content type: tutorial Language: en Sources: [WorkOS Blog]() Topics: [coding](), [context window](), [long-context]() Tags: [agent](), [coding](), [context-window](), [generation](), [model](), [recovery](), [tokens]() ### AI overview The article explains how to derive a coding agent's effective context window from compaction thresholds and the response runway required to complete generation. It discusses threshold behavior, model metadata, route limits, generation clamping, overflow detection, and recovery. ### Source excerpt Derive your coding agent's effective context window from two numbers: the compaction threshold you want, and the response runway the model needs to finish. ## Grok 4.6: 500K Context, Pricing, and Agentic Coding Features DevFeed: [Grok 4.6: 500K Context, Pricing, and Agentic Coding Features]() Original publisher: [Read original article]() Author: Aishwari Pahwa Published: 2026-08-13T00:00:00Z Content type: article Language: en Sources: [Appwrite Blog]() Topics: [Frontier Model](), [long-context](), [coding](), [API](), [Fine-tuning](), [Reinforcement learning]() Tags: [agentic](), [agents](), [ai](), [api](), [benchmarks](), [coding](), [context-window](), [fine-tuning](), [frontier-model](), [reinforcement-learning](), [token](), [web-development]() ### AI overview This article describes Grok 4.6, focusing on long-running agents, coding, interactive and visual work, its 500K-token context window, pricing, benchmarks, and API availability. It also outlines claimed training changes, longer task trajectories, and self-testing or verification during multi-step work. ### Source excerpt Grok 4.6 is SpaceXAI's frontier model for long-running agents and coding. See full benchmarks, token pricing, the 500K context window, and API setup. ## Grok 4.6 now available on AI Gateway DevFeed: [Grok 4.6 now available on AI Gateway]() Original publisher: [Read original article]() Author: Jerilyn Zheng Published: 2026-08-12T00:00:00Z Content type: release Language: en Sources: [Vercel News]() Topics: [AI, ML & Data Engineering](), [vercel ai sdk](), [long-context](), [AI-assisted coding]() Tags: [agent](), [ai](), [ai-gateway](), [ai-models](), [api](), [model](), [reasoning](), [vercel]() ### AI overview Grok 4.6 is now available through AI Gateway, with a 500K-token context window, text and image inputs, and configurable reasoning levels. The announcement also describes AI Gateway features for model access, usage and cost tracking, routing, reliability, and reporting. ### Source excerpt Grok 4.6 from SpaceXAI is now available on AI Gateway. The model has a 500K token context window and accepts text and image inputs. Grok 4.6 supports low, medium, high, and xhigh reasoning levels and defaults to high. To use Grok 4.6, set model to xai/grok-4.6 in the AI SDK: To use it in a coding agent, run vercel ai-gateway coding-agents setup to connect Claude Code, Codex, OpenCode, or Pi, then select xai/grok-4.6 inside the agent. To try Grok 4.5 with no code, try the model in the model playground. AI Gateway provides a unified API for calling models, tracking usage and cost, and configuring retries, failover, and performance optimizations for higher-than-provider uptime. It includes built-in custom reporting, Zero Data Retention support, budgets for API keys, routing rules, and more. AI Gateway reflects provider pricing with no markup and does not charge a platform fee on inference, including on Bring Your Own Key (BYOK) requests. Read more ## 🍔🧠 What's inside an LLM's KV cache DevFeed: [🍔🧠 What's inside an LLM's KV cache]() Original publisher: [Read original article]() Author: Alexandre Zajac Published: 2026-08-10T15:30:59Z Content type: article Language: en Sources: [Hungry Minds]() Topics: [Cache](), [Caching](), [Large Language Model](), [Inference Performance](), [long-context](), [Inference]() Tags: [cache](), [caching](), [inference](), [inference-performance](), [latency](), [llm](), [models](), [performance](), [quantization]() ### AI overview The article explains that the KV cache stores key and value vectors for input tokens during LLM inference. Its memory use and bandwidth demands grow with context length and batch size, creating a bottleneck for long-context serving. It discusses grouped-query attention, latent attention, KV cache quantization, paged attention, and prefix caching as ways to reduce memory use, fragmentation, cost, or latency. ### Source excerpt PLUS: Google's AI depth decline 🧠, Real-time agent load balancing ⚡, Semantic search ranking at scale 👨💻 ## Run Local Agentic AI Workflows with Meta's Muse Glimmer on NVIDIA DevFeed: [Run Local Agentic AI Workflows with Meta's Muse Glimmer on NVIDIA]() Original publisher: [Read original article]() Author: Michelle Horton Published: 2026-08-10T13:27:19Z Content type: article Language: en Sources: [NVIDIA Developer](), [NVIDIA Technical Blog]() Topics: [Artificial Intelligence](), [Local AI](), [Large Language Model](), [long-context](), [Inference](), [Meta](), [Open Source](), [GPU](), [NVIDIA DGX](), [NVLink](), [Jetson](), [Automation]() Tags: [agentic-ai](), [agentic-ai-generative-ai](), [ai](), [automation](), [context-window](), [data-center-cloud](), [dgx-spark](), [dgx-station](), [edge-computing](), [featured](), [gpu](), [inference](), [jetson](), [local-ai](), [nemoclaw](), [nvidia](), [nvidia-dgx](), [nvlink](), [open-source](), [top-stories]() ### AI overview Meta's Muse Glimmer is a 30B open-weight dense model designed for local agentic AI workflows. With a 120K+ context window and performance of up to 20K tokens per second on a single GPU, it supports sustained, multi-step tool use and local processing of sensitive data. ### Source excerpt Meta returns to the open source ecosystem with the release of Muse Glimmer, a 30B open-weight dense model with a 120K+ context window built for local AI... ## Ling 3.0 Tiny is now available on AI Gateway DevFeed: [Ling 3.0 Tiny is now available on AI Gateway]() Original publisher: [Read original article]() Author: Jerilyn Zheng Published: 2026-08-06T00:00:00Z Content type: release Language: en Sources: [Vercel News]() Topics: [Artificial Intelligence](), [moe](), [vercel ai sdk](), [long-context](), [SDKs]() Tags: [ai](), [ai-gateway](), [api](), [caching](), [coding-agents](), [context-window](), [cost](), [models](), [moe](), [sdk]() ### AI overview Ling 3.0 Tiny from ANT Group is available on Vercel AI Gateway, initially in a free slot. It is a Mixture-of-Experts model with 7.9B total parameters, approximately 1.3B active parameters per token, a 256K-token context window, and support for up to 32K output tokens. The article also explains how to use it through the AI SDK, the model playground, and coding agents. ### Source excerpt Ling 3.0 Tiny from ANT Group is now on AI Gateway, free to use till 8:00am PT on 8/14. Ling 3.0 Tiny takes the free slot from Ling 3.0 Flash. Ling 3.0 Tiny is a MOE model with 7.9B total parameters and about 1.3B active per token, a 256K token context window, and up to 32K output tokens. The model is built for responsive agents, instruction following, and multi-turn conversation, with native function calling and prompt caching. To use Ling 3.0 Tiny, set model to inclusionai/ling-3.0-tiny-free in the AI SDK. On August 14th, the new model name will be inclusionai/ling-3.0-tiny. Try Ling 3.0 Tiny in the model playground. To use it in a coding agent, run vercel ai-gateway coding-agents setup and select inclusionai/ling-3.0-tiny-free inside the agent. AI Gateway provides a unified API for calling models, tracking usage and cost, and configuring retries, failover, and performance optimizations for higher-than-provider uptime. It includes built-in custom reporting, Zero Data Retention support, budgets for API keys, routing rules, and more. AI Gateway reflects provider pricing with no markup and does not charge a platform fee on inference, including on Bring Your Own Key (BYOK) requests. Read more ## Co-Designing AI Model Attention for Fast, Interactive Long-Context Inference DevFeed: [Co-Designing AI Model Attention for Fast, Interactive Long-Context Inference]() Original publisher: [Read original article]() Author: Tanya Lenz Published: 2026-07-31T22:16:17Z Content type: article Language: en Sources: [NVIDIA Developer](), [NVIDIA Technical Blog]() Topics: [Inference Performance](), [long-context](), [Artificial Intelligence](), [GPU](), [GPU optimization](), [Large Language Model](), [Nvidia]() Tags: [agentic-ai-generative-ai](), [ai](), [ai-inference](), [data-center-cloud](), [developer-tools-techniques](), [featured](), [gpu](), [inference](), [inference-performance](), [long-context](), [memory](), [model-architecture](), [nvidia](), [performance]() ### AI overview This article examines how co-designing dense attention with GPU execution can improve throughput and interactivity for long-context inference. It analyzes group size, head dimension, sequence length, and the different compute and memory behavior of prefill and decode, including the effects of speculative decoding and prefix caching. ### Source excerpt As agentic and long-context workloads become common, the context lengths increase and attention consumes a larger share of inference time (Figure 1). Because... [Next page]()