# Tsinghua AI Open-Sources AKV, a Cache Engineering Server for LLM Agents on SGLang

DevFeed: [Tsinghua AI Open-Sources AKV, a Cache Engineering Server for LLM Agents on SGLang](<https://devfeed.tech/articles/tsinghua-ai-open-sources-akv-a-cache-engineering-server-for-llm-agents-on-sglang-100319.md>)

Original publisher: [Read original article](<https://www.marktechpost.com/2026/10/26/tsinghua-akv-kv-cache-engineering-llm-agents/>)

Author: Asif Razzaq

Published: 2026-10-26T07:56:45Z

Content type: news

Language: en

Sources: [MarkTechPost](<https://devfeed.tech/sources/marktechpost.md>)

Topics: [Cache](<https://devfeed.tech/topics/cache.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [ai-infrastructure](<https://devfeed.tech/tags/ai-infrastructure.md>), [ai-shorts](<https://devfeed.tech/tags/ai-shorts.md>), [applications](<https://devfeed.tech/tags/applications.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [cache](<https://devfeed.tech/tags/cache.md>), [editors-pick](<https://devfeed.tech/tags/editors-pick.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [for-devs](<https://devfeed.tech/tags/for-devs.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-agents](<https://devfeed.tech/tags/llm-agents.md>), [new-releases](<https://devfeed.tech/tags/new-releases.md>), [open](<https://devfeed.tech/tags/open.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [server](<https://devfeed.tech/tags/server.md>), [sglang](<https://devfeed.tech/tags/sglang.md>), [software-engineering](<https://devfeed.tech/tags/software-engineering.md>), [staff](<https://devfeed.tech/tags/staff.md>), [tech-news](<https://devfeed.tech/tags/tech-news.md>), [technology](<https://devfeed.tech/tags/technology.md>), [uncategorized](<https://devfeed.tech/tags/uncategorized.md>)

## AI overview

AKV is an open-source, Apache-2.0 SGLang serving system that lets LLM agents drop selected KV-cache entries and reposition retained entries while leaving the text history unchanged. Its API accepts drop and reposition rules, and its serving layer tracks those operations to avoid incorrect cache reuse; it can restore evicted states in one prefill pass. Across three models and four agent benchmarks, the excerpt reports 27.7% lower average cost and a 1.7-point higher average task score than threshold summarization. In serving tests on long SWE-bench tasks, AKV also showed higher input and output throughput and lower P95 latency; at concurrency 16, reported P95 latency was 30.63 seconds versus 229.90 for Standard. Results vary by model and task: MiniMax-M2.7 scored below Text Engineering on BrowseComp-Plus, and its cost rose when cache control was exposed as a tool. The excerpt says current support is limited to Qwen, GPT-OSS, and MiniMax text models; reported comparisons use different setups and are not directly comparable.

## Source excerpt

Tsinghua researchers released AKV, an SGLang-based serving system for cache engineering. LLM agents drop and reposition KV cache instead of summarizing history, cutting average cost 27.7% across 4 agent benchmarks. The post Tsinghua AI Open-Sources AKV, a Cache Engineering Server for LLM Agents on SGLang appeared first on MarkTechPost.