# radix

Published articles for radix.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Octopus: One Brain, Many Branches for Latent-KV Multi-Agent Reinforcement Learning

DevFeed: [Octopus: One Brain, Many Branches for Latent-KV Multi-Agent Reinforcement Learning](<https://devfeed.tech/articles/octopus-one-brain-many-branches-for-latent-kv-multi-agent-reinforcement-learning-120541.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.06956>)

Author: Tianming Sha

Published: 2026-11-06T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [grpo](<https://devfeed.tech/topics/grpo.md>), [Multi Agent Systems](<https://devfeed.tech/topics/multi-agent-systems.md>)

Tags: [agent-frameworks](<https://devfeed.tech/tags/agent-frameworks.md>), [agents](<https://devfeed.tech/tags/agents.md>), [attention](<https://devfeed.tech/tags/attention.md>), [bm25](<https://devfeed.tech/tags/bm25.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [dag](<https://devfeed.tech/tags/dag.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [grpo](<https://devfeed.tech/tags/grpo.md>), [llm](<https://devfeed.tech/tags/llm.md>), [mamba-2](<https://devfeed.tech/tags/mamba-2.md>), [memory](<https://devfeed.tech/tags/memory.md>), [multi-agent](<https://devfeed.tech/tags/multi-agent.md>), [octopus](<https://devfeed.tech/tags/octopus.md>), [qwen3-5-4b](<https://devfeed.tech/tags/qwen3-5-4b.md>), [radix](<https://devfeed.tech/tags/radix.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [rl](<https://devfeed.tech/tags/rl.md>), [sparse-attention](<https://devfeed.tech/tags/sparse-attention.md>), [throughput](<https://devfeed.tech/tags/throughput.md>), [trainer](<https://devfeed.tech/tags/trainer.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

Octopus coordinates multi-agent reinforcement learning through a GPU-resident DAG of shared KV states, letting branches share a model backbone and reuse cache state. Its three mechanisms are zero-copy radix forks, attention-only mounting of sibling KV blocks (bypassing recurrent states), and a packed DAG-GRPO trainer that assigns branch credit using leave-one-branch-out rewards. The abstract reports that mounting reduces off-task hijacking from 57.9% to 2.7% while retaining 92.2% of full-text mutual information with no extra KV allocation. On Qwen3.5-4B with sparse BM25 retrieval, Octopus reports 49.6% BrowseComp-Plus accuracy--17.5 points above single-agent GRPO and 12.5 above text-summary multi-agent RL--and 51.2% on OpenResearcher. It also reports 3.3x higher training throughput and 2.8x lower peak KV memory. These are abstract-reported results; the supplied text does not provide evaluation details.

### Source excerpt

arXiv:2611.06956v1 Announce Type: new Abstract: Multi-agent LLM systems explore in parallel by spawning sub-agents with isolated context windows, whereas single-agent systems append every search and verification step into one growing trajectory. However, existing multi-agent frameworks duplicate prefix KV caches across branches, exchange findings by re-prefilling text summaries, and flatten or drop cross-branch dependencies during reinforcement learning (RL). We introduce Octopus, a single-model multi-agent RL framework in which all threads share one backbone and coordinate over a GPU-resident directed acyclic graph (DAG) of KV states through five primitives (fork, read_thread, wait_threads, rewind, commit). Octopus combines three co-designed mechanisms: (1) Zero-Copy Radix Forking, which spawns branches as $\mathcal{O}(1)$ cursor views over shared physical KV pages; (2) Attention-Only KV Mounting, which mounts a sibling thread's terminal KV blocks directly into full-attention layers while bypassing linear-recurrent states (Gated DeltaNet / Mamba-2), cutting off-task hijacking by $21\times$ ($57.9\% \to 2.7\%$) while retaining $92.2\%$ of full-text mutual information at zero extra KV allocation; and (3) a Packed DAG-GRPO Trainer, which packs all episode branches into one block-sparse attention pass and assigns counterfactual branch credit via leave-one-branch-out marginal rewards. Trained end-to-end with GRPO on Qwen3.5-4B over a sparse BM25 corpus, Octopus reaches $49.6\%$ accuracy on BrowseComp-Plus ($+17.5$ points over single-agent GRPO at 143K context and $+12.5$ points over text-summary multi-agent RL) and $51.2\%$ on OpenResearcher, outperforming dense-retrieval agents up to $250\times$ larger while increasing training throughput $3.3\times$ and cutting peak KV memory $2.8\times$.