# HARTS: A Prefix-Sharing Training System for Agentic RL with Hybrid-Attention Models

DevFeed: [HARTS: A Prefix-Sharing Training System for Agentic RL with Hybrid-Attention Models](<https://devfeed.tech/articles/harts-a-prefix-sharing-training-system-for-agentic-rl-with-hybrid-attention-models-101588.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2608.28158>)

Author: Boyuan Meng, Peihua Bao, Hong Liu, Xiaowei Zhu, Chao Wang, Gen Li, Zhenxuan Pan

Published: 2026-10-27T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [distributed-training](<https://devfeed.tech/topics/distributed-training.md>), [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [gradient\_accumulation](<https://devfeed.tech/topics/gradient-accumulation.md>)

Tags: [acceleration](<https://devfeed.tech/tags/acceleration.md>), [activation](<https://devfeed.tech/tags/activation.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [attention](<https://devfeed.tech/tags/attention.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [distributed-training](<https://devfeed.tech/tags/distributed-training.md>), [execution](<https://devfeed.tech/tags/execution.md>), [hybrid](<https://devfeed.tech/tags/hybrid.md>), [mixture-of-experts](<https://devfeed.tech/tags/mixture-of-experts.md>), [mixture-of-experts-moe](<https://devfeed.tech/tags/mixture-of-experts-moe.md>), [objectives](<https://devfeed.tech/tags/objectives.md>), [online-learning](<https://devfeed.tech/tags/online-learning.md>), [planner](<https://devfeed.tech/tags/planner.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [reuse](<https://devfeed.tech/tags/reuse.md>), [rl](<https://devfeed.tech/tags/rl.md>), [routing](<https://devfeed.tech/tags/routing.md>), [sharing](<https://devfeed.tech/tags/sharing.md>), [system](<https://devfeed.tech/tags/system.md>), [training](<https://devfeed.tech/tags/training.md>)

## AI overview

HARTS accelerates agentic RL training by reusing shared prefixes across arbitrary rollout trees on hybrid-attention models. A two-level planner forms and refines microbatches for sharing and execution; gradients accumulate across dependent batches before one backward pass through shared computation. Differentiable state handoff and bounded replay preserve each trajectory's chunkwise numerical path, while a common compact runtime layout supports dense and MoE layers, distributed parallelism, and activation recomputation. On SWE-bench, HARTS reports 4.77-10.45x update throughput versus trajectory-expanded training across two sampling schemes, on 7.9B- and 124B-parameter models using 8-64 GPUs. For the 7.9B model, logit differences were comparable to baseline self-rerun variation; the 124B model showed stable online learning through step 150 on SWE-bench and τ³-Bench.

## Source excerpt

arXiv:2608.28158v2 Announce Type: replace-cross Abstract: Agentic reinforcement learning (RL) generates irregular rollout trees, but trajectory-by-trajectory training recomputes their shared interaction histories. Existing training-time prefix-reuse systems do not jointly address state recovery, dense execution, and distributed training for hybrid-attention models. We present HARTS (Hybrid-Attention RL over Tree Structures), to our knowledge the first system to demonstrate training acceleration from arbitrary rollout-tree prefix sharing on real hybrid-attention models. A two-level planner constructs compression-oriented microbatches, then jointly refines their membership, cross-microbatch sharing, placement, and execution order. Within each sharing window, gradients from dependent microbatches accumulate before one backward pass through the shared prefix computation. Differentiable boundary-state handoff and bounded replay preserve the per-trajectory chunkwise numerical path, while a linear-time planner attains the global minimum number of sequential linear-attention calls under our packed chunkwise-call model. The runtime uses a common compact layout across attention and dense or mixture-of-experts (MoE) layers, with support for distributed parallelism and activation recomputation. Recovering log-probabilities for each original trajectory position preserves RL objectives; for deterministic top-k routing without token dropping, occurrence counts restore MoE objective weights and expert-load statistics. On SWE-bench, HARTS delivers 4.77-10.45x the update throughput of trajectory-expanded training across two sampling schemes on 7.9B- and 124B-parameter hybrid-attention MoE models using 8-64 GPUs. On the 7.9B model, full-model logit differences are comparable to baseline self-rerun variation. The 124B model shows stable online learning trends through step 150 on SWE-bench and $\tau^3$-Bench.