# ModHarness-RL: Recasting Harnesses as Learnable and Evolving Programs for Agentic RL

DevFeed: [ModHarness-RL: Recasting Harnesses as Learnable and Evolving Programs for Agentic RL](<https://devfeed.tech/articles/modharness-rl-recasting-harnesses-as-learnable-and-evolving-programs-for-agentic-rl-135212.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.18533>)

Author: Zelin He, Siyu Zhu, Jacob Helwig, Yujian Liu, Li An, Rohit Jain, Shiyu Chang, Alborz Geramifard

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [AI Context Management](<https://devfeed.tech/topics/ai-context-management.md>), [rlvr](<https://devfeed.tech/topics/rlvr.md>), [Agentic AI](<https://devfeed.tech/topics/what-is-agentic-ai.md>)

Tags: [agentic](<https://devfeed.tech/tags/agentic.md>), [behavior](<https://devfeed.tech/tags/behavior.md>), [blocks](<https://devfeed.tech/tags/blocks.md>), [candidate](<https://devfeed.tech/tags/candidate.md>), [compression](<https://devfeed.tech/tags/compression.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [execution](<https://devfeed.tech/tags/execution.md>), [fixed](<https://devfeed.tech/tags/fixed.md>), [guidance](<https://devfeed.tech/tags/guidance.md>), [harness](<https://devfeed.tech/tags/harness.md>), [history](<https://devfeed.tech/tags/history.md>), [model](<https://devfeed.tech/tags/model.md>), [performance](<https://devfeed.tech/tags/performance.md>), [program](<https://devfeed.tech/tags/program.md>), [rl](<https://devfeed.tech/tags/rl.md>), [sql](<https://devfeed.tech/tags/sql.md>), [state](<https://devfeed.tech/tags/state.md>), [steering](<https://devfeed.tech/tags/steering.md>), [study](<https://devfeed.tech/tags/study.md>)

## AI overview

ModHarness-RL treats an agentic RL harness as programs that can adapt alongside the model, rather than as a fixed, hand-written scaffold. It stores interactions as typed State blocks and learns two programs: Compression assembles interaction history into the model's context, while Steering offers guidance when behavior deviates. During training, candidate programs are proposed from recent trajectories, checked by execution, and replayed over past trajectories. In experiments on CodeSearch, with further analysis on ALFWorld and BIRD SQL and models from 1.5B to 8B, Compression and Steering improve held-out performance over a fixed-harness baseline in most settings. The supplied abstract does not quantify gains or specify which settings did not improve.

## Source excerpt

arXiv:2611.18533v1 Announce Type: new Abstract: Agentic reinforcement learning (RL) increasingly relies on a harness, the scaffold around the model that decides what it sees of its own interaction and what guidance it receives. Most existing approaches train under a hand-written harness that stays fixed throughout. However, an effective harness depends on the domain, the data and the model, and keeps changing as the model trains. We introduce ModHarness-RL, which recasts the harness as programs that are learned from data and evolve as the model trains with RL. ModHarness-RL records the interaction as structured State, a set of typed blocks, and two core components of the harness are programs over these blocks, a Compression program that assembles the accumulated history into the model's context and a Steering program that supplies guidance when behavior deviates. During RL training, candidate programs are proposed from recent trajectories, verified by execution, and replayed over past trajectories, so the harness tracks how the model's behavior shifts. We study CodeSearch in depth and further analyze ALFWorld and BIRD SQL, with models from 1.5B to 8B. The proposed Compression and Steering mechanisms improve held-out performance over the fixed-harness baseline in most settings. These results suggest that a harness for agentic RL can be learned from the task's data and evolved with the model.