# rlvr

A reinforcement-learning training approach for language-model agents that uses automatically verifiable task outcomes as rewards.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## ROAM-GRPO: Enforcing Reasoning Diversity in GRPO via Routed Off-mode Approach Mining

DevFeed: [ROAM-GRPO: Enforcing Reasoning Diversity in GRPO via Routed Off-mode Approach Mining](<https://devfeed.tech/articles/roam-grpo-enforcing-reasoning-diversity-in-grpo-via-routed-off-mode-approach-mining-145610.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25926>)

Author: Jeff Duan, Matt Thomson

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [grpo](<https://devfeed.tech/topics/grpo.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [rlvr](<https://devfeed.tech/topics/rlvr.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [algorithm](<https://devfeed.tech/tags/algorithm.md>), [creativity](<https://devfeed.tech/tags/creativity.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [diversity](<https://devfeed.tech/tags/diversity.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [exploration](<https://devfeed.tech/tags/exploration.md>), [grpo](<https://devfeed.tech/tags/grpo.md>), [math](<https://devfeed.tech/tags/math.md>), [mining](<https://devfeed.tech/tags/mining.md>), [post-training](<https://devfeed.tech/tags/post-training.md>), [problems](<https://devfeed.tech/tags/problems.md>), [qwen3](<https://devfeed.tech/tags/qwen3.md>), [qwen3-5](<https://devfeed.tech/tags/qwen3-5.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [rlvr](<https://devfeed.tech/tags/rlvr.md>), [sft](<https://devfeed.tech/tags/sft.md>), [strategies](<https://devfeed.tech/tags/strategies.md>), [supervised-fine-tuning](<https://devfeed.tech/tags/supervised-fine-tuning.md>), [well](<https://devfeed.tech/tags/well.md>)

### AI overview

ROAM-GRPO aims to counter reasoning-mode collapse in GRPO by routing well-solved problems to an "off-mode" track. The model generates distinct solution strategies, an LLM judge checks their logical validity and distinctness, and accepted strategies are self-distilled into the policy. In the reported experiment, training Qwen3.5-2b on 1,000 MATH level-5 problems increased peak pass@16 by 4.7% and correct embedding cosine distance by approximately 1.5-2x on a difficult evaluation set. The supplied text does not detail the evaluation setup or broader limitations.

### Source excerpt

arXiv:2611.25926v1 Announce Type: new Abstract: Supervised fine-tuning (SFT) paired with Reinforcement Learning with Verifiable Rewards (RLVR) has become the dominant recipe for post-training LLMs on complex reasoning tasks, but has been shown to collapse the policy into a low-diversity mode. Repeated LLM outputs are nearly identical, and pass@k accuracy converges or degrades as a result. We propose ROAM-GRPO (Routed Off-mode Approach Mining for GRPO), an augmentation to the GRPO algorithm which incentivizes exploration of distinct reasoning strategies and aims to increase output creativity during evaluation. ROAM-GRPO detects well-solved problems during training and transfers them to an "off-mode" track. The model is prompted to produce distinct solution strategies to such problems, which are verified for logical validity and distinctness by an LLM judge and subsequently self-distilled into the policy. Training Qwen3.5-2b on a 1,000-problem subset of the MATH lvl5 dataset, ROAM-GRPO implementation increases peak pass@16 by 4.7% and correct embedding cosine distance by approximately 1.5-2x on a difficult evaluation set.

## Structured Role-Aware Policy Optimization for Multimodal Reasoning

DevFeed: [Structured Role-Aware Policy Optimization for Multimodal Reasoning](<https://devfeed.tech/articles/structured-role-aware-policy-optimization-for-multimodal-reasoning-144892.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2605.07274>)

Author: Bingqing Jiang, Guoxi Zhang, Jasper Wang, Bingning Wang, Difan Zou

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [grpo](<https://devfeed.tech/topics/grpo.md>), [rlvr](<https://devfeed.tech/topics/rlvr.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>)

Tags: [capabilities](<https://devfeed.tech/tags/capabilities.md>), [consistency](<https://devfeed.tech/tags/consistency.md>), [credit-assignment](<https://devfeed.tech/tags/credit-assignment.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [dependency](<https://devfeed.tech/tags/dependency.md>), [evidence-based](<https://devfeed.tech/tags/evidence-based.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [grounded](<https://devfeed.tech/tags/grounded.md>), [group-relative-policy-optimization](<https://devfeed.tech/tags/group-relative-policy-optimization.md>), [grpo](<https://devfeed.tech/tags/grpo.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [policy](<https://devfeed.tech/tags/policy.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [reasoning-tokens](<https://devfeed.tech/tags/reasoning-tokens.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [reward](<https://devfeed.tech/tags/reward.md>), [rlvr](<https://devfeed.tech/tags/rlvr.md>), [roles](<https://devfeed.tech/tags/roles.md>), [structured](<https://devfeed.tech/tags/structured.md>)

### AI overview

SRPO refines GRPO's sequence-level advantage into token-level credit for perception and reasoning, aiming to improve whether correct multimodal answers rely on relevant visual evidence. It uses on-policy contrasts between original and corrupted images to weight perception tokens by visual dependence, and weights reasoning tokens by consistency with the generated perception. A shared trajectory-level baseline combines these signals into positive weights that adjust update magnitudes while preserving the original reward and advantage signs. The method requires neither external reward models nor separate teachers. The abstract reports improved evidence-grounded reasoning across diverse multimodal benchmarks; it gives no specific benchmark results or quantitative gains.

### Source excerpt

arXiv:2605.07274v2 Announce Type: replace-cross Abstract: Reinforcement learning from verifiable rewards (RLVR), especially with Group Relative Policy Optimization (GRPO), has shown strong potential for improving the reasoning capabilities of large vision-language models (LVLMs). However, in multimodal reasoning, final-answer rewards are typically assigned at the sequence level and do not distinguish the functional roles of different tokens, making it difficult to determine whether a correct answer is supported by task-relevant visual evidence. In this paper, we revisit multimodal RLVR from the perspective of role-aware token-level credit assignment, where structured responses are decomposed into perception tokens for extracting visual evidence and reasoning tokens for deriving answers from that evidence. Based on this perspective, we propose Structured Role-aware Policy Optimization (SRPO), which refines the sequence-level GRPO advantage into role-aware token-level advantages without changing the reward function. Specifically, SRPO assigns role-specific credit by using self-distilled on-policy contrasts: perception tokens are emphasized according to their visual dependency under original versus corrupted visual inputs, while reasoning tokens are emphasized according to their consistency with the generated perception. These role-specific signals are further unified through a shared trajectory-level baseline, yielding positive token weights that adjust relative update magnitudes while preserving the original GRPO reward and advantage signs, without requiring external reward models or separate teachers. Experiments across diverse multimodal reasoning benchmarks show that SRPO improves evidence-grounded reasoning, highlighting the importance of moving beyond uniform sequence-level credit toward role-aware optimization for reliable multimodal reasoning.

## How Complete Must an Environment Be to Train a Capable GUI Agent?

DevFeed: [How Complete Must an Environment Be to Train a Capable GUI Agent?](<https://devfeed.tech/articles/how-complete-must-an-environment-be-to-train-a-capable-gui-agent-142730.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23946>)

Author: Jingxing Wang, Teng Wang, Xingyu Lou, Zihan Guo, Jun Wang, Weiwen Liu, Weinan Zhang, Jianghao Lin

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [GUI](<https://devfeed.tech/topics/gui.md>), [AI Agent](<https://devfeed.tech/topics/ai-agent.md>), [rlvr](<https://devfeed.tech/topics/rlvr.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agents](<https://devfeed.tech/tags/agents.md>), [also](<https://devfeed.tech/tags/also.md>), [application](<https://devfeed.tech/tags/application.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [away](<https://devfeed.tech/tags/away.md>), [batches](<https://devfeed.tech/tags/batches.md>), [build](<https://devfeed.tech/tags/build.md>), [but](<https://devfeed.tech/tags/but.md>), [complete](<https://devfeed.tech/tags/complete.md>), [construction](<https://devfeed.tech/tags/construction.md>), [cost](<https://devfeed.tech/tags/cost.md>), [coverage](<https://devfeed.tech/tags/coverage.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [dimension](<https://devfeed.tech/tags/dimension.md>), [dimensions](<https://devfeed.tech/tags/dimensions.md>), [environment](<https://devfeed.tech/tags/environment.md>), [gui](<https://devfeed.tech/tags/gui.md>), [train](<https://devfeed.tech/tags/train.md>)

### AI overview

In MobileGym, training Qwen3-VL-4B-Instruct with a verifier that rewards only one reference path reduced success by 10.53 points, and removing task-irrelevant application records also hurt performance. Pruning unused features raised success from 51.61% to 56.13%, but the gain was confined to tasks whose environments were unchanged; it reversed under equal episode weighting and disappeared when balancing only the pruned tasks. The authors attribute an additional weighting issue to long detour successes reinforcing skills that eroded performance elsewhere. Their MobileGym-Lite keeps reward and data coverage complete while pruning unused features, achieving 55.48% versus Full's 51.61% success in the original environment at lower construction cost.

### Source excerpt

arXiv:2611.23946v1 Announce Type: cross Abstract: Graphical user interface (GUI) agents are widely trained in executable GUI environments. To lower cost, build environments in batches, and reduce manual effort, each work trades away part of the environment through its construction choices. We describe these choices along three dimensions: reward coverage, the goal-reaching trajectories the verifier rewards; data coverage, the application data the agent is exposed to beyond what the task reads or writes; and feature coverage, the application features beyond what the task uses. Existing works set each dimension on engineering grounds, but they do not measure its effect on training. How complete must an environment be to train a capable agent, and along which dimensions? In MobileGym, we adjust each dimension separately, train Qwen3-VL-4B-Instruct, and evaluate every policy in the original full environment. A verifier that rewards only one reference path lowers success by 10.53 points, and removing unneeded records also lowers success. Pruning unneeded features instead raises success from 51.61% to 56.13%, entirely on tasks whose environments it left unchanged. The gain reverses when each episode is weighted equally and vanishes when only episodes on the pruned tasks are balanced. Without pruning, length weighting there reinforces long detour successes that erode skills on other tasks. Following these findings, we build MobileGym-Lite, which keeps reward and data coverage complete and prunes unneeded features. It outperforms Full in the full environment (55.48% versus 51.61% success) with lower construction cost.

## Seeing with You: Perception-Reasoning Coevolution for Multimodal Reasoning

DevFeed: [Seeing with You: Perception-Reasoning Coevolution for Multimodal Reasoning](<https://devfeed.tech/articles/seeing-with-you-perception-reasoning-coevolution-for-multimodal-reasoning-139762.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2603.28618>)

Author: Ziqi Miao, Haonan Jia, Lijun Li, Chen Qian, Yuan Xiong, Wenting Yan, Yongyao Wang, Zaoying Huang, Ken Deng, Jing Shao

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [rlvr](<https://devfeed.tech/topics/rlvr.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>)

Tags: [bottleneck](<https://devfeed.tech/tags/bottleneck.md>), [credit-assignment](<https://devfeed.tech/tags/credit-assignment.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [framework](<https://devfeed.tech/tags/framework.md>), [improvements](<https://devfeed.tech/tags/improvements.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [observer](<https://devfeed.tech/tags/observer.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [policy](<https://devfeed.tech/tags/policy.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [rlvr](<https://devfeed.tech/tags/rlvr.md>), [roles](<https://devfeed.tech/tags/roles.md>)

### AI overview

PRCO targets a perception bottleneck in multimodal reasoning: shared final-answer rewards can improve reasoning without reliably improving visual evidence extraction. Its shared-policy framework assigns two roles: an Observer creates a question-focused evidence caption, and a Solver answers from that caption. The Solver receives verifiable rewards for the final answer; the Observer receives a utility reward based on the Solver's downstream success. The abstract reports consistent gains across eight challenging benchmarks and model scales, averaging over 7 accuracy points above the base model and outperforming prior open-source RL-tuned baselines.

### Source excerpt

arXiv:2603.28618v3 Announce Type: replace Abstract: Reinforcement learning with verifiable rewards (RLVR) has substantially enhanced the reasoning capabilities of multimodal large language models (MLLMs). However, existing RLVR approaches typically rely on outcome-driven optimization that updates both perception and reasoning using a shared reward based solely on the final answer. This shared reward blurs credit assignment, frequently improving reasoning patterns while failing to reliably enhance the accuracy of upstream visual evidence extraction. To address this perception bottleneck, we introduce PRCO (Perception-Reasoning Coevolution), a dual-role RLVR framework with a shared policy. PRCO consists of two cooperative roles: an Observer that generates an evidence caption tailored to the question and a Solver that predicts the final answer based on this caption. Crucially, PRCO employs role-specific reward signals: the Solver is optimized using verifiable outcome rewards on the final answer, while the Observer receives a utility reward derived from the Solver's downstream success. Extensive experiments across eight challenging multimodal reasoning benchmarks demonstrate that PRCO yields consistent improvements across model scales by over 7 points on average accuracy compared to the base model, outperforming prior open-source RL-tuned baselines.

## CrossEnv: Synthesizing Verifiable Cross-Environment Tasks for Long-Horizon Agent Training

DevFeed: [CrossEnv: Synthesizing Verifiable Cross-Environment Tasks for Long-Horizon Agent Training](<https://devfeed.tech/articles/crossenv-synthesizing-verifiable-cross-environment-tasks-for-long-horizon-agent-training-140077.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22777>)

Author: Zhengfeng Yu, Tian Lan, Zhijing Wu, Enbo Zhao, Xuannan Liu, Feihu Jiang, Qianghuai Jia, Longyue Wang, Zhao Xu, Weihua Luo, Xian-Ling Mao

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [rlvr](<https://devfeed.tech/topics/rlvr.md>), [agentic workflows](<https://devfeed.tech/topics/agentic-workflows.md>), [Multi Agent Systems](<https://devfeed.tech/topics/multi-agent-systems.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>)

Tags: [actions](<https://devfeed.tech/tags/actions.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agent-training](<https://devfeed.tech/tags/agent-training.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [business](<https://devfeed.tech/tags/business.md>), [cross](<https://devfeed.tech/tags/cross.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [data](<https://devfeed.tech/tags/data.md>), [domains](<https://devfeed.tech/tags/domains.md>), [environment](<https://devfeed.tech/tags/environment.md>), [environments](<https://devfeed.tech/tags/environments.md>), [external](<https://devfeed.tech/tags/external.md>), [feedback](<https://devfeed.tech/tags/feedback.md>), [fewer](<https://devfeed.tech/tags/fewer.md>), [framework](<https://devfeed.tech/tags/framework.md>), [generate](<https://devfeed.tech/tags/generate.md>), [graph](<https://devfeed.tech/tags/graph.md>), [horizon](<https://devfeed.tech/tags/horizon.md>), [information](<https://devfeed.tech/tags/information.md>), [tasks](<https://devfeed.tech/tags/tasks.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

CrossEnv synthesizes executable, verifiable tasks that require agents to coordinate across services, addressing the consistency challenge in aligning data generated across environments. It uses an entity-tool graph to find connections between environments and align task-specific data. CrossEnv-Curriculum advances from single-environment tool use to coordination across more environments and business domains, then adds user interaction. On four external benchmarks, combining cross-environment training data with this curriculum achieved competitive performance using fewer training examples. The abstract also reports evidence of transfer to new environment combinations, less similar external tasks, and longer workflows.

### Source excerpt

arXiv:2611.22777v1 Announce Type: new Abstract: Executable synthetic environments provide scalable training tasks and verifiable feedback for long-horizon agents. Existing environment synthesis methods primarily generate tasks within individual environments. However, these tasks do not adequately capture real-world interactions across services, where actions in one service depend on information obtained from another. Synthesizing such tasks is challenging because data generated across environments must remain consistent to support executable and verifiable interactions. To address this challenge, we introduce CrossEnv, a framework that uses an entity-tool graph to identify connections between environments and aligns task-specific data to synthesize executable, verifiable cross-environment tasks. We further propose CrossEnv-Curriculum, which progresses from single-environment tool use to coordination across more environments and business domains, before introducing user interaction. Across four external benchmarks, combining cross-environment training data with this curriculum achieves competitive performance with fewer training examples. Further analyses support the benefits of cross-environment data and curriculum training, and provide evidence of transfer to new environment combinations, external tasks less similar to the training data, and longer workflows.

## CORE: Concept-Oriented Reinforcement for Bridging the Definition-Application Gap in Mathematical Reasoning

DevFeed: [CORE: Concept-Oriented Reinforcement for Bridging the Definition-Application Gap in Mathematical Reasoning](<https://devfeed.tech/articles/core-concept-oriented-reinforcement-for-bridging-the-definition-application-gap-in-mathematical-reasoning-139754.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2512.18857>)

Author: Zijun Gao, Zhikun Xu, Xiao Ye, Ben Zhou

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [grpo](<https://devfeed.tech/topics/grpo.md>), [rlvr](<https://devfeed.tech/topics/rlvr.md>)

Tags: [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [concepts](<https://devfeed.tech/tags/concepts.md>), [core](<https://devfeed.tech/tags/core.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [grpo](<https://devfeed.tech/tags/grpo.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [large-language-models](<https://devfeed.tech/tags/large-language-models.md>), [large-language-models-llms](<https://devfeed.tech/tags/large-language-models-llms.md>), [llms](<https://devfeed.tech/tags/llms.md>), [math](<https://devfeed.tech/tags/math.md>), [mathematical-reasoning](<https://devfeed.tech/tags/mathematical-reasoning.md>), [problem-solving](<https://devfeed.tech/tags/problem-solving.md>), [quizzes](<https://devfeed.tech/tags/quizzes.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [rl](<https://devfeed.tech/tags/rl.md>), [rlvr](<https://devfeed.tech/tags/rlvr.md>), [sft](<https://devfeed.tech/tags/sft.md>), [supervision](<https://devfeed.tech/tags/supervision.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

CORE addresses a gap in reinforcement learning with verifiable rewards (RLVR), which trains final-answer correctness but supplies little direct feedback about applying concepts. The framework uses textbook-linked exercises and concise definitions to create concept-aligned quizzes, and can inject concept snippets into training rollouts. It reinforces concept use through trajectory replacement after group failures, a lightweight forward-KL constraint aligning unguided and concept-primed policies, or standard GRPO on concept-aligned quizzes. The abstract reports consistent gains over vanilla and supervised fine-tuning baselines across in-domain concept exercises and diverse out-of-domain math benchmarks. It presents CORE as compatible with different algorithms and verifiers; detailed experimental results are not included in the supplied excerpt.

### Source excerpt

arXiv:2512.18857v4 Announce Type: replace Abstract: Large language models (LLMs) often solve challenging math exercises yet fail to apply the concept right when the problem requires genuine understanding. Popular Reinforcement Learning with Verifiable Rewards (RLVR) pipelines reinforce final answers but provide little fine-grained conceptual signal, so models improve at pattern reuse rather than conceptual applications. We introduce CORE (Concept-Oriented REinforcement), an RL training framework that turns explicit concepts into a controllable supervision signal. Starting from a high-quality, low-contamination textbook resource that links verifiable exercises to concise concept descriptions, we run a sanity probe showing LLMs can restate definitions but fail concept-linked quizzes, quantifying the conceptual reasoning gap. CORE then (i) synthesizes concept-aligned quizzes, (ii) injects brief concept snippets during rollouts to elicit concept-primed trajectories, and (iii) reinforces conceptual reasoning via trajectory replacement after group failures, a lightweight forward-KL constraint that aligns unguided with concept-primed policies, or standard GRPO directly on concept-aligned quizzes. Across several models, CORE delivers consistent gains over vanilla and SFT baselines on both in-domain concept-exercise suites and diverse out-of-domain math benchmarks. CORE unifies direct training on concept-aligned quizzes and concept-injected rollouts under outcome regularization. It provides fine-grained conceptual supervision that bridges problem-solving competence and genuine conceptual reasoning, while remaining algorithm- and verifier-agnostic.

## Learning to Edit IFC Building Models: Verifiable Checkers as the Training Signal for Dependency-Aware Editing Agents

DevFeed: [Learning to Edit IFC Building Models: Verifiable Checkers as the Training Signal for Dependency-Aware Editing Agents](<https://devfeed.tech/articles/learning-to-edit-ifc-building-models-verifiable-checkers-as-the-training-signal-for-dependency-aware-editing-agents-140010.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22307>)

Author: Jiaxin Zhang, Yao Luo, Yuze Dan, Yunqin Li, Bowen Wang

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [rlvr](<https://devfeed.tech/topics/rlvr.md>), [grpo](<https://devfeed.tech/topics/grpo.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [AI Development](<https://devfeed.tech/topics/ai-development.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [after](<https://devfeed.tech/tags/after.md>), [agents](<https://devfeed.tech/tags/agents.md>), [all](<https://devfeed.tech/tags/all.md>), [allowed](<https://devfeed.tech/tags/allowed.md>), [also](<https://devfeed.tech/tags/also.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [bim](<https://devfeed.tech/tags/bim.md>), [building](<https://devfeed.tech/tags/building.md>), [changes](<https://devfeed.tech/tags/changes.md>), [classes](<https://devfeed.tech/tags/classes.md>), [code](<https://devfeed.tech/tags/code.md>), [comparisons](<https://devfeed.tech/tags/comparisons.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [dependency](<https://devfeed.tech/tags/dependency.md>), [ifc](<https://devfeed.tech/tags/ifc.md>), [learning](<https://devfeed.tech/tags/learning.md>), [models](<https://devfeed.tech/tags/models.md>), [signal](<https://devfeed.tech/tags/signal.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

IfcGym trains IFC editing agents with verifiable task outcomes: four checkers assess validity, intent, dependency handling and preservation, and all must pass for strict success. IfcEdit first uses supervised fine-tuning on verified teacher episodes, then GRPO with strict success as its reward. On 1,990 IfcEditBench tasks across 13 held-out projects, the Qwen3.5-9B student rises from 79.15% strict success after SFT to 86.78% after GRPO; its paired confidence interval versus the teacher includes zero. The same recipe reaches 91.76% on a 27B model and exceeds the teacher. On IfcEditBench, students beat prompted models except the teacher with matched tool access, while both evaluated frameworks score lower. Using all four checkers as reward yields 4.83 points more strict success than rewarding intent and validity alone. Scores on BIM-Edit and BIBIMBAP are 55.19 and 54.50.

### Source excerpt

arXiv:2611.22307v1 Announce Type: new Abstract: Editing a building information model in the Industry Foundation Classes (IFC) schema requires propagating changes to dependent elements while preserving unrelated entities. Existing language-model agents for IFC editing use prompted models without learning from edit outcomes. Target-file comparisons do not specify which alternative edits are allowed or which unrelated entities must remain unchanged. We present IfcGym, a CodeAct environment in which a language model edits IFC through executable code and four checkers verify validity relative to the input model, intent, dependency and preservation against a task specification. Their conjunction is strict success. IfcEdit distils verified teacher episodes into a Qwen3.5-9B student through supervised fine-tuning (SFT), then applies Group Relative Policy Optimization (GRPO) with strict success as the reward. On IfcEditBench, 1,990 editing tasks on 13 held-out real projects, the final student reaches 86.78% strict success, from 79.15% after SFT. The paired confidence interval for the difference from its teacher includes zero; the same recipe on a 27B model reaches 91.76% and exceeds the teacher. On the public benchmarks BIM-Edit and BIBIMBAP the final student scores 55.19 and 54.50. On IfcEditBench, the students outperform all prompted models except the teacher under matched tool access; both evaluated frameworks also score lower. Rewarding all four checkers yields 4.83 points more strict success than rewarding intent and validity alone.

## Optimization-like Reasoning in LLMs over Expanding Search Spaces

DevFeed: [Optimization-like Reasoning in LLMs over Expanding Search Spaces](<https://devfeed.tech/articles/optimization-like-reasoning-in-llms-over-expanding-search-spaces-139778.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2606.05464>)

Author: Nicol\\'as Astorga, Nabeel Seedat, Mihaela van der Schaar

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Combinatorial optimization](<https://devfeed.tech/topics/combinatorial-optimization.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [rlvr](<https://devfeed.tech/topics/rlvr.md>), [Operations research and optimization](<https://devfeed.tech/topics/operations-research-and-optimization.md>)

Tags: [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [llms](<https://devfeed.tech/tags/llms.md>), [offline](<https://devfeed.tech/tags/offline.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [search](<https://devfeed.tech/tags/search.md>), [spaces](<https://devfeed.tech/tags/spaces.md>), [tasks](<https://devfeed.tech/tags/tasks.md>)

### AI overview

OPT* is a scalable family of optimization tasks for training and evaluating LLMs that must build high-value feasible plans among many valid alternatives. A feasibility checker and evaluator support verification, while a complexity parameter expands the search space without requiring new human labels. The work studies solver-guided online policy optimization, where a solver scores partial states and rank-based reward shaping encourages better next steps, and search-based offline RL for cases without such solvers. The authors relate large-search-space success to information gained per unit of search budget, ablate components for efficient search, and report that OPT* training improves step-by-step optimization-like reasoning.

### Source excerpt

arXiv:2606.05464v2 Announce Type: replace Abstract: Verifiable reward training has improved mathematical and coding reasoning, but these domains capture only part of step-by-step decision making. Many real-world tasks require finding a high-value feasible plan among many valid alternatives. We introduce OPT*, a scalable family of optimization-style tasks for training and evaluating LLM step-by-step optimization-like reasoning along a complexity axis: each task provides a feasibility checker and evaluator, while a complexity parameter expands the search space without requiring new human labels. This motivates studying these tasks in two regimes: (i) solver-guided online policy optimization, which uses a solver as a value oracle for partial states and applies rank-based reward shaping to reinforce better next steps, and (ii) search-based offline RL when such solvers are unavailable. Theoretically, we relate success in large search spaces to the information a reasoner extracts per unit of search budget. Empirically, we ablate the ingredients that make search efficient on OPT* and show that training on OPT* improves step-by-step optimization-like reasoning.

## Video-HopChain: Multi-Hop Questions and Confidence-Gated Exploration for Video Reasoning Models

DevFeed: [Video-HopChain: Multi-Hop Questions and Confidence-Gated Exploration for Video Reasoning Models](<https://devfeed.tech/articles/video-hopchain-multi-hop-questions-and-confidence-gated-exploration-for-video-reasoning-models-139819.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.25773>)

Author: Trung Nguyen Quang, Yuhao Dong, Shuo Sun, Shuai Liu, Shulin Tian, Kim-Hui Yap, Ziwei Liu

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [grpo](<https://devfeed.tech/topics/grpo.md>), [rlvr](<https://devfeed.tech/topics/rlvr.md>)

Tags: [benchmark](<https://devfeed.tech/tags/benchmark.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [budget](<https://devfeed.tech/tags/budget.md>), [chain-of-thought](<https://devfeed.tech/tags/chain-of-thought.md>), [checkpoint](<https://devfeed.tech/tags/checkpoint.md>), [code](<https://devfeed.tech/tags/code.md>), [compute](<https://devfeed.tech/tags/compute.md>), [confidence](<https://devfeed.tech/tags/confidence.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [data](<https://devfeed.tech/tags/data.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [errors](<https://devfeed.tech/tags/errors.md>), [exploration](<https://devfeed.tech/tags/exploration.md>), [framework](<https://devfeed.tech/tags/framework.md>), [generation](<https://devfeed.tech/tags/generation.md>), [gradient](<https://devfeed.tech/tags/gradient.md>), [models](<https://devfeed.tech/tags/models.md>), [questions](<https://devfeed.tech/tags/questions.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [reasoning-models](<https://devfeed.tech/tags/reasoning-models.md>), [video](<https://devfeed.tech/tags/video.md>)

### AI overview

Video-HopChain uses multi-step questions about moments in a video to train video reasoning models with verifiable rewards. Its 22,550 questions span 13,378 videos, with a held-out benchmark of 1,000; each chains three to six yes/no questions and scores the summed integer answers by exact match. For Qwen3-VL-8B, a second GRPO training stage on this dataset raises the mean across eight benchmarks from 55.4 to 57.9, improving all eight. The authors address GRPO groups with no reward variation using Confidence-Gated Exploration: after four initial rollouts are all correct or all incorrect, four more are sampled with the policy's most confident reasoning token masked, and those masked positions are excluded from the loss. At the same compute budget, CGE raises the mean to 59.3. The abstract says the dataset, checkpoint, and generation and training code are released.

### Source excerpt

arXiv:2609.25773v3 Announce Type: replace-cross Abstract: HopChain has shown on still images that multi-hop data synthesis improves vision-language reasoning, because long chain-of-thought reasoning exposes errors that compound across steps, while most data used for reinforcement learning with verifiable rewards (RLVR) rarely demands a chain of visual evidence, so these weaknesses are likely to stay unexposed. We observe the same problem in video, where this framework has not yet been explored. We therefore build Video-HopChain, a dataset of 22,550 multi-hop video questions over 13,378 videos, together with a held-out benchmark of 1,000 questions. Each question chains three to six yes/no questions about moments in one video, and each yields one of two integers depending on its answer. The final answer is the sum of these integers, so an exact match on that sum gives the verifiable reward that RLVR needs. We first train Qwen3-VL-8B with GRPO on a standard video dataset, and a second stage on Video-HopChain then raises the mean over eight video understanding and reasoning benchmarks from 55.4 to 57.9 and improves every one of them. Training on such a dataset, however, exposes a known limitation of GRPO: its learning signal comes from the reward variance within a group, so hard questions whose rollouts are all incorrect and easy questions whose rollouts are all correct both leave the group with no gradient. To recover these groups at the same compute budget, we introduce Confidence-Gated Exploration (CGE). With 8 rollouts per question, CGE samples the first 4 as usual. If these 4 are either all correct or all incorrect, it samples the last 4 with the policy's most confident token masked inside the reasoning span, and removes the masked positions from the loss while all 8 rollouts enter the advantage. With CGE, the mean rises further to 59.3. We release the dataset, the checkpoint, and the data generation and training code.

## CoreWeave connects custom training loops to model serving

DevFeed: [CoreWeave connects custom training loops to model serving](<https://devfeed.tech/articles/coreweave-connects-custom-training-loops-to-model-serving-138246.md>)

Original publisher: [Read original article](<https://thenewstack.io/coreweave-forge-training-loops/>)

Author: Urvashi Chowdhary

Published: 2026-11-17T16:30:00Z

Content type: article

Language: en

Sources: [The New Stack](<https://devfeed.tech/sources/the-new-stack.md>)

Topics: [post-training](<https://devfeed.tech/topics/post-training.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Post-training optimization](<https://devfeed.tech/topics/post-training-optimization.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [rlvr](<https://devfeed.tech/topics/rlvr.md>), [distributed-training](<https://devfeed.tech/topics/distributed-training.md>), [model-deployment](<https://devfeed.tech/topics/model-deployment.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>)

Tags: [ai-engineering](<https://devfeed.tech/tags/ai-engineering.md>), [ai-infrastructure](<https://devfeed.tech/tags/ai-infrastructure.md>), [ai-operations](<https://devfeed.tech/tags/ai-operations.md>), [api](<https://devfeed.tech/tags/api.md>), [coreweave](<https://devfeed.tech/tags/coreweave.md>), [coreweave-forge](<https://devfeed.tech/tags/coreweave-forge.md>), [distillation](<https://devfeed.tech/tags/distillation.md>), [model-serving](<https://devfeed.tech/tags/model-serving.md>), [post-contributed](<https://devfeed.tech/tags/post-contributed.md>), [sponsor-coreweave](<https://devfeed.tech/tags/sponsor-coreweave.md>), [sponsored-post-contributed](<https://devfeed.tech/tags/sponsored-post-contributed.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

CoreWeave Forge connects production inference, model training, and serving so teams can turn production failures into evaluated model updates. The excerpt distinguishes distillation for reproducing a stronger model's behavior in a smaller one, supervised fine-tuning when good examples are available, and reinforcement learning when outcomes can be scored but ideal responses are hard to specify; it cautions that weight changes will not fix retrieval, prompts, tools, or serving problems. Model Distillation offers a managed workflow from captured traffic through relabeling and candidate comparison. A programmable Python API lets teams control training logic while CoreWeave manages infrastructure, and RL Rollouts hot-loads checkpoints into live inference while preserving requests and returning training metadata. In a reported Nemotron web-search run with you.com, BrowseComp accuracy rose from 36.97% to 45.45%, average tool calls fell 30.24%, and hot-loading was about 15 times faster than traditional redeployment. The programmable API was in limited preview, with launch planned for later in 2026.

### Source excerpt

Production traffic shows failures that benchmarks or pre-deployment evaluations can't fully reveal. It tells you which edge cases matter and The post CoreWeave connects custom training loops to model serving appeared first on The New Stack.

## Where Should LLM Agents Adapt at Test Time? In Context, in Weights, or Both

DevFeed: [Where Should LLM Agents Adapt at Test Time? In Context, in Weights, or Both](<https://devfeed.tech/articles/where-should-llm-agents-adapt-at-test-time-in-context-in-weights-or-both-137485.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.19700>)

Author: Parsa Mirtaheri, Mohammadreza Armandpour, Keivan Alizadeh, Iman Mirzadeh, Parshin Shojaee, Aryo Lotfi, David Harrison, Mehrdad Farajtabar

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [rlvr](<https://devfeed.tech/topics/rlvr.md>), [AI Context Management](<https://devfeed.tech/topics/ai-context-management.md>), [grpo](<https://devfeed.tech/topics/grpo.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [adapting](<https://devfeed.tech/tags/adapting.md>), [agents](<https://devfeed.tech/tags/agents.md>), [context](<https://devfeed.tech/tags/context.md>), [context-management](<https://devfeed.tech/tags/context-management.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [grpo](<https://devfeed.tech/tags/grpo.md>), [knowledge](<https://devfeed.tech/tags/knowledge.md>), [learning](<https://devfeed.tech/tags/learning.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-agents](<https://devfeed.tech/tags/llm-agents.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [repository](<https://devfeed.tech/tags/repository.md>), [reuse](<https://devfeed.tech/tags/reuse.md>), [structure](<https://devfeed.tech/tags/structure.md>), [swe-bench](<https://devfeed.tech/tags/swe-bench.md>), [tasks](<https://devfeed.tech/tags/tasks.md>), [test](<https://devfeed.tech/tags/test.md>), [time](<https://devfeed.tech/tags/time.md>), [updating](<https://devfeed.tech/tags/updating.md>), [verification](<https://devfeed.tech/tags/verification.md>), [weight](<https://devfeed.tech/tags/weight.md>)

### AI overview

Across the evaluated task streams, retaining experience in context (ICL) produced the largest standalone improvement in agent success, while adding reinforcement learning with verifiable rewards (GRPO) improved it further. Using Qwen3.5-9B on 109 tasks across four environments and five task orders per environment, the best ICL policy raised mean success from 28.8% for the stateless baseline to 49.4%; ICL plus GRPO reached 51.6%. Self-distillation (SDPO) had mixed effects. Trace analyses suggest ICL reduced environment lookups and shortened attempts, but agents often skipped inspection and verification of the current task. Adding GRPO helped restore those behaviors while retaining fewer lookups. The comparison uses an online protocol in which agents are evaluated before adapting and learn from their own attempts and environment feedback; findings are limited to the tested model, tasks, and environments.

### Source excerpt

arXiv:2611.19700v1 Announce Type: cross Abstract: LLM agents operating repeatedly in the same environment, such as a code repository, a database, or an application suite, encounter tasks that share latent structure. Experience can reveal this structure, allowing agents to reuse environment knowledge and learn from environmental feedback. Agents can adapt at test-time using this experience through two channels: retaining it in context and updating their weights. These channels are usually studied separately, leaving unclear what each contributes and when they complement each other. We compare in-context learning (ICL), reinforcement learning with verifiable rewards using GRPO, and self-distillation using SDPO, alone and in combination, using a shared online protocol and modular framework. Agents encounter matched task streams, are evaluated on each task before adapting to it, and learn only from their own attempts and environment feedback. We evaluate 14 adaptation mechanisms and a stateless baseline with Qwen3.5-9B on 109 tasks from four environments drawn from SWE-bench Verified, AppWorld, ScienceWorld, and BIRD-Critic-SQLite, using five task orders per environment. ICL provides the largest standalone gain: its best policy raises mean success across environments from 28.8% to 49.4%. Adding GRPO to this policy achieves the highest mean success, 51.6%. While adding GRPO improves the overall mean under every ICL policy, SDPO has mixed effects. Trace analyses show fewer lookups and shorter attempts under ICL, consistent with reusing knowledge of the shared structure. However, ICL also frequently skips inspection and verification for the current task. Adding GRPO to ICL helps restore these behaviors while preserving its reduced environment lookups. These observations suggest that reinforcement learning with verifiable rewards can refine how agents use retained experience, motivating the joint study of context management and weight updates.

## SpeCal: Spectral Calibration for Unsupervised RLVR

DevFeed: [SpeCal: Spectral Calibration for Unsupervised RLVR](<https://devfeed.tech/articles/specal-spectral-calibration-for-unsupervised-rlvr-137603.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.20691>)

Author: Kai Tang, Xing Zheng, Haobo Wang, Guangcheng Zhu, Shenzhi Yang, Zhengqing Zang, Yingfan Ma, Cheng Peng, Bowen Song, Weiqiang Wang

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [rlvr](<https://devfeed.tech/topics/rlvr.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [conditional](<https://devfeed.tech/tags/conditional.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [effective](<https://devfeed.tech/tags/effective.md>), [entropy](<https://devfeed.tech/tags/entropy.md>), [framework](<https://devfeed.tech/tags/framework.md>), [gaussian](<https://devfeed.tech/tags/gaussian.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [reliability](<https://devfeed.tech/tags/reliability.md>), [reward](<https://devfeed.tech/tags/reward.md>), [rlvr](<https://devfeed.tech/tags/rlvr.md>), [state](<https://devfeed.tech/tags/state.md>), [structure](<https://devfeed.tech/tags/structure.md>), [support](<https://devfeed.tech/tags/support.md>)

### AI overview

SpeCal calibrates the advantages used in unsupervised RLVR by assessing rollout reliability from hidden-state spectra, while leaving the underlying unlabeled reward unchanged. At the group level, it combines spectral mass with a Gaussian band-pass gate on normalized entropy-based effective rank to favor groups with substantial representation variation and effective dimensionality in a preferred range. At the individual rollout level, that group rank sets the number of dominant directions retained in a leave-one-out subspace built from the other rollouts; SpeCal measures the held-out rollout's conditional support there. These group and rollout signals adjust reward-induced advantages, complementing decoded outcomes and reward statistics.

### Source excerpt

arXiv:2611.20691v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) has substantially improved the reasoning abilities of large language models, but its dependence on ground-truth answers or external verifiers limits its ability to exploit abundant unlabeled reasoning data. Unsupervised RLVR alleviates this bottleneck by deriving scalar rewards from the model's own rollouts. However, the reliability of such self-generated supervision varies substantially across rollout groups and individual responses, while existing methods construct or refine their rewards mainly from decoded outcomes and reward statistics. These endpoint signals reveal what answers the model produces, but provide limited information about whether the underlying rollout representations form a reliable structure for learning. Our key observation is that hidden-state spectra offer a complementary view of supervision reliability by characterizing both the organization of a rollout group and the support received by each rollout within that group. Based on this observation, we propose SpeCal, a bi-level spectral calibration framework for unsupervised RLVR. At the group level, SpeCal combines spectral mass with a Gaussian band-pass gate over normalized entropy-based effective rank, favoring rollout groups with substantial representation variation whose effective dimensionality lies in a preferred regime. At the rollout level, the same group effective rank determines how many dominant directions are retained in a leave-one-out spectral subspace formed by the remaining rollouts, within which the conditional support of each held-out rollout is measured. The resulting signals calibrate the original reward-induced advantages at the group and rollout levels without redefining the underlying unlabeled reward.

## Policy Optimization With Multiple Verifiers via Pareto Rankings

DevFeed: [Policy Optimization With Multiple Verifiers via Pareto Rankings](<https://devfeed.tech/articles/policy-optimization-with-multiple-verifiers-via-pareto-rankings-137387.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.19064>)

Author: Kai Yamashita, Shohei Taniguchi, Hiroki Furuta, Masahiro Suzuki, Yutaka Matsuo

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [rlvr](<https://devfeed.tech/topics/rlvr.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [dpo](<https://devfeed.tech/topics/dpo.md>)

Tags: [code-generation](<https://devfeed.tech/tags/code-generation.md>), [comparisons](<https://devfeed.tech/tags/comparisons.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [direct-preference-optimization](<https://devfeed.tech/tags/direct-preference-optimization.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [feedback](<https://devfeed.tech/tags/feedback.md>), [improvements](<https://devfeed.tech/tags/improvements.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [learning](<https://devfeed.tech/tags/learning.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [policy](<https://devfeed.tech/tags/policy.md>), [probability](<https://devfeed.tech/tags/probability.md>), [rankings](<https://devfeed.tech/tags/rankings.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [reward](<https://devfeed.tech/tags/reward.md>), [rlvr](<https://devfeed.tech/tags/rlvr.md>), [supervision](<https://devfeed.tech/tags/supervision.md>), [tasks](<https://devfeed.tech/tags/tasks.md>), [verification](<https://devfeed.tech/tags/verification.md>)

### AI overview

POLPO trains language models from multiple verifier outcomes without collapsing them into a scalar reward. It uses Pareto dominance to order responses that satisfy additional requirements while preserving already satisfied ones, leaving conflicting responses incomparable. Its objective maximizes the probability of complete rankings consistent with these partial orders, extending the listwise likelihood underlying Direct Preference Optimization. The abstract reports improved learning efficiency and higher complete-requirement satisfaction than scalar-reward training on code generation and instruction-following experiments.

### Source excerpt

arXiv:2611.19064v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) has become an important approach to improving the reasoning capabilities of large language models. In tasks such as code generation and instruction following, multiple verifiers evaluate whether a response satisfies different requirements. Their outcomes provide detailed feedback about partial success, yet this feedback is commonly aggregated into a scalar reward for policy optimization. Rewarding only complete success discards distinctions among partial solutions, while counts and weighted sums impose comparisons between responses that satisfy different subsets of requirements. We introduce Partial-Order Likelihood Policy Optimization (POLPO), which learns from the comparisons established by verification without first constructing an aggregate reward. Pareto dominance identifies responses that satisfy additional requirements without losing any already satisfied, while leaving responses with conflicting outcomes incomparable. POLPO extends the listwise likelihood underlying Direct Preference Optimization by maximizing the total probability of all complete rankings consistent with these partial orders. This objective provides supervision from verified improvements even before complete solutions are available, without requiring a prescribed ordering of incomparable responses. Experiments on code generation and instruction following demonstrate improved learning efficiency and higher rates of complete requirement satisfaction compared with scalar-reward training. These results support partial-order likelihood as an effective approach to learning from multiple verifiers.

## Optimistic Verifiers Get Hacked: Reward Hacking Under an Exact Process Oracle

DevFeed: [Optimistic Verifiers Get Hacked: Reward Hacking Under an Exact Process Oracle](<https://devfeed.tech/articles/optimistic-verifiers-get-hacked-reward-hacking-under-an-exact-process-oracle-137713.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.21538>)

Author: Arunkumar V, Manoranjan Gandhudi, Gangadharan G. R., Arun Prakash, S. Senthilkumar

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [rlvr](<https://devfeed.tech/topics/rlvr.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>), [audit](<https://devfeed.tech/topics/audit.md>)

Tags: [audit](<https://devfeed.tech/tags/audit.md>), [authorization](<https://devfeed.tech/tags/authorization.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [hacked](<https://devfeed.tech/tags/hacked.md>), [hacking](<https://devfeed.tech/tags/hacking.md>), [instrumentation](<https://devfeed.tech/tags/instrumentation.md>), [llm](<https://devfeed.tech/tags/llm.md>), [magnitude](<https://devfeed.tech/tags/magnitude.md>), [noise](<https://devfeed.tech/tags/noise.md>), [opa](<https://devfeed.tech/tags/opa.md>), [oracle](<https://devfeed.tech/tags/oracle.md>), [padding](<https://devfeed.tech/tags/padding.md>), [policies](<https://devfeed.tech/tags/policies.md>), [process](<https://devfeed.tech/tags/process.md>), [proxy](<https://devfeed.tech/tags/proxy.md>), [ranking](<https://devfeed.tech/tags/ranking.md>), [recall](<https://devfeed.tech/tags/recall.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [reward](<https://devfeed.tech/tags/reward.md>), [testbed](<https://devfeed.tech/tags/testbed.md>), [train](<https://devfeed.tech/tags/train.md>)

### AI overview

The study tests reward hacking in verifiable-reward training using executable authorization policies across 124,416 contexts, where an oracle can identify every minimal sufficient reason for each decision. Training against seven degraded verifiers revealed four patterns: optimistic sufficiency proxies encouraged padded explanations, raising proxy scores to 0.940 while held-out true faithfulness fell from 0.731 to 0.667; outcome-only verifiers saturated, extraction noise diluted the signal, and ranking-preserving gaps caused no harm. A model-free audit of selection error and gap sign, plus a short saturation pilot, predicted the observed pattern. The patterns recurred across nine configurations; sufficiency damage was positive in all 10 replication cells and exceeded two standard errors in seven. However, its magnitude did not replicate. The authors release the testbed, training instrumentation, and 14 public policies converted to executable oracles.

### Source excerpt

arXiv:2611.21538v1 Announce Type: cross Abstract: Reinforcement learning from verifiable rewards inherits whatever its verifier gets wrong. Outside mathematics and code the verifier is a proxy, a rubric, a learned judge or a hand-written checker, and its distance from the true objective is never measured, so a rising reward cannot be told apart from a hack. We build a setting where the true reward is exactly computable at the process level: executable authorization policies over 124,416 contexts admit an oracle that lists every minimal sufficient reason for every decision. We compile these policies into exact process rewards, degrade the verifier along seven realistic axes (outcome-only, sufficiency, recall-only, extraction noise, coarse taxonomy, stale policy, LLM judge), and train against each while the exact verifier scores every rollout. Four phenotypes appear. Optimistic verifiers are hacked by padding: the sufficiency proxy ends at 0.940 while true faithfulness falls from 0.731 to 0.667 on 96 held-out episodes. Outcome-only verifiers saturate and stop teaching, noise dilutes, and ranking-preserving gaps do no harm. A model-free audit of the verifier, its selection error and gap sign, together with a short saturation pilot, anticipates which phenotype a run will produce. Across nine configurations spanning two frameworks, two model vendors, two adapter sizes, three precisions, two initialisations and two seeds, the phenotypes recur, sufficiency damage is positive in all 10 replication cells and beyond two standard errors in 7, and optimistic gaps remain the steeper branch. The damage magnitude does not replicate, and five pre-registered attempts to explain it are reported as they fell. We release the testbed, the training instrumentation, and 14 real public policies (Cedar, OPA, Gatekeeper) ported to executable oracles.

## Finding Side-Effect Loopholes in AppWorld and WorkArena Verifiers

DevFeed: [Finding Side-Effect Loopholes in AppWorld and WorkArena Verifiers](<https://devfeed.tech/articles/finding-side-effect-loopholes-in-appworld-and-workarena-verifiers-137293.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.09142>)

Author: Richard Abrich

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [mutation-testing](<https://devfeed.tech/topics/mutation-testing.md>), [rlvr](<https://devfeed.tech/topics/rlvr.md>), [AI Bots](<https://devfeed.tech/topics/ai-bots.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>)

Tags: [agents](<https://devfeed.tech/tags/agents.md>), [automated](<https://devfeed.tech/tags/automated.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [change](<https://devfeed.tech/tags/change.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [false-positives](<https://devfeed.tech/tags/false-positives.md>), [fields](<https://devfeed.tech/tags/fields.md>), [mutation](<https://devfeed.tech/tags/mutation.md>), [records](<https://devfeed.tech/tags/records.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [reward](<https://devfeed.tech/tags/reward.md>), [tasks](<https://devfeed.tech/tags/tasks.md>), [tests](<https://devfeed.tech/tags/tests.md>)

### AI overview

Mutation tests found that AppWorld and WorkArena verifiers can reward unrequested side effects even when agents complete the requested task. AppWorld accepted duplicate writes in all six tasks across two templates; adding one count assertion per template rejected the duplicates while preserving all correct runs. Three RL methods training on one template would fully reward the duplicate. WorkArena's verifier, called once after submission, accepted all 36 edited forms with an extra field. Checking after each field caught extras entered first or through drop-downs, but still missed all eight cases where a text or reference field was filled last. The tests found no false positives from planted omissions or incorrect values. The results indicate that checking requested changes alone does not reliably ensure other state remains untouched.

### Source excerpt

arXiv:2610.09142v3 Announce Type: replace Abstract: Language-model agents act for people inside software, sending payments or editing records. Benchmarks such as AppWorld and WorkArena grade them with automated verifiers, whose verdicts also become reinforcement learning (RL) rewards, so a lenient verifier rewards mistakes. Verifier audits mostly hunt for runs that pass without doing the task; we ask about runs that do it and also change something nobody asked for. We mutation-test both verifiers with hand-written edits to correct runs, designed from the verifier source; an end-state check confirms each false positive. All confirmed false positives were unrequested changes; planted omissions and wrong values produced none. AppWorld's tests passed a duplicate write in all six tasks of two templates, and three RL methods that train on one of these templates would fully reward it; one count assertion per template rejected all six duplicates while passing every correct run. WorkArena's form verifier, called once after submission, passed all 36 edited runs that filled an unrequested field. Checked after every field, it caught extra fields filled first or in drop-downs but still passed all 8 runs that filled a text or reference field last. Both verifiers check that the requested change happened but not reliably that nothing else changed.

## Let Execution Do the Teaching

DevFeed: [Let Execution Do the Teaching](<https://devfeed.tech/articles/let-execution-do-the-teaching-137508.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.19875>)

Author: Abhishek Mishra, Sandeep Srinivasa, Arko Chattopadhyay, Enrique Martinho Ferrao, Guda Pranav Reddy

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [rlvr](<https://devfeed.tech/topics/rlvr.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [grpo](<https://devfeed.tech/topics/grpo.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Test automation](<https://devfeed.tech/topics/test-automation.md>)

Tags: [c-plus-plus](<https://devfeed.tech/tags/c-plus-plus.md>), [coding](<https://devfeed.tech/tags/coding.md>), [coding-tasks](<https://devfeed.tech/tags/coding-tasks.md>), [compiler](<https://devfeed.tech/tags/compiler.md>), [contracts](<https://devfeed.tech/tags/contracts.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [execution](<https://devfeed.tech/tags/execution.md>), [feedback](<https://devfeed.tech/tags/feedback.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [glm](<https://devfeed.tech/tags/glm.md>), [program](<https://devfeed.tech/tags/program.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [repair](<https://devfeed.tech/tags/repair.md>), [structured](<https://devfeed.tech/tags/structured.md>), [test](<https://devfeed.tech/tags/test.md>), [tests](<https://devfeed.tech/tags/tests.md>), [training](<https://devfeed.tech/tags/training.md>), [workflow](<https://devfeed.tech/tags/workflow.md>)

### AI overview

Starting from GLM-4.7-Flash, the study trains coding models using whole-file supervised fine-tuning, targeted C++ repair practice based on compiler errors and failing tests, and group-relative reinforcement learning with executable verifiers that convert program behavior into rewards. On 26 C++ tasks, the strongest reported configuration reached 45.2% success in the first test round and 61.5% within two rounds. The results are from an assisted development suite with evolving prompt contracts; that configuration used four selected evaluation trials and included six tasks also used in training.

### Source excerpt

arXiv:2611.19875v1 Announce Type: cross Abstract: Improving a coding model is a sequence of decisions: find the limiting behavior, create useful practice, and determine which attempts deserve reinforcement. We investigate a practical way to make those decisions increasingly repeatable by grounding feedback in execution. Starting from GLM-4.7-Flash, the work combines whole-file supervised fine-tuning, targeted C++ repair curricula, group-relative reinforcement learning, and verifiers that turn program behavior into structured rewards. Compiler errors and failing tests provide specific contexts for repair practice, while executable checks score the resulting programs. We evaluate the trained models on 26 C++ coding tasks, measuring both initial success and completion after test feedback. The strongest reported post-training configuration achieves 45.2% success in the first test round and 61.5% within two rounds. These results come from an assisted development suite with evolving prompt contracts; the strongest configuration uses four selected evaluation trials and includes six tasks also used in training. The study describes an execution-guided workflow for improving coding models, including how repair tasks are constructed, how program behavior determines rewards, and how the resulting models are evaluated.

## Procedure without Judgment: What Small Language Models Learn under an Exact Verifier

DevFeed: [Procedure without Judgment: What Small Language Models Learn under an Exact Verifier](<https://devfeed.tech/articles/procedure-without-judgment-what-small-language-models-learn-under-an-exact-verifier-137712.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.21529>)

Author: Arunkumar V, Manoranjan Gandhudi, Gangadharan G. R., Arun Prakash, S. Senthilkumar

Published: 2026-11-17T05:00:00Z

Content type: news

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [rlvr](<https://devfeed.tech/topics/rlvr.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>)

Tags: [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [frontier-models](<https://devfeed.tech/tags/frontier-models.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [small-language-models](<https://devfeed.tech/tags/small-language-models.md>)

### AI overview

An exact verifier makes it possible to score whether an explanation names only the facts sufficient to block an action. In the described task, the explainer never sees the policy and gets 30 turns to query it. Frontier models find exact reasons about half the time with tools alone; a prompt to shrink the answer raises Haiku 4.5 from 58% to 75%. A 4B model fine-tuned on 2,000 deletion-algorithm traces reaches 92% on its training policy, outperforming frontier-taught students across all three reported seeds. But it appears to learn a fixed checking schedule: on three unseen public policies, its answers remain sufficient 94% of the time but are minimal only 48% of the time, while Haiku with the same prompt is exact in all 47 episodes. Reinforcement learning from the exact reward does not improve held-out accuracy and shifts behavior from policy probing to repeated sufficiency checks. The results show that a perfect verifier clarifies what training teaches, but does not ensure transferable learning.

### Source excerpt

arXiv:2611.21529v1 Announce Type: new Abstract: When an authorization policy blocks an agent's tool call, the audit log should name the facts that forced the decision and nothing more. For an executable policy over an enumerable context space this minimal sufficient reason can be computed exactly, so an explanation is scored against ground truth rather than a proxy for it. We use that exactness to ask what a model can learn from a verifier that has no error in it. The explainer sees one request and one decision, never the policy, and has thirty turns in which it may call the policy on altered requests and ask whether a candidate set of facts is sufficient. Frontier models find the exact reason about half the time with the tools alone, and one sentence telling them to shrink the answer lifts Haiku 4.5 from 58% to 75%. A 4B model fine-tuned on 2000 traces of the deletion algorithm reaches 92% on the policy it was trained on. Teacher kind is what moves this: at matched data size the algorithm-taught student leads the frontier-taught one in all three training seeds, and the two arms never overlap. What the student learns is a fixed schedule of sufficiency checks, and the schedule does not travel. On three public policies it never saw, its answers stay sufficient (94%) and stop being minimal (48%), while Haiku under the same sentence is exact on all 47 episodes. Reinforcement learning from the exact reward leaves held-out accuracy where it was and inverts the strategy, from probing the policy to spending the entire budget on sufficiency checks. The oracle made the effect of training legible, but it did not make training better.

## Post-Training NVIDIA Nemotron to Increase Accuracy in Your Scientific Domain

DevFeed: [Post-Training NVIDIA Nemotron to Increase Accuracy in Your Scientific Domain](<https://devfeed.tech/articles/post-training-nvidia-nemotron-to-increase-accuracy-in-your-scientific-domain-136679.md>)

Original publisher: [Read original article](<https://developer.nvidia.com/blog/post-training-nvidia-nemotron-to-increase-accuracy-in-your-scientific-domain/>)

Author: Gautham Dharuman

Published: 2026-11-16T16:00:00Z

Content type: tutorial

Language: en

Sources: [NVIDIA Developer](<https://devfeed.tech/sources/nvidia-developer.md>), [NVIDIA Technical Blog](<https://devfeed.tech/sources/nvidia-technical-blog.md>)

Topics: [post-training](<https://devfeed.tech/topics/post-training.md>), [Nemotron](<https://devfeed.tech/topics/nemotron.md>), [rlvr](<https://devfeed.tech/topics/rlvr.md>), [grpo](<https://devfeed.tech/topics/grpo.md>), [agentic workflows](<https://devfeed.tech/topics/agentic-workflows.md>), [AI for science](<https://devfeed.tech/topics/ai-for-science.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [NeMo](<https://devfeed.tech/topics/nemo.md>)

Tags: [17](<https://devfeed.tech/tags/17.md>), [adapter](<https://devfeed.tech/tags/adapter.md>), [adapters](<https://devfeed.tech/tags/adapters.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [agentic-ai-generative-ai](<https://devfeed.tech/tags/agentic-ai-generative-ai.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-models](<https://devfeed.tech/tags/ai-models.md>), [autonomous](<https://devfeed.tech/tags/autonomous.md>), [autonomous-agents](<https://devfeed.tech/tags/autonomous-agents.md>), [computational-chemistry-materials-science](<https://devfeed.tech/tags/computational-chemistry-materials-science.md>), [developer-tools-techniques](<https://devfeed.tech/tags/developer-tools-techniques.md>), [featured](<https://devfeed.tech/tags/featured.md>), [innovation](<https://devfeed.tech/tags/innovation.md>), [machine-learning-artificial-intelligence](<https://devfeed.tech/tags/machine-learning-artificial-intelligence.md>), [natural-language](<https://devfeed.tech/tags/natural-language.md>), [nemo](<https://devfeed.tech/tags/nemo.md>), [nemotron](<https://devfeed.tech/tags/nemotron.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [nvidia-nemotron](<https://devfeed.tech/tags/nvidia-nemotron.md>), [post-training](<https://devfeed.tech/tags/post-training.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [research](<https://devfeed.tech/tags/research.md>), [sc26](<https://devfeed.tech/tags/sc26.md>), [scientific-computing](<https://devfeed.tech/tags/scientific-computing.md>), [scientific-research](<https://devfeed.tech/tags/scientific-research.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

The excerpt describes adapting Nemotron 3.5 Super VL to scientific tasks with reinforcement learning from verifiable rewards. Experimental records become prompts paired with reference outcomes; a task-specific verifier scores model submissions, so the reward comes from data without human preference labels. In the drug-metabolism example, Morgan fingerprints and Tanimoto similarity score predicted metabolites. The recipe uses NeMo Gym for the environment and NeMo RL for GRPO training with LoRA. It reports SFT improved accuracy by 12%, with RL adding 17% at a 16,384-token output budget. An optional warm start distills teacher traces that meet a similarity threshold; the excerpt reports retaining 1,123 of 1,934 prompts at a threshold of at least 0.7. It emphasizes choosing a reward that reflects the scientific outcome and evaluating on held-out data. These figures and implementation details are specific to the described example.

### Source excerpt

Agentic AI is reshaping scientific research and innovation. Researchers communicate complex scientific hypotheses in natural language and let autonomous agents...

## Benchmarks Don't Talk Back: Learning Through Self-Disagreement

DevFeed: [Benchmarks Don't Talk Back: Learning Through Self-Disagreement](<https://devfeed.tech/articles/benchmarks-don-t-talk-back-learning-through-self-disagreement-135080.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.17563>)

Author: Soheil Abbasloo, Peng Cheng

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [rlvr](<https://devfeed.tech/topics/rlvr.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Ground truth / benchmark quality](<https://devfeed.tech/topics/ground-truth-benchmark-quality.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

The paper presents dialectical competence--the ability to hold a correct answer against faulty counterarguments while yielding to stronger ones--as distinct from first-answer accuracy. It proposes Self-Disagreement training: a model states a position, generates the strongest opposing case, judges whether it succeeds, and receives reinforcement learning from verifiable rewards. The same reference-based check scores both the answer and the judgment, with no reward model, preference data, human annotation, or second model. The abstract reports that this capability is weak in the base model, can improve without sacrificing first-answer accuracy, and transfers to unseen challengers, including stronger models. It also says accuracy training alone may not improve this competence, and inference-time reflection or additional sampling does not provide it.

### Source excerpt

arXiv:2611.17563v1 Announce Type: cross Abstract: A benchmark usually measures a language model alone with a question, and deployment rarely looks like that: answers get contested, and models give up correct ones under counter-reasoning that is itself wrong. We show that what fails there is not a shortfall of accuracy but a capability in its own right -- knowing when to hold a position and when to yield to a better one -- which we call dialectical competence. It stands on first-answer accuracy without following from it: a model can be made materially better at answering and no better at this, and training for accuracy alone can drift against it. Nor can it be bought at inference, by self-reflection or by sampling more. Taking a cue from the dialectical tradition in philosophy, in which productive opposition arises within a position rather than being imposed on it from outside, we hypothesize that this capability can be trained from opposition the model supplies itself. It states a position, generates the strongest case against that position -- its antithesis -- rules on whether the case stands, and is trained on that ruling, with no second model at training or at deployment. We call this learning through Self-Disagreement: reinforcement learning from verifiable rewards throughout, where the same check that scores an answer against its reference scores the ruling too, so no reward model, no preference data and no human annotation enter anywhere. The capability is trained rather than elicited: weak in the base model, it arrives with the reward, is not paid for out of first-answer accuracy, and holds against challengers the model never trained against, including ones stronger than itself. The opposition a model needs in order to improve, it turns out, need not come from outside it.

## HARDTESTGEN: A High-Quality RL Verifier Generation Pipeline for LLM Algorithmic Coding

DevFeed: [HARDTESTGEN: A High-Quality RL Verifier Generation Pipeline for LLM Algorithmic Coding](<https://devfeed.tech/articles/hardtestgen-a-high-quality-rl-verifier-generation-pipeline-for-llm-algorithmic-coding-134947.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2505.24098>)

Author: Zhongmou He, Yee Man Choi, Kexun Zhang, Jiabao Ji, Junting Zhou, Dejia Xu, Ivan Bercovich, Aidan Zhang, Yixiao Zeng, Lei Li

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [rlvr](<https://devfeed.tech/topics/rlvr.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Data Quality](<https://devfeed.tech/topics/data-quality.md>)

Tags: [coding](<https://devfeed.tech/tags/coding.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [generation](<https://devfeed.tech/tags/generation.md>), [llm](<https://devfeed.tech/tags/llm.md>), [quality](<https://devfeed.tech/tags/quality.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [rl](<https://devfeed.tech/tags/rl.md>), [synthesis](<https://devfeed.tech/tags/synthesis.md>), [synthetic](<https://devfeed.tech/tags/synthetic.md>), [tests](<https://devfeed.tech/tags/tests.md>)

### AI overview

HARDTESTGEN synthesizes high-quality test cases to improve verifiers used for reinforcement learning of LLM-generated algorithmic code, where subtle wrong solutions can evade existing tests. The authors curate HARDTESTS, a dataset of 26.6k problems with synthetic tests. Compared with existing tests, these tests improve precision by 11.22 percentage points--the share of predicted-correct programs that are actually correct. The authors also report improved code-generation performance when HARDTESTS verifier results guide rejection sampling and reinforcement learning. The dataset and synthesis pipeline are open-sourced.

### Source excerpt

arXiv:2505.24098v2 Announce Type: cross Abstract: Verifiers provide important reward signals for reinforcement learning of large language models (LLMs). However, it is challenging to develop or create reliable verifiers, especially for code generation tasks. A well-disguised wrong solution program may only be detected by carefully human-written edge cases that are difficult to synthesize automatically. To address this issue, we propose HARDTESTGEN, an approach to synthesize high-quality test cases for algorithmic coding problems. We curate a comprehensive algorithmic programming dataset HARDTESTS with 26.6k problems and high-quality synthetic tests. Compared with existing tests, HARDTESTGEN tests demonstrate significantly higher accuracy in verifying LLM-generated code (+11.22 percentage points in precision, the percentage of actually correct code within the predicted correct ones). We also show that downstream post-training -- including rejection sampling and reinforcement learning (RL) -- using HARDTESTS verifier results in improved performance of LLM code generation. We open-source our dataset and synthesis pipeline at https://leililab.github.io/HardTests/.

## Training Language Model Agents to Find Vulnerabilities with CTF-Dojo

DevFeed: [Training Language Model Agents to Find Vulnerabilities with CTF-Dojo](<https://devfeed.tech/articles/training-language-model-agents-to-find-vulnerabilities-with-ctf-dojo-135644.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2508.18370>)

Author: Terry Yue Zhuo, Dingmin Wang, Hantian Ding, Varun Kumar, Zijian Wang

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>), [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [rlvr](<https://devfeed.tech/topics/rlvr.md>), [ctf](<https://devfeed.tech/topics/ctf.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>)

Tags: [486](<https://devfeed.tech/tags/486.md>), [absolute](<https://devfeed.tech/tags/absolute.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agents](<https://devfeed.tech/tags/agents.md>), [artifacts](<https://devfeed.tech/tags/artifacts.md>), [automated](<https://devfeed.tech/tags/automated.md>), [bench](<https://devfeed.tech/tags/bench.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [best](<https://devfeed.tech/tags/best.md>), [capture](<https://devfeed.tech/tags/capture.md>), [capture-the-flag](<https://devfeed.tech/tags/capture-the-flag.md>), [challenges](<https://devfeed.tech/tags/challenges.md>), [containerized](<https://devfeed.tech/tags/containerized.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [ctf](<https://devfeed.tech/tags/ctf.md>), [deepseek-v3](<https://devfeed.tech/tags/deepseek-v3.md>), [docker](<https://devfeed.tech/tags/docker.md>), [dojo](<https://devfeed.tech/tags/dojo.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [training](<https://devfeed.tech/tags/training.md>), [vulnerabilities](<https://devfeed.tech/tags/vulnerabilities.md>)

### AI overview

CTF-Dojo provides 658 reproducible, Docker-containerized CTF challenges for training language model agents with verifiable execution feedback. Its CTF-Forge pipeline turns public artifacts into ready-to-use environments in minutes, reducing configuration that the authors say traditionally took weeks. Agents trained on 486 execution-verified trajectories gained up to 9.9 percentage points over strong baselines across InterCode-CTF, NYU CTF Bench, and Cybench. The best 32B model scored 31.9% Pass@1; the authors report this as an open-weight state of the art rivaling DeepSeek-V3-0324 and Gemini-2.5-Flash.

### Source excerpt

arXiv:2508.18370v3 Announce Type: replace-cross Abstract: Large language models (LLMs) have demonstrated exceptional capabilities when trained within executable runtime environments, notably excelling at software engineering tasks through verified feedback loops. Yet, scalable and generalizable execution-grounded environments remain scarce, limiting progress in training more capable ML agents. We introduce CTF-Dojo, the first large-scale executable runtime tailored for training LLMs with verifiable feedback, featuring 658 fully functional Capture-The-Flag (CTF)-style challenges containerized in Docker with guaranteed reproducibility. To enable rapid scaling without manual intervention, we develop CTF-Forge, an automated pipeline that transforms publicly available artifacts into ready-to-use execution environments in minutes, eliminating weeks of expert configuration traditionally required. We trained LLM-based agents on just 486 high-quality, execution-verified trajectories from CTF-Dojo, achieving up to 9.9% absolute gains over strong baselines across three competitive benchmarks: InterCode-CTF, NYU CTF Bench, and Cybench. Our best-performing 32B model reaches 31.9% Pass@1, establishing a new open-weight state-of-the-art that rivals frontier models like DeepSeek-V3-0324 and Gemini-2.5-Flash. By framing CTF-style tasks as a benchmark for executable-agent learning, CTF-Dojo demonstrates that execution-grounded training signals are not only effective but pivotal in advancing high-performance ML agents without dependence on costly proprietary systems.

## ModHarness-RL: Recasting Harnesses as Learnable and Evolving Programs for Agentic RL

DevFeed: [ModHarness-RL: Recasting Harnesses as Learnable and Evolving Programs for Agentic RL](<https://devfeed.tech/articles/modharness-rl-recasting-harnesses-as-learnable-and-evolving-programs-for-agentic-rl-135212.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.18533>)

Author: Zelin He, Siyu Zhu, Jacob Helwig, Yujian Liu, Li An, Rohit Jain, Shiyu Chang, Alborz Geramifard

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [AI Context Management](<https://devfeed.tech/topics/ai-context-management.md>), [rlvr](<https://devfeed.tech/topics/rlvr.md>), [Agentic AI](<https://devfeed.tech/topics/what-is-agentic-ai.md>)

Tags: [agentic](<https://devfeed.tech/tags/agentic.md>), [behavior](<https://devfeed.tech/tags/behavior.md>), [blocks](<https://devfeed.tech/tags/blocks.md>), [candidate](<https://devfeed.tech/tags/candidate.md>), [compression](<https://devfeed.tech/tags/compression.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [execution](<https://devfeed.tech/tags/execution.md>), [fixed](<https://devfeed.tech/tags/fixed.md>), [guidance](<https://devfeed.tech/tags/guidance.md>), [harness](<https://devfeed.tech/tags/harness.md>), [history](<https://devfeed.tech/tags/history.md>), [model](<https://devfeed.tech/tags/model.md>), [performance](<https://devfeed.tech/tags/performance.md>), [program](<https://devfeed.tech/tags/program.md>), [rl](<https://devfeed.tech/tags/rl.md>), [sql](<https://devfeed.tech/tags/sql.md>), [state](<https://devfeed.tech/tags/state.md>), [steering](<https://devfeed.tech/tags/steering.md>), [study](<https://devfeed.tech/tags/study.md>)

### AI overview

ModHarness-RL treats an agentic RL harness as programs that can adapt alongside the model, rather than as a fixed, hand-written scaffold. It stores interactions as typed State blocks and learns two programs: Compression assembles interaction history into the model's context, while Steering offers guidance when behavior deviates. During training, candidate programs are proposed from recent trajectories, checked by execution, and replayed over past trajectories. In experiments on CodeSearch, with further analysis on ALFWorld and BIRD SQL and models from 1.5B to 8B, Compression and Steering improve held-out performance over a fixed-harness baseline in most settings. The supplied abstract does not quantify gains or specify which settings did not improve.

### Source excerpt

arXiv:2611.18533v1 Announce Type: new Abstract: Agentic reinforcement learning (RL) increasingly relies on a harness, the scaffold around the model that decides what it sees of its own interaction and what guidance it receives. Most existing approaches train under a hand-written harness that stays fixed throughout. However, an effective harness depends on the domain, the data and the model, and keeps changing as the model trains. We introduce ModHarness-RL, which recasts the harness as programs that are learned from data and evolve as the model trains with RL. ModHarness-RL records the interaction as structured State, a set of typed blocks, and two core components of the harness are programs over these blocks, a Compression program that assembles the accumulated history into the model's context and a Steering program that supplies guidance when behavior deviates. During RL training, candidate programs are proposed from recent trajectories, verified by execution, and replayed over past trajectories, so the harness tracks how the model's behavior shifts. We study CodeSearch in depth and further analyze ALFWorld and BIRD SQL, with models from 1.5B to 8B. The proposed Compression and Steering mechanisms improve held-out performance over the fixed-harness baseline in most settings. These results suggest that a harness for agentic RL can be learned from the task's data and evolved with the model.

## Honesty Is a Race: Predicting When Verifiable-Reward RL Teaches a Model to Bluff

DevFeed: [Honesty Is a Race: Predicting When Verifiable-Reward RL Teaches a Model to Bluff](<https://devfeed.tech/articles/honesty-is-a-race-predicting-when-verifiable-reward-rl-teaches-a-model-to-bluff-132171.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.15779>)

Author: Nathaniel Bottman, Alexei Oblomkov

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [rlvr](<https://devfeed.tech/topics/rlvr.md>), [grpo](<https://devfeed.tech/topics/grpo.md>)

Tags: [code](<https://devfeed.tech/tags/code.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [learning](<https://devfeed.tech/tags/learning.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [math](<https://devfeed.tech/tags/math.md>), [model](<https://devfeed.tech/tags/model.md>), [models](<https://devfeed.tech/tags/models.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [reward](<https://devfeed.tech/tags/reward.md>), [rl](<https://devfeed.tech/tags/rl.md>)

### AI overview

The abstract's central finding is that equal initial expected reward can lead verifiable-reward RL to either honesty or bluffing, depending on how accepted certificates and reward ratios evolve. In the authors' first-order GRPO condition, honesty improves when the checker's acceptance rate on still-ambiguous items, weighted by certificate reward, exceeds the pull of items where an unverifiable verdict is correct. A checker can improve acceptance during training, letting runs just below the initial threshold recover; random rewards with the same mean do not. Runs just above it may first become less honest before recovering. The authors report that, across propositional Datalog, two code models, and math abstention, runs above initial p-r/b=1 did not collapse, while checker-based runs sometimes survived slightly below. At p-r/b=3, both code models mislabeled about half of correct programs as buggy. In depth-3 Datalog, training with a higher payoff and then reducing it retained at least 67% honesty across three seeds, compared with 4-15% for training from the start at the lower ratio. In depth-2 Datalog, a pre-RL linear probe still represented truth after behavioral honesty collapsed.

### Source excerpt

arXiv:2611.15779v1 Announce Type: cross Abstract: Two rewards with the same expected payoff at the start of training can end a reinforcement-learning run in opposite places, with an honest model or one that bluffs. We study rewards that pay $b$ for a verdict that a checker cannot verify ("this program is correct") and $r$ for a certificate that it accepts (an input revealing a bug). To first order, a GRPO step makes the model more honest exactly when $\tilde{p}\,r/b>1+\kappa$. Here $\tilde{p}$ is the rate at which the checker accepts the model's certificates, weighted toward the items on which the model still gives both verdicts, and $\kappa$ is the pull of the items on which the unverifiable verdict is right. Both terms move during training. A checker raises $\tilde{p}$, so near the threshold honesty is a race, which a run just below the threshold can win and a run under a lottery (random pay with the same mean) cannot. The term $\kappa$ starts large and falls as the other items settle, so a run just above the threshold first dips toward bluffing, on code nearly to zero, before climbing back. Paying the honest verdict also pulls the other items toward it, which predicts a ceiling on the payoff ratio, above which the model gives the honest verdict where it is wrong. On propositional Datalog, two code models, and abstention on grade-school math, with $p$ the model's success rate before RL, no run above $pr/b=1$ ends collapsed, and runs under the checker can survive slightly below it. At $pr/b=3$, above the ceiling, both code models call about half of correct programs buggy. On depth-3 Datalog, training generously under the checker and then cutting the payoff keeps the model at least 67% honest in three seeds, at a ratio where the same model trained from the start falls to 4-15% and is at most 47% honest after 160 steps. On depth-2 Datalog, when honesty collapses, a linear probe fit before RL still reads the truth.

## Reward-Driven Collider Physics Challenges for Self-Improving Agentic AI via Reinforcement Learning

DevFeed: [Reward-Driven Collider Physics Challenges for Self-Improving Agentic AI via Reinforcement Learning](<https://devfeed.tech/articles/reward-driven-collider-physics-challenges-for-self-improving-agentic-ai-via-reinforcement-learning-132181.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.15885>)

Author: Alexander Yue, Qibin Liu, Julia Gonski

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [rlvr](<https://devfeed.tech/topics/rlvr.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [automate](<https://devfeed.tech/tags/automate.md>), [automated](<https://devfeed.tech/tags/automated.md>), [automatically](<https://devfeed.tech/tags/automatically.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [challenges](<https://devfeed.tech/tags/challenges.md>), [continuous](<https://devfeed.tech/tags/continuous.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [environments](<https://devfeed.tech/tags/environments.md>), [feedback](<https://devfeed.tech/tags/feedback.md>), [frontier](<https://devfeed.tech/tags/frontier.md>), [frontier-models](<https://devfeed.tech/tags/frontier-models.md>), [improvement](<https://devfeed.tech/tags/improvement.md>), [learning](<https://devfeed.tech/tags/learning.md>), [physics](<https://devfeed.tech/tags/physics.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [reward](<https://devfeed.tech/tags/reward.md>)

### AI overview

The paper proposes open-ended high-energy physics challenges with continuous, automatically evaluated rewards as reinforcement-learning environments for improving agentic AI. Unlike completion-focused benchmarks, the tasks target difficult problems where frontier models still struggle to produce correct, complete solutions. The authors benchmark frontier reasoning models on several tasks and report an initial proof of concept: a small language model improves through RL with automated feedback and no human intervention. The abstract presents this as an early step toward agents that discover better scientific solutions; it provides no detailed task descriptions or quantitative results.

### Source excerpt

arXiv:2611.15885v1 Announce Type: new Abstract: Recent advances in agentic AI have demonstrated the potential to automate complex high-energy physics (HEP) workflows. We focus on the regime of HEP tasks where frontier models still struggle to provide correct and complete solutions, and introduce an open-ended optimization task format with continuous, automatically evaluated rewards. Unlike conventional benchmarks focused on task completion, these tasks target reinforcement learning (RL) environments to generate agentic improvement on state- of-the-art HEP challenges. We benchmark frontier reasoning models on several HEP tasks and demonstrate an initial proof of concept in which a small language model improves through RL using automated feedback without human intervention. By connecting challenging HEP problems with verifiable scientific rewards and RL, we aim to advance agentic AI from reproducing established workflows to autonomously discovering better scientific solutions, paving the way toward self-improving scientific agents.

[Next page](<https://devfeed.tech/topics/rlvr.md?cursor=WyIyMDI2LTExLTEzVDA1OjAwOjAwKzAwOjAwIiwgImM0MDcyN2NhLWQ4MmMtNDE2OC05Y2I0LTQ1NmFiOWQ1Y2IzYyJd>)