# Phase-Conditioned Reward Shaping for Token-Efficient LLM Reasoning

DevFeed: [Phase-Conditioned Reward Shaping for Token-Efficient LLM Reasoning](<https://devfeed.tech/articles/phase-conditioned-reward-shaping-for-token-efficient-llm-reasoning-125308.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.11024>)

Author: Zhe Chen, Xiaolin Zhou, Xiyang Hu

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [classification](<https://devfeed.tech/tags/classification.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [execution](<https://devfeed.tech/tags/execution.md>), [hidden](<https://devfeed.tech/tags/hidden.md>), [implementation](<https://devfeed.tech/tags/implementation.md>), [inference](<https://devfeed.tech/tags/inference.md>), [language](<https://devfeed.tech/tags/language.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-reasoning](<https://devfeed.tech/tags/llm-reasoning.md>), [models](<https://devfeed.tech/tags/models.md>), [native](<https://devfeed.tech/tags/native.md>), [oracle](<https://devfeed.tech/tags/oracle.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [reduce](<https://devfeed.tech/tags/reduce.md>), [responses](<https://devfeed.tech/tags/responses.md>), [revision](<https://devfeed.tech/tags/revision.md>), [reward](<https://devfeed.tech/tags/reward.md>), [token](<https://devfeed.tech/tags/token.md>), [weight](<https://devfeed.tech/tags/weight.md>)

## AI overview

Phase-Conditioned Reward Shaping (PCRS) allocates training-time token penalties according to reasoning phases, aiming to curb redundant output without penalizing useful steps uniformly. The main PCRS-T configuration uses a frozen text classifier to assign overlapping reasoning, execution, and revision labels to native chunks, combined with exact repetition to adjust existing penalties; PCRS-H uses a hidden-state classifier. Across six mathematical and scientific reasoning benchmarks, default PCRS-T reduced mean response length by 3.13% versus token-level length penalties. Sampled Pass@1 was about 62.07% for both, while oracle Pass@16 increased from 86.08% to 87.67%. The abstract reports that this approach needs no inference-time classification.

## Source excerpt

arXiv:2611.11024v1 Announce Type: new Abstract: Extended reasoning can help language models solve complex problems, but long responses often contain both useful computations and redundant checks. Conventional length penalties encourage shorter responses without distinguishing the functions of intermediate steps. We propose Phase-Conditioned Reward Shaping (PCRS), a framework that combines phase identification with training-time reward allocation. We investigate two classification routes: PCRS-H with a hidden-state classifier and PCRS-T with a frozen text classifier. Our main text classifier implementation assigns overlapping reasoning, execution, and revision labels to native chunks, then uses these labels and exact repetition to adjust existing token penalties. We compare the completed configurations on six mathematical and scientific reasoning benchmarks and examine four reward-weight settings with the text classifier. Compared with a baseline using token-level length penalties, the default text classifier configuration reduces mean response length by 3.13%; sampled Pass@1 is approximately 62.07% for both methods, while oracle Pass@16 rises from 86.08% to 87.67%. These results show how phase-informed penalty allocation can reduce output volume and improve multi-attempt coverage without inference-time classification.