# Reasoning or Pattern Exploitation? Mechanistic Insights from RL-Trained Language Models

DevFeed: [Reasoning or Pattern Exploitation? Mechanistic Insights from RL-Trained Language Models](<https://devfeed.tech/articles/reasoning-or-pattern-exploitation-mechanistic-insights-from-rl-trained-language-models-108161.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.39182>)

Author: Aishik Nagar, Cheston Tan

Published: 2026-10-30T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [grpo](<https://devfeed.tech/topics/grpo.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [rlvr](<https://devfeed.tech/topics/rlvr.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [post-training](<https://devfeed.tech/topics/post-training.md>)

Tags: [autoencoders](<https://devfeed.tech/tags/autoencoders.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [input](<https://devfeed.tech/tags/input.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [mechanistic-interpretability](<https://devfeed.tech/tags/mechanistic-interpretability.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [rl](<https://devfeed.tech/tags/rl.md>), [tic-tac-toe](<https://devfeed.tech/tags/tic-tac-toe.md>)

## AI overview

In Tic-Tac-Toe models trained with Group Relative Policy Optimization (GRPO), reinforcement learning produced high accuracy and robustness across prompt variations, but mechanistic analysis suggests these gains rely mainly on making better use of information already explicit in the prompt--such as turn, game progression, and board occupancy. Sparse autoencoder features for board symmetries and strategic forks remained weakly represented. The authors report that post-training reorganizes which features dominate rather than adding or removing concept information, with causal interventions locating the decision downstream in a late-layer state. Their results suggest task-specific adaptation may be mistaken for reasoning, though the study examines a finite logical domain.

## Source excerpt

arXiv:2610.39182v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly described as acquiring "reasoning" skills after reinforcement learning with verifiable rewards. However, whether these gains reflect genuine reasoning or more superficial adaptations remains underexplored. In this work, we probe LLMs trained in a finite and exhaustively analyzable logical domain, namely Tic-Tac-Toe, and trace how internal representations evolve across reinforcement learning (RL) with Group Relative Policy Optimization (GRPO). Quantitatively, RL yields high accuracy and robustness across prompt variations, suggesting the emergence of game reasoning. Mechanistic interpretability, however, paints a different picture: features extracted with sparse autoencoders (SAEs) reveal that models primarily adapt to rely more heavily on information already explicit in the prompt, such as whose turn it is, game progression and board occupancy. In comparison, high-level concepts like board symmetries and strategic forks remain weakly represented, raising concerns that reasoning benchmarks overestimate model abilities. This suggests that RL-driven "reasoning" may conflate task-specific adaptations to input information with reasoning ability. We provide, for the first time: (i) a systematic interpretability pipeline tracing representation dynamics across RL training in small-scale LLMs; (ii) an extension of SAE-based feature discovery to hypothesis-driven testing in a finite logical domain; and (iii) an interpretability-based demonstration that RL post-training neither adds nor removes concept information but reorganizes which features dominate the representation, with causal interventions locating the decision downstream of it, in the late-layer state.