# Parallel reasoning

A method in which multiple diverse reasoning paths are generated and compared for the same problem, used in large language model research.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## No-Harm-Jury A Jury-Based Multi-Dimensional Harm Evaluation Framework for Medical AI Responses

DevFeed: [No-Harm-Jury A Jury-Based Multi-Dimensional Harm Evaluation Framework for Medical AI Responses](<https://devfeed.tech/articles/no-harm-jury-a-jury-based-multi-dimensional-harm-evaluation-framework-for-medical-ai-responses-145451.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24972>)

Author: Antonio Puertas Gallardo, Mario Ceresa

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [AI Chat](<https://devfeed.tech/topics/ai-chat.md>), [Parallel reasoning](<https://devfeed.tech/topics/parallel-reasoning.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-responses](<https://devfeed.tech/tags/ai-responses.md>), [automated](<https://devfeed.tech/tags/automated.md>), [autonomy](<https://devfeed.tech/tags/autonomy.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [dimensions](<https://devfeed.tech/tags/dimensions.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [framework](<https://devfeed.tech/tags/framework.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [llm](<https://devfeed.tech/tags/llm.md>), [median](<https://devfeed.tech/tags/median.md>), [medical](<https://devfeed.tech/tags/medical.md>), [medical-ai](<https://devfeed.tech/tags/medical-ai.md>), [models](<https://devfeed.tech/tags/models.md>), [privacy](<https://devfeed.tech/tags/privacy.md>), [qa](<https://devfeed.tech/tags/qa.md>), [reliability](<https://devfeed.tech/tags/reliability.md>), [responses](<https://devfeed.tech/tags/responses.md>), [robust](<https://devfeed.tech/tags/robust.md>), [social](<https://devfeed.tech/tags/social.md>)

### AI overview

No-Harm-Jury evaluates medical AI responses across seven harm dimensions--informational, social, psychological, autonomy, economic, privacy, and epistemic--using five diverse language models as independent jurors. The study applies it to Ministral-14B responses for 3,000 questions from PubMedQA, MedQA, and MedMCQA; 151 empty responses are excluded, leaving 2,849 samples. Median aggregation is intended to reduce individual juror bias while retaining coverage through complementary specialisation, with a reliability-weighted extension also described. The supplied abstract does not report comparative performance results.

### Source excerpt

arXiv:2611.24972v1 Announce Type: cross Abstract: We present No-Harm-Jury, a jury-based automated evaluation framework that scores AI-generated medical responses across seven conceptually distinct harm dimensions using a panel of five diverse language models as independent jurors. Responses generated by Ministral-14B on 3,000 questions drawn from PubMedQA, MedQA, and MedMCQA are scored by five jurors across dimensions of informational, social, psychological, autonomy, economic, privacy, and epistemic harm; the 151 empty responses (98 MedQA, 53 MedM- CQA) are excluded, leaving 2,849 valid samples. The median aggregation robustly suppresses individual juror biases while maintaining coverage through complementary specialisation. No existing system combines all five prop- erties of No-Harm-Jury: automated scalable evaluation, five diverse LLM jurors, robust me- dian aggregation (with a reliability-weighted extension), seven conceptually distinct harm dimensions, and application to medical QA benchmarks.

## DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning

DevFeed: [DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning](<https://devfeed.tech/articles/dlwm-diverse-latent-world-models-for-efficient-multimodal-reasoning-143275.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2606.15160>)

Author: David Huang, Lianlei Shan

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Parallel reasoning](<https://devfeed.tech/topics/parallel-reasoning.md>), [World models](<https://devfeed.tech/topics/world-models.md>)

Tags: [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [large-language-models](<https://devfeed.tech/tags/large-language-models.md>), [memory](<https://devfeed.tech/tags/memory.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [orthogonality](<https://devfeed.tech/tags/orthogonality.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [sequential-decision](<https://devfeed.tech/tags/sequential-decision.md>), [world-models](<https://devfeed.tech/tags/world-models.md>)

### AI overview

DLWM represents ambiguous visual input with multiple candidate latent interpretations and reasons along each independently. An orthogonality-based regularizer discourages those hypotheses from collapsing into one. A resource-aware reinforcement learning policy then allocates computation by expanding or pruning hypotheses, merging paths, or producing an answer. The abstract reports 2-5 point accuracy gains over existing methods across multiple multimodal reasoning benchmarks, alongside 24% lower memory usage; it does not specify benchmark names or the accuracy metric.

### Source excerpt

arXiv:2606.15160v2 Announce Type: replace-cross Abstract: Reasoning capabilities of multimodal large language models (MLLMs) have improved considerably in recent years. Existing approaches typically rely on explicit chain-of-thought or continuous latent-space trajectories to enhance multi-step reasoning. However, these methods generally assume that an input admits a single latent interpretation and unfold reasoning along a fixed path or under a uniform computation budget. In real-world multimodal settings, visual observations are often subject to occlusion, blur, viewpoint variation, or semantic ambiguity, giving rise to multiple plausible interpretations. A uniform reasoning strategy not only limits the model's ability to explore multiple hypotheses but also incurs high memory usage and rollout cost. We present DLWM (Diverse Latent World Models), a multimodal reasoning framework that combines latent-space reasoning with reinforcement learning. First, we construct a set of diverse latent world hypotheses in continuous latent space, each representing a candidate interpretation of the visual input, and unfold latent reasoning independently on each hypothesis. An orthogonality-based diversity regularizer discourages hypothesis collapse. Second, we formulate the latent reasoning process as a resource-constrained sequential decision problem and introduce a resource-aware reinforcement learning policy that adaptively allocates computation across hypotheses, dynamically deciding whether to expand a reasoning path, prune a hypothesis, merge reasoning paths, or produce the final answer, thereby substantially reducing memory footprint and improving rollout efficiency. Experiments on multiple multimodal reasoning benchmarks demonstrate that DLWM outperforms existing methods by 2-5 points in accuracy while reducing memory usage by 24%.

## From Ambiguous Tickets to Ranked Hypotheses: Multi-Agent Root-Cause Analysis for Autonomous Heavy-Vehicle Software

DevFeed: [From Ambiguous Tickets to Ranked Hypotheses: Multi-Agent Root-Cause Analysis for Autonomous Heavy-Vehicle Software](<https://devfeed.tech/articles/from-ambiguous-tickets-to-ranked-hypotheses-multi-agent-root-cause-analysis-for-autonomous-heavy-vehicle-software-142159.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23294>)

Author: Marten Seegers, Maximilian Kneissl, Enrique Aguayo-Lara, Chih-Hong Cheng

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [Multi Agent Systems](<https://devfeed.tech/topics/multi-agent-systems.md>), [Parallel reasoning](<https://devfeed.tech/topics/parallel-reasoning.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [observability](<https://devfeed.tech/topics/observability.md>)

Tags: [autonomous](<https://devfeed.tech/tags/autonomous.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [multi-agent](<https://devfeed.tech/tags/multi-agent.md>), [root-cause-analysis](<https://devfeed.tech/tags/root-cause-analysis.md>), [software](<https://devfeed.tech/tags/software.md>)

### AI overview

The workflow turns sparse incident tickets and large vehicle recordings into telemetry-grounded event chains, ranked causal hypotheses, and candidate code locations. Non-neural tools select telemetry once, then an LLM creates shared evidence artifacts, cutting per-incident cost by more than half versus per-analyst querying. Specialist agents independently use fault tree analysis, Kepner-Tregoe analysis, and call-chain tracing; hypotheses reached by multiple analysts rank higher. A separate analyst session examines any suspected ticket cause to reduce anchoring risk. On 12 held-out incidents, the workflow found the faulty file in 10 cases versus 8 for a single agent using the same model and tools, at higher cost. Two domain experts accepted a top-three hypothesis in 4 and 9 cases, respectively, while two LLM judges did not resolve the disagreement, underscoring the difficulty of evaluating causal explanations.

### Source excerpt

arXiv:2611.23294v1 Announce Type: new Abstract: Autonomous haulage systems run safety-critical software on driverless mining trucks, with no onboard driver to detect or mitigate abnormal behavior. Every failure in testing or operation must therefore be traced to its root cause to keep the safety case valid and limit downtime. An investigation often starts from a sparse ticket containing an approximate timestamp, sometimes a suspected cause, and a multimodal vehicle recording of several gigabytes. We present a multi-agent root-cause analysis (RCA) workflow that turns this input into a telemetry-grounded event chain, ranked causal hypotheses, and candidate code locations. A suspected cause in the ticket, which could anchor the analysis on a false theory, goes to a single LLM analyst in a separate agent session. Non-neural tools select the relevant telemetry once, before analysis, and an LLM condenses it into shared evidence artifacts; this more than halved the cost per incident compared with per-analyst querying. Specialist analyst agents then apply established RCA techniques in parallel, including fault tree analysis, Kepner-Tregoe analysis, and call-chain tracing, and hypotheses that several LLM analysts reach independently rank higher. We evaluated the workflow on 12 held-out industrial incidents against a single agent with the same model and tools. It located the faulty file more often (10 vs. 8 cases) at higher cost. Two human domain experts accepted a top-three hypothesis in 4 and 9 cases, respectively. The gap, which two LLM judges did not resolve, shows that rating causal explanations is itself a hard evaluation problem.

## Atomic MathReasoner: Instance-Level Composition of Atomic Reasoning Skills for Mathematical LLMs

DevFeed: [Atomic MathReasoner: Instance-Level Composition of Atomic Reasoning Skills for Mathematical LLMs](<https://devfeed.tech/articles/atomic-mathreasoner-instance-level-composition-of-atomic-reasoning-skills-for-mathematical-llms-143572.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24846>)

Author: Jiashu Li, Tianyu Gao, Qinglong Liu, Jinping Wang, Ye Wei, Zhiqiang Gao

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Parallel reasoning](<https://devfeed.tech/topics/parallel-reasoning.md>), [lora](<https://devfeed.tech/topics/lora.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>)

Tags: [amr](<https://devfeed.tech/tags/amr.md>), [atomic](<https://devfeed.tech/tags/atomic.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [capabilities](<https://devfeed.tech/tags/capabilities.md>), [composition](<https://devfeed.tech/tags/composition.md>), [confidence](<https://devfeed.tech/tags/confidence.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [gradient](<https://devfeed.tech/tags/gradient.md>), [inference](<https://devfeed.tech/tags/inference.md>), [llms](<https://devfeed.tech/tags/llms.md>), [performance](<https://devfeed.tech/tags/performance.md>), [prototypes](<https://devfeed.tech/tags/prototypes.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [router](<https://devfeed.tech/tags/router.md>), [skills](<https://devfeed.tech/tags/skills.md>)

### AI overview

Atomic MathReasoner (AMR) composes specialized reasoning skills per problem instead of treating mathematical reasoning as one capability. It trains capability-specific LoRA experts, searches held-out validation subsets without gradients for low-loss composition prototypes, scores them by token-level negative log-likelihood of reference solutions, filters uncertain assignments with a confidence margin, and distills the resulting labels into a lightweight router. At inference, the router selects a prototype without costly online optimization. On AtomicMath, the abstract reports consistent gains across general and math-specialized LLMs; for Qwen2.5-Math-Instruct-7B, average field accuracy rose from 64.4% to 68.1% at Level 1 and 59.3% to 63.7% at Level 2, while average logical reasoning performance rose from 39.4% to 46.8%.

### Source excerpt

arXiv:2611.24846v1 Announce Type: new Abstract: Most existing approaches enhance mathematical reasoning in large language models (LLMs) as a monolithic capability, despite the fact that different mathematical problems require different reasoning skills. Recent work has explored a more fine-grained perspective by decomposing mathematical reasoning into atomic skills and developing specialized modules for individual capabilities. However, how to effectively utilize these atomic abilities remains underexplored. To address this challenge, we propose Atomic MathReasoner (AMR), an instance-level framework for composing atomic reasoning skills. AMR first trains a library of capability-specific LoRA experts and performs gradient-free optimization over held-out validation subsets to discover a diverse set of low-loss composition prototypes. It evaluates the preference of each prototype using the token-level negative log-likelihood of reference solutions, filters ambiguous assignments through a confidence margin, and distills the resulting pseudo-labels into a lightweight router. During inference, the router selects an appropriate composition prototype for each problem, enabling problem-specific utilization of atomic skills without costly online optimization. Experiments on the AtomicMath benchmark across general and mathematics-specialized LLMs demonstrate consistent improvements. On Qwen2.5-Math-Instruct-7B, AMR increases average field accuracy from 64.4% to 68.1% at Level 1 and from 59.3% to 63.7% at Level 2, while improving average logical reasoning performance from 39.4% to 46.8%. These results demonstrate that effectively composing specialized atomic skills provides a promising approach to strengthening mathematical reasoning in LLMs.

## Gemini 4 Neon Launches Same Day at Three Times Carbon's Speed, Ranks Third Overall With AA Score of 61

DevFeed: [Gemini 4 Neon Launches Same Day at Three Times Carbon's Speed, Ranks Third Overall With AA Score of 61](<https://devfeed.tech/articles/gemini-4-neon-launches-same-day-at-three-times-carbon-s-speed-ranks-third-overall-with-aa-score-of-61-141943.md>)

Original publisher: [Read original article](<https://joho-todai.com/en/gemini-4-neon-carbon-three-times-faster/>)

Author: Toru Misaki

Published: 2026-11-18T23:08:47Z

Content type: news

Language: en

Sources: [Joho no Todai](<https://devfeed.tech/sources/joho-no-todai.md>)

Topics: [open-llm-leaderboard](<https://devfeed.tech/topics/open-llm-leaderboard.md>), [Parallel reasoning](<https://devfeed.tech/topics/parallel-reasoning.md>), [Audit-then-score](<https://devfeed.tech/topics/audit-then-score.md>), [Cryptography](<https://devfeed.tech/topics/cryptography.md>), [Claude](<https://devfeed.tech/topics/claude.md>)

Tags: [api](<https://devfeed.tech/tags/api.md>), [app](<https://devfeed.tech/tags/app.md>), [carbon](<https://devfeed.tech/tags/carbon.md>), [cost](<https://devfeed.tech/tags/cost.md>), [family](<https://devfeed.tech/tags/family.md>), [gemini](<https://devfeed.tech/tags/gemini.md>), [gemini-4](<https://devfeed.tech/tags/gemini-4.md>), [gemini-app](<https://devfeed.tech/tags/gemini-app.md>), [independent](<https://devfeed.tech/tags/independent.md>), [index](<https://devfeed.tech/tags/index.md>), [intelligence](<https://devfeed.tech/tags/intelligence.md>), [intelligence-index](<https://devfeed.tech/tags/intelligence-index.md>), [model](<https://devfeed.tech/tags/model.md>), [neon](<https://devfeed.tech/tags/neon.md>), [score](<https://devfeed.tech/tags/score.md>), [speed](<https://devfeed.tech/tags/speed.md>), [times](<https://devfeed.tech/tags/times.md>), [united-states](<https://devfeed.tech/tags/united-states.md>), [was](<https://devfeed.tech/tags/was.md>), [with](<https://devfeed.tech/tags/with.md>)

### AI overview

Gemini 4 Neon began rolling out on its November 18 announcement day through Google AI Studio's paid API, Gemini for enterprises, Antigravity for Pro and Ultra subscribers, and the Gemini app for Pro and Ultra subscribers. Adapted from Carbon, it scored 61 and ranked third on Artificial Analysis' Intelligence Index v5.01 at Xhigh; the text cautions that the updated benchmark and newly evaluated models make direct comparison with Carbon's earlier score unreliable. Google reports three times Carbon's speed and 32% lower cost per task at the default high setting; API rates are $3 per million input tokens and $15 per million output tokens, 25% below Carbon's standard rates. The excerpt also reports strong coding, business, science, and agent benchmark results, while noting that harnesses affect outcomes and many scores are Google-reported. Its 0.0% prompt-injection attack success result applies specifically with Xhigh reasoning and real-time safeguards. Serious biological and cybersecurity risks cannot be ruled out, according to the cited safety framework.

### Source excerpt

Neon, the fourth model in the Gemini 4 family, was announced November 18 in the United States and began rolling out the same day through the API and Gemini app. Adapted from Carbon, it promises three times the speed and 40% lower cost while scoring 61 and ranking third overall on the independent AA Intelligence Index.

## EvoDisco: Strategy Discovery for Evolving Agent Memory

DevFeed: [EvoDisco: Strategy Discovery for Evolving Agent Memory](<https://devfeed.tech/articles/evodisco-strategy-discovery-for-evolving-agent-memory-139963.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22102>)

Author: Mingyang Liu, Qingcan Kang, Shixiong Kai, Kaichao Liang, Xiaojin Fu, Tao Zhong, Linqi Song

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Retrieval Augmented Generation (RAG)](<https://devfeed.tech/topics/retrieval-augmented-generation-rag.md>), [Parallel reasoning](<https://devfeed.tech/topics/parallel-reasoning.md>), [model architecture](<https://devfeed.tech/topics/model-architecture.md>)

Tags: [agent-memory](<https://devfeed.tech/tags/agent-memory.md>), [agents](<https://devfeed.tech/tags/agents.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [discovery](<https://devfeed.tech/tags/discovery.md>), [large-language-model](<https://devfeed.tech/tags/large-language-model.md>), [memory](<https://devfeed.tech/tags/memory.md>)

### AI overview

EvoDisco lets agents learn which memory transformation to use, when to use it, and how to apply it. A policy conditions on the current memory and recent task history, selecting among Abstract (compress related memories), Refine (decompose overloaded entries), Bridge (connect weakly linked experiences), and Reframe (reorganize retrieved reasoning chains). Its bi-level fitness rewards retrieval-quality gains while regularizing memory complexity, and evaluates longer-term strategy through operator-utility regret, prediction error, and selection diversity. In the reported one-pass online and frozen-transfer evaluations on ALFWorld, WebShop, and LifelongAgentBench (OS and DB), it outperformed MemP and MemRL; the largest reported gains were on ALFWorld. Ablations credited both fitness levels and the operator library. These are abstract-level claims; detailed methods and results are not provided here.

### Source excerpt

arXiv:2611.22102v1 Announce Type: new Abstract: Large language model agents accumulate experience in memory, and how that memory evolves during interaction is as important as what it stores. Recent methods update memory online, yet both the operators that transform memory and the rules that select them remain fixed by design. We propose EvoDisco, which formulates agent memory evolution as evolutionary strategy discovery: the agent must discover which operator to apply, when to apply it, and how to instantiate it, rather than rewriting memory under a static operator schedule. A learnable policy conditions on the current memory state and recent task history, then selects and instantiates one of four semantic operators---Abstract, Refine, Bridge, and Reframe---which respectively compress related memories, decompose overloaded entries, connect weakly linked experiences, and reorganize retrieved reasoning chains. The policy is updated with a bi-level fitness. An inner signal scores each transformation using a retrieval-quality improvement regularized by memory complexity. An outer signal scores strategic efficacy over a window of past decisions, combining historical operator-utility regret, utility-prediction error, and selection diversity. Under a one-pass online plus frozen-transfer protocol on ALFWorld, WebShop, and LifelongAgentBench (OS and DB), EvoDisco improves over strong memory baselines including MemP and MemRL. The largest gains are on ALFWorld ($+8.8$ online / $+12.9$ frozen-transfer points over MemP; $+16.4$ / $+17.9$ over MemRL). Ablations attribute these gains to both fitness levels and the operator library.

## From Imperfective Paradox to Moorean Absurdity: Single-Chain Reasoning Lock-In and Multi-Candidate Verbalization Across Aspectual and Pragmatic Ambiguity

DevFeed: [From Imperfective Paradox to Moorean Absurdity: Single-Chain Reasoning Lock-In and Multi-Candidate Verbalization Across Aspectual and Pragmatic Ambiguity](<https://devfeed.tech/articles/from-imperfective-paradox-to-moorean-absurdity-single-chain-reasoning-lock-in-and-multi-candidate-verbalization-across-aspectual-and-pragmatic-ambiguity-139981.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22188>)

Author: An-Sheng Jhang

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Parallel reasoning](<https://devfeed.tech/topics/parallel-reasoning.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [ablation](<https://devfeed.tech/tags/ablation.md>), [abstract](<https://devfeed.tech/tags/abstract.md>), [all](<https://devfeed.tech/tags/all.md>), [alpha](<https://devfeed.tech/tags/alpha.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [call](<https://devfeed.tech/tags/call.md>), [candidate](<https://devfeed.tech/tags/candidate.md>), [cf](<https://devfeed.tech/tags/cf.md>), [code](<https://devfeed.tech/tags/code.md>), [coherence](<https://devfeed.tech/tags/coherence.md>), [completion](<https://devfeed.tech/tags/completion.md>), [confidence](<https://devfeed.tech/tags/confidence.md>), [control](<https://devfeed.tech/tags/control.md>), [cross](<https://devfeed.tech/tags/cross.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [dev](<https://devfeed.tech/tags/dev.md>), [effects](<https://devfeed.tech/tags/effects.md>), [english](<https://devfeed.tech/tags/english.md>), [lock](<https://devfeed.tech/tags/lock.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>)

### AI overview

Across four preregistered experiments on Qwen3-8B-Q8_0, the study finds that language-model entailment and coherence judgments vary with prompt and readout protocol. Direct-CoT and MCU differed by 25 percentage points on telic high-confidence completion, while NLI-CoT fell 91.25 points on uninterrupted atelic items. The proposed Multi-Candidate Verbalization elicits five weighted estimates in one call, distinguishing prompt wording from unweighted MCU and self-weighted MCW readouts. Mandarin and English experiments report an interruption-anchored realis paradox under NLI-CoT; in implicit Moorean sentences, MCU-CF increased perceived coherence by 26.64 points over Direct-CoT. An OpenAI ablation and a development-stage cross-model extension also found discrete NLI-CoT and scalar Direct-CoT judgments dissociated across models. The confirmatory results concern observable prompt-and-readout protocols; premature internal reasoning lock-in remains a hypothesis.

### Source excerpt

arXiv:2611.22188v1 Announce Type: cross Abstract: Natural language entailment and coherence judgments often depend on whether an event merely occurred, reached a telic culmination, satisfied a plan, or was evaluated under multiple pragmatic standards. We test observable consequences of the single-chain premature lock-in hypothesis across four preregistered, sealed formal experiments on Qwen3-8B-Q8_0 (16,640 formal units, temperature=0.0): 15 of 16 hypothesis slots retain confirmatory status (12 rejecting H0 under within-experiment Holm control at alpha_fam=0.05; E3_H3 demoted to descriptive status). In Experiment 1, Direct-CoT vs. MCU differs by 25.00 pp on telic high-confidence completion (Holm p=1.31239e-05), while NLI-CoT collapses by -91.25 pp on uninterrupted atelic items (Holm p=1.81805e-21). Multi-Candidate Verbalization (MCV) elicits K=5 weighted estimates in one call, separating the prompt from unweighted (MCU) and self-weighted (MCW) readouts. Mandarin (E2) and English (E3) expose an Interruption-Anchored Realis Paradox under NLI-CoT. In Experiment 4, MCU-CF raises perceived coherence on implicit Moorean sentences by 26.64 pp over Direct-CoT (Holm p=0.0006), alongside Direct-CF - Direct-CoT = 29.15 pp and MCU-CoT - Direct-CoT = 17.31 pp. A 1,800-call OpenAI ablation and a stage="dev" P16.7 cross-model extension (4,240 planned/collected units across five external models: 4,222 valid, 18 format-missing, 0 transport-missing; <2% missing per slice; OpenAI gpt-4o-mini at top_logprobs=20 achieves 1,000/1,000 valid units) show that discrete NLI-CoT and scalar Direct-CoT judgments dissociate across models. Confirmatory effects concern observable prompt-and-readout protocols; internal premature lock-in remains a hypothesis. All code, stimuli, preregistered analysis plans, and verification tools are available at https://github.com/asj252/llm-judgment-studies.

## When Does Confidence Reveal Problem Difficulty?

DevFeed: [When Does Confidence Reveal Problem Difficulty?](<https://devfeed.tech/articles/when-does-confidence-reveal-problem-difficulty-137411.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.19173>)

Author: Xiyu Bao, Langyuan Cui, Hwee Tou Ng, Chun Kai Ling

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [computational complexity](<https://devfeed.tech/topics/computational-complexity.md>), [Parallel reasoning](<https://devfeed.tech/topics/parallel-reasoning.md>)

Tags: [applications](<https://devfeed.tech/tags/applications.md>), [confidence](<https://devfeed.tech/tags/confidence.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [entropy](<https://devfeed.tech/tags/entropy.md>), [inference](<https://devfeed.tech/tags/inference.md>), [model](<https://devfeed.tech/tags/model.md>), [problems](<https://devfeed.tech/tags/problems.md>), [token](<https://devfeed.tech/tags/token.md>), [utility](<https://devfeed.tech/tags/utility.md>)

### AI overview

The study finds that perturbation inconsistency, semantic entropy, and solution length are more strongly associated with population-level problem difficulty than token-level uncertainty. These associations persist among problems the focal model answers correctly, suggesting the signals reflect difficulty beyond predicting that model's correctness. The authors explore using them to guide inference-time decisions, describing them as informative but imperfect proxies. The supplied text gives no specific application results or comparative measurements.

### Source excerpt

arXiv:2611.19173v1 Announce Type: new Abstract: While confidence signals are commonly evaluated for their ability to predict a model's own correctness, their relationship with problem difficulty remains less understood. In this work, we investigate a wide range of confidence-related signals and whether they can serve as indicators of problem difficulty across various task domains. We find that perturbation inconsistency, semantic entropy, and solution length exhibit strong associations with population difficulty than token-level uncertainty. Notably, these relationships persist even when restricting the analysis to problems correctly answered by the focal model, suggesting that such signals capture information about difficulty beyond their ability to predict the model's own correctness. We further explore applications of these signals, illustrating the potential and limitations of using confidence-related signals to guide inference-time decisions. Our findings highlight the utility of model-specific signals as imperfect but informative proxies for population-level problem difficulty.

## Joint Consistency: A Unified Test-Time Aggregation Framework via Energy Minimization

DevFeed: [Joint Consistency: A Unified Test-Time Aggregation Framework via Energy Minimization](<https://devfeed.tech/articles/joint-consistency-a-unified-test-time-aggregation-framework-via-energy-minimization-137250.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2605.06219>)

Author: Yunzhen Yao, Hongye Wang, Yahong Wang, Michael C. Gastpar, Bo Jiang, Lie He

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Parallel reasoning](<https://devfeed.tech/topics/parallel-reasoning.md>)

Tags: [aggregation](<https://devfeed.tech/tags/aggregation.md>), [approximation](<https://devfeed.tech/tags/approximation.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [code](<https://devfeed.tech/tags/code.md>), [comparisons](<https://devfeed.tech/tags/comparisons.md>), [consistency](<https://devfeed.tech/tags/consistency.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [energy](<https://devfeed.tech/tags/energy.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [framework](<https://devfeed.tech/tags/framework.md>), [generation](<https://devfeed.tech/tags/generation.md>), [interaction](<https://devfeed.tech/tags/interaction.md>), [ising](<https://devfeed.tech/tags/ising.md>), [llm-as-a-judge](<https://devfeed.tech/tags/llm-as-a-judge.md>), [modeling](<https://devfeed.tech/tags/modeling.md>), [problems](<https://devfeed.tech/tags/problems.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [trace](<https://devfeed.tech/tags/trace.md>), [voting](<https://devfeed.tech/tags/voting.md>)

### AI overview

Joint Consistency (JC) aggregates multiple reasoning traces by minimizing a constrained Ising-type energy that combines individual evaluation signals with pairwise LLM-judge comparisons. This lets it model how candidate traces compare, while treating common voting and weighted aggregation methods as special cases. The authors describe an efficient approximation for scaling interaction modeling. They report that JC outperforms existing baselines on math and code reasoning benchmarks across judge models, trace budgets, and generation settings, with larger gains on challenging problems.

### Source excerpt

arXiv:2605.06219v2 Announce Type: replace Abstract: This paper studies test-time aggregation, an approach that generates multiple reasoning traces and aggregates them into a final answer. Most existing methods rely on evaluation signals collected from candidate traces in isolation or answer frequencies, while ignoring comparative interactions among candidates. We propose Joint Consistency (JC), formulated as a constrained Ising-type energy minimization problem, which employs both independent evaluation signals and pairwise comparisons using LLM-as-a-judge. JC provides a unified framework for test-time aggregation that subsumes typical voting and weighted aggregation methods as special cases. Moreover, we develop an efficient approximation strategy that makes interaction modeling practical for large-scale test-time aggregation. Experiments on math and code reasoning benchmarks show that JC outperforms existing baselines across tasks, judge models, trace budgets, and trace-generation settings, with particularly pronounced gains on challenging problems.

## Anytime-Valid PAC-Bayes Certificates for Adaptive Test-Time Scaling

DevFeed: [Anytime-Valid PAC-Bayes Certificates for Adaptive Test-Time Scaling](<https://devfeed.tech/articles/anytime-valid-pac-bayes-certificates-for-adaptive-test-time-scaling-139143.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.21241>)

Author: Xiaoyu Li, Jiangxuan Long

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Parallel reasoning](<https://devfeed.tech/topics/parallel-reasoning.md>)

Tags: [adaptive](<https://devfeed.tech/tags/adaptive.md>), [adaptive-test](<https://devfeed.tech/tags/adaptive-test.md>), [aggregation](<https://devfeed.tech/tags/aggregation.md>), [amplify](<https://devfeed.tech/tags/amplify.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [audit](<https://devfeed.tech/tags/audit.md>), [bayes](<https://devfeed.tech/tags/bayes.md>), [budget](<https://devfeed.tech/tags/budget.md>), [certificates](<https://devfeed.tech/tags/certificates.md>), [compute](<https://devfeed.tech/tags/compute.md>), [compute-allocation](<https://devfeed.tech/tags/compute-allocation.md>), [consistency](<https://devfeed.tech/tags/consistency.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [distribution](<https://devfeed.tech/tags/distribution.md>), [error](<https://devfeed.tech/tags/error.md>), [scaling](<https://devfeed.tech/tags/scaling.md>), [stat-ml](<https://devfeed.tech/tags/stat-ml.md>), [test](<https://devfeed.tech/tags/test.md>), [time](<https://devfeed.tech/tags/time.md>)

### AI overview

Adaptive choices of sampling budget, prompt, verifier, or aggregation rule can make ordinary validation error bars unreliable. The paper models each inference recipe as a randomized predictor with random cost and proves a time-uniform PAC-Bayes certificate covering all calibration times, posterior mixtures, and data-dependent stopping rules, including the recipe search before deployment. For a finite grid, the penalty is log M/n; non-uniform priors and localized mixtures may sharpen bounds for structured searches. For self-consistency, plurality improves exponentially with sample budget only when the correct answer is the generator's unique mode; otherwise more compute can amplify errors. The margin analysis also gives an oracle water-filling allocation rule. In reported experiments, selected-best error bars fell below out-of-split GSM8K risk in up to 22.5% of random splits at nominal 5%, while the certificate did not; on the full GSM8K test split, the certificate was 0.109 against held-out risk 0.039.

### Source excerpt

arXiv:2611.21241v1 Announce Type: new Abstract: Test-time scaling systems often choose how much to sample, which prompt to use, and which verifier or aggregation rule to trust after repeatedly inspecting a calibration set. Standard validation error bars are not valid after this adaptive selection. We model a test-time scaling recipe as a randomized predictor with a random cost and prove a time-uniform PAC-Bayes certificate that holds simultaneously over all calibration times, all posterior mixtures over recipes, and all data-dependent stopping rules. Thus the same event covers the recipe search that precedes deployment, not merely the fixed recipe finally reported. For a finite grid of budgets and verifiers, the penalty is the familiar $\log M/n$, which in the worst case no certificate valid after selection can avoid, while non-uniform priors and localized posterior mixtures can give sharper certificates for structured searches. We specialize the result to self-consistency and show a complementary margin law: plurality voting improves exponentially in the sample budget only on examples where the correct answer is the generator's unique mode; otherwise more compute can provably amplify errors. The same margin analysis yields an oracle water-filling law for compute allocation. A synthetic audit quantifies the optimism caused by adaptive budget selection, and a 192-example GSM8K audit using gpt-4.1-mini generations illustrates risk-cost reporting for a real inference pipeline. At scale, with two models on GSM8K and MATH-500, selected-best error bars fall below the out-of-split GSM8K risk in up to 22.5% of random splits at a nominal 5% while the certificate never does, and on the full GSM8K test split it tightens to 0.109 (held-out risk 0.039). The results give a lightweight, distribution-free way to report risk and compute claims for adaptive inference systems.

## Which Mathematical Context Supports Target-Aligned Conjecture Generation? A Paired Ablation Study

DevFeed: [Which Mathematical Context Supports Target-Aligned Conjecture Generation? A Paired Ablation Study](<https://devfeed.tech/articles/which-mathematical-context-supports-target-aligned-conjecture-generation-a-paired-ablation-study-137306.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.18621>)

Author: Austin LaHue, Michael Burr, Luis David Garcia Puente, Vinita Gangaram Jansari, Benjamin Nye, Carlos Toxtli Hernandez

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Parallel reasoning](<https://devfeed.tech/topics/parallel-reasoning.md>)

Tags: [ablation](<https://devfeed.tech/tags/ablation.md>), [alignment](<https://devfeed.tech/tags/alignment.md>), [components](<https://devfeed.tech/tags/components.md>), [context](<https://devfeed.tech/tags/context.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [domains](<https://devfeed.tech/tags/domains.md>), [generation](<https://devfeed.tech/tags/generation.md>), [study](<https://devfeed.tech/tags/study.md>), [target](<https://devfeed.tech/tags/target.md>)

### AI overview

Across 61 theorems from 12 papers in four mathematical domains, the study generated 793 conjectures under paired context ablations. Removing one of six context components usually had little effect on alignment with the expected theorem, while keeping only one component substantially reduced alignment. Local Mathematical Objects gave the strongest standalone signal; removing Structural Constraints or Conditions caused the largest alignment drops. The findings suggest these signals work complementarily: objects identify what a claim concerns, while constraints narrow which claims fit. The evidence is limited to the study described in the supplied abstract.

### Source excerpt

arXiv:2611.18621v1 Announce Type: cross Abstract: Generating mathematical conjectures from natural-language context is challenging because theorem-relevant information is distributed across multiple forms of mathematical context, including definitions, objects, assumptions, and constructions. We study which of these contextual signals help a model recover the specific claim supported by the surrounding setup, using 61 main theorems from 12 papers across four mathematical domains. For each theorem, we redact the result and represent the remaining setup using six context components. A paired ablation design compares full context, six leave-one-out conditions, and six single-component conditions, yielding 793 generated conjectures. Context components were largely complementary: removing one component usually had little effect, whereas retaining only one substantially reduced expected-theorem alignment. Local Mathematical Objects provided the strongest standalone signal, while Structural Constraints and Conditions produced the largest alignment decrease when removed. These findings suggest that target-aligned conjecture generation depends on combining information that identifies relevant objects with information that constrains admissible claims.

## CompoundingKernel: Self-Improving from Zero to First

DevFeed: [CompoundingKernel: Self-Improving from Zero to First](<https://devfeed.tech/articles/compoundingkernel-self-improving-from-zero-to-first-137420.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.19230>)

Author: Qian Qu, Hongyang Wang, Wenbo Chen, Yichuan Li, Jingguo Liu, Xian Li, Xuan Wang, Ruochen Wen, Leo Fan, Yiming Yan, Zheshen Wang, Benoit Dumoulin

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [kernels](<https://devfeed.tech/topics/kernels.md>), [Parallel reasoning](<https://devfeed.tech/topics/parallel-reasoning.md>), [ab-mcts](<https://devfeed.tech/topics/ab-mcts.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [search](<https://devfeed.tech/tags/search.md>), [self-improving-agent](<https://devfeed.tech/tags/self-improving-agent.md>)

### AI overview

CompoundingKernel searches for GPU kernels from each problem's PyTorch definition, without reference kernels or official scores. An LLM controller selects an archived correct kernel as a parent or starts a new design, then assigns one direction to each parallel child agent; external checks determine correctness, and the archive preserves successful kernels and their lineage. On NVIDIA B200, all 235 problems had a site-verified correct kernel, and results were 4.40x faster than the reference baseline across 209 public-board problems. The system ranked first on SOL-ExecBench's public L1, L2, and Quantization boards as of 2026-11-11. At equal wall time, it beat a coded UCB-style tree on 35 of 38 problems, producing 2.4x as many correct attempts. More than 1,000 agents ran concurrently, but the authors identify reliable measurement rate--not agent count--as the scaling limit.

### Source excerpt

arXiv:2611.19230v1 Announce Type: new Abstract: We present CompoundingKernel, a self-improving agent system that writes GPU kernels by search. As of 2026-11-11, its generated kernels rank #1 on the public L1, L2 and Quantization boards of SOL-ExecBench (NVIDIA B200). They have held #1 on L1 since 2026-10-27, albeit a short loss on 2026-11-07. All 235 problems have a site-verified correct kernel; on the 209 public-board problems they are 4.40x faster than the benchmark's reference baseline. The search starts from zero: agents get each problem's PyTorch definition but no reference kernels or official scores. A per-problem archive outside the model holds every correct kernel and its lineage. An LLM controller reads it, picks a parent or a new design, and gives each parallel child agent one direction. Checks outside the model decide correctness. At equal wall time, it beat a coded UCB style tree on 35 of 38 problems by making 2.4x as many correct attempts. Over a thousand agents ran at once, and one warm restart fix raised their evaluation rate 3.6x. We find that the rate of reliable measurements, not the number of agents, limits how far such a search scales. Our code and kernels are available on URL.

## Beyond Benchmarks: Evaluating Large Language Models and Agentic Patterns for Static Code Analysis Warning Classification

DevFeed: [Beyond Benchmarks: Evaluating Large Language Models and Agentic Patterns for Static Code Analysis Warning Classification](<https://devfeed.tech/articles/beyond-benchmarks-evaluating-large-language-models-and-agentic-patterns-for-static-code-analysis-warning-classification-137981.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.19086>)

Author: Lakmal Deshapriya, Sherlock A. Licorish, Brendon J. Woodford, Grant Dick

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [Static code analysis](<https://devfeed.tech/topics/static-code-analysis.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [Chain-of-thought](<https://devfeed.tech/topics/chain-of-thought.md>), [Parallel reasoning](<https://devfeed.tech/topics/parallel-reasoning.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-patterns](<https://devfeed.tech/tags/agentic-patterns.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [applications](<https://devfeed.tech/tags/applications.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [classification](<https://devfeed.tech/tags/classification.md>), [code](<https://devfeed.tech/tags/code.md>), [code-analysis](<https://devfeed.tech/tags/code-analysis.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [language](<https://devfeed.tech/tags/language.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [large](<https://devfeed.tech/tags/large.md>), [large-language-models](<https://devfeed.tech/tags/large-language-models.md>), [models](<https://devfeed.tech/tags/models.md>), [patterns](<https://devfeed.tech/tags/patterns.md>), [static](<https://devfeed.tech/tags/static.md>), [static-code-analysis](<https://devfeed.tech/tags/static-code-analysis.md>), [warning](<https://devfeed.tech/tags/warning.md>)

### AI overview

The study finds that LLM performance on static code analysis warning classification depends strongly on the dataset and warning type: best-model F1 scores ranged from 0.52 to 0.96 across four manually annotated datasets. It evaluates three contemporary LLMs selected using LiveCodeBench with Structured Chain-of-Thought prompting, and incorporates coding standards and agentic patterns such as Debating and Self-Consistency. Those patterns did not improve performance in most scenarios. The authors recommend evaluation on real-world applications because results on public datasets or benchmarks may not reflect performance on unseen data.

### Source excerpt

arXiv:2611.19086v1 Announce Type: new Abstract: Context: Static Code Analysis (SCA) tools support software verification and validation, reducing the time and cost of code reviews. However, the best SCA tools in the practitioner community generate a high volume of false positives, which developers tend to ignore, leading to genuine warnings being overlooked. Accordingly, studies increasingly adopt Large Language Models (LLMs) to identify and reduce false positives. Existing studies have produced promising results on public datasets, but underperformed on unseen datasets. Moreover, researchers do not leverage the full potential of Agentic patterns in LLMs or incorporate established code quality standards for warning classification. Objectives: This study utilises four manually annotated datasets to evaluate the performance of three contemporary LLMs for SCA warning classification in real-world applications, and incorporates Agentic patterns and established coding standards to enhance models' performance, addressing gaps identified in existing studies. Method: Three LLMs were selected using the LiveCodeBench benchmark, and the Structured Chain-of-Thought (SCoT) prompting technique was used to evaluate these LLMs. Furthermore, we incorporated Agentic patterns, such as Debating and Self-Consistency, to potentially improve performance in SCA warning classification. Results: Our results show that the performance of different LLMs varies with the characteristics of the dataset, ranging from 0.52 to 0.96 F1-Score for the best models. Additionally, the use of Agentic patterns does not improve LLMs' performance in most scenarios. Conclusions: SCA warning classification using LLMs works best only with certain warning types. We recommend evaluating LLMs in real-world applications rather than using publicly available datasets/benchmarks to understand their full utility.

## Simplify, Agree, Distill: Label-Free Self-Distillation with Stepping Stones

DevFeed: [Simplify, Agree, Distill: Label-Free Self-Distillation with Stepping Stones](<https://devfeed.tech/articles/simplify-agree-distill-label-free-self-distillation-with-stepping-stones-138804.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.18916>)

Author: Dingyang Chen, Jinfeng Xiao, Zhenyu Zhang, William Hu, Ziyu Wang, Yang Liu, Shirley Zhang, Yuan Ling, Xinyang Shen, Qi Zhang, Peijie Qiu

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Post-training optimization](<https://devfeed.tech/topics/post-training-optimization.md>), [Chain-of-thought](<https://devfeed.tech/topics/chain-of-thought.md>), [Parallel reasoning](<https://devfeed.tech/topics/parallel-reasoning.md>)

Tags: [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [distillation](<https://devfeed.tech/tags/distillation.md>), [language](<https://devfeed.tech/tags/language.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [math](<https://devfeed.tech/tags/math.md>), [qwen3](<https://devfeed.tech/tags/qwen3.md>), [thinking](<https://devfeed.tech/tags/thinking.md>)

### AI overview

STONE replaces voting over a model's answers with agreement on easier related problems: the model generates "stepping stones," retains those it solves consistently, and gives their worked solutions--without final answers--to the teacher during distillation. The student still sees only the original problem; the method uses no labels and requires no extra context or model calls at inference. The abstract reports the highest average accuracy across five math benchmarks for Qwen3-4B and Qwen3-8B in thinking mode. Without thinking mode, it reports gains of 13.2 and 22.7 points, respectively, described as 2.5 and 2.7 times the gains of OPSD trained with ground-truth solutions. The supplied text is an abstract, so it does not establish further experimental details.

### Source excerpt

arXiv:2611.18916v1 Announce Type: new Abstract: Can a language model teach itself to reason without labels? On-policy self-distillation (OPSD) uses the model as its own teacher, but strengthens that teacher with a ground-truth solution. The natural label-free substitute, voting over the model's attempts, fails precisely on hard problems: attempts disagree, and their most common answer is often wrong. We propose STONE, a label-free alternative that moves agreement to easier related problems. The model writes these stepping stones, keeps those it solves consistently, and shows the teacher their worked solutions, with final answers removed; the student sees only the original problem. No step reads a label, and inference needs no extra context or model calls. Across five math benchmarks, the proposed method achieves the highest average accuracy for Qwen3-4B and Qwen3-8B in thinking mode. Without thinking mode, it gains 13.2 and 22.7 points, 2.5 and 2.7 times the gains of OPSD with ground-truth solutions.

## Taxonomy of multi-agent design patterns, anti-patterns, and lessons from manufacturing

DevFeed: [Taxonomy of multi-agent design patterns, anti-patterns, and lessons from manufacturing](<https://devfeed.tech/articles/taxonomy-of-multi-agent-design-patterns-anti-patterns-and-lessons-from-manufacturing-136364.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/taxonomy-of-multi-agent-design-patterns-anti-patterns-and-lessons-from-manufacturing/>)

Author: Malini Sethi

Published: 2026-11-16T21:53:46Z

Content type: article

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [Multi Agent Systems](<https://devfeed.tech/topics/multi-agent-systems.md>), [AI Architecture](<https://devfeed.tech/topics/ai-architecture.md>), [software-architecture](<https://devfeed.tech/topics/software-architecture.md>), [agent to agent communication](<https://devfeed.tech/topics/agent-to-agent-communication.md>), [AI Integration Strategies](<https://devfeed.tech/topics/ai-integration-strategies.md>), [supervisor agent pattern](<https://devfeed.tech/topics/supervisor-agent-pattern.md>), [AI Context Management](<https://devfeed.tech/topics/ai-context-management.md>), [Multi-step AI processes](<https://devfeed.tech/topics/multi-step-ai-processes.md>), [hierarchical agent systems](<https://devfeed.tech/topics/hierarchical-agent-systems.md>), [Physical AI](<https://devfeed.tech/topics/physical-ai.md>), [Parallel reasoning](<https://devfeed.tech/topics/parallel-reasoning.md>)

Tags: [advanced-300](<https://devfeed.tech/tags/advanced-300.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agent-loops](<https://devfeed.tech/tags/agent-loops.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [amazon-bedrock](<https://devfeed.tech/tags/amazon-bedrock.md>), [amazon-bedrock-guardrails](<https://devfeed.tech/tags/amazon-bedrock-guardrails.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [automotive](<https://devfeed.tech/tags/automotive.md>), [autonomous](<https://devfeed.tech/tags/autonomous.md>), [aws](<https://devfeed.tech/tags/aws.md>), [best-practices](<https://devfeed.tech/tags/best-practices.md>), [cost](<https://devfeed.tech/tags/cost.md>), [design](<https://devfeed.tech/tags/design.md>), [design-patterns](<https://devfeed.tech/tags/design-patterns.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [distributed-systems](<https://devfeed.tech/tags/distributed-systems.md>), [framework](<https://devfeed.tech/tags/framework.md>), [manufacturing](<https://devfeed.tech/tags/manufacturing.md>), [multi-agent](<https://devfeed.tech/tags/multi-agent.md>), [patterns](<https://devfeed.tech/tags/patterns.md>), [taxonomy](<https://devfeed.tech/tags/taxonomy.md>)

### AI overview

The post recommends starting with deterministic Tier 1 workflows and using autonomous multi-agent coordination only when subtasks emerge during the work, since flexibility adds token cost, latency, and debugging burden. It distinguishes pipelines, conditional workflows, routers, and parallel workflows from plan-based orchestration, handoffs, shared conversations, and debate. It advises hard retry and token limits, schema-based handoffs, bounded agents, and approval gates for actions affecting customers or equipment. For manufacturing, it emphasizes p99 latency budgets and deterministic fallbacks, keeping PLC/SCADA/MES control deterministic, validating recommendations against current OT state before approved actuation, processing time-sensitive sensor data at the edge, and structuring agent memory and handoffs around shifts. AWS IoT Greengrass is named for edge monitoring; the supplied text does not detail the promised broader service mapping.

### Source excerpt

Multi-agent coordination is a distributed systems problem, not a model problem. This post gives a field-tested taxonomy of multi-agent design patterns and anti-patterns, a framework for deciding when multiple agents are warranted, and how to build cost-efficient, reliable systems on AWS.

## DECODE: Diagnosing Interpretability Failures in Automated Interpretation Process

DevFeed: [DECODE: Diagnosing Interpretability Failures in Automated Interpretation Process](<https://devfeed.tech/articles/decode-diagnosing-interpretability-failures-in-automated-interpretation-process-135129.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.17856>)

Author: Dohyun Kim, Jaeung Lee, Donghee Hong, Jaemin Jo

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [AI Chat](<https://devfeed.tech/topics/ai-chat.md>), [human review](<https://devfeed.tech/topics/human-review.md>), [Ground truth / benchmark quality](<https://devfeed.tech/topics/ground-truth-benchmark-quality.md>), [Parallel reasoning](<https://devfeed.tech/topics/parallel-reasoning.md>)

Tags: [analysis](<https://devfeed.tech/tags/analysis.md>), [analytics](<https://devfeed.tech/tags/analytics.md>), [autoencoder](<https://devfeed.tech/tags/autoencoder.md>), [automated](<https://devfeed.tech/tags/automated.md>), [automation](<https://devfeed.tech/tags/automation.md>), [case-study](<https://devfeed.tech/tags/case-study.md>), [consensus](<https://devfeed.tech/tags/consensus.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-hc](<https://devfeed.tech/tags/cs-hc.md>), [decode](<https://devfeed.tech/tags/decode.md>), [defects](<https://devfeed.tech/tags/defects.md>), [diagnostic](<https://devfeed.tech/tags/diagnostic.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [failure](<https://devfeed.tech/tags/failure.md>), [failures](<https://devfeed.tech/tags/failures.md>), [features](<https://devfeed.tech/tags/features.md>), [generate](<https://devfeed.tech/tags/generate.md>), [human-oversight](<https://devfeed.tech/tags/human-oversight.md>), [inspection](<https://devfeed.tech/tags/inspection.md>), [interpretability](<https://devfeed.tech/tags/interpretability.md>), [manual-review](<https://devfeed.tech/tags/manual-review.md>), [process](<https://devfeed.tech/tags/process.md>)

### AI overview

DECODE adds human oversight to automated interpretation of LLM representations, where explanations are generated and evaluated by LLMs and ground truth is lacking. Its three-stage diagnostic workflow helps separate structural defects from explanation-level misalignments. It uses agreement among independently generated explanations and their evaluation metrics as a proxy for reliability, then applies a three-phase active seeding protocol to recommend instances for manual review. A case study involving 16,384 Sparse Autoencoder features reports that the system can identify and attribute interpretability failures at scale.

### Source excerpt

arXiv:2611.17856v1 Announce Type: new Abstract: Tools for interpreting the internal representations of Large Language Models (LLMs) increasingly rely on LLMs themselves to both generate and evaluate explanations, creating a self-referential automated interpretation process with minimal human oversight. This automation raises significant reliability concerns, which are exacerbated by the absence of ground truth. In practice, it is difficult to validate explanations or to attribute interpretability failures to their source -- whether they originate from the tool's extracted representations or from the LLM used in the process. To address these challenges, we present DECODE, a visual analytics system that reintroduces human oversight into the automated interpretation process. Our approach decomposes the diagnostic workflow into three progressive stages, isolating failure sources ranging from structural defects to explanation-level misalignments. Drawing inspiration from inter-coder consensus in qualitative analysis, we utilize consensus among independently generated explanations and their evaluation metrics as a proxy for reliability. To further scale human inspection, we employ a three-phase active seeding protocol that strategically recommends high-priority instances for manual review. Finally, we demonstrate the effectiveness of our approach through a case study on 16,384 Sparse Autoencoder features, showing that it can efficiently identify and attribute interpretability failures at scale.

## Shared Item-Level Error Limits the Aggregation of Small Language Models in Fermi Estimation

DevFeed: [Shared Item-Level Error Limits the Aggregation of Small Language Models in Fermi Estimation](<https://devfeed.tech/articles/shared-item-level-error-limits-the-aggregation-of-small-language-models-in-fermi-estimation-132218.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.16315>)

Author: Carolina Fortuna, Blaz Bertalanic

Published: 2026-11-13T05:00:00Z

Content type: news

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Multi Agent Systems](<https://devfeed.tech/topics/multi-agent-systems.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Parallel reasoning](<https://devfeed.tech/topics/parallel-reasoning.md>)

Tags: [aggregation](<https://devfeed.tech/tags/aggregation.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [bias](<https://devfeed.tech/tags/bias.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-ma](<https://devfeed.tech/tags/cs-ma.md>), [error](<https://devfeed.tech/tags/error.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [multi-agent](<https://devfeed.tech/tags/multi-agent.md>), [multi-agent-systems](<https://devfeed.tech/tags/multi-agent-systems.md>), [open-weight-models](<https://devfeed.tech/tags/open-weight-models.md>)

### AI overview

Across the tested Fermi estimation tasks, adding more language-model agents offered little accuracy gain because most error was shared at the item level. In 2.51 million answers from 15 open-weight models, this component accounted for a median 90.6% of error variance; teams of 30 reduced single-agent error by a median 1.7% (maximum 5.4%). Three debate rounds cut within-team dispersion by 93% but left shared error unchanged, and solo revision achieved the same small gain. Estimates tended to compress toward 10^6; recalibration removed overall bias but not item-level error. Mixing five similarly accurate models helped against their own homogeneous teams but did not beat the most accurate model. Temperature changes had at most about a 1% effect, while contrastive activation steering had no detectable effect. The findings are conditional on the models, benchmarks, prompts and settings tested.

### Source excerpt

arXiv:2611.16315v1 Announce Type: cross Abstract: Multi-agent systems of language models are often motivated by an averaging argument: if the agents' errors are unbiased and independent, the error of an aggregate of N answers falls as 1/sqrt(N). Fermi estimation, whose continuous answers are aggregated on the logarithmic scale, permits a direct test. We decompose the log10 error of each answer into the mean error of its model on the item, common to all its draws on the item, and a residual, and estimate both from 2.51M answers given before any communication by 15 open-weight models (3B-72B) on realFP, with replication under a single prompt pair on realFP, Science Olympiad Fermi questions and a guesstimation set. Given the item, teammates' errors are uncorrelated and teams behave as independent draws, but the item-level component accounts for a median 0.906 of the error variance over items. Teams of thirty therefore reduce a single agent's error by a median 1.7% (at most 5.4%), whereas draws that were also unbiased on each item would reduce it by 74-79%. All models compress their estimates toward about 10^6; this trend explains a median 17% of the item-level component, and a cross-validated linear recalibration on labelled items removes the overall bias but not that component. Three rounds of debate reduce within-team dispersion by 93% and leave the item-level component unchanged; revision without peers attains the same small gain. Mixing five models of similar accuracy reduces error by 6.6% relative to their own homogeneous teams, but no mixture improves on the most accurate model. Higher sampling temperature changes team error by about 1% at most, and contrastive activation steering not detectably; per-agent scaffold prompts (not significant after multiplicity correction) and skipping one layer in a single-model screen improve it. The results are conditional on the models, benchmarks, prompts and settings tested.

## Early Signals, Strong Decisions: Prefix-Guided Sampling for Parallel Test-Time Scaling

DevFeed: [Early Signals, Strong Decisions: Prefix-Guided Sampling for Parallel Test-Time Scaling](<https://devfeed.tech/articles/early-signals-strong-decisions-prefix-guided-sampling-for-parallel-test-time-scaling-132075.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.14965>)

Author: Jie Ren, Jonathan Rosenfeld, Neil Thompson

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Parallel reasoning](<https://devfeed.tech/topics/parallel-reasoning.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>)

Tags: [aggregation](<https://devfeed.tech/tags/aggregation.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [boundary](<https://devfeed.tech/tags/boundary.md>), [budget](<https://devfeed.tech/tags/budget.md>), [compute](<https://devfeed.tech/tags/compute.md>), [confidence](<https://devfeed.tech/tags/confidence.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [decision](<https://devfeed.tech/tags/decision.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [independent](<https://devfeed.tech/tags/independent.md>), [language](<https://devfeed.tech/tags/language.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [large-language-models](<https://devfeed.tech/tags/large-language-models.md>), [margin](<https://devfeed.tech/tags/margin.md>), [model](<https://devfeed.tech/tags/model.md>), [multiple](<https://devfeed.tech/tags/multiple.md>), [parallel](<https://devfeed.tech/tags/parallel.md>), [performance](<https://devfeed.tech/tags/performance.md>), [sampling](<https://devfeed.tech/tags/sampling.md>), [scaling](<https://devfeed.tech/tags/scaling.md>)

### AI overview

Prefix-Guided Sampling (PreG) aims to improve parallel test-time scaling by avoiding full completion of low-quality reasoning paths. It first generates short prefixes, ranks them with internal confidence scores, and completes only the most promising candidates, reducing token use while maintaining gains in path quality. The abstract reports that confidence matters most for questions near a decision boundary and that even small signals from short prefixes can affect aggregation there. Its theoretical analysis says PreG widens the decision-boundary margin under a fixed compute budget; experiments reportedly outperform standard parallel TTS across benchmarks while improving token efficiency and reducing computation. The excerpt does not specify benchmarks or quantitative results.

### Source excerpt

arXiv:2611.14965v1 Announce Type: new Abstract: Parallel test-time scaling (TTS) has been shown to enhance reasoning in large language models by generating and aggregating multiple independent reasoning paths. Internal confidence scores can further improve this process by identifying paths of varying quality. However, existing methods remain computationally inefficient, as low-quality paths are fully expanded before being evaluated. In this work, we aim to improve the use of internal confidence signals for better model performance and computational efficiency. We begin by offering a new perspective on the role of confidence in aggregation, showing that it is most impactful for questions near the decision boundary. Notably, when amplified by multi-sample aggregation, even small confidence signals derived from short prefixes can meaningfully influence outcomes in these boundary cases. Building on this insight, we introduce Prefix-Guided Sampling (PreG), a method that reallocates computation by first generating short prefixes, ranking them using internal confidence scores, and then completing only the most promising candidates. This strategy reduces token usage while maintaining gains in path quality. We provide theoretical analysis demonstrating that, under a fixed compute budget, PreG strengthens the effective signal by widening the decision boundary margin. Empirically, our method consistently outperforms standard parallel TTS approaches across benchmarks, while significantly improving token efficiency and reducing overall computation.

## From Noise to Diversity: Random Embedding Injection in LLM Reasoning

DevFeed: [From Noise to Diversity: Random Embedding Injection in LLM Reasoning](<https://devfeed.tech/articles/from-noise-to-diversity-random-embedding-injection-in-llm-reasoning-131912.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2605.11936>)

Author: Heejun Kim, Seungpil Lee, Jewon Yeom, Jaewon Sok, Seonghyeon Park, Jeongjae Park, Taesup Kim, Sundong Kim

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Parallel reasoning](<https://devfeed.tech/topics/parallel-reasoning.md>), [prompt](<https://devfeed.tech/topics/prompt.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Temperature settings in AI](<https://devfeed.tech/topics/temperature-settings-in-ai.md>), [grpo](<https://devfeed.tech/topics/grpo.md>)

Tags: [attention](<https://devfeed.tech/tags/attention.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [diversity](<https://devfeed.tech/tags/diversity.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [inference](<https://devfeed.tech/tags/inference.md>), [llm](<https://devfeed.tech/tags/llm.md>), [noise](<https://devfeed.tech/tags/noise.md>), [prompt](<https://devfeed.tech/tags/prompt.md>), [random](<https://devfeed.tech/tags/random.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [sampling](<https://devfeed.tech/tags/sampling.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

Random Soft Prompts (RSPs) append a freshly sampled sequence of random embedding vectors to an LLM input, avoiding prompt training while reaching accuracy comparable to optimized soft prompts on math reasoning benchmarks in several settings. The vectors are drawn from an isotropic Gaussian fitted to the pretrained embedding table's entrywise mean and variance. The authors propose that attention to the unfamiliar positions flattens the distribution over early generated tokens, branching reasoning trajectories; that effect then dilutes as generation continues. At inference, RSPs increase early token diversity and, with temperature sampling, widen Pass@N--the chance that at least one of N attempts is correct. The authors also report practical gains when applying the effect in DAPO training. The supplied text does not specify benchmark names or numerical results.

### Source excerpt

arXiv:2605.11936v3 Announce Type: replace Abstract: Recent soft prompt research has tried to improve reasoning by inserting trained vectors into LLM inputs, yet whether the gain comes from the learned content or from the act of injection itself has not been carefully separated. We study Random Soft Prompts (RSPs), which drop the training step entirely and append a freshly drawn sequence of random embedding vectors to the input. Each RSP vector is sampled from an isotropic Gaussian fitted to the entrywise mean and variance of the pretrained embedding table; the sequence carries no learned content, and yet reaches accuracy comparable to optimized soft prompts on math reasoning benchmarks in several settings. The mechanism unfolds in two stages: because attention has to absorb a never-seen-before random position, the distribution over the first few generated tokens flattens and reasoning trajectories branch, and as generation continues this influence dilutes naturally so the response commits to a single completion. We show that during inference RSPs lift early-stage token diversity and, combined with temperature sampling, widen Pass@N, the probability that at least one out of N attempts is correct. Beyond inference, we carry the same effect into DAPO training and demonstrate practical gains. Our contributions are: (i) RSP isolates the simplest form of soft prompt -- training-free, freshly resampled -- providing a unified lens for the structural effect of injection that variants otherwise differing in training and form all share; (ii) a theoretical and empirical validation of the underlying mechanism; and (iii) an extension from inference to training.

## Same Task, Different Statement: When and How Reformulation Helps LLM Problem Solving

DevFeed: [Same Task, Different Statement: When and How Reformulation Helps LLM Problem Solving](<https://devfeed.tech/articles/same-task-different-statement-when-and-how-reformulation-helps-llm-problem-solving-132078.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.14997>)

Author: Wentao Zhang, Xiaoyu Zhang, Chengwei Liu, Yutao Hu, Tao Zhang, Junsheng Wu, Hongping Gan, Yang Liu

Published: 2026-11-13T05:00:00Z

Content type: news

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Parallel reasoning](<https://devfeed.tech/topics/parallel-reasoning.md>)

Tags: [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [chain-of-thought](<https://devfeed.tech/tags/chain-of-thought.md>), [code](<https://devfeed.tech/tags/code.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [effect](<https://devfeed.tech/tags/effect.md>), [fixed](<https://devfeed.tech/tags/fixed.md>), [frame](<https://devfeed.tech/tags/frame.md>), [hidden](<https://devfeed.tech/tags/hidden.md>), [llm](<https://devfeed.tech/tags/llm.md>), [logic](<https://devfeed.tech/tags/logic.md>), [math](<https://devfeed.tech/tags/math.md>), [oracle](<https://devfeed.tech/tags/oracle.md>), [patterns](<https://devfeed.tech/tags/patterns.md>), [problem-solving](<https://devfeed.tech/tags/problem-solving.md>), [question](<https://devfeed.tech/tags/question.md>), [reduction](<https://devfeed.tech/tags/reduction.md>), [rewrite](<https://devfeed.tech/tags/rewrite.md>), [structure](<https://devfeed.tech/tags/structure.md>), [study](<https://devfeed.tech/tags/study.md>)

### AI overview

The study finds that reformulating a problem can improve LLM accuracy, but the benefit depends on matching the rewrite to the problem type. It tests decomposition (making subgoals explicit), typed reformulation (placing a task in a suitable solving frame), and concept reduction (removing surface detail) across eight math, logic, and code benchmarks and eight models under the same chain-of-thought prompt. An oracle choosing the best rewrite after the fact improves accuracy by an average of 27.6%, but no rewrite helps every problem and a mismatched rewrite can hurt. A fixed rule learned on one problem set transfers some benefit: it gains 11.0% on cases it selects for rewriting, while blind rewriting loses 5.6% on a similar share. The findings suggest selective, problem-dependent reformulation rather than rewriting everything.

### Source excerpt

arXiv:2611.14997v1 Announce Type: new Abstract: When people face a hard problem, they often restate it in a clearer way before solving it. We study whether the same habit helps large language models by rewriting the statement of a problem. A legal rewrite must keep the meaning, the constraints, and the correct answer unchanged. We test three kinds of rewrites. Decomposition makes hidden subgoals explicit. Typed Reformulation casts the task into a more suitable solving frame. Concept Reduction removes surface detail and exposes the task structure. We evaluate them on eight math, logic, and code benchmarks with eight models, all under the same chain-of-thought solving prompt. The effect is large but depends on the problem type. An oracle that picks the best rewrite after the fact raises accuracy by +27.6% on average. Yet no single rewrite helps every problem, and a mismatched rewrite can hurt. Still, the effect follows clear patterns. Each kind of rewrite fits certain problem types. A simple fixed rule learned from one set of problems transfers part of the gain to new problems. It gains +11.0% on the problems it chooses to rewrite, while blind rewriting loses 5.6% on a similar share of problems. Rewriting is therefore not a universal way to solve problems. The question is when and how to rewrite.

## Contrastive Discovery: Open-Ended Scientific Discovery over Competing Explanations

DevFeed: [Contrastive Discovery: Open-Ended Scientific Discovery over Competing Explanations](<https://devfeed.tech/articles/contrastive-discovery-open-ended-scientific-discovery-over-competing-explanations-128438.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.14514>)

Author: Ziang Liu, James J. Kim, Yijia Dai, Jennifer J. Sun

Published: 2026-11-11T05:00:00Z

Content type: news

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [AI for science](<https://devfeed.tech/topics/ai-for-science.md>), [Parallel reasoning](<https://devfeed.tech/topics/parallel-reasoning.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agents](<https://devfeed.tech/tags/agents.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [budget](<https://devfeed.tech/tags/budget.md>), [but](<https://devfeed.tech/tags/but.md>), [can](<https://devfeed.tech/tags/can.md>), [change](<https://devfeed.tech/tags/change.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [data-driven](<https://devfeed.tech/tags/data-driven.md>), [discovery](<https://devfeed.tech/tags/discovery.md>), [discovery-agents](<https://devfeed.tech/tags/discovery-agents.md>), [domains](<https://devfeed.tech/tags/domains.md>), [exclusive](<https://devfeed.tech/tags/exclusive.md>), [exhaustive](<https://devfeed.tech/tags/exhaustive.md>), [existing](<https://devfeed.tech/tags/existing.md>), [experiment](<https://devfeed.tech/tags/experiment.md>), [experts](<https://devfeed.tech/tags/experts.md>), [open](<https://devfeed.tech/tags/open.md>), [over](<https://devfeed.tech/tags/over.md>)

### AI overview

Contrastive Discovery helps scientific discovery agents identify which explanation evidence supports, instead of only rejecting a single hypothesis. It represents a question as mutually exclusive, collectively exhaustive alternatives and scores experiments by how they shift belief across those alternatives; a question is resolved when belief concentrates on one. The approach can reassess existing evidence or guide new experiments. Across six domains, contrast sets identified a specific alternative for 73% of candidates rejected by pointwise evaluation. Under the same experiment budget, guided search resolved 67% more questions than pointwise surprisal. In a blinded study, 38 experts agreed with the agent's explanation on 78% of resolved questions, including 82% of cases where the pointwise agent had rejected its hypothesis and stopped.

### Source excerpt

arXiv:2611.14514v1 Announce Type: new Abstract: LLM agents are increasingly used for open-ended, data-driven scientific discovery, yet most of them evaluate hypotheses pointwise, testing an empirical claim $h$ only against its complement $\neg h$. When evidence rejects $h$, the agent learns that one explanation is unlikely but not which alternative the evidence supports. To recover that alternative, we introduce Contrastive Discovery, which replaces the single hypothesis with a contrast set, a scientific question paired with mutually exclusive and collectively exhaustive competing explanations. Each experiment is scored by how its evidence redistributes belief across the set, and the question is resolved when belief concentrates on one explanation. Because contrast sets change only the unit an agent proposes and scores, an existing agent can use them to re-evaluate evidence it has already collected or to guide its search. Across six domains, $89\%$ of the pointwise candidates with the largest belief shifts are rejections. On the same evidence, contrast sets resolve $73\%$ of these candidates to a specific alternative. When contrast sets guide search, the agent resolves $67\%$ more questions than pointwise surprisal under the same experiment budget. In a blinded study, $38$ human experts chose the same explanation on $78\%$ of resolved questions and on $82\%$ of questions where the pointwise agent rejected its hypothesis and stopped. Together, these results support explicit competing explanations as a way for discovery agents to resolve more scientific questions from the same evidence and validation budget.

## Cognitive Mapping of Learning by Thinking Mechanisms in Large Language Models

DevFeed: [Cognitive Mapping of Learning by Thinking Mechanisms in Large Language Models](<https://devfeed.tech/articles/cognitive-mapping-of-learning-by-thinking-mechanisms-in-large-language-models-128425.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.14382>)

Author: Dong Han, Jin Zhang, Kunyue Yang

Published: 2026-11-11T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Chain-of-thought](<https://devfeed.tech/topics/chain-of-thought.md>), [Parallel reasoning](<https://devfeed.tech/topics/parallel-reasoning.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [chain-of-thought](<https://devfeed.tech/tags/chain-of-thought.md>), [concept-drift](<https://devfeed.tech/tags/concept-drift.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [framework](<https://devfeed.tech/tags/framework.md>), [generator](<https://devfeed.tech/tags/generator.md>), [grounding](<https://devfeed.tech/tags/grounding.md>), [improvement](<https://devfeed.tech/tags/improvement.md>), [inference](<https://devfeed.tech/tags/inference.md>), [input](<https://devfeed.tech/tags/input.md>), [instructions](<https://devfeed.tech/tags/instructions.md>), [large-language-models-llms](<https://devfeed.tech/tags/large-language-models-llms.md>), [learning](<https://devfeed.tech/tags/learning.md>), [limits](<https://devfeed.tech/tags/limits.md>), [llms](<https://devfeed.tech/tags/llms.md>), [loops](<https://devfeed.tech/tags/loops.md>), [memory](<https://devfeed.tech/tags/memory.md>), [model](<https://devfeed.tech/tags/model.md>), [monitor](<https://devfeed.tech/tags/monitor.md>), [performance](<https://devfeed.tech/tags/performance.md>), [thinking](<https://devfeed.tech/tags/thinking.md>)

### AI overview

The paper proposes a framework connecting four Learning by Thinking mechanisms in humans--reflective reasoning, synthetic memory formation, internal simulation, and conceptual self-organization--to processes in large language models. It argues that methods such as chain-of-thought, self-consistency, Tree-of-Thoughts, and Self-Instruct can approximate aspects of these mechanisms and improve some tasks without new labeled examples, but may not produce durable learning in model parameters. As one reported example, Self-Instruct generated about 52,000 instructions from 175 seed tasks and achieved a 33% absolute improvement on Super-NaturalInstructions. The framework also links hallucination diffusion, model collapse, concept drift, and high computation costs to structural differences from human cognition, including closed loops without external grounding and inference-time computation. The abstract presents testable propositions for when these methods may succeed or fail.

### Source excerpt

arXiv:2611.14382v1 Announce Type: new Abstract: Large language models (LLMs) increasingly exhibit capabilities that resemble human Learning by Thinking (LbT), the acquisition of new insight through internal processes such as explanation and reasoning, without new external input. A systematic account of the cognitive mechanisms behind these capabilities, and of their limits, remains underdeveloped. This paper proposes a Cognitive LbT Framework that maps four mechanisms (Reflective Reasoning, Synthetic Memory Formation, Internal Simulation, and Conceptual Self-Organization) onto computational processes in LLMs. Drawing on representative methods, including chain-of-thought prompting, self-consistency, self-reflection, Tree-of-Thoughts search, Self-Instruct, and self-taught reasoning, the analysis argues that selected methods can functionally approximate aspects of these mechanisms and can improve performance on some tasks without additional labeled examples, although this does not always constitute durable learning in model parameters. For example, Self-Instruct generated approximately 52,000 instructions from 175 seed tasks and reported a 33% absolute improvement on Super-NaturalInstructions (Y. Wang et al., 2023). The same mechanisms also produce characteristic failures, including hallucination diffusion, model collapse, concept drift, and high computational cost. These failures are not isolated technical defects but follow from four structural asymmetries between LLMs and human cognition: the identity of generator and monitor, closed learning loops without external grounding, symbolic rather than sensorimotor simulation, and the cost of inference-time computation. The framework yields testable propositions about when LbT in LLMs should succeed or fail and offers a structured basis for designing more reliable autonomous learning systems.

## BARS: Budget-Aware Sequential Sampling with Refusal for Majority Identification under Unknown Distributions

DevFeed: [BARS: Budget-Aware Sequential Sampling with Refusal for Majority Identification under Unknown Distributions](<https://devfeed.tech/articles/bars-budget-aware-sequential-sampling-with-refusal-for-majority-identification-under-unknown-distributions-128403.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.14171>)

Author: Shenxi Liu, Yiwei Li, Mingyang Zhao, Yuhang Tian, Kan Li

Published: 2026-11-11T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Chinchilla scaling law](<https://devfeed.tech/topics/chinchilla-scaling-law.md>), [Parallel reasoning](<https://devfeed.tech/topics/parallel-reasoning.md>)

Tags: [about](<https://devfeed.tech/tags/about.md>), [abstract](<https://devfeed.tech/tags/abstract.md>), [also](<https://devfeed.tech/tags/also.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [behavior](<https://devfeed.tech/tags/behavior.md>), [budget](<https://devfeed.tech/tags/budget.md>), [but](<https://devfeed.tech/tags/but.md>), [confidence](<https://devfeed.tech/tags/confidence.md>), [consistency](<https://devfeed.tech/tags/consistency.md>), [cost](<https://devfeed.tech/tags/cost.md>), [costs](<https://devfeed.tech/tags/costs.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [distribution](<https://devfeed.tech/tags/distribution.md>), [distributions](<https://devfeed.tech/tags/distributions.md>), [effective](<https://devfeed.tech/tags/effective.md>), [error](<https://devfeed.tech/tags/error.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [fewer](<https://devfeed.tech/tags/fewer.md>), [sampling](<https://devfeed.tech/tags/sampling.md>), [under](<https://devfeed.tech/tags/under.md>), [with](<https://devfeed.tech/tags/with.md>)

### AI overview

BARS uses sequential sampling to certify whether the leading answer has a sufficient gap over the runner-up, then returns an early stop, a budgeted majority vote, or a refusal if feasibility cannot be established. Refusal signals a limit of the rule and available budget, not that the question is unanswerable. The proposed certified mode provides proven bounds on joint answering error and on refusing cases the budget could handle; the practical mode used in experiments substitutes empirical calibration. On GSM8K, practical BARS used 22.3% fewer samples than fixed-8 sampling, including refused questions, while declining 18.8% of questions. On MATH, output accuracy reached 82.5%, while the method refused about half of the difficult questions the model could not solve.

### Source excerpt

arXiv:2611.14171v1 Announce Type: new Abstract: Repeated sampling with majority selection is an effective means of enforcing self-consistency in large language models. Recently, a variant has also been introduced that terminates early to save costs when the output distribution is favorable. But it still cannot yield a correct confidence estimate from the answer distribution. For this issue, we formalize this as majority identification under an unknown distribution with a budget and a target error rate, and propose BARS, a sequential rule certifying the top-two gap and returning an early stop, a budgeted majority vote, or a refusal when feasibility cannot be established. A refusal is a statement about the rule and its budget, not a claim that a question is unanswerable, and it tells practitioners when to widen the budget or switch models. A certified mode carries proven bounds on the joint answering error and on refusing an instance the budget could handle, while a practical mode, used in our experiments, replaces them with empirical calibration. On synthetic instances drawn from known distributions over thousands of seeds, the difficulty-driven behavior is recovered and the price of the certified mode is measured.On GSM8K the practical mode draws 22.3\% fewer samples than fixed-8 sampling over the same pool, refused questions included, at the cost of declining 18.8\% of the questions. On MATH, it greatly improves the accuracy of its output answers into 82.5\% at the cost of honestly refusing to answer half of the difficult questions that the model itself is unable to solve.

## MedMMV: A Controllable Multimodal Multi-Agent Framework for Reliable and Verifiable Clinical Reasoning

DevFeed: [MedMMV: A Controllable Multimodal Multi-Agent Framework for Reliable and Verifiable Clinical Reasoning](<https://devfeed.tech/articles/medmmv-a-controllable-multimodal-multi-agent-framework-for-reliable-and-verifiable-clinical-reasoning-125824.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2509.24314>)

Author: Hongjun Liu, Yinghao Zhu, Yuhui Wang, Yitao Long, Lequan Yu, Chen Zhao

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Parallel reasoning](<https://devfeed.tech/topics/parallel-reasoning.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [Uncertainty quantification LLMs](<https://devfeed.tech/topics/uncertainty-quantification-llms.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agent-framework](<https://devfeed.tech/tags/agent-framework.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [framework](<https://devfeed.tech/tags/framework.md>), [multi-agent](<https://devfeed.tech/tags/multi-agent.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [post-training](<https://devfeed.tech/tags/post-training.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>)

### AI overview

MedMMV aims to reduce hallucinations in multimodal clinical reasoning by treating reasoning as a search among candidate trajectories: it constructs, refines, and selects evidence-grounded paths, using early uncertainty as a signal of trajectory instability. The selected paths also provide supervision for post-training. Reject sampling and pairwise preference learning train smaller models to favor preferred trajectories in a single pass, rather than simply imitate reasoning traces. The abstract reports accuracy rising from 10.5% to 29.0% with richer supervision; it does not specify the evaluation setting or comparative baselines.

### Source excerpt

arXiv:2509.24314v2 Announce Type: replace Abstract: Current multimodal large language models (MLLMs) suffer from unstable reasoning due to the lack of mechanisms for controlling reasoning trajectories, often leading to hallucinated conclusions. Our analysis reveals that early-stage uncertainty induces trajectory switching, significantly increasing hallucination risk. We propose MedMMV, an evidence-grounded trajectory control framework that treats reasoning as a search-and-selection process over candidate trajectories. By constructing, refining, and selecting among multiple reasoning paths, MedMMV improves reliability and produces structured supervision signals over trajectory quality. We further convert these signals into learnable objectives for post-training, including reject sampling and pairwise preference learning, enabling smaller models to learn trajectory selection as an efficient single-pass capability. Instead of merely imitating reasoning traces, the model is trained to assign higher likelihood to preferred trajectories. Experiments show that richer supervision leads to stronger transfer, with accuracy improving from 10.5% to 29.0%. Our results demonstrate that trajectory selection is the key transferable capability for reliable reasoning.

[Next page](<https://devfeed.tech/topics/parallel-reasoning.md?cursor=WyIyMDI2LTExLTEwVDA1OjAwOjAwKzAwOjAwIiwgImVmNjliNmFiLWViOTUtNGY3Mi1iMWM1LThmMzEzNDEyNTRiOCJd>)