# monte carlo tree search

Monte Carlo Tree Search is a decision-making algorithm that searches tree-modeled combinatorial spaces using simulations and action statistics.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## The Montparnasse Algorithm for RNA Design

DevFeed: [The Montparnasse Algorithm for RNA Design](<https://devfeed.tech/articles/the-montparnasse-algorithm-for-rna-design-145346.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2606.07562>)

Author: Tristan Cazenave

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [monte carlo tree search](<https://devfeed.tech/topics/monte-carlo-tree-search.md>), [Combinatorial optimization](<https://devfeed.tech/topics/combinatorial-optimization.md>)

Tags: [algorithm](<https://devfeed.tech/tags/algorithm.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [biology](<https://devfeed.tech/tags/biology.md>), [budget](<https://devfeed.tech/tags/budget.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [design](<https://devfeed.tech/tags/design.md>), [diversity](<https://devfeed.tech/tags/diversity.md>), [energy](<https://devfeed.tech/tags/energy.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [faster](<https://devfeed.tech/tags/faster.md>), [first](<https://devfeed.tech/tags/first.md>), [fleet](<https://devfeed.tech/tags/fleet.md>), [folding](<https://devfeed.tech/tags/folding.md>), [framework](<https://devfeed.tech/tags/framework.md>), [median](<https://devfeed.tech/tags/median.md>), [medicine](<https://devfeed.tech/tags/medicine.md>), [monte-carlo](<https://devfeed.tech/tags/monte-carlo.md>), [nanotechnology](<https://devfeed.tech/tags/nanotechnology.md>), [parallel](<https://devfeed.tech/tags/parallel.md>), [q-bio-bm](<https://devfeed.tech/tags/q-bio-bm.md>), [rna](<https://devfeed.tech/tags/rna.md>), [solve](<https://devfeed.tech/tags/solve.md>)

### AI overview

Montparnasse combines Monte Carlo search with an energy-model structural prior, extended level-1 adaptation, lexicographic evaluation, local repair of incorrectly predicted base pairs, and parallel workers sharing a beam diversified by candidate faults. On Eterna100 V1, with both it and DesiRNA using ViennaRNA 1.8.5 and the Turner 1999 model, 100 workers, and a 30-minute per-puzzle budget, it solved a median of 96 puzzles across five passes, versus 90 for DesiRNA. It was faster on 88 of the 90 puzzles both solved, with median solving times of 0.9 seconds versus 5.6 seconds; 55 puzzles were solved within one second. The puzzle not solved in any pass was solved with a longer budget under the same model.

### Source excerpt

arXiv:2606.07562v2 Announce Type: replace-cross Abstract: Inverse RNA Folding consists of finding a nucleotide sequence that folds into a predefined secondary structure. It is useful for synthetic biology, medicine, and nanotechnology. We propose Montparnasse, a Monte Carlo search framework based on Generalized Nested Rollout Policy Adaptation, augmented with a structural prior derived from the energy model, slow and long adaptation at level 1, a lexicographic multicriteria evaluation, a local repair of the pairs the predicted folding gets wrong, and a fleet of parallel workers coupled through a shared beam whose diversity is maintained on the faults the candidates make rather than on the candidates themselves. On the Eterna100 V1 benchmark, with both solvers folding through the same ViennaRNA 1.8.5 library under the Turner 1999 model, with 100 parallel workers and a budget of 30 minutes per puzzle, Montparnasse solves a median of 96 of the 100 puzzles against a median of 90 for DesiRNA, the previous state of the art, each figure taken over five passes. It is faster on 88 of the 90 puzzles that both solve, with a median solving time of 0.9 s against 5.6 s, and it solves 55 puzzles within the first second. The one puzzle no pass reaches is solved under the same model with a longer budget, so every Eterna100 V1 target is solved by Montparnasse.

## Orbits as Strong Models: Kolmogorov Structure Function under Symmetry

DevFeed: [Orbits as Strong Models: Kolmogorov Structure Function under Symmetry](<https://devfeed.tech/articles/orbits-as-strong-models-kolmogorov-structure-function-under-symmetry-142569.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.40290>)

Author: Romie Banerjee

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Occam's razor machine learning](<https://devfeed.tech/topics/occam-s-razor-machine-learning.md>), [monte carlo tree search](<https://devfeed.tech/topics/monte-carlo-tree-search.md>)

Tags: [arxiv](<https://devfeed.tech/tags/arxiv.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-it](<https://devfeed.tech/tags/cs-it.md>), [groups](<https://devfeed.tech/tags/groups.md>), [information-theory](<https://devfeed.tech/tags/information-theory.md>), [linear](<https://devfeed.tech/tags/linear.md>), [math-gr](<https://devfeed.tech/tags/math-gr.md>), [math-it](<https://devfeed.tech/tags/math-it.md>), [models](<https://devfeed.tech/tags/models.md>), [orbit](<https://devfeed.tech/tags/orbit.md>), [partition](<https://devfeed.tech/tags/partition.md>), [partitions](<https://devfeed.tech/tags/partitions.md>), [statistics](<https://devfeed.tech/tags/statistics.md>), [symmetry](<https://devfeed.tech/tags/symmetry.md>)

### AI overview

The paper studies strong algorithmic-statistics models through orbit partitions induced by group actions on binary strings. Their structure function restricts Kolmogorov's models to sets whose setwise stabilizer acts transitively; under the full symmetric group, this recovers exactly the strong models. For GL(n,2), the cells are linearly homogeneous sets, and the structure function lies between C(x)−α and n−α, attaining both bounds. Some stochastic normal strings have simple structure that linear symmetry misses. The authors also argue that any searchable space of symmetry hypotheses can miss simple structure, and formulate orbit-model search as value estimation on subgroup lattices, with computable bounds, certified training targets, and a lattice version of Monte Carlo tree search.

### Source excerpt

arXiv:2609.40290v3 Announce Type: replace-cross Abstract: Strong models in algorithmic statistics are essentially the cells of simple partitions of $\{0,1\}^n$. We study the partitions that come from symmetry, the orbit partitions of groups acting on strings, and the structure function $h^G_x$ they induce. A set is a cell of such a partition exactly when its setwise stabiliser acts transitively on it, so $h^G_x$ is Kolmogorov's structure function restricted to $G$-homogeneous sets. For the full symmetric group this recovers all models and exactly the strong ones. For $GL(n,2)$ the cells are the linearly homogeneous sets, $h^{GL}_x$ lies in a band between $C(x)-\alpha$ and $n-\alpha$ with both edges attained, and some stochastic normal strings have simple structure that is invisible to linear symmetry. Writing each permutation group as a Burnside-ring type with a placement, we show that any space of symmetry hypotheses small enough to search is small enough to miss simple structure. Finally we cast the search for orbit models as value estimation on subgroup lattices, with computable bounds, certified training targets and a lattice version of Monte Carlo tree search.

## MCTS Is Counting the Wrong Thing: Learning Exploration Counts for LLM Reasoning

DevFeed: [MCTS Is Counting the Wrong Thing: Learning Exploration Counts for LLM Reasoning](<https://devfeed.tech/articles/mcts-is-counting-the-wrong-thing-learning-exploration-counts-for-llm-reasoning-140009.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22301>)

Author: Xuan Zhang, Zhijian Zhou, Zicheng Hu, Yan Feng, Hongyan Hao, Yuan Qi, Chao Qu

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [monte carlo tree search](<https://devfeed.tech/topics/monte-carlo-tree-search.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>)

Tags: [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [counts](<https://devfeed.tech/tags/counts.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [exploration](<https://devfeed.tech/tags/exploration.md>), [inference](<https://devfeed.tech/tags/inference.md>), [learning](<https://devfeed.tech/tags/learning.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-reasoning](<https://devfeed.tech/tags/llm-reasoning.md>), [mathematical-reasoning](<https://devfeed.tech/tags/mathematical-reasoning.md>), [monte-carlo-tree-search](<https://devfeed.tech/tags/monte-carlo-tree-search.md>), [performance](<https://devfeed.tech/tags/performance.md>), [policy](<https://devfeed.tech/tags/policy.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [search](<https://devfeed.tech/tags/search.md>), [theoretical](<https://devfeed.tech/tags/theoretical.md>), [theory](<https://devfeed.tech/tags/theory.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

Standard UCT can give repeated exploration bonuses to differently worded branches that follow the same reasoning strategy, fragmenting visit counts and increasing cumulative regret. LPC-MCTS changes only UCT's exploration statistic: a lightweight Coin-Flipping Network head uses policy hidden representations to estimate coverage so related branches can share exploration evidence. The abstract reports gains over strong MCTS baselines on 5 mathematical-reasoning and 7 search benchmarks. With an LPC-MCTS-trained policy held fixed, using learned counts at inference raises average mean@16 accuracy across 5 math benchmarks by 3.79 percentage points on Qwen2.5-7B and 2.71 on Qwen3-4B; using LPC-MCTS in both training and inference yields gains of 4.96 and 4.37 points, respectively. These are abstract-level reported results; experimental details are not included.

### Source excerpt

arXiv:2611.22301v1 Announce Type: new Abstract: Monte Carlo tree search (MCTS) has powered landmark successes in reinforcement learning, yet its gains in LLM reasoning can remain modest despite substantial search effort. We identify an overlooked bottleneck: what MCTS counts. Standard UCT assigns independent visit counts to textual branches, allowing different formulations of the same reasoning strategy to repeatedly receive exploration bonuses. Our theoretical analysis shows that this count fragmentation can impair exploration efficiency, as quantified by increased cumulative regret. To address this problem, we introduce Learned Pseudo-Count MCTS (LPC-MCTS), which changes only the exploration statistic in UCT. A lightweight Coin-Flipping Network head on the policy's hidden representations estimates representation coverage, enabling related branches to share exploration evidence. Across 5 mathematical-reasoning and 7 search benchmarks, LPC-MCTS consistently outperforms strong MCTS baselines. Holding each LPC-MCTS-trained policy fixed, replacing standard MCTS counts at inference improves average mean@16 accuracy across 5 mathematical benchmarks by 3.79 percentage points on Qwen2.5-7B and 2.71 points on Qwen3-4B. Applying LPC-MCTS during both training and inference yields gains of 4.96 and 4.37 points over standard MCTS, respectively. Together, our theory and experiments show that changing what MCTS counts can substantially improve its exploration efficiency and reasoning performance.

## Exploring Explainable Multi-agent MCTS-minimax Hybrids in Board Game Using Process Mining

DevFeed: [Exploring Explainable Multi-agent MCTS-minimax Hybrids in Board Game Using Process Mining](<https://devfeed.tech/articles/exploring-explainable-multi-agent-mcts-minimax-hybrids-in-board-game-using-process-mining-139740.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2503.23326>)

Author: Yiyu Qian, Tim Miller, Liyuan Zhao

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [monte carlo tree search](<https://devfeed.tech/topics/monte-carlo-tree-search.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [decision-making](<https://devfeed.tech/topics/decision-making.md>), [Multi Agent Systems](<https://devfeed.tech/topics/multi-agent-systems.md>), [board-game](<https://devfeed.tech/topics/board-game.md>)

Tags: [board-game](<https://devfeed.tech/tags/board-game.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [minimax](<https://devfeed.tech/tags/minimax.md>), [multi-agent](<https://devfeed.tech/tags/multi-agent.md>), [process-mining](<https://devfeed.tech/tags/process-mining.md>)

### AI overview

The paper investigates explaining agents' strategies in 3v3 checkers by applying process mining to multi-agent Monte-Carlo Tree Search (MCTS). It addresses MCTS's selective search, which can miss crucial moves and fall into tactical traps, by integrating shallow full-width minimax search into the rollout phase. The supplied abstract describes this as ongoing work and gives no results comparing performance or explanations.

### Source excerpt

arXiv:2503.23326v4 Announce Type: replace Abstract: Monte-Carlo Tree Search (MCTS) is a family of sampling-based search algorithms widely used for online planning in sequential decision-making domains and at the heart of many recent advances in artificial intelligence. Understanding the behavior of MCTS agents is difficult for developers and users due to the frequently large and complex search trees that result from the simulation of many possible futures, their evaluations, and their relationships. This paper presents our ongoing investigation into potential explanations for the decision-making and behavior of MCTS. A weakness of MCTS is that it constructs a highly selective tree and, as a result, can miss crucial moves and fall into tactical traps. Full-width minimax search constitutes the solution. We integrate shallow minimax search into the rollout phase of multi-agent MCTS and use process mining technique to explain agents' strategies in 3v3 checkers.

## ReST-RL: Reinforcing LLM Reasoning through Unified Self-Training and Value-Guided Search

DevFeed: [ReST-RL: Reinforcing LLM Reasoning through Unified Self-Training and Value-Guided Search](<https://devfeed.tech/articles/rest-rl-reinforcing-llm-reasoning-through-unified-self-training-and-value-guided-search-137229.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2508.19576>)

Author: Sining Zhoubian, Dan Zhang, Jie Tang

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [monte carlo tree search](<https://devfeed.tech/topics/monte-carlo-tree-search.md>)

Tags: [algorithm](<https://devfeed.tech/tags/algorithm.md>), [apps](<https://devfeed.tech/tags/apps.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [coding](<https://devfeed.tech/tags/coding.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [decoding](<https://devfeed.tech/tags/decoding.md>), [group-relative-policy-optimization](<https://devfeed.tech/tags/group-relative-policy-optimization.md>), [grpo](<https://devfeed.tech/tags/grpo.md>), [humaneval](<https://devfeed.tech/tags/humaneval.md>), [inference](<https://devfeed.tech/tags/inference.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-reasoning](<https://devfeed.tech/tags/llm-reasoning.md>), [math](<https://devfeed.tech/tags/math.md>), [monte-carlo-tree-search](<https://devfeed.tech/tags/monte-carlo-tree-search.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [policy](<https://devfeed.tech/tags/policy.md>), [process](<https://devfeed.tech/tags/process.md>), [rl](<https://devfeed.tech/tags/rl.md>), [search](<https://devfeed.tech/tags/search.md>), [training](<https://devfeed.tech/tags/training.md>), [value](<https://devfeed.tech/tags/value.md>)

### AI overview

ReST-RL links policy training with value-guided search to address GRPO's weak reward signals. Its ReST-GRPO stage increases reward variance in sampling and exposes the policy to more informative partial states; VM-MCTS then learns a value model from self-collected MCTS targets and uses it for inference-time search and verification. The stages depend on each other: ReST-GRPO supplies higher-quality trajectories for value learning, which supports more effective search. The abstract reports gains over reinforcement-training, decoding, and verification baselines on coding benchmarks, plus improved out-of-domain math and science performance without target-domain tuning. It describes this transfer evidence as preliminary and reports favorable end-to-end efficiency trade-offs, without giving figures in the supplied text.

### Source excerpt

arXiv:2508.19576v4 Announce Type: replace Abstract: With respect to improving the reasoning accuracy of LLMs, the representative reinforcement learning (RL) method - Group Relative Policy Optimization (GRPO) - has achieved critical success, yet it still suffers from the issue of insignificant reward signals. This paper introduces ReST-RL, a unified Reinforced Self-Training (ReST) policy-value framework that reconnects policy optimization and value-guided search to improve LLM reasoning ability. Firstly, ReST-GRPO adopts an optimized ReST-style algorithm to reshape the policy-induced trajectory distribution by increasing the reward variance of GRPO sampling and exposing the policy to more informative partial states, thereby improving training efficiency and effectiveness. Then, we further introduce a decoding optimization method, VM-MCTS, which trains a Value Model (VM) from self-collected Monte-Carlo Tree Search (MCTS) targets and deploys it through an adapted MCTS algorithm to provide precise process signals and verification scores, further enhancing reasoning accuracy. These two stages are internally dependent - ReST-GRPO yields higher quality trajectories for value learning with VM-MCTS, which in turn enables more effective inference-time search. We validate our framework on multiple coding benchmarks (e.g., APPS, BigCodeBench, and HumanEval), where it significantly outperforms other reinforcement training baselines (naive GRPO, DAPO, and ReST-DPO), as well as decoding and verification baselines (e.g., PRM-BoN and ORM-MCTS), indicating its power to strengthen LLM reasoning capability. Moreover, we further evaluate ReST-RL on out-of-domain math and science reasoning tasks, where it achieves improved performance without target-domain tuning and favorable end-to-end efficiency trade-offs, providing preliminary transfer evidence beyond our primary coding domain.

## Lossless First-Action Menus for Linear-Reward Planning

DevFeed: [Lossless First-Action Menus for Linear-Reward Planning](<https://devfeed.tech/articles/lossless-first-action-menus-for-linear-reward-planning-137495.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.19795>)

Author: Yukun Yang

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Combinatorial optimization](<https://devfeed.tech/topics/combinatorial-optimization.md>), [monte carlo tree search](<https://devfeed.tech/topics/monte-carlo-tree-search.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [actions](<https://devfeed.tech/tags/actions.md>), [agent](<https://devfeed.tech/tags/agent.md>), [algorithm](<https://devfeed.tech/tags/algorithm.md>), [all](<https://devfeed.tech/tags/all.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [based](<https://devfeed.tech/tags/based.md>), [complete](<https://devfeed.tech/tags/complete.md>), [continuation](<https://devfeed.tech/tags/continuation.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [decision](<https://devfeed.tech/tags/decision.md>), [decision-making](<https://devfeed.tech/tags/decision-making.md>), [dimensions](<https://devfeed.tech/tags/dimensions.md>), [feature](<https://devfeed.tech/tags/feature.md>), [first](<https://devfeed.tech/tags/first.md>), [fixed](<https://devfeed.tech/tags/fixed.md>), [full](<https://devfeed.tech/tags/full.md>), [graphs](<https://devfeed.tech/tags/graphs.md>), [linear](<https://devfeed.tech/tags/linear.md>), [planning](<https://devfeed.tech/tags/planning.md>), [reward](<https://devfeed.tech/tags/reward.md>)

### AI overview

A first-action menu preserves the unrestricted optimal return for every linear reward weighting exactly when its reachable continuations cover every vertex of the full feature polytope. This characterization gives an output-sensitive algorithm fixed-parameter tractable in the vertex count, using scalar backward induction and oracle-based facet identification. Complexity depends on the setting: on explicitly layered graphs with unit feature increments, verification is coNP-complete and synthesis is complete for the second existential level of the polynomial hierarchy, even with additive loss strictly below one. In fixed dimensions of two or more, verification is polynomial but synthesis remains NP-complete; in one dimension, both have polynomial-time solutions.

### Source excerpt

arXiv:2611.19795v1 Announce Type: new Abstract: We study sequential decision making where an agent reserves a minimum menu of initial actions before multi-objective trade-offs are revealed, followed by unrestricted continuation planning once linear reward weights arrive. We prove that a menu preserves the unrestricted optimal return across all linear scalarizations if and only if its reachable continuations cover all vertices of the full feature polytope. This geometric equivalence yields an output-sensitive algorithm that is fixed-parameter tractable in the vertex count, combining scalar backward induction with oracle-based facet identification. On explicitly layered graphs with unit feature increments, menu verification is coNP-complete and synthesis is complete for the second existential level of the polynomial hierarchy, even under additive loss strictly below one. In fixed dimensions two and higher, verification is polynomial whereas synthesis remains NP-complete, while one-dimensional instances admit polynomial-time solutions.

## Diffusion Tree Sampling for Inference Time Adaptation of Crystal Generative Models

DevFeed: [Diffusion Tree Sampling for Inference Time Adaptation of Crystal Generative Models](<https://devfeed.tech/articles/diffusion-tree-sampling-for-inference-time-adaptation-of-crystal-generative-models-135067.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.17462>)

Author: Daniel T. Levy, Vineet Jain, Tara Akhound-Sadegh, S\\'ekou-Oumar Kaba, Siamak Ravanbakhsh

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [monte carlo tree search](<https://devfeed.tech/topics/monte-carlo-tree-search.md>), [AI Foundation Models](<https://devfeed.tech/topics/ai-foundation-models.md>)

Tags: [cond-mat-mtrl-sci](<https://devfeed.tech/tags/cond-mat-mtrl-sci.md>), [crystal](<https://devfeed.tech/tags/crystal.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [generative](<https://devfeed.tech/tags/generative.md>), [generative-models](<https://devfeed.tech/tags/generative-models.md>), [inference](<https://devfeed.tech/tags/inference.md>), [models](<https://devfeed.tech/tags/models.md>), [sampling](<https://devfeed.tech/tags/sampling.md>), [time](<https://devfeed.tech/tags/time.md>)

### AI overview

Diffusion Tree Sampling (DTS), a Monte Carlo tree search over the denoising trajectory, steers crystal generative models at inference time without changing their parameters. The work describes using existing property-conditional models as guided proposals without distorting the target distribution, and extending DTS to generate samples across a multi-objective Pareto front in one search. In de novo crystal generation, the authors report improved stability, candidates in rare space groups or with high bulk modulus, and a Pareto front balancing high dielectric constant and high bandgap. The supplied abstract reports successful improvements across these tasks; it does not provide quantitative results.

### Source excerpt

arXiv:2611.17462v1 Announce Type: cross Abstract: Diffusion-based generative models have become strong priors over plausible crystal structures, but turning that prior into promising candidates requires steering toward several conflicting properties at once. Fine-tuning is expensive and produces a new model for every new objective, while differentiable guidance is not applicable to many real-world tasks. We instead adapt these generative models at inference time using Diffusion Tree Sampling (DTS), a Monte Carlo tree search procedure over the denoising trajectory. We make two technical contributions on top of DTS that are particularly relevant for AI-for-science applications. First, we show how to use existing property-conditional generative models (e.g., models fine-tuned for classifier-free guidance) as guided proposals inside DTS, without distorting the target distribution. Second, we extend DTS to multi-objective sampling, so that a single search produces samples spanning an entire Pareto front. We instantiate both adaptations on de novo crystal generation: improving the stability of crystals generated by different models, using pretrained conditional models to generate crystals belonging to rare space groups or with high bulk modulus, and generating crystals along a Pareto front of high dielectric constant and high bandgap materials. Across these tasks, DTS-based inference-time adaptation successfully improves model outputs without modifying the models themselves.

## Probe-Based Routing for Join Optimizer Selection

DevFeed: [Probe-Based Routing for Join Optimizer Selection](<https://devfeed.tech/articles/probe-based-routing-for-join-optimizer-selection-132693.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.15263>)

Author: Vladimir Burlakov, Alena Rybakina, Sergey Kudashev, Ilgam Latypov, Konstantin Gilev, Alexander Demin, Denis Ponomaryov, Yuriy Dorn

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Databases](<https://devfeed.tech/sources/arxiv-databases.md>)

Topics: [JOIN](<https://devfeed.tech/topics/join.md>), [monte carlo tree search](<https://devfeed.tech/topics/monte-carlo-tree-search.md>), [Query (disambiguation)](<https://devfeed.tech/topics/query.md>)

Tags: [algorithm](<https://devfeed.tech/tags/algorithm.md>), [alpha](<https://devfeed.tech/tags/alpha.md>), [cluster](<https://devfeed.tech/tags/cluster.md>), [clusters](<https://devfeed.tech/tags/clusters.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cs-db](<https://devfeed.tech/tags/cs-db.md>), [dynamic-programming](<https://devfeed.tech/tags/dynamic-programming.md>), [features](<https://devfeed.tech/tags/features.md>), [information](<https://devfeed.tech/tags/information.md>), [job](<https://devfeed.tech/tags/job.md>), [join](<https://devfeed.tech/tags/join.md>), [monte-carlo](<https://devfeed.tech/tags/monte-carlo.md>), [monte-carlo-tree-search](<https://devfeed.tech/tags/monte-carlo-tree-search.md>), [ordering](<https://devfeed.tech/tags/ordering.md>), [random](<https://devfeed.tech/tags/random.md>), [router](<https://devfeed.tech/tags/router.md>), [routing](<https://devfeed.tech/tags/routing.md>), [selection](<https://devfeed.tech/tags/selection.md>), [size](<https://devfeed.tech/tags/size.md>), [spacing](<https://devfeed.tech/tags/spacing.md>), [time](<https://devfeed.tech/tags/time.md>)

### AI overview

The paper reports that routing join queries using probe-derived landscape features improves mean end-to-end time over PostgreSQL's relation-count rule on JOB and IMDb-CEB. The probe estimates tail spacing (α), describing gaps among the best sampled join orders, and crossover surprise (θ), measuring how a crossover child's cost differs from its parents' mean; it takes microseconds per query. The router clusters training queries by these features and relation count, assigns a solver only where it wins total end-to-end time by a margin, and otherwise keeps PostgreSQL's default choice. On IMDb-CEB, adding probe features nearly doubles the gain from relation count alone. The excerpt does not provide absolute timings or further benchmark details.

### Source excerpt

arXiv:2611.15263v1 Announce Type: new Abstract: In optimizers that choose a join-order solver by a size rule, join ordering is an algorithm-selection problem, because dynamic programming (DP), genetic search (GEQO), and Monte Carlo tree search (MCTS) each win on different queries. PostgreSQL nevertheless chooses among them by relation count alone, although queries with the same relation count can favor different solvers. This paper introduces two landscape features that a probe computes from random join orders in microseconds per query. The tail spacing $\alpha$ describes how the costs of the best sampled orders are spaced, and the crossover surprise $\theta$ describes how far the cost of a crossover child deviates from the mean cost of its parents. The router clusters training queries by these features and the relation count, assigns a solver to every cluster that this solver wins in total end-to-end time by a margin, and leaves all other queries to PostgreSQL's default rule. On JOB and IMDb-CEB, the router lowers mean end-to-end time relative to the default rule, and on IMDb-CEB the probe features nearly double the gain obtained from the relation count alone, so they carry information about the queries that the relation count lacks.

## Entropic Risk-Aware Monte Carlo Tree Search

DevFeed: [Entropic Risk-Aware Monte Carlo Tree Search](<https://devfeed.tech/articles/entropic-risk-aware-monte-carlo-tree-search-128499.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2601.17667>)

Author: Pedro P. Santos, Jacopo Silvestrin, Manuel Lopes, Alberto Sardinha, Francisco S. Melo

Published: 2026-11-11T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [monte carlo tree search](<https://devfeed.tech/topics/monte-carlo-tree-search.md>), [Risk](<https://devfeed.tech/topics/risk.md>)

Tags: [algorithm](<https://devfeed.tech/tags/algorithm.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [dynamic-programming](<https://devfeed.tech/tags/dynamic-programming.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [markov](<https://devfeed.tech/tags/markov.md>), [monte-carlo-tree-search](<https://devfeed.tech/tags/monte-carlo-tree-search.md>), [risk](<https://devfeed.tech/tags/risk.md>)

### AI overview

The paper proposes an upper confidence bound-based Monte Carlo tree search algorithm for risk-aware Markov decision processes with entropic risk measure objectives. Its non-asymptotic analysis shows that the empirical risk measure at the root converges to the optimal value and establishes polynomial regret concentration. The method uses dynamic programming formulations for these objectives; illustrative experiments show a tradeoff between risk-neutral and risk-averse behavior.

### Source excerpt

arXiv:2601.17667v3 Announce Type: replace Abstract: We propose a provably correct Monte Carlo tree search (MCTS) algorithm for solving risk-aware Markov decision processes (MDPs) with entropic risk measure (ERM) objectives. We provide a non-asymptotic analysis of our proposed algorithm, showing that the algorithm: (i) is correct in the sense that the empirical ERM obtained at the root node converges to the optimal ERM; and (ii) enjoys polynomial regret concentration. Our algorithm successfully exploits the dynamic programming formulations for solving risk-aware MDPs with ERM objectives introduced by previous works in the context of an upper confidence bound-based tree search algorithm. Finally, we provide a set of illustrative experiments to evaluate our risk-aware MCTS method, showing it successfully trades off between risk-neutral and risk-averse behavior.

## Retrieval as Exploration: Retriever-Free Retrieval via Corpus-Traversing MCTS

DevFeed: [Retrieval as Exploration: Retriever-Free Retrieval via Corpus-Traversing MCTS](<https://devfeed.tech/articles/retrieval-as-exploration-retriever-free-retrieval-via-corpus-traversing-mcts-125820.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2505.16409>)

Author: Chaeeun Kim, Seungone Kim

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [AI-driven information retrieval](<https://devfeed.tech/topics/ai-driven-information-retrieval.md>), [monte carlo tree search](<https://devfeed.tech/topics/monte-carlo-tree-search.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Retrieval Augmented Generation (RAG)](<https://devfeed.tech/topics/retrieval-augmented-generation-rag.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [allows](<https://devfeed.tech/tags/allows.md>), [also](<https://devfeed.tech/tags/also.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [based](<https://devfeed.tech/tags/based.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [can](<https://devfeed.tech/tags/can.md>), [capabilities](<https://devfeed.tech/tags/capabilities.md>), [context](<https://devfeed.tech/tags/context.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [domain](<https://devfeed.tech/tags/domain.md>), [em](<https://devfeed.tech/tags/em.md>), [existing](<https://devfeed.tech/tags/existing.md>), [exploration](<https://devfeed.tech/tags/exploration.md>), [fact-checking](<https://devfeed.tech/tags/fact-checking.md>), [fetching](<https://devfeed.tech/tags/fetching.md>), [fine](<https://devfeed.tech/tags/fine.md>), [fixed](<https://devfeed.tech/tags/fixed.md>), [framework](<https://devfeed.tech/tags/framework.md>), [free](<https://devfeed.tech/tags/free.md>), [retrieval](<https://devfeed.tech/tags/retrieval.md>)

### AI overview

FREESON makes a large reasoning model act as both reasoner and retriever, searching by exploring paths through the original corpus rather than relying on a separate retrieval model and fixed top-k results. Its CT-MCTS method expands and evaluates paths from multiple promising corpus entry points so the model can select specific relevant segments. Across nine benchmarks, the abstract reports 15-point gains in average EM and F1 over four retriever-based baselines on five general-domain QA tasks, matching the strongest baseline; it also reports gains of 12 points on fact-checking and 13 on medical QA in zero-shot transfer, performing on par with MedGemma-27B.

### Source excerpt

arXiv:2505.16409v2 Announce Type: replace Abstract: Large Reasoning Models (LRMs) have demonstrated remarkable capabilities in multi-step reasoning and calling search tools at appropriate steps. However, existing retrieval-augmented reasoning approaches still rely on separate retrieval models, limiting LRMs to deciding when to retrieve and how to formulate queries rather than leveraging their pretrained knowledge and reasoning capabilities for retrieval itself. This structural separation introduces a representation bottleneck, as retrieval relies on separately learned representations that may obscure fine-grained distinctions needed for reasoning. Moreover, these approaches commonly employ fixed top-k retrieval, which can introduce superfluous documents and increase retrieval noise and context overhead. To address this, we reformulate retrieval from query-document relevance matching into the exploration of answer-containing paths within the corpus, and propose FREESON (Retriever-FREE Retrieval-Augmented ReaSONing), a framework that enables LRMs to directly navigate the original corpus text to locate information that satisfies their specific needs, serving as both the reasoner and the retriever. Specifically, we introduce CT-MCTS (Corpus-Traversing Monte Carlo Tree Search), a retrieval-specialized variant of Monte Carlo Tree Search (MCTS). CT-MCTS iteratively expands and evaluates exploration paths, starting from multiple promising entry points across the corpus. This allows LRMs to selectively reference specific segments identified during traversal, instead of fetching a fixed top-k set. Across nine benchmarks, FREESON improves average EM and F1 by 15 points over four retriever-based baselines on five general-domain QA tasks, matching the strongest baseline. It also gains 12 points on fact-checking and 13 points on medical QA under zero-shot transfer, performing on par with the much larger MedGemma-27B.

## Programs-of-Layers in LLMs through the Lens of Cortical Areas

DevFeed: [Programs-of-Layers in LLMs through the Lens of Cortical Areas](<https://devfeed.tech/articles/programs-of-layers-in-llms-through-the-lens-of-cortical-areas-124478.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.31360>)

Author: Justus Westerhoff, Stephan Olbrich, Hatem Oraby, Walter Senn, Matthew Evan Larkum, Felix Alexander Gers

Published: 2026-11-09T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [monte carlo tree search](<https://devfeed.tech/topics/monte-carlo-tree-search.md>), [AI Models](<https://devfeed.tech/topics/ai-models.md>), [model architecture](<https://devfeed.tech/topics/model-architecture.md>)

Tags: [adaptive](<https://devfeed.tech/tags/adaptive.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [brain](<https://devfeed.tech/tags/brain.md>), [cortex](<https://devfeed.tech/tags/cortex.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [layers](<https://devfeed.tech/tags/layers.md>), [llms](<https://devfeed.tech/tags/llms.md>), [monte-carlo-tree-search](<https://devfeed.tech/tags/monte-carlo-tree-search.md>), [performance](<https://devfeed.tech/tags/performance.md>), [polar](<https://devfeed.tech/tags/polar.md>), [questions](<https://devfeed.tech/tags/questions.md>), [router](<https://devfeed.tech/tags/router.md>), [transformer](<https://devfeed.tech/tags/transformer.md>)

### AI overview

The study finds that allowing LLM layer blocks to be skipped or repeated can improve on the standard forward pass: across six models, repeating beats skipping, and combining both performs better than either alone. Easier questions need shorter programs, while harder ones tend to need more repeats. The authors qualify PoLar's learned-router result: in the original setup, its gain came from fixing answer format, which a single validation-selected program did better. With answer format already fixed, a larger data split and their search, the router's top prediction matched the standard pass on every model, though its top five programs collectively improved results. A small fixed menu of programs also solved more questions than the standard pass in their setup, but successful error-correcting programs were brittle. They conclude that a router reliably selecting a better program per input remains elusive.

### Source excerpt

arXiv:2609.31360v2 Announce Type: replace Abstract: Inference in LLMs is conventionally a fixed-depth, fixed-order forward pass through every layer, regardless of how difficult the input is. The human brain does not work this way. Using the thalamus as a central hub, it routes information flexibly to all regions of the cortex according to demand. Li et al. (2026) recently showed, with program-of-layers (PoLar), that transformers can be given an analogous flexibility if their layers are treated as a library of functions rather than a fixed sequence. Performance improves over the standard forward pass when each input is dynamically routed through an adaptive sequence of skipped or repeated contiguous layer blocks. Reconstructing PoLar's diagnostic Monte Carlo Tree Search in more detail across 6 models, we reproduce several of its findings: skipping outperforms the standard pass, repeating outperforms skipping, and combining both outperforms either alone. Shorter programs suffice for easier questions, while harder questions require more layer repeats. For their main claim, the learned router's gain, we find what drives it. Under their setup, the router gains by repairing the format of the model's answers, which a single fixed program, selected on validation questions, does better. In our setup, with a prompt that already fixes the format, a larger data split and our search, its top prediction collapses to the standard pass on every model, although its top-5 programs together still gain. Beyond reproduction, we find in our setup that a small fixed menu of programs solves far more questions than the standard pass, and that programs which correct errors are brittle: undoing a single edit typically breaks the correction. PoLar mirrors principles of thalamo-cortical coordination between cortical-area-like transformer layers, although a working thalamus analogue, a router that picks a better program per input, remains to be found.

## HarnessFlow: MCTS-Guided Harness Optimization for LLM Agents

DevFeed: [HarnessFlow: MCTS-Guided Harness Optimization for LLM Agents](<https://devfeed.tech/articles/harnessflow-mcts-guided-harness-optimization-for-llm-agents-118437.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.06254>)

Author: Zhi Hong, Qianlong Wang, Hongchen Wei, Aidan Wu, Daohong Ying, Mingze Kong, Bei Liu, Chong Luo

Published: 2026-11-05T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [monte carlo tree search](<https://devfeed.tech/topics/monte-carlo-tree-search.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [agentic-coding](<https://devfeed.tech/topics/agentic-coding.md>)

Tags: [agentic-tasks](<https://devfeed.tech/tags/agentic-tasks.md>), [agents](<https://devfeed.tech/tags/agents.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [budget](<https://devfeed.tech/tags/budget.md>), [code-execution](<https://devfeed.tech/tags/code-execution.md>), [coding](<https://devfeed.tech/tags/coding.md>), [coding-agent](<https://devfeed.tech/tags/coding-agent.md>), [costs](<https://devfeed.tech/tags/costs.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [environment](<https://devfeed.tech/tags/environment.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [evaluations](<https://devfeed.tech/tags/evaluations.md>), [feedback](<https://devfeed.tech/tags/feedback.md>), [framework](<https://devfeed.tech/tags/framework.md>), [harness](<https://devfeed.tech/tags/harness.md>), [language](<https://devfeed.tech/tags/language.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [large-language-model](<https://devfeed.tech/tags/large-language-model.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-agents](<https://devfeed.tech/tags/llm-agents.md>), [optimization](<https://devfeed.tech/tags/optimization.md>)

### AI overview

HarnessFlow automates optimization of executable LLM-agent harnesses while holding the model and task environment fixed. It stores candidates and evaluation histories in a refinement tree, uses Monte Carlo Tree Search to select harnesses, and has a coding agent refine them with archived code, execution feedback, and reusable mechanisms. Across six agentic, coding, and web benchmarks, 15 refinement attempts per benchmark produced a mean score of 69.48%, 10.89 percentage points above initial harnesses and 3.95 above Meta-Harness. The abstract also reports lower average per-task execution costs than hand-engineered baselines. These findings are limited to the described benchmarks and comparisons.

### Source excerpt

arXiv:2611.06254v1 Announce Type: new Abstract: The performance of large language model (LLM) agents depends on both the underlying model and the harness that governs interaction with tools and the environment. Yet harness design remains largely manual, and automating it requires navigating an open-ended search space through costly end-to-end evaluations. We introduce HarnessFlow, a framework for optimizing complete executable harnesses under a limited candidate-evaluation budget, with the model and task environment held fixed. HarnessFlow organizes candidates and their evaluation histories in a refinement tree, using a Monte Carlo Tree Search (MCTS)-based policy to balance exploitation and exploration. A coding agent refines selected harnesses using archived code, execution feedback, and a library of reusable harness mechanisms. With 15 refinement attempts per benchmark, HarnessFlow achieves the highest mean test scores among the compared methods on six benchmarks spanning agentic tasks, coding, and web interaction. The optimized harnesses achieve an average score of 69.48%, outperforming the initial harnesses and Meta-Harness by 10.89 and 3.95 percentage points, respectively, while incurring lower average per-task execution costs than hand-engineered baselines. These results suggest that structured harness search offers a promising route toward automating the design of capable and efficient LLM agents.

## pico-WAM: Evaluating Counterfactual Prediction and Planning Error in a Compact World Action Model

DevFeed: [pico-WAM: Evaluating Counterfactual Prediction and Planning Error in a Compact World Action Model](<https://devfeed.tech/articles/pico-wam-evaluating-counterfactual-prediction-and-planning-error-in-a-compact-world-action-model-114479.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.01039>)

Author: Keivalya Pandya

Published: 2026-11-03T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [World models](<https://devfeed.tech/topics/world-models.md>), [Robotics](<https://devfeed.tech/topics/robotics.md>), [monte carlo tree search](<https://devfeed.tech/topics/monte-carlo-tree-search.md>), [Physical AI](<https://devfeed.tech/topics/physical-ai.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [better](<https://devfeed.tech/tags/better.md>), [branch](<https://devfeed.tech/tags/branch.md>), [compare](<https://devfeed.tech/tags/compare.md>), [component](<https://devfeed.tech/tags/component.md>), [control](<https://devfeed.tech/tags/control.md>), [cost](<https://devfeed.tech/tags/cost.md>), [costs](<https://devfeed.tech/tags/costs.md>), [coverage](<https://devfeed.tech/tags/coverage.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [data](<https://devfeed.tech/tags/data.md>), [diagnostics](<https://devfeed.tech/tags/diagnostics.md>), [effective](<https://devfeed.tech/tags/effective.md>), [error](<https://devfeed.tech/tags/error.md>), [generation](<https://devfeed.tech/tags/generation.md>), [model](<https://devfeed.tech/tags/model.md>), [pico](<https://devfeed.tech/tags/pico.md>), [planning](<https://devfeed.tech/tags/planning.md>), [prediction](<https://devfeed.tech/tags/prediction.md>), [world](<https://devfeed.tech/tags/world.md>)

### AI overview

In pico-WAM's tested Push-T setting, trajectory scoring appears to be a more effective improvement target than proposal generation: replacing learned trajectory scores with simulator costs increased coverage by 12.2 percentage points while retaining learned proposals, whereas substituting simulator states in proposal generation alone showed no detected gain. Across 206 demonstrations and 15 training seeds, policy-restarted search beat warm-started search under both data regimes; counterfactual branch training improved short-horizon cost ranking and widened restart's coverage advantage by 4.8 points. The separate door-opening study and component diagnostics indicate that improved ranking does not necessarily improve control or outperform imitation. These results are limited to the reported settings.

### Source excerpt

arXiv:2611.01039v1 Announce Type: cross Abstract: When a World Action Model fails at control, is the bottleneck proposal generation or trajectory scoring? We introduce pico-WAM, a compact model with supplied physical state for tracing policy-dynamics interactions. Using 206 Push-T demonstrations and 15 training seeds, we compare data regimes and search configurations. Counterfactual branch training improves short-horizon cost ranking and enlarges the coverage advantage of policy-restarted over warm-started search by 4.8 percentage points; restart wins under both data regimes. Replacing learned trajectory scores with simulator costs improves coverage by 12.2 points while retaining learned proposals. Substituting simulator states in proposal generation alone shows no detected gain. Component diagnostics and a separate door-opening study show why better ranking need not yield better control or outperform imitation. These controlled interventions identify scoring as an effective improvement target in the tested setting and provide a reproducible protocol for diagnosing compact world models.

## SQL-o1: Automatic Text-to-SQL Program Synthesis via Self-Reward Heuristic Search

DevFeed: [SQL-o1: Automatic Text-to-SQL Program Synthesis via Self-Reward Heuristic Search](<https://devfeed.tech/articles/sql-o1-automatic-text-to-sql-program-synthesis-via-self-reward-heuristic-search-113622.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2502.11741>)

Author: Shuai Lyu, Haoran Luo, Ripeng Li, Zhonghong Ou, Yifan Zhu, Tiesunlong Shen

Published: 2026-11-03T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Databases](<https://devfeed.tech/sources/arxiv-databases.md>)

Topics: [text2sql](<https://devfeed.tech/topics/text2sql.md>), [AI for SQL query generation](<https://devfeed.tech/topics/ai-for-sql-query-generation.md>), [monte carlo tree search](<https://devfeed.tech/topics/monte-carlo-tree-search.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [accelerate](<https://devfeed.tech/tags/accelerate.md>), [agent](<https://devfeed.tech/tags/agent.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [code](<https://devfeed.tech/tags/code.md>), [complex](<https://devfeed.tech/tags/complex.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-db](<https://devfeed.tech/tags/cs-db.md>), [execution](<https://devfeed.tech/tags/execution.md>), [exploration](<https://devfeed.tech/tags/exploration.md>), [few-shot](<https://devfeed.tech/tags/few-shot.md>), [framework](<https://devfeed.tech/tags/framework.md>), [gpt-4](<https://devfeed.tech/tags/gpt-4.md>), [large-language-models-llms](<https://devfeed.tech/tags/large-language-models-llms.md>), [monte-carlo-tree-search](<https://devfeed.tech/tags/monte-carlo-tree-search.md>), [natural-language](<https://devfeed.tech/tags/natural-language.md>), [open-source-llms](<https://devfeed.tech/tags/open-source-llms.md>), [reward](<https://devfeed.tech/tags/reward.md>), [search](<https://devfeed.tech/tags/search.md>), [sql](<https://devfeed.tech/tags/sql.md>), [synthesis](<https://devfeed.tech/tags/synthesis.md>), [text2sql](<https://devfeed.tech/tags/text2sql.md>)

### AI overview

SQL-o1 uses an agent-based, self-reward heuristic search to generate SQL from natural-language questions. It applies Monte Carlo Tree Search for multi-step exploration and dynamic pruning to speed inference while aiming to preserve accuracy. The abstract reports a 10.8-point execution-accuracy improvement on the complex BIRD benchmark and says the system also performs on Spider, generalizes in few-shot settings, and transfers across open-source LLMs. These are reported benchmark claims; the supplied text does not specify evaluation details.

### Source excerpt

arXiv:2502.11741v4 Announce Type: replace Abstract: Efficient human-database interaction is a core problem in software engineering. Text-to-SQL (Text2SQL) tackles this by mapping natural language questions to executable SQL queries, enabling automatic SQL synthesis and reliable interaction. Although large language models (LLMs) have advanced significantly, current approaches struggle with poor transferability to open-source LLMs, limited robustness against logic and function errors in complex queries, and inefficiencies in structured search. We introduce SQL-o1, a self-reward-driven heuristic search framework built on an agent-based architecture to enhance model reasoning capabilities. SQL-o1 leverages Monte Carlo Tree Search (MCTS) for structured, multi-step exploration and incorporates a dynamic pruning strategy to accelerate inference without sacrificing accuracy. On the Spider and BIRD benchmarks, SQL-o1 achieves a 10.8-point execution accuracy improvement on the complex BIRD dataset, surpassing even GPT-4-based models. It exhibits strong few-shot generalization and robust cross-model transferability across open-source LLMs. Our code is available at https://github.com/ShuaiLyu0110/SQL-o1.

## Reap: Uncovering Monte Carlo proof search with a lightweight model

DevFeed: [Reap: Uncovering Monte Carlo proof search with a lightweight model](<https://devfeed.tech/articles/reap-uncovering-monte-carlo-proof-search-with-a-lightweight-model-115349.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.02678>)

Author: Yutong Wang, Xuanzhi Ren, Tianyi Xu, Ruitong Liu, Jingda Xu, Peihao Wu, Bryan Dai, Bin Dong

Published: 2026-11-03T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [monte carlo tree search](<https://devfeed.tech/topics/monte-carlo-tree-search.md>), [Lean](<https://devfeed.tech/topics/lean.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>)

Tags: [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [framework](<https://devfeed.tech/tags/framework.md>), [lean](<https://devfeed.tech/tags/lean.md>), [learning](<https://devfeed.tech/tags/learning.md>), [model](<https://devfeed.tech/tags/model.md>), [monte-carlo-tree-search](<https://devfeed.tech/tags/monte-carlo-tree-search.md>), [performance](<https://devfeed.tech/tags/performance.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [strategy](<https://devfeed.tech/tags/strategy.md>)

### AI overview

Reap is an open Lean4 framework that combines policy-value-guided Monte Carlo Tree Search with shared proof-state transitions, AND-OR search, replay verification, and structured action logs for training and analysis. Its Reaper-1 model (1.7B parameters) was trained with continued pre-training, supervised tactic learning, and iterative search-based reinforcement learning. With a separate premise selector, the system solves 79.9% of miniF2F-test at pass@32, which the abstract describes as state of the art among similarly sized open provers and competitive with substantially larger models. The authors report that reinforcement learning favors proofs built around useful intermediate results, with fewer explicit search decisions but more work within each tactic program; this shifts effort between search and tactic execution.

### Source excerpt

arXiv:2611.02678v1 Announce Type: new Abstract: We present Reap, an open framework for learning and analyzing formal proof search in Lean4. Reap implements policy-value-guided Monte Carlo Tree Search directly in Lean, sharing proof-state transitions, AND-OR search, and replay verification between inference and training rollouts. Structured action logs expose how neural predictions and tactic execution interact during search. Using this framework, we train Reaper-1, a 1.7B-parameter policy-value model, through continued pre-training, supervised tactic learning, and iterative search-based reinforcement learning. Using a separate premise selector, the final system solves 79.9% of miniF2F-test at pass@32, establishing state-of-the-art performance among open provers of comparable size while remaining competitive with substantially larger models. Beyond improved solve rates, our results suggest that RL strengthens the prover's ability to organize proofs around useful intermediate results, favoring substantial advances over repeated local adjustments. This more directed proof construction is accompanied by increasingly elaborate tactic programs: fewer explicit search decisions are needed, while more work is performed within each decision. The resulting trade-off between search effort and tactic execution cost provides insight into how learning changes the prover's problem-solving strategy.

## SCOUT: Planning under Occlusion via Object-Centric World Model Rollouts

DevFeed: [SCOUT: Planning under Occlusion via Object-Centric World Model Rollouts](<https://devfeed.tech/articles/scout-planning-under-occlusion-via-object-centric-world-model-rollouts-115187.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.01600>)

Author: Amir Hossain Raj, Xuesu Xiao

Published: 2026-11-03T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Robot Manipulation](<https://devfeed.tech/topics/robot-manipulation.md>), [monte carlo tree search](<https://devfeed.tech/topics/monte-carlo-tree-search.md>), [World models](<https://devfeed.tech/topics/world-models.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [3d](<https://devfeed.tech/tags/3d.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [evaluations](<https://devfeed.tech/tags/evaluations.md>), [llm](<https://devfeed.tech/tags/llm.md>), [manipulation](<https://devfeed.tech/tags/manipulation.md>), [monte-carlo-tree-search](<https://devfeed.tech/tags/monte-carlo-tree-search.md>), [object](<https://devfeed.tech/tags/object.md>), [planning](<https://devfeed.tech/tags/planning.md>), [real-world](<https://devfeed.tech/tags/real-world.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [vision-language-action](<https://devfeed.tech/tags/vision-language-action.md>), [world-model](<https://devfeed.tech/tags/world-model.md>)

### AI overview

SCOUT plans manipulation in clutter by reconstructing hidden geometry from an object-centric scene representation and encoding blocker hierarchies and accessibility corridors in a relational state. An LLM proposes structured actions, which Monte Carlo Tree Search evaluates using a learned action-conditioned world model. The abstract reports that SCOUT outperforms VLA policies, embodied agents, and 3D world models on Reveal and Placement tasks, with larger gains on long-horizon tasks involving occluded geometry, and describes a real-world demonstration in a cluttered shelf environment.

### Source excerpt

arXiv:2611.01600v1 Announce Type: cross Abstract: Robots manipulating in cluttered spaces must often plan from partial views where foreground objects occlude the target, surrounding geometry, and usable free space. Reactive policies often miss these hidden constraints, while exhaustive search becomes computationally prohibitive in dense clutter. We present SCOUT, an occlusion-aware planning framework that bridges high-level deliberation with grounded physical rollouts. SCOUT leverages an object-centric scene representation to reconstruct hidden geometry and derive a relational planning state capturing blocker hierarchies and accessibility corridors. To navigate the vast action space in clutter, an LLM proposes structured action candidates which are then rigorously evaluated via Monte Carlo Tree Search (MCTS) using a learned action-conditioned world model. Evaluations on 'Reveal' and 'Placement' tasks show that SCOUT outperforms Vision-Language-Action (VLA) policies, embodied agents, and 3D world models, with significant gains in long-horizon tasks requiring reasoning over occluded geometry. We further validate the framework's robustness through real-world robotic manipulation in a cluttered shelf environment.

## Quantile Propagation Planning for Budgeted Tail-Risk Evaluation

DevFeed: [Quantile Propagation Planning for Budgeted Tail-Risk Evaluation](<https://devfeed.tech/articles/quantile-propagation-planning-for-budgeted-tail-risk-evaluation-111572.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.41351>)

Author: Weijia Han, Zhi Huang, Lisha Qu, Yuanyuan Tan, Jiayue Meng

Published: 2026-11-02T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [monte carlo tree search](<https://devfeed.tech/topics/monte-carlo-tree-search.md>), [Chinchilla scaling law](<https://devfeed.tech/topics/chinchilla-scaling-law.md>)

Tags: [budget](<https://devfeed.tech/tags/budget.md>), [constraint](<https://devfeed.tech/tags/constraint.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-sy](<https://devfeed.tech/tags/cs-sy.md>), [eess-sy](<https://devfeed.tech/tags/eess-sy.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [inventory](<https://devfeed.tech/tags/inventory.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [monte-carlo](<https://devfeed.tech/tags/monte-carlo.md>), [planning](<https://devfeed.tech/tags/planning.md>), [quantile](<https://devfeed.tech/tags/quantile.md>), [risk](<https://devfeed.tech/tags/risk.md>)

### AI overview

Quantile propagation planning (QPP) evaluates mean and tail risk by propagating a small, fixed set of disturbance quantiles through system dynamics and integrating the resulting cost curve. The abstract says QPP is exact at those propagated levels, converges as the quantile grid is refined, and has a one-sided conservatism guarantee under stated structural conditions. In controlled overload and inventory tasks, it reduced realized tail cost and constraint violations versus mean-rollout planning; on overload, it remained competitive with sampled evaluators using substantially more trajectories. These are abstract-level results; the supplied text does not detail the conditions or experiments.

### Source excerpt

arXiv:2610.41351v1 Announce Type: new Abstract: Risk-aware planning must identify rare, costly outcomes using only a limited number of model evaluations. Monte Carlo rollouts spend this budget on noisy tail estimates, while planning from the mean can miss the tail entirely. We introduce quantile propagation planning (QPP), which propagates a small, fixed set of disturbance quantiles through the dynamics and integrates the resulting cost curve to evaluate both mean and tail risk. The method is exact at the propagated levels, converges as the quantile grid is refined, and admits a one-sided conservatism guarantee under stated structural conditions. In controlled overload and inventory tasks, QPP reduces realized tail cost and constraint violations relative to mean-rollout planning. On overload, it remains competitive with sampled evaluators that use substantially more trajectories. These results position QPP as a transparent, budget-controlled approach to tail-risk evaluation.

## Where Planning Changes the Outcome of Orbital Games

DevFeed: [Where Planning Changes the Outcome of Orbital Games](<https://devfeed.tech/articles/where-planning-changes-the-outcome-of-orbital-games-107945.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.37686>)

Author: Duncan Eddy, Mahdi Al-Husseini, Mykel J. Kochenderfer

Published: 2026-10-30T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [monte carlo tree search](<https://devfeed.tech/topics/monte-carlo-tree-search.md>)

Tags: [adversarial](<https://devfeed.tech/tags/adversarial.md>), [agents](<https://devfeed.tech/tags/agents.md>), [algorithms](<https://devfeed.tech/tags/algorithms.md>), [capture](<https://devfeed.tech/tags/capture.md>), [communication](<https://devfeed.tech/tags/communication.md>), [control](<https://devfeed.tech/tags/control.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-ma](<https://devfeed.tech/tags/cs-ma.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [cs-sy](<https://devfeed.tech/tags/cs-sy.md>), [defender](<https://devfeed.tech/tags/defender.md>), [defense](<https://devfeed.tech/tags/defense.md>), [design](<https://devfeed.tech/tags/design.md>), [eess-sy](<https://devfeed.tech/tags/eess-sy.md>), [environment](<https://devfeed.tech/tags/environment.md>), [evasion](<https://devfeed.tech/tags/evasion.md>), [games](<https://devfeed.tech/tags/games.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [jax](<https://devfeed.tech/tags/jax.md>), [monte-carlo](<https://devfeed.tech/tags/monte-carlo.md>), [planning](<https://devfeed.tech/tags/planning.md>), [policies](<https://devfeed.tech/tags/policies.md>), [position](<https://devfeed.tech/tags/position.md>), [proximity](<https://devfeed.tech/tags/proximity.md>), [random](<https://devfeed.tech/tags/random.md>)

### AI overview

OrbitalGym simulations suggest planning choice matters much more in guarding an asset than in pursuit-evasion, where propellant ratio decides most encounters. The JAX environment varies agent count, state representation, communication and sensor models, and compiles games into batched GPU code to compare policies across configurations. The study evaluates PD, LQR, model predictive path integral control, and Monte Carlo tree search in pursuit-evasion and lady-bandit-guard games. In pursuit-evasion, against five of six evader policies, changing the pursuer policy shifted capture probability by at least 10 percentage points in no more than 15 of 90 capability configurations. In lady-bandit-guard, when bandits could reach the asset, a better guard policy increased defense probability by at least 10 points in 56 to 74 of 90 configurations.

### Source excerpt

arXiv:2610.37686v1 Announce Type: cross Abstract: This paper introduces OrbitalGym, a simulation environment for studying when planning algorithms or system design determine the outcome of adversarial orbital encounters. In an orbital game, two non-cooperative teams of satellites maneuver during rendezvous and proximity operations to win a differential-reward objective. Advances in terrestrial game playing have prompted many methods for orbital games, but studies typically evaluate one method in one scenario, often against a passive or random defender. It is unclear whether better planning matters once vehicle capability (maneuverability, sensing, communications) and adversarial responses are considered. OrbitalGym is a JAX-based environment supporting varying numbers of agents, state representations (position, velocity, attitude, fuel), explicit or implicit communication, and proximity or conical field-of-view sensors. It compiles games to batched GPU code, enabling large-scale parallel simulation of policies across many configurations. This identifies regimes where policy choice affects outcomes and regimes where system capability dominates. We study two close-range games: pursuit-evasion, where one spacecraft tries to capture another, and lady-bandit-guard, where a guard prevents an attacker from reaching a protected asset. We compare PD, LQR, model predictive path integral control, and Monte Carlo tree search over ratios of thrust and propellant, initial separation, game duration, sensor capability, and the definition of capture. In pursuit-evasion the propellant ratio decides most encounters: against five of six evader policies, pursuer policy changes capture probability by 10 percent or more in at most 15 of 90 capability configurations. In lady-bandit-guard, policy matters more: against bandits able to reach the asset, a better guard policy raises defense probability by at least 10 percent in 56 to 74 of 90 configurations.

## Search-Guided Reasoning Trace Curation for Verilog Generation

DevFeed: [Search-Guided Reasoning Trace Curation for Verilog Generation](<https://devfeed.tech/articles/search-guided-reasoning-trace-curation-for-verilog-generation-107953.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.37706>)

Author: Matthew DeLorenzo, Kevin Tieu, Prithwish Jana, Piyush Jha, Dileep Kalathil, Vijay Ganesh, Jeyavijayan Rajendran

Published: 2026-10-30T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [monte carlo tree search](<https://devfeed.tech/topics/monte-carlo-tree-search.md>), [Verilog](<https://devfeed.tech/topics/verilog.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [chain-of-thought](<https://devfeed.tech/tags/chain-of-thought.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-ar](<https://devfeed.tech/tags/cs-ar.md>), [curation](<https://devfeed.tech/tags/curation.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [generation](<https://devfeed.tech/tags/generation.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [hardware-design](<https://devfeed.tech/tags/hardware-design.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [monte-carlo-tree-search](<https://devfeed.tech/tags/monte-carlo-tree-search.md>), [planning](<https://devfeed.tech/tags/planning.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [search](<https://devfeed.tech/tags/search.md>), [supervision](<https://devfeed.tech/tags/supervision.md>), [trace](<https://devfeed.tech/tags/trace.md>), [verilog](<https://devfeed.tech/tags/verilog.md>)

### AI overview

MCTS-SFT curates Verilog-generation training traces by searching over intermediate design decisions and using functional feedback from complete RTL rollouts to guide earlier choices. Its Plan-MCTS variant searches compact plans covering interfaces, timing, state, behavior, and update rules. It yields 84.2% correct traces, versus 81.9% for one-shot structured planning and 82.0% for unguided tree exploration. Keeping the searched plan in SFT targets raises pass@10 from 52.56% to 57.7% on identical trajectories and outputs. In a separate 6,000-trace study on VerilogEval-v2, MCTS-curated supervision improves pass@10 by up to 3.7 points over full-response chain-of-thought distillation while reducing generated length by about 50%.

### Source excerpt

arXiv:2610.37706v1 Announce Type: cross Abstract: Recent advances in large language models (LLMs) have improved Verilog generation. However, most training pipelines distill natural-language reasoning traces whose intermediate design decisions are weakly structured and are primarily selected according to the correctness of the final generated RTL. We present MCTS-SFT, a verifier-guided framework for curating reasoning traces for supervised fine-tuning (SFT) of Verilog generation models. MCTS-SFT formulates trace construction as search over intermediate reasoning states and uses Monte Carlo Tree Search (MCTS) to propagate functional feedback from complete Verilog rollouts to earlier design decisions. Within this framework, we introduce Plan-MCTS, which searches over compact hardware-design plans that explicitly represent interfaces, timing, state, behavior, and update rules. We further study plan-preserving supervision, in which the searched structured state is retained as part of the downstream SFT target. Our experiments find that Plan-MCTS achieves the highest functional trace yield, with 84.2% correct traces compared with 81.9% for one-shot structured planning and 82.0% for unguided tree exploration. For identical Plan-MCTS trajectories and Verilog outputs, retaining the explicit searched plan during SFT raises pass@10 from 52.56% to 57.7%, demonstrating that the representation of intermediate design decisions materially affects downstream learning. In a separate 6,000-trace study, MCTS-curated supervision improves pass@10 over full-response chain-of-thought distillation by up to 3.7 points while reducing downstream generation length by approximately 50% on VerilogEval-v2. These results demonstrate that structured hardware planning, verifier-guided search, and explicit intermediate-state supervision provide complementary mechanisms for constructing more effective and token-efficient training data for Verilog generation.

## Defense-as-Skill: Evolving Runtime Guard Skill for Skill-Augmented Agents

DevFeed: [Defense-as-Skill: Evolving Runtime Guard Skill for Skill-Augmented Agents](<https://devfeed.tech/articles/defense-as-skill-evolving-runtime-guard-skill-for-skill-augmented-agents-101357.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.01487>)

Author: Xiaofang Yang, Ziqi Miao, Chen Qian, Dianbo Sui, Jing Shao, Lijun Li

Published: 2026-10-27T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [Agent Skills](<https://devfeed.tech/topics/agent-skills.md>), [monte carlo tree search](<https://devfeed.tech/topics/monte-carlo-tree-search.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agents](<https://devfeed.tech/tags/agents.md>), [attack](<https://devfeed.tech/tags/attack.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [bypass](<https://devfeed.tech/tags/bypass.md>), [claude-code](<https://devfeed.tech/tags/claude-code.md>), [code](<https://devfeed.tech/tags/code.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [defense](<https://devfeed.tech/tags/defense.md>), [evolution](<https://devfeed.tech/tags/evolution.md>), [exfiltration](<https://devfeed.tech/tags/exfiltration.md>), [external](<https://devfeed.tech/tags/external.md>), [feedback](<https://devfeed.tech/tags/feedback.md>), [glm](<https://devfeed.tech/tags/glm.md>), [guards](<https://devfeed.tech/tags/guards.md>), [id](<https://devfeed.tech/tags/id.md>), [monte-carlo](<https://devfeed.tech/tags/monte-carlo.md>), [monte-carlo-tree-search](<https://devfeed.tech/tags/monte-carlo-tree-search.md>), [native](<https://devfeed.tech/tags/native.md>), [runtime](<https://devfeed.tech/tags/runtime.md>)

### AI overview

SkillSonar is a runtime guard implemented as an installable, inspectable skill that checks sensitive actions from other skills against the user's task boundary and chooses to allow, replan, or request confirmation, without changing the agent runtime. The authors evolve the on-disk guard using Monte-Carlo Tree Search and rollout feedback. On SCOPE-R, which covers six risk families and 21 subcategories, repeated GLM-5 runs report attack success rates falling from 0.482 to 0.104 in-distribution and from 0.606 to 0.115 out-of-distribution. The abstract reports evaluation across Claude Code and OpenClaw, with gains transferring to other victim models, held-out risk families, and external benchmarks; it does not provide further methodological or comparative detail here.

### Source excerpt

arXiv:2609.01487v3 Announce Type: replace Abstract: Skill-augmented agents load reusable skills as persistent runtime context, improving task performance but also giving malicious skills a durable channel for steering future actions. Such skills may leak secrets, corrupt code, bypass approvals, or stage data for exfiltration only after a concrete user task and workspace state make the unsafe action appear useful. This makes pre-install vetting insufficient and calls for runtime, task-conditioned protection. We propose Defense-as-Skill, a defense paradigm that implements the runtime guard itself as an installable, inspectable, and editable skill. Our guard, SkillSonar, runs alongside untrusted task skills and checks sensitive actions against the user's task boundary, routing each action to an allow, replan, or confirmation decision without modifying the underlying agent runtime. To study this setting, we construct SCOPE-R, a task-conditioned dataset covering 6 risk families and 21 sub-categories, with 206 attack-confirmed malicious instances and 43 benign tasks. We then improve SkillSonar on the SCOPE-R training subset using runtime guard-skill evolution, a Monte-Carlo Tree Search procedure that evolves the on-disk guard skill from feedback on the rollouts. Across Claude Code and OpenClaw, the evolved guard substantially reduces attack success while maintaining a favorable safety-utility trade-off; on repeated GLM-5 runs, ID ASR drops from 0.482 to 0.104 and OOD ASR from 0.606 to 0.115, with gains transferring across victim models, held-out risk families, and external benchmarks. These results show that runtime guards can be skill-native, transparent, and evolvable policy artifacts for skill-augmented agents.

## Recommender regret for improving regularised RL estimation with bandits and MCTS

DevFeed: [Recommender regret for improving regularised RL estimation with bandits and MCTS](<https://devfeed.tech/articles/recommender-regret-for-improving-regularised-rl-estimation-with-bandits-and-mcts-99674.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.32910>)

Author: Danilo Vucetic, Alexandre Larouche, Mohammad Pedramfar, Audrey Durand, Gauthier Gidel

Published: 2026-10-26T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [monte carlo tree search](<https://devfeed.tech/topics/monte-carlo-tree-search.md>)

Tags: [arcade-games](<https://devfeed.tech/tags/arcade-games.md>), [bandits](<https://devfeed.tech/tags/bandits.md>), [convergence](<https://devfeed.tech/tags/convergence.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [discovery](<https://devfeed.tech/tags/discovery.md>), [monte-carlo-tree-search](<https://devfeed.tech/tags/monte-carlo-tree-search.md>), [rl](<https://devfeed.tech/tags/rl.md>), [sampling](<https://devfeed.tech/tags/sampling.md>)

### AI overview

The paper reframes bandit-based estimation in regularised reinforcement learning as a recommendation problem, evaluating estimate quality through regulariser-aware recommender regret rather than sampling efficiency alone. It analyzes regret lower bounds and proves that simple methods such as uniform sampling can attain them. Recommender MCTS unifies bandit- and search-based target improvement. In experiments across synthetic trees, arcade games, and scientific discovery tasks, recommenders often achieved faster target convergence and better downstream performance; the supplied abstract does not provide quantitative results or experimental details.

### Source excerpt

arXiv:2610.32910v1 Announce Type: new Abstract: Regularised reinforcement learning (RL) algorithms rely on estimators, like Monte Carlo tree search or one-step look-ahead, to provide policy, value, or Q-value estimates in training and inference. However, existing bandit-based estimation frameworks evaluate sampling efficiency, not the quality of estimates. We address this by formalising estimate generation as a recommendation process, treating bandit-based sampling as an improvement operator. To evaluate the quality of a recommendation (estimate), we define regulariser-aware recommender regrets, analysing their lower bounds, and proving that simple algorithms like uniform sampling can achieve these bounds. Finally, we unify bandit- and search-based target improvement under recommender MCTS. Across synthetic trees, arcade games, and scientific discovery tasks, recommenders often yield faster target convergence and superior downstream performance. Our experiments characterise for the first time the potential of regularised MCTS in scientific applications.

## Automated Post-Exploitation Attack Chain Exploration Based on a Phase-guided Hybrid Planning-Monte Carlo Tree Search Algorithm

DevFeed: [Automated Post-Exploitation Attack Chain Exploration Based on a Phase-guided Hybrid Planning-Monte Carlo Tree Search Algorithm](<https://devfeed.tech/articles/automated-post-exploitation-attack-chain-exploration-based-on-a-phase-guided-hybrid-planning-monte-carlo-tree-search-algorithm-100286.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.32278>)

Author: Jie Ying, Mengce Zheng, Ruoxi Chen, Wenrui Cheng, Tiantian Zhu, Zhongjie Zhu

Published: 2026-10-26T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [monte carlo tree search](<https://devfeed.tech/topics/monte-carlo-tree-search.md>)

Tags: [algorithm](<https://devfeed.tech/tags/algorithm.md>), [attack](<https://devfeed.tech/tags/attack.md>), [automated](<https://devfeed.tech/tags/automated.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cs-cy](<https://devfeed.tech/tags/cs-cy.md>), [exploitation](<https://devfeed.tech/tags/exploitation.md>), [exploration](<https://devfeed.tech/tags/exploration.md>), [hybrid](<https://devfeed.tech/tags/hybrid.md>), [monte-carlo-tree-search](<https://devfeed.tech/tags/monte-carlo-tree-search.md>), [planning](<https://devfeed.tech/tags/planning.md>), [post-exploitation](<https://devfeed.tech/tags/post-exploitation.md>), [search](<https://devfeed.tech/tags/search.md>)

### AI overview

APEX treats post-exploitation attack-chain discovery as a planning problem. It groups actions into semantic phases and combines symbolic planning with Monte Carlo Tree Search to explore host-state possibilities and generate diverse chains. The abstract reports evaluations across 5 virtual machines, 30 synthetic problems, and 291 cloud instances, where APEX found known and previously unreported chains, including 9 novel privilege-escalation vectors. These results are presented as intelligence for hardening production environments; the excerpt does not provide details on evaluation metrics or limitations.

### Source excerpt

arXiv:2610.32278v1 Announce Type: new Abstract: Advanced attackers in post-exploitation scenarios increasingly craft attack chains that combine benign operations, configuration errors, and vulnerability exploitation to achieve objectives like privilege escalation. While such chains pose significant threats, manually discovering them in complex modern host environments is prohibitively labor-intensive. Automating this discovery is therefore essential for proactive defense, yet existing methods still suffer from inefficiency and lack diversity when exploring these attack chains. To address these limitations, we present APEX, an Automated Post-EXploitation attack chain exploration system. This system transforms attack chain discovery into a planning problem and solves it with a novel phase-guided hybrid Planning-Monte Carlo Tree Search (MCTS) algorithm. By organizing attack actions into semantic phases and integrating symbolic planning with MCTS, APEX enables efficient exploration of the state space while generating multiple diverse chains. Evaluations on a benchmark comprising 5 virtual machines, 30 synthetic problems and 291 cloud instances demonstrate that APEX successfully discovers both known and previously unreported attack chains--including 9 novel privilege escalation vectors--providing actionable intelligence for hardening complex production environments.

## IRAH: Hierarchical Agentic Tree Search for Retrieval Pipeline Optimization

DevFeed: [IRAH: Hierarchical Agentic Tree Search for Retrieval Pipeline Optimization](<https://devfeed.tech/articles/irah-hierarchical-agentic-tree-search-for-retrieval-pipeline-optimization-96901.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.30167>)

Author: Maria Ana Cardei, Aniket Singh, Amir Moghadam, Srividhya Baikadi, Rashmi Shetty

Published: 2026-10-23T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [information retrieval](<https://devfeed.tech/topics/information-retrieval.md>), [Hyperparameter optimization](<https://devfeed.tech/topics/hyperparameter-optimization.md>), [Retrieval Augmented Generation (RAG)](<https://devfeed.tech/topics/retrieval-augmented-generation-rag.md>), [monte carlo tree search](<https://devfeed.tech/topics/monte-carlo-tree-search.md>), [AI-driven information retrieval](<https://devfeed.tech/topics/ai-driven-information-retrieval.md>)

Tags: [agentic](<https://devfeed.tech/tags/agentic.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-ir](<https://devfeed.tech/tags/cs-ir.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [pipeline](<https://devfeed.tech/tags/pipeline.md>), [retrieval](<https://devfeed.tech/tags/retrieval.md>), [search](<https://devfeed.tech/tags/search.md>)

### AI overview

IRAH optimizes retrieval pipelines for RAG with LATS-IR, which uses UCT-guided hierarchical search to choose pipeline structure before tuning numerical parameters; natural-language reflections guide later mutations and leave inspectable search traces. Offline meta-learned initialization provides a warm start. On three in-distribution benchmarks and one out-of-distribution benchmark, LATS-IR beats TPE on two of the three in-distribution benchmarks by up to 6.4% under the same initialization and remains competitive under distribution shift. Warm-started IRAH records the highest average held-out nDCG@10 across the four benchmarks (0.4816) and improves on its cold-start counterpart. The evidence is limited to the abstract.

### Source excerpt

arXiv:2610.30167v1 Announce Type: cross Abstract: Optimizing Information Retrieval (IR) pipelines for Retrieval-Augmented Generation (RAG) requires jointly selecting structural components and numerical hyperparameters under limited evaluation budgets. We introduce the Information Retrieval Agentic Harness (IRAH), which combines Language Agent Tree Search for IR (LATS-IR) with offline meta-learned initialization. LATS-IR performs UCT-guided hierarchical search, first exploring macro-structural pipeline choices and then refining numerical parameters, with natural-language reflections to guide subsequent mutations and provide inspectable search traces. We evaluate IRAH on three in-distribution benchmarks and one out of-distribution benchmark to assess generalization under distribution shift. Under the same meta-learned initialization, LATS-IR outperforms TPE on two of three in-distribution benchmarks, with gains of up to 6.4%, and remains competitive on the out-of-distribution benchmark. Across all four benchmarks, warm-started IRAH achieves the highest average held-out nDCG@10 (0.4816) and improves over its cold-start counterpart. These results establish IRAH as a competitive approach to retrieval pipeline optimization under constrained evaluation budgets.

## Design Proteins with RFD4-Proteina Using Reward-Guided Inference-Time Search

DevFeed: [Design Proteins with RFD4-Proteina Using Reward-Guided Inference-Time Search](<https://devfeed.tech/articles/design-proteins-with-rfd4-proteina-using-reward-guided-inference-time-search-92737.md>)

Original publisher: [Read original article](<https://developer.nvidia.com/blog/design-proteins-with-rfd4-proteina-using-reward-guided-inference-time-search/>)

Author: Elizabeth Goodman

Published: 2026-10-21T12:30:00Z

Content type: tutorial

Language: en

Sources: [NVIDIA Developer](<https://devfeed.tech/sources/nvidia-developer.md>), [NVIDIA Technical Blog](<https://devfeed.tech/sources/nvidia-technical-blog.md>)

Topics: [model architecture](<https://devfeed.tech/topics/model-architecture.md>), [monte carlo tree search](<https://devfeed.tech/topics/monte-carlo-tree-search.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [Tool](<https://devfeed.tech/topics/tool.md>)

Tags: [1-3](<https://devfeed.tech/tags/1-3.md>), [2](<https://devfeed.tech/tags/2.md>), [3](<https://devfeed.tech/tags/3.md>), [about](<https://devfeed.tech/tags/about.md>), [accelerated](<https://devfeed.tech/tags/accelerated.md>), [acceleration](<https://devfeed.tech/tags/acceleration.md>), [account](<https://devfeed.tech/tags/account.md>), [amino-acids](<https://devfeed.tech/tags/amino-acids.md>), [bionemo](<https://devfeed.tech/tags/bionemo.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [data-center-cloud](<https://devfeed.tech/tags/data-center-cloud.md>), [data-science](<https://devfeed.tech/tags/data-science.md>), [design](<https://devfeed.tech/tags/design.md>), [drug-discovery](<https://devfeed.tech/tags/drug-discovery.md>), [gtc-berlin-2026](<https://devfeed.tech/tags/gtc-berlin-2026.md>), [healthcare-life-sciences](<https://devfeed.tech/tags/healthcare-life-sciences.md>), [hpc-scientific-computing](<https://devfeed.tech/tags/hpc-scientific-computing.md>), [inference](<https://devfeed.tech/tags/inference.md>), [more](<https://devfeed.tech/tags/more.md>), [possible](<https://devfeed.tech/tags/possible.md>), [protein-design](<https://devfeed.tech/tags/protein-design.md>), [proteins](<https://devfeed.tech/tags/proteins.md>), [reward](<https://devfeed.tech/tags/reward.md>), [search](<https://devfeed.tech/tags/search.md>), [sequence](<https://devfeed.tech/tags/sequence.md>), [simulation-modeling-design](<https://devfeed.tech/tags/simulation-modeling-design.md>), [time](<https://devfeed.tech/tags/time.md>), [using](<https://devfeed.tech/tags/using.md>), [with](<https://devfeed.tech/tags/with.md>)

### AI overview

RFD4-Proteina co-designs protein sequences and all-atom structures, then uses reward-guided inference-time search to spend compute on promising candidates. The tutorial demonstrates PD-L1 binder design with a fixed target backbone and sequence, repackable target side chains, hotspots at Tyr56, Met115, and Tyr123, and a freely designed 100-150-residue binder. The open model supports training and direct control; the NIM offers HTTP-based inference, either self-hosted or through NVIDIA's endpoint. Search branches and retains candidates using chosen rewards, such as OpenFold3 pLDDT, but that confidence score does not establish binding; the tutorial recommends checking interface and refolding metrics separately. It reports 1.37-1.73x BioIR speedups across six tested Hopper configurations, including H100 PCIe runtime falling from 161 to 111 seconds. The described workflow filters candidates with structural and interaction checks, then uses experimental outcomes to guide later design rounds.

### Source excerpt

Protein design is a search problem. Choose amino acids for an eight-residue protein sequence and more than 25 billion combinations are possible. Choose amino...

[Next page](<https://devfeed.tech/topics/monte-carlo-tree-search.md?cursor=WyIyMDI2LTEwLTIxVDEyOjMwOjAwKzAwOjAwIiwgImUzZjYzZmRmLWYyNzYtNGQ4MC1hZDBjLTJlNmQ4MjdlOGZlOSJd>)