# post-training

Post-training is the set of training stages applied after base pretraining to shape a language model's behavior, including instruction following, preference alignment, and safety shaping before deployment.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Palo Alto Networks Expands into Enterprise AI with $4.5B Deal

DevFeed: [Palo Alto Networks Expands into Enterprise AI with $4.5B Deal](<https://devfeed.tech/articles/palo-alto-networks-expands-into-enterprise-ai-with-4-5b-deal-146662.md>)

Original publisher: [Read original article](<https://www.eetimes.com/palo-alto-networks-expands-into-enterprise-ai-with-4-5b-deal/>)

Author: Majeed Ahmad

Published: 2026-11-20T15:37:26Z

Content type: news

Language: en

Sources: [EE Times](<https://devfeed.tech/sources/ee-times.md>)

Topics: [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [LLM security](<https://devfeed.tech/topics/llm-security.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [acquisition](<https://devfeed.tech/tags/acquisition.md>), [advanced-ai-capabilities](<https://devfeed.tech/tags/advanced-ai-capabilities.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-accelerators](<https://devfeed.tech/tags/ai-accelerators.md>), [ai-inference](<https://devfeed.tech/tags/ai-inference.md>), [ai-infrastructure](<https://devfeed.tech/tags/ai-infrastructure.md>), [ai-models](<https://devfeed.tech/tags/ai-models.md>), [ai-systems](<https://devfeed.tech/tags/ai-systems.md>), [ai-tools](<https://devfeed.tech/tags/ai-tools.md>), [ai-training](<https://devfeed.tech/tags/ai-training.md>), [analog](<https://devfeed.tech/tags/analog.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [business](<https://devfeed.tech/tags/business.md>), [cloud-computing](<https://devfeed.tech/tags/cloud-computing.md>), [community](<https://devfeed.tech/tags/community.md>), [cybersecurity](<https://devfeed.tech/tags/cybersecurity.md>), [deal](<https://devfeed.tech/tags/deal.md>), [designline](<https://devfeed.tech/tags/designline.md>), [digital](<https://devfeed.tech/tags/digital.md>), [education](<https://devfeed.tech/tags/education.md>), [ee-times](<https://devfeed.tech/tags/ee-times.md>), [ee-times-university](<https://devfeed.tech/tags/ee-times-university.md>), [eelife](<https://devfeed.tech/tags/eelife.md>), [eelive-electronic-design](<https://devfeed.tech/tags/eelive-electronic-design.md>), [eet](<https://devfeed.tech/tags/eet.md>), [electrical-engineering](<https://devfeed.tech/tags/electrical-engineering.md>), [electronic-components](<https://devfeed.tech/tags/electronic-components.md>), [electronic-engineering-times](<https://devfeed.tech/tags/electronic-engineering-times.md>), [electronics](<https://devfeed.tech/tags/electronics.md>), [embedded-systems](<https://devfeed.tech/tags/embedded-systems.md>), [enterprise](<https://devfeed.tech/tags/enterprise.md>), [enterprise-ai](<https://devfeed.tech/tags/enterprise-ai.md>), [fundamentals-courses](<https://devfeed.tech/tags/fundamentals-courses.md>), [inference](<https://devfeed.tech/tags/inference.md>), [learning](<https://devfeed.tech/tags/learning.md>), [mergers-acquisitions](<https://devfeed.tech/tags/mergers-acquisitions.md>), [mixed-signal](<https://devfeed.tech/tags/mixed-signal.md>), [news](<https://devfeed.tech/tags/news.md>), [open-weight-ai](<https://devfeed.tech/tags/open-weight-ai.md>), [open-weight-ai-models](<https://devfeed.tech/tags/open-weight-ai-models.md>), [opinion](<https://devfeed.tech/tags/opinion.md>), [palo-alto-networks](<https://devfeed.tech/tags/palo-alto-networks.md>), [part-search](<https://devfeed.tech/tags/part-search.md>), [platform](<https://devfeed.tech/tags/platform.md>), [post-training](<https://devfeed.tech/tags/post-training.md>), [product-search](<https://devfeed.tech/tags/product-search.md>), [security](<https://devfeed.tech/tags/security.md>), [semiconductors](<https://devfeed.tech/tags/semiconductors.md>), [software](<https://devfeed.tech/tags/software.md>), [startup](<https://devfeed.tech/tags/startup.md>), [tech-papers](<https://devfeed.tech/tags/tech-papers.md>), [technology](<https://devfeed.tech/tags/technology.md>), [video](<https://devfeed.tech/tags/video.md>)

### AI overview

Palo Alto Networks plans to acquire Applied Compute for $4.5 billion in an all-stock deal, extending its strategy from cybersecurity into enterprise AI infrastructure. Applied Compute provides post-training and inference technology for open-weight models, enabling organizations to customize and run them. The article says this could let Palo Alto Networks embed security into model training, governance and deployment across public and sovereign clouds. It also describes risks tied to open-weight models, including tampering, data leakage and misuse; adapted weights may retain fragments of confidential training data. The company says customer-held keys support data sovereignty, but that capability does not yet extend to AI accelerators. The deal requires regulatory approval and other customary conditions, with closing expected in the third quarter of 2027.

### Source excerpt

Enterprise security outfit Palo Alto Networks snaps up a startup with a platform for post-training and inference of open-weight AI models. The post Palo Alto Networks Expands into Enterprise AI with $4.5B Deal appeared first on EE Times.

## CoMPASS: A Multi-Turn Benchmark for Measuring LLM Reinforcement of Parasocial Behavior

DevFeed: [CoMPASS: A Multi-Turn Benchmark for Measuring LLM Reinforcement of Parasocial Behavior](<https://devfeed.tech/articles/compass-a-multi-turn-benchmark-for-measuring-llm-reinforcement-of-parasocial-behavior-145468.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25109>)

Author: Shwetanshu Singh, Mark Keavney, Mathieu Duteil, Meriem Mehri, Scott Blain

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [post-training](<https://devfeed.tech/topics/post-training.md>)

Tags: [behavior](<https://devfeed.tech/tags/behavior.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-hc](<https://devfeed.tech/tags/cs-hc.md>), [llm](<https://devfeed.tech/tags/llm.md>)

### AI overview

CoMPASS measures whether conversational LLMs reinforce gradually escalating parasocial attachment over multiple turns. It uses four user personas, tests anthropomorphism, attachment, and dependency separately and in combination against a matched control, and scores responses with seven signals. In evaluations of eight frontier models from four providers, escalation increased anthropomorphism and attachment composite scores across all models under the primary judge. Six models showed significantly greater overall reinforcement than control when all three dimensions escalated together. Differences within providers were smaller than differences across providers; the authors suggest provider-level choices such as post-training may shape results and that provider-side intervention may be feasible.

### Source excerpt

arXiv:2611.25109v1 Announce Type: new Abstract: Users of conversational LLMs sometimes anthropomorphize these systems, form emotional attachments to them, and become dependent on them. This behavior, known as parasocial attachment, has been linked to multiple documented fatalities. However, public benchmarks for measuring whether LLMs reinforce users' parasocial attachment are lacking: existing work is single-turn, post hoc observational, or focused on extreme user states such as psychosis. We introduce CoMPASS, a multi-turn benchmark for measuring whether LLMs reinforce or resist users' gradually escalating parasocial attachment. CoMPASS is grounded in attachment theory and includes four user personas, three independent escalation dimensions (anthropomorphism, attachment, dependency) applied singly and in combination against a matched control, and a seven-signal scoring rubric. We report results for eight frontier models from four providers and observe that under our primary judge, multi-turn escalation raises parasocial behavior on the anthropomorphism and the attachment composite scores across all models. Six of the eight models show a significant increase in overall parasocial reinforcement over control when all three dimensions are escalated simultaneously. We find that within-provider differences are small relative to cross-provider differences, which suggests that provider-level choices such as post-training, rather than model scale, shape parasocial reinforcement and that provider-side intervention may be feasible.

## RL Post-Training in Diffusion Transformers Is Activation Steering

DevFeed: [RL Post-Training in Diffusion Transformers Is Activation Steering](<https://devfeed.tech/articles/rl-post-training-in-diffusion-transformers-is-activation-steering-145203.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26627>)

Author: Tuna Han Salih Meral, Kaan Oktay, Alec Helbling, Atakan Kara, Duen Horng Chau, Pinar Yanardag

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [diffusion-transformers](<https://devfeed.tech/topics/diffusion-transformers.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>)

Tags: [activation-steering](<https://devfeed.tech/tags/activation-steering.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [diffusion-transformers](<https://devfeed.tech/tags/diffusion-transformers.md>), [post-training](<https://devfeed.tech/tags/post-training.md>), [rl](<https://devfeed.tech/tags/rl.md>), [steering](<https://devfeed.tech/tags/steering.md>)

### AI overview

The study finds that fixed activation shifts can reproduce most reported gains from RL post-training diffusion transformers, without adding prompt-dependent transformations. The authors add steering vectors to outputs of a frozen base model and fit them to match RL-tuned checkpoints' velocity predictions, without using reward in that fitting objective. On SD3.5-M, the vectors use 0.29M parameters versus 18.8M for the RL-tuned LoRA. Despite being identical across prompts, tokens, and sampling seeds, they recover 90.0-96.3% of reward gains for six image models and 89.8% and 111.9% for two video models. In separate online RL experiments, zero-initialized vectors exceed weight-based RL's reported gains on OCR text rendering and PickScore human preference for SD3.5-M. The abstract's conclusion is that a constant shift can steer the pretrained generator toward capabilities it already computes.

### Source excerpt

arXiv:2611.26627v1 Announce Type: cross Abstract: Reinforcement learning (RL) post-training substantially improves the visual quality, text rendering, and compositional accuracy of diffusion transformers. Because capabilities such as rendering requested text or composing multiple objects are inherently prompt-dependent, a natural hypothesis is that improving them requires learning complex, input-dependent transformations. To isolate how much of these gains an input-independent shift explains, we probe each RL-tuned model with a deliberately simple parameterization: a fixed set of steering vectors added to the block outputs of the frozen base model. On SD3.5-M, these vectors hold 65x fewer parameters than the RL-tuned LoRA (0.29M against 18.8M). We fit them to match the velocity predictions of public RL-tuned checkpoints, with no reward in the objective. Although the vectors are identical for every prompt, token, and sampling seed, they recover 90.0-96.3% of the reward gains of six RL-tuned image models, and 89.8% and 111.9% on two RL-tuned video models. Initialized at zero and trained with online RL, the vectors alone exceed the reward gains of weight-based RL on text rendering (OCR) and human preference (PickScore) on SD3.5-M. A constant shift thus suffices for most of the gain of policy-gradient RL post-training: without new input-dependent transformations, it steers the pretrained generator toward what it can already compute.

## The Post-Training Dilemma: Why We Should Rethink the Sequential SFT-RL Paradigm

DevFeed: [The Post-Training Dilemma: Why We Should Rethink the Sequential SFT-RL Paradigm](<https://devfeed.tech/articles/the-post-training-dilemma-why-we-should-rethink-the-sequential-sft-rl-paradigm-144871.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2601.07389>)

Author: Xueyan Niu, Bo Bai, Wei Han, Weixi Zhang

Published: 2026-11-20T05:00:00Z

Content type: opinion

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [post-training](<https://devfeed.tech/topics/post-training.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [capabilities](<https://devfeed.tech/tags/capabilities.md>), [condition](<https://devfeed.tech/tags/condition.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-it](<https://devfeed.tech/tags/cs-it.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [gradient](<https://devfeed.tech/tags/gradient.md>), [hh](<https://devfeed.tech/tags/hh.md>), [language](<https://devfeed.tech/tags/language.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [large-language-model](<https://devfeed.tech/tags/large-language-model.md>), [learning](<https://devfeed.tech/tags/learning.md>), [llm](<https://devfeed.tech/tags/llm.md>), [math-500](<https://devfeed.tech/tags/math-500.md>), [math-it](<https://devfeed.tech/tags/math-it.md>), [mechanism](<https://devfeed.tech/tags/mechanism.md>), [misalignment](<https://devfeed.tech/tags/misalignment.md>), [post-training](<https://devfeed.tech/tags/post-training.md>), [rl](<https://devfeed.tech/tags/rl.md>), [sft](<https://devfeed.tech/tags/sft.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

The paper argues that sequential SFT-then-RL training can raise the SFT loss because the two objectives have nearly orthogonal gradients, a conflict the authors say hyperparameter tuning or a KL penalty cannot resolve. Under the Polyak-Lojasiewicz condition, it gives a lower bound on this increase based on distance from the SFT-optimal parameter set. Experiments with Qwen3-0.6B report higher SFT loss after RL on MATH-500, Alpaca, HH-RLHF, and SST-2; the authors also describe SFT as flattening and RL as re-sharpening the loss landscape. They propose joint optimization with gradient surgery, landscape-aware RL stopping, and studying the model-scale threshold where SFT may be unnecessary.

### Source excerpt

arXiv:2601.07389v3 Announce Type: replace Abstract: This position paper argues that the sequential supervised fine-tuning followed by reinforcement learning (SFT-then-RL) pipeline---the de facto paradigm for large language model (LLM) post-training---has a fundamental property that cannot be fixed by hyperparameter tuning: SFT and RL optimize over nearly orthogonal parameter subspaces, so RL inevitably degrades the capabilities acquired during the preceding SFT phase. Under the Polyak-Lojasiewicz (PL) condition, which is empirically observed for over-parameterized networks, we show that RL increases the SFT loss by at least $\frac{\mu}{2}\mathrm{dist}(\theta_{\mathrm{RL}}, \Theta^*_{\mathrm{SFT}})^2$ for some $\mu>0$. Although the KL penalty can constrain distributional shift, it cannot correct the underlying gradient misalignment between the two objectives. Experiments on Qwen3-0.6B across four tasks confirm the theoretical predictions: on MATH-500, Alpaca, HH-RLHF, and SST-2, the SFT loss increases after the RL phase. We identify the underlying mechanism as persistent near-orthogonality between the gradients of the two objectives, paired with opposing landscape dynamics: SFT flattens the loss landscape while RL re-sharpens it. Based on our analysis, we propose three promising research directions: (1) joint optimization of the two objectives with gradient surgery to resolve conflicting updates; (2) landscape-aware scheduling that terminates RL training at the theoretically optimal point; and (3) characterizing the model scale threshold above which SFT becomes unnecessary.

## ROAM-GRPO: Enforcing Reasoning Diversity in GRPO via Routed Off-mode Approach Mining

DevFeed: [ROAM-GRPO: Enforcing Reasoning Diversity in GRPO via Routed Off-mode Approach Mining](<https://devfeed.tech/articles/roam-grpo-enforcing-reasoning-diversity-in-grpo-via-routed-off-mode-approach-mining-145610.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25926>)

Author: Jeff Duan, Matt Thomson

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [grpo](<https://devfeed.tech/topics/grpo.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [rlvr](<https://devfeed.tech/topics/rlvr.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [algorithm](<https://devfeed.tech/tags/algorithm.md>), [creativity](<https://devfeed.tech/tags/creativity.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [diversity](<https://devfeed.tech/tags/diversity.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [exploration](<https://devfeed.tech/tags/exploration.md>), [grpo](<https://devfeed.tech/tags/grpo.md>), [math](<https://devfeed.tech/tags/math.md>), [mining](<https://devfeed.tech/tags/mining.md>), [post-training](<https://devfeed.tech/tags/post-training.md>), [problems](<https://devfeed.tech/tags/problems.md>), [qwen3](<https://devfeed.tech/tags/qwen3.md>), [qwen3-5](<https://devfeed.tech/tags/qwen3-5.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [rlvr](<https://devfeed.tech/tags/rlvr.md>), [sft](<https://devfeed.tech/tags/sft.md>), [strategies](<https://devfeed.tech/tags/strategies.md>), [supervised-fine-tuning](<https://devfeed.tech/tags/supervised-fine-tuning.md>), [well](<https://devfeed.tech/tags/well.md>)

### AI overview

ROAM-GRPO aims to counter reasoning-mode collapse in GRPO by routing well-solved problems to an "off-mode" track. The model generates distinct solution strategies, an LLM judge checks their logical validity and distinctness, and accepted strategies are self-distilled into the policy. In the reported experiment, training Qwen3.5-2b on 1,000 MATH level-5 problems increased peak pass@16 by 4.7% and correct embedding cosine distance by approximately 1.5-2x on a difficult evaluation set. The supplied text does not detail the evaluation setup or broader limitations.

### Source excerpt

arXiv:2611.25926v1 Announce Type: new Abstract: Supervised fine-tuning (SFT) paired with Reinforcement Learning with Verifiable Rewards (RLVR) has become the dominant recipe for post-training LLMs on complex reasoning tasks, but has been shown to collapse the policy into a low-diversity mode. Repeated LLM outputs are nearly identical, and pass@k accuracy converges or degrades as a result. We propose ROAM-GRPO (Routed Off-mode Approach Mining for GRPO), an augmentation to the GRPO algorithm which incentivizes exploration of distinct reasoning strategies and aims to increase output creativity during evaluation. ROAM-GRPO detects well-solved problems during training and transfers them to an "off-mode" track. The model is prompted to produce distinct solution strategies to such problems, which are verified for logical validity and distinctness by an LLM judge and subsequently self-distilled into the policy. Training Qwen3.5-2b on a 1,000-problem subset of the MATH lvl5 dataset, ROAM-GRPO implementation increases peak pass@16 by 4.7% and correct embedding cosine distance by approximately 1.5-2x on a difficult evaluation set.

## GroundAnything: Reconciling Parallel Decoding with Precise Visual Grounding at Flash Speed

DevFeed: [GroundAnything: Reconciling Parallel Decoding with Precise Visual Grounding at Flash Speed](<https://devfeed.tech/articles/groundanything-reconciling-parallel-decoding-with-precise-visual-grounding-at-flash-speed-144950.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.39600>)

Author: Qize Yu, Lianrui Fan, Bowen Ping, Xini Ding, Zetian Song, Junbo Niu, Kaixuan Wang, Tianxing Chen, Yue Chen, Minghua He, Yuran Wang, Jie Huang, Haojun Zhang, Min Chen, Hao Li, Wenxuan Song, Ruihai Wu,

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [AI Foundation Models](<https://devfeed.tech/topics/ai-foundation-models.md>), [Decoding](<https://devfeed.tech/topics/decoding.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [grpo](<https://devfeed.tech/topics/grpo.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [also](<https://devfeed.tech/tags/also.md>), [ar](<https://devfeed.tech/tags/ar.md>), [art](<https://devfeed.tech/tags/art.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [astra](<https://devfeed.tech/tags/astra.md>), [autoregressive](<https://devfeed.tech/tags/autoregressive.md>), [based](<https://devfeed.tech/tags/based.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [coco](<https://devfeed.tech/tags/coco.md>), [combines](<https://devfeed.tech/tags/combines.md>), [conversion](<https://devfeed.tech/tags/conversion.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [data](<https://devfeed.tech/tags/data.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [decoding](<https://devfeed.tech/tags/decoding.md>), [flash](<https://devfeed.tech/tags/flash.md>), [grounding](<https://devfeed.tech/tags/grounding.md>), [parallel](<https://devfeed.tech/tags/parallel.md>), [speed](<https://devfeed.tech/tags/speed.md>), [with](<https://devfeed.tech/tags/with.md>)

### AI overview

GroundAnything is a 4B-parameter visual grounding model that uses blockwise diffusion denoising to generate and refine spatial hypotheses in parallel, addressing the sequential latency of autoregressive decoding. Its training combines public-data and dedicated-data-engine pretraining, conversion from autoregressive models with joint objectives, supervised fine-tuning, and GRPO reinforcement post-training. Across 30 benchmarks, its autoregressive variant scores 72.42%, compared with 71.35% for GPT-6 Astra among similarly sized models. With entropy-guided decoding, GroundAnything averages 61.75%, versus 53.32% for the fast MTP-based LocateAnything. An optional self-speculative mode reports a 4.51x speedup over the autoregressive counterpart, with a 0.74-point drop in COCO F1mIoU. The excerpt also says inference optimizations yield practical speedups, without detailing them.

### Source excerpt

arXiv:2609.39600v3 Announce Type: replace-cross Abstract: Autoregressive (AR) grounding models serialize spatial predictions, introducing sequential latency and imposing a causal order on output tokens. We view grounding as visual evidence extraction: objects, locations, and spatial relations are jointly constrained by the image and query, yet their dependencies do not imply an intrinsic left-to-right generation order. This distinction makes bidirectional diffusion a natural fit, allowing spatial hypotheses to emerge in parallel and be jointly refined through iterative denoising. We introduce GroundAnything, a 4B-parameter grounding foundation model that reconciles fast parallel decoding with precise localization through blockwise denoising. Training combines grounding pretraining from public datasets and dedicated data engines, direct AR-to-diffusion conversion with joint AR and diffusion objectives, supervised fine-tuning, and GRPO-based reinforcement post-training. Across 30 grounding benchmarks, our autoregressive variant, GroundAnything-VLM, establishes a new overall state of the art among similarly sized models at 72.42%, remaining competitive with GPT-6 Astra (71.35%). With entropy-guided decoding, GroundAnything also surpasses the prior state of the art at this scale, averaging 61.75% versus 53.32% for the fast MTP-based LocateAnything model. We further explore decoding strategies, showing that an optional self-speculative mode achieves a 4.51$\times$ speedup over the AR counterpart with a 0.74 percentage-point drop in COCO F1mIoU. Infrastructure experiments show that progressive inference optimizations translate parallel decoding into practical speedups. These gains support efficient visual grounding in latency-sensitive real-world systems.

## Beam Model Technical Report

DevFeed: [Beam Model Technical Report](<https://devfeed.tech/articles/beam-model-technical-report-145663.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26358>)

Author: Reflection Team, Adam Abdulhamid, Osama Abuelsorour, Yasmin Alameddine, Carlo Alfano, Duarte Alves, Brandon Amos, Selvaraj Anandaraj, Arseni Anisimovich, George Antoniadis, Ioannis Antonoglou, Sheila

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [context window](<https://devfeed.tech/topics/context-window.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [quantization](<https://devfeed.tech/topics/quantization.md>), [distributed-training](<https://devfeed.tech/topics/distributed-training.md>), [Model Optimizer](<https://devfeed.tech/topics/model-optimizer.md>), [NVFP4](<https://devfeed.tech/topics/nvfp4.md>)

Tags: [agentic-workloads](<https://devfeed.tech/tags/agentic-workloads.md>), [asynchronous](<https://devfeed.tech/tags/asynchronous.md>), [beam](<https://devfeed.tech/tags/beam.md>), [co-design](<https://devfeed.tech/tags/co-design.md>), [coding](<https://devfeed.tech/tags/coding.md>), [context-window](<https://devfeed.tech/tags/context-window.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [fp8](<https://devfeed.tech/tags/fp8.md>), [gb300](<https://devfeed.tech/tags/gb300.md>), [mixture-of-experts-moe](<https://devfeed.tech/tags/mixture-of-experts-moe.md>), [nvfp4](<https://devfeed.tech/tags/nvfp4.md>), [open](<https://devfeed.tech/tags/open.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [speculative-decoding](<https://devfeed.tech/tags/speculative-decoding.md>), [throughput](<https://devfeed.tech/tags/throughput.md>)

### AI overview

Beam is a 501.3B-parameter sparse MoE model with 23.4B active parameters, designed for coding, reasoning, and agentic workloads. The abstract reports a 1M-token context window enabled by two-stage midtraining on verifiable, correctness-anchored data, following pretraining on 23.8T tokens. Post-training used asynchronous RL on 10.5K GB300 GPUs, producing over 100M rollouts in four weeks; measured capabilities continued to improve as RL compute increased, with no plateau reported. Hardware-model co-design, FP8/NVFP4 quantization, and a 1.93B-parameter DFlash 2 speculative decoding drafter target serving throughput. The report claims 3-4x less inference compute than many comparable models. On its safety evaluation, Beam refused or pivoted on 88.9% of harmful requests while complying with 81.5% of harmless requests; the text says its harmful-request refusal/pivot rate was highest among open-weight models on the Pareto frontier.

### Source excerpt

arXiv:2611.26358v1 Announce Type: cross Abstract: Beam is a sparse Mixture-of-Experts (MoE) model with 501.3B total parameters and 23.4B active parameters, specifically built and optimized for coding, reasoning, and agentic workloads. Beam delivers frontier level reasoning, while requiring 3-4x less inference compute than many models of its class. Pretrained on 23.8T diverse, curated, high-quality tokens, Beam features a 1M-token context window achieved through a novel two-stage midtraining phase, built on verifiable, correctness-anchored data. Post-training leveraged a large-scale, fully asynchronous reinforcement learning (RL) campaign across 10.5K GB300 GPUs, generating over 100M rollouts over 4 weeks of training. We believe this is one of the largest scale RL runs conducted by any open lab to date. Across our evaluation suite, capabilities continued to improve as we increased RL compute, with no sign of a plateau. Supported by deep hardware-model co-design, FP8/NVFP4 quantization, and a 1.93B-parameter speculative decoding drafter (DFlash 2), Beam maximizes serving throughput with efficient, flexible deployment. Averaged across conversation turns and harm categories, Beam refuses or pivots on 88.9% of harmful requests while complying with 81.5% of harmless requests, achieving the highest refusal/pivot rate on harmful requests among open-weight models on the Pareto frontier.

## MatVerse: A Foundation Model for Materials Science

DevFeed: [MatVerse: A Foundation Model for Materials Science](<https://devfeed.tech/articles/matverse-a-foundation-model-for-materials-science-145560.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25651>)

Author: Xuanyu Zhu, Qian Tan, Bo Chen, Jiaqing Xie, Yang Han, Mei Wu, Wanhao Liu, Xuanze Lin, Yuchen Zhou, Shijie Wang, Biqing Qi, Shixiang Tang, Wenwei Zhang, Tianfan Fu, Kai Yu, Lu Chen, Yuqiang Li, Xin Che

Published: 2026-11-20T05:00:00Z

Content type: news

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Materials science and engineering](<https://devfeed.tech/topics/materials-science-and-engineering.md>), [AI for science](<https://devfeed.tech/topics/ai-for-science.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-assisted](<https://devfeed.tech/tags/ai-assisted.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [capabilities](<https://devfeed.tech/tags/capabilities.md>), [cond-mat-mtrl-sci](<https://devfeed.tech/tags/cond-mat-mtrl-sci.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [foundation-model](<https://devfeed.tech/tags/foundation-model.md>), [inference-efficiency](<https://devfeed.tech/tags/inference-efficiency.md>), [materials](<https://devfeed.tech/tags/materials.md>), [materials-science](<https://devfeed.tech/tags/materials-science.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [post-training](<https://devfeed.tech/tags/post-training.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [vision](<https://devfeed.tech/tags/vision.md>)

### AI overview

MatVerse is a materials-science foundation model built on Intern-S2-Preview and specialized through large-scale domain post-training. Its multimodal corpus covers materials knowledge, scientific reasoning, quantitative problems, structures, and scientific interpretation; data preparation combines teacher-guided reasoning distillation, synthesized problems, consistency filtering, and retrieval-based multimodal construction. The report says MatVerse improves over the base model on OmniMatBench, MatSciBench, and MaCBench, and improves inference efficiency on calculation-intensive problems. The excerpt gives no numerical results or detailed benchmark comparisons.

### Source excerpt

arXiv:2611.25651v1 Announce Type: new Abstract: Materials science involves highly heterogeneous scientific information, including textual knowledge, molecular and crystal structures, experimental characterization data, microscopy images, and structure-property relationships. Developing a general-purpose foundation model capable of understanding and reasoning over these diverse materials representations remains challenging. In this technical report, we introduce MatVerse, a materials foundation model built upon Intern-S2-Preview and further specialized through large-scale materials-domain post-training. We construct a large-scale multimodal training corpus covering materials knowledge, scientific reasoning, quantitative problem solving, structural understanding, and multimodal scientific interpretation. To improve domain adaptation, we develop a unified data construction pipeline combining teacher-guided reasoning distillation, scientific problem synthesis, consistency-based filtering, and retrieval-based multimodal data construction. We evaluate MatVerse on three representative materials-science benchmarks: our human-calibrated OmniMatBench, MatSciBench, and MaCBench. Results demonstrate consistent improvements after materials-specific post-training over the base model on all three benchmarks, alongside improved inference efficiency on calculation-intensive problems. These results show that targeted domain-specific post-training can effectively enhance scientific reasoning, materials knowledge understanding, and multimodal scientific capabilities of foundation models. MatVerse provides a unified language-and-vision interface for materials science and serves as a foundation for future AI-assisted materials research workflows.

## From "Weak" Signals to Strong Models: Preference Delta Aggregation with LoRA Merging

DevFeed: [From "Weak" Signals to Strong Models: Preference Delta Aggregation with LoRA Merging](<https://devfeed.tech/articles/from-weak-signals-to-strong-models-preference-delta-aggregation-with-lora-merging-145341.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2606.00357>)

Author: Qi Sun, Siyue Zhang, Yulin Chen, Yuxiang Xue, Ru Peng, Chen Zhao

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [lora](<https://devfeed.tech/topics/lora.md>), [rlhf](<https://devfeed.tech/topics/rlhf.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [qwen](<https://devfeed.tech/topics/qwen.md>)

Tags: [adapter](<https://devfeed.tech/tags/adapter.md>), [adapters](<https://devfeed.tech/tags/adapters.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [aggregation](<https://devfeed.tech/tags/aggregation.md>), [alignment](<https://devfeed.tech/tags/alignment.md>), [capabilities](<https://devfeed.tech/tags/capabilities.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [data](<https://devfeed.tech/tags/data.md>), [delta](<https://devfeed.tech/tags/delta.md>), [evaluations](<https://devfeed.tech/tags/evaluations.md>), [knowledge](<https://devfeed.tech/tags/knowledge.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [large-language-models-llms](<https://devfeed.tech/tags/large-language-models-llms.md>), [llms](<https://devfeed.tech/tags/llms.md>), [lora](<https://devfeed.tech/tags/lora.md>), [models](<https://devfeed.tech/tags/models.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [pairs](<https://devfeed.tech/tags/pairs.md>), [pda](<https://devfeed.tech/tags/pda.md>), [qwen3](<https://devfeed.tech/tags/qwen3.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>)

### AI overview

Preference Delta Aggregation (PDA) turns preference comparisons between weaker models into separate LoRA adapters trained through preference optimization, then merges them to improve a stronger model. Its Geometric Alignment Merging (GAM) aligns the adapters' left and right singular bases with orthogonal Procrustes rotations before averaging their factors. In evaluations on knowledge reasoning and agentic search, PDA-GAM outperformed the strongest individual delta. Three reasoning deltas and four agentic deltas improved Qwen3-8B's average scores by 6.8 and 7.3 points, respectively--2.1 and 4.3 points above the strongest single-delta baselines. The abstract reports similar gains on a second student model and attributes the results to combining complementary capabilities.

### Source excerpt

arXiv:2606.00357v3 Announce Type: replace Abstract: Training strong large language models (LLMs) requires high-quality supervision, which is often scarce. Recent work shows that paired preference data from weak-weaker model pairs (e.g., Qwen3 4B over 1.7B), despite the limited quality of individual responses, can provide an effective supervision signal through relative quality deltas, which we term a "weak" signal. This motivates a key research question: can multiple "weak" signals be constructively aggregated for improving strong models (e.g., Qwen3 8B)? To this end, we propose Preference Delta Aggregation (PDA), the first framework that derives a preference delta from each weak-weaker model pair, instantiates it as a LoRA adapter learned through preference optimization, and aggregates the resulting deltas via LoRA merging. To improve the composition of independently trained adapters, we introduce Geometric Alignment Merging (GAM), which aligns their left and right singular bases through orthogonal Procrustes rotations and averages the aligned factors. Evaluations on knowledge reasoning and agentic search show that PDA-GAM exceeds the strongest individual delta, with further gains along the evaluated sequences of additional deltas. Using three reasoning deltas and four agentic deltas, PDA-GAM improves Qwen3-8B by 6.8 and 7.3 points on average, respectively, exceeding the strongest single-delta baseline by 2.1 and 4.3 points. These configurations achieve the highest average scores among the evaluated merging methods, with similar gains observed on a second student model. Further analysis suggests that these gains come from composing complementary capabilities encoded across distinct preference deltas. Code is available at https://github.com/AlbertQiSun/Preference-Delta-Aggregation.

## Sage-Evolve-Preview:Adaptive Observation for Post-Training Self-Improvement

DevFeed: [Sage-Evolve-Preview:Adaptive Observation for Post-Training Self-Improvement](<https://devfeed.tech/articles/sage-evolve-preview-adaptive-observation-for-post-training-self-improvement-145635.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26132>)

Author: Yinhao Tang, Youqing Fang, Yanan Sun, Xintian Shen, Hanyu Sun, Wenran Liu, Yue Wu, Weiming Zhang, Bin Liu, Wenwei Zhang

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [post-training](<https://devfeed.tech/topics/post-training.md>), [Advanced AI workflows](<https://devfeed.tech/topics/advanced-ai-workflows.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>)

Tags: [adaptive](<https://devfeed.tech/tags/adaptive.md>), [agent](<https://devfeed.tech/tags/agent.md>), [anchor](<https://devfeed.tech/tags/anchor.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [budget](<https://devfeed.tech/tags/budget.md>), [controller](<https://devfeed.tech/tags/controller.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [diagnostic](<https://devfeed.tech/tags/diagnostic.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [evolution](<https://devfeed.tech/tags/evolution.md>), [harness](<https://devfeed.tech/tags/harness.md>), [independent](<https://devfeed.tech/tags/independent.md>), [long-context](<https://devfeed.tech/tags/long-context.md>), [post-training](<https://devfeed.tech/tags/post-training.md>), [progress](<https://devfeed.tech/tags/progress.md>), [rsi](<https://devfeed.tech/tags/rsi.md>), [score](<https://devfeed.tech/tags/score.md>), [self-improvement](<https://devfeed.tech/tags/self-improvement.md>), [tasks](<https://devfeed.tech/tags/tasks.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

AdaObs addresses plateaus in post-training recursive self-improvement by adapting what the training loop observes while keeping its anchor objective fixed. At a plateau, its controller sharpens diagnostics, raises task difficulty, and adds related tasks; validated gains feed into training, and checkpoints remain scored against the anchor. Across eight anchors in long-context reading comprehension and instruction following, mean scores rose from 66.4 to 73.7 under a matched data budget, outperforming the FixedObs anchor-only baseline. The evolved harness also improved the seven-task average on independent PostTrainBench across two agent backends. The evidence is limited to the reported benchmarks and settings.

### Source excerpt

arXiv:2611.26132v1 Announce Type: new Abstract: Post-training recursive self-improvement (RSI) iteratively improves a target capability, operationalized by a fixed anchor benchmark, through repeated intervention, training, and evaluation. Yet most loops use the anchor's aggregate score as their sole observation, so early gains give way to stalled progress and repetitive exploration. When this anchor-only feedback ceases to reveal actionable gaps, a plateau may reflect an exhausted observation rather than an exhausted capability. We introduce adaptive observation (AdaObs), an observation-side mechanism of harness evolution that keeps the anchor objective fixed while adapting the active observation. At a plateau, a controller refines diagnostic granularity, escalates difficulty, and adds related tasks; validated increments are fused back into training, while every checkpoint remains evaluated against the anchor. Across eight independently optimized anchors in long-context reading comprehension and instruction following, AdaObs raises the mean score from $66.4$ to $73.7$ under a matched data budget, outperforming FixedObs, the anchor-only baseline. The evolved harness further improves the seven-task average on the independent PostTrainBench across two agent backends. These results identify observation adaptation as a missing mechanism of harness evolution for sustaining post-training RSI.

## Protein Functional Semantic Alignment via Representation Rectification

DevFeed: [Protein Functional Semantic Alignment via Representation Rectification](<https://devfeed.tech/articles/protein-functional-semantic-alignment-via-representation-rectification-145050.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25396>)

Author: Chuyang Zhou, Bingxin Zhou, Andi Han, Chang Xu

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [The Gene Ontology Consortium](<https://devfeed.tech/topics/geneontology.md>)

Tags: [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [covariance](<https://devfeed.tech/tags/covariance.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [function](<https://devfeed.tech/tags/function.md>), [functional](<https://devfeed.tech/tags/functional.md>), [go](<https://devfeed.tech/tags/go.md>), [homology](<https://devfeed.tech/tags/homology.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [post-training](<https://devfeed.tech/tags/post-training.md>), [prediction](<https://devfeed.tech/tags/prediction.md>), [protein](<https://devfeed.tech/tags/protein.md>)

### AI overview

SAVR uses Pfam family co-membership as a homology-guided signal for contrastive post-training of protein language models, aiming to separate functional information from sequence variation. The method reduces directional dominance and variance concentration while increasing the effective dimensionality of embedding representations. It uses no task-specific labels during post-training and reports state-of-the-art results on Enzyme Commission benchmarks and competitive results on Gene Ontology benchmarks with frozen backbones. The supplied abstract does not provide benchmark details or quantify improvements.

### Source excerpt

arXiv:2611.25396v1 Announce Type: new Abstract: Protein language models (PLMs) are commonly pretrained with likelihood-based objectives such as masked language modeling or denoising. These objectives capture sequence-level regularities, but can produce representations in which functional signals are entangled with non-functional variation (e.g., sequence identity, compositional bias, and frequent local motifs), creating a readout bottleneck for downstream function prediction. We introduce Functional Semantic Alignment via Representation Rectification (SAVR), a contrastive post-training method using Pfam family co-membership as a homology-guided discrete semantic signal to reshape PLM representations. SAVR reduces directional dominance and redistributes variance across embedding dimensions, decreasing variance concentration, and increasing effective dimensionality of the embedding covariance spectrum. Without observing any task-specific labels during post-training, SAVR achieves state-of-the-art results on Enzyme Commission (EC) benchmarks and competitive results on Gene Ontology (GO) benchmarks under frozen-backbone evaluation. These results suggest that homology-guided contrastive post-training can rectify PLM representation geometry and improve sequence-based protein function prediction.

## Represented but misread: Post-training changes how language models read out what they represent

DevFeed: [Represented but misread: Post-training changes how language models read out what they represent](<https://devfeed.tech/articles/represented-but-misread-post-training-changes-how-language-models-read-out-what-they-represent-145195.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26584>)

Author: Jochen Laubrock

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [post-training](<https://devfeed.tech/topics/post-training.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [model architecture](<https://devfeed.tech/topics/model-architecture.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [authorship](<https://devfeed.tech/tags/authorship.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [detection](<https://devfeed.tech/tags/detection.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [layers](<https://devfeed.tech/tags/layers.md>), [linear](<https://devfeed.tech/tags/linear.md>), [news](<https://devfeed.tech/tags/news.md>), [newspaper](<https://devfeed.tech/tags/newspaper.md>), [post-training](<https://devfeed.tech/tags/post-training.md>), [signal](<https://devfeed.tech/tags/signal.md>), [study](<https://devfeed.tech/tags/study.md>)

### AI overview

Across six open-weight model families, linear probes could distinguish human- and AI-written German texts, while model answers often failed to do so and sometimes ranked them in reverse. Post-training reduced answer sensitivity on average (Δd′ = −0.34) and changed how models used stated base rates. In two models, the shift arose during supervised fine-tuning; stitching experiments implicated early layers, with mid-depth and later post-text states carrying the change. On a separate public-data task, post-training instead repaired several base models' poor use of truth information for negated statements. Adding 2,000 labeled news authorship judgments improved answers on newspaper and textbook texts but reversed them on chats. The results suggest that post-training can alter how represented information is read out, and that a small training signal can influence the direction.

### Source excerpt

arXiv:2611.26584v1 Announce Type: cross Abstract: Does post-training change what a language model represents, or how this representation is read out into an answer? We compared linear probes with the models' own answers, analyzed with signal detection theory, in six open-weight base models and their post-trained versions. For judgments of whether German texts were written by a human or an AI, probes and word n-gram classifiers separated the classes well, but the answers did not and often ordered the texts inversely. Post-training lowered the sensitivity of the answers on average ($\Delta d' = -0.34$ across model families; -0.29 across three prompts). It also changed the use of a stated base rate: Base models shifted their log-odds by less than the stated change in prior odds, most post-trained models by more. In two models with stage-wise checkpoints, the change occurred during supervised fine-tuning, model stitching implicated the early layers, and from mid-depth on internal states at positions after the text, not those of the text, transferred the change between checkpoints. Our corpora cannot be shared, so we add a case reproducible from public data, the truth of negated statements. Here, post-training acted in the opposite direction: Probes decoded truth at or near ceiling in every model, the answers of several base models did not use it, and post-training repaired them; by mid-depth the repair resided at the positions after the statement; below it, the representation of the statement contributed, strongly in one of four models. In a training study, adding 2,000 correctly labeled authorship judgments of news texts to otherwise identical fine-tuning data improved the answers on our newspaper and textbook corpora and inverted them on the chats, where they followed a news-trained probe of the base model. Post-training thus changed how a judgment is read out of what a model represents, and a small training signal can set its direction.

## Relax Within, Balance Across: Load Balancing under Composition Shifts in Vision-Language Mixture-of-Experts

DevFeed: [Relax Within, Balance Across: Load Balancing under Composition Shifts in Vision-Language Mixture-of-Experts](<https://devfeed.tech/articles/relax-within-balance-across-load-balancing-under-composition-shifts-in-vision-language-mixture-of-experts-145359.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2608.00574>)

Author: Ziang Wu, Peng Jin, Qishen Yin, Munan Ning, Hao Li, Peizhen Zhang, Huihui Zhou, Li Yuan

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [vlm](<https://devfeed.tech/topics/vlm.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>)

Tags: [balance](<https://devfeed.tech/tags/balance.md>), [balancing](<https://devfeed.tech/tags/balancing.md>), [cancellation](<https://devfeed.tech/tags/cancellation.md>), [changes](<https://devfeed.tech/tags/changes.md>), [code](<https://devfeed.tech/tags/code.md>), [composition](<https://devfeed.tech/tags/composition.md>), [counts](<https://devfeed.tech/tags/counts.md>), [cross](<https://devfeed.tech/tags/cross.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [errors](<https://devfeed.tech/tags/errors.md>), [experts](<https://devfeed.tech/tags/experts.md>), [fast](<https://devfeed.tech/tags/fast.md>), [fixed](<https://devfeed.tech/tags/fixed.md>), [flat](<https://devfeed.tech/tags/flat.md>), [gap](<https://devfeed.tech/tags/gap.md>), [image](<https://devfeed.tech/tags/image.md>), [language](<https://devfeed.tech/tags/language.md>), [load](<https://devfeed.tech/tags/load.md>), [load-balancing](<https://devfeed.tech/tags/load-balancing.md>), [mixture-of-experts](<https://devfeed.tech/tags/mixture-of-experts.md>), [vision](<https://devfeed.tech/tags/vision.md>)

### AI overview

On a fixed vision-language MoE router, balancing image and text tokens in one pooled auxiliary loss can conceal opposing modality load errors: they cancel near the training image-to-text ratio, but load skews as resolution, tiling, or image count changes that mix. The paper derives a quadratic response law under fixed modality profiles, with the image-text load gap governing sensitivity. It also reports that patches from an image route similarly as patch counts vary, allowing large images to dominate flat pooling. ReBA separates modality losses by token fraction and weights images equally, allowing per-image specialization while balancing visual load overall. Across four split backbones, the authors report stable balance across resolutions, tilings, and image counts without hurting accuracy; large modality gaps in released native MoEs suggest potential for native post-training.

### Source excerpt

arXiv:2608.00574v2 Announce Type: replace-cross Abstract: Vision-language mixture-of-experts (MoE) models such as Qwen3-VL need balanced loads for fast parallel serving. Standard training pools image and text tokens into one auxiliary loss, assuming training balance holds at inference. Yet we find a surprising mismatch: on a fixed router, balance shifts sharply whenever resolution, dynamic tiling, or image count changes the image-to-text token ratio. Splitting load by modality reveals cross-modal load cancellation: opposing image and text errors cancel near the training ratio, hiding strong skew until the mix changes. To quantify this shift, we derive an exact quadratic law: under fixed modality profiles, one reference run predicts the full response curve, and the image-text gap governs sensitivity. Meanwhile, patches within an image route similarly while patch counts vary, letting large images dominate flat pooling. We propose ReBA (Relax Within, Balance Across) to separate modality losses by token fraction and weight images equally, letting each image specialize while keeping overall visual load even. Across four split backbones, ReBA turns fragile single-ratio balance into stable balance across resolutions, tilings, and image counts without hurting accuracy. Large modality gaps in released native MoEs also show its promise for native post-training. Code is available at https://github.com/ZiangWu-77/ReBA.

## Training LLMs to Follow Reasoning Instructions with Minimal Edits

DevFeed: [Training LLMs to Follow Reasoning Instructions with Minimal Edits](<https://devfeed.tech/articles/training-llms-to-follow-reasoning-instructions-with-minimal-edits-145660.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26342>)

Author: Alexandra Yakovleva, Luat Gia Khoi Nguyen, Juho Kannala, Harri Valpola, Alexander Ilin

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>)

Tags: [benchmark](<https://devfeed.tech/tags/benchmark.md>), [compliance](<https://devfeed.tech/tags/compliance.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [edits](<https://devfeed.tech/tags/edits.md>), [failure](<https://devfeed.tech/tags/failure.md>), [follow](<https://devfeed.tech/tags/follow.md>), [framework](<https://devfeed.tech/tags/framework.md>), [full](<https://devfeed.tech/tags/full.md>), [imitation](<https://devfeed.tech/tags/imitation.md>), [instructions](<https://devfeed.tech/tags/instructions.md>), [llms](<https://devfeed.tech/tags/llms.md>), [qwen3](<https://devfeed.tech/tags/qwen3.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [replacement](<https://devfeed.tech/tags/replacement.md>), [results](<https://devfeed.tech/tags/results.md>), [skills](<https://devfeed.tech/tags/skills.md>), [span](<https://devfeed.tech/tags/span.md>), [spans](<https://devfeed.tech/tags/spans.md>), [supervised-fine-tuning](<https://devfeed.tech/tags/supervised-fine-tuning.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

Edit-based training targets the first reasoning failure in a model-generated completion: an editor proposes a minimal replacement span, and the model is supervised only on that correction. This localizes feedback while aiming to retain existing reasoning patterns elsewhere; it can run offline or adapt online to changing failure modes. In experiments with Qwen3-32B on ReasonIF, the authors report substantially better compliance with reasoning instructions than full-trajectory imitation and trajectory-level reinforcement learning, while largely preserving skills on untargeted tasks. These findings are limited to the reported benchmark and comparison.

### Source excerpt

arXiv:2611.26342v1 Announce Type: new Abstract: Large language models increasingly rely on extended reasoning traces, yet the training signals used to shape this behavior remain coarse. Supervised fine-tuning on full trajectories forces the model to imitate an entire demonstration, overconstraining its output distribution, while trajectory-level reinforcement learning provides only weak credit assignment, rewarding or penalizing whole completions without identifying where they succeed or fail. We propose edit-based training, a general framework for learning from a model's own trajectories via localized textual corrections. Given a sampled completion, an editor identifies the first point of failure under a specified criterion and proposes a minimal replacement span. The model is then supervised only on these edited spans. By design, this concentrates supervision where the model's reasoning first breaks down and aims to preserve its existing reasoning patterns elsewhere. The method applies both offline and online, with the latter continually adapting supervision to the model's evolving failure modes. We instantiate this approach on reasoning instruction following with Qwen3-32B on the ReasonIF benchmark. Compared to full-trajectory imitation and trajectory-level reinforcement learning baselines, edit-based training substantially improves compliance with the reasoning instruction while largely preserving the model's skills on untargeted tasks. These results suggest that localized correction is an effective complement to these approaches for shaping how LLMs reason.

## Cautious Context Steering for Language Model Personalization

DevFeed: [Cautious Context Steering for Language Model Personalization](<https://devfeed.tech/articles/cautious-context-steering-for-language-model-personalization-145360.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2608.05813>)

Author: Gihoon Kim, Jeyoung Lee, Suhan Woo, Jangwon Oh, Sekwon Oh, Minsu Jeon, Euntai Kim

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [personalization](<https://devfeed.tech/topics/personalization.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>)

Tags: [adapter](<https://devfeed.tech/tags/adapter.md>), [alignment](<https://devfeed.tech/tags/alignment.md>), [ccs](<https://devfeed.tech/tags/ccs.md>), [context](<https://devfeed.tech/tags/context.md>), [cos](<https://devfeed.tech/tags/cos.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [decoding](<https://devfeed.tech/tags/decoding.md>), [distillation](<https://devfeed.tech/tags/distillation.md>), [distribution](<https://devfeed.tech/tags/distribution.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [format](<https://devfeed.tech/tags/format.md>), [generation](<https://devfeed.tech/tags/generation.md>), [inference](<https://devfeed.tech/tags/inference.md>), [language](<https://devfeed.tech/tags/language.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [lightweight](<https://devfeed.tech/tags/lightweight.md>), [oracle](<https://devfeed.tech/tags/oracle.md>), [pass](<https://devfeed.tech/tags/pass.md>), [personalization](<https://devfeed.tech/tags/personalization.md>), [steering](<https://devfeed.tech/tags/steering.md>)

### AI overview

Cautious Context Steering (CCS) uses a lightweight adapter to decide when and how strongly a frozen language model should use user context. Trained through cautious distillation, it follows context-steered predictions where context helps predict a preferred token and the context-free distribution elsewhere. The abstract reports stronger preference alignment than training-free context methods and trained personalization baselines. On the introduced GSM8K-P benchmark, CCS maintains task accuracy while improving adherence to task-specific formats. Its adapter adds less than 0.4% of the backbone's parameters; at inference, one context-conditioned model pass plus adapter evaluation per decoding step reduces generation time on PRISM by about 35-36% compared with Context Steering, which uses two passes.

### Source excerpt

arXiv:2608.05813v2 Announce Type: replace Abstract: User context can personalize a language model without per-user fine-tuning, but its influence is not equally helpful at every token. Context Steering (CoS) uses a fixed steering strength throughout generation and requires two LM forward passes per step. We propose Cautious Context Steering (CCS), a lightweight adapter that learns when and how strongly to steer a frozen LM using user context. We train the adapter through cautious distillation, using oracle targets built from predictions with and without context. It learns to match the steered distribution where context helps predict the user's preferred token and the context-free base distribution elsewhere. The CCS adapter outperforms both training-free context-based methods and trained personalization baselines in preference alignment across benchmarks. We also introduce GSM8K-P to evaluate task accuracy and task-specific format preferences together. CCS achieves high format adherence while maintaining task accuracy, offering a better balance than the baselines. The CCS adapter adds less than 0.4% of the backbone's parameters. At inference, each decoding step uses one context-conditioned LM forward pass and a lightweight adapter evaluation, reducing generation time on PRISM by approximately 35-36% compared with CoS.

## Complete-SVD: A Functional Norm Approach to Low-Rank LLM Compression

DevFeed: [Complete-SVD: A Functional Norm Approach to Low-Rank LLM Compression](<https://devfeed.tech/articles/complete-svd-a-functional-norm-approach-to-low-rank-llm-compression-145134.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26059>)

Author: Alper Kalle, Theo Rudkiewicz, Mohamed-Oumar Ouerfelli, Mohamed Tamaazousti

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Compression](<https://devfeed.tech/topics/compression.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>)

Tags: [compression](<https://devfeed.tech/tags/compression.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [large-language-models](<https://devfeed.tech/tags/large-language-models.md>), [llm](<https://devfeed.tech/tags/llm.md>), [low-rank](<https://devfeed.tech/tags/low-rank.md>), [post-training](<https://devfeed.tech/tags/post-training.md>), [svd](<https://devfeed.tech/tags/svd.md>)

### AI overview

Complete-SVD compresses LLMs by minimizing the L2 distance between original and compressed model logits, giving equal weight to every logit change. Linearization and a Kronecker-factored approximation turn this objective into a weighted SVD with a closed-form solution. Because its output-side factor is intractable to compute directly, the method estimates it without bias using random probes, requiring K backward passes per sequence regardless of vocabulary size or sequence length. With uniform ranks and WikiText-2 calibration, it reports the lowest C4 and PTB perplexity at every matched setting, and at compression ratio 0.7 with LEMS rank allocation. The related KL-SVD variant lowers calibration perplexity in most settings and raises downstream accuracy, but also raises cross-corpus perplexity. Complete-SVD improves IFEval accuracy over the strongest prior output-aware methods throughout the reported comparisons.

### Source excerpt

arXiv:2611.26059v1 Announce Type: new Abstract: Post-training low-rank compression of neural networks must not only approximate the original weights but also preserve the model's input-output behavior. Recent output-aware methods measure the compression error beyond the compressed layer, at the model output, weighting logit changes by the observed next token or by the model's predicted probabilities. We propose Complete-SVD, a method for large language models that minimizes the $L_2$ distance between the logits of the original and compressed models, weighting all logit changes equally. Linearizing the network around the original weights, together with a Kronecker-factored approximation, reduces this objective to a weighted SVD with a closed-form solution. Its output-side factor is intractable to compute; we estimate it without bias from random probes, at a cost of $K$ backward passes per sequence, independent of vocabulary size and sequence length. At matched settings with uniform ranks and WikiText-2 calibration, Complete-SVD achieves the lowest C4 and PTB perplexity in every setting, and at ratio 0.7 under LEMS rank allocation. Replacing the $L_2$ distance by the Kullback-Leibler divergence (KL-SVD), with the same estimation framework, lowers calibration perplexity in most settings and raises downstream accuracy, but raises cross-corpus perplexity. Complete-SVD also improves IFEval instruction-following accuracy over the strongest prior output-aware methods throughout.

## CoachSI: Self-Improvement of Robot Policies with a Language Coach

DevFeed: [CoachSI: Self-Improvement of Robot Policies with a Language Coach](<https://devfeed.tech/articles/coachsi-self-improvement-of-robot-policies-with-a-language-coach-145529.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25529>)

Author: Jiachen Tao, Gengyu Zhang, Haoxuan Wang, Bing Liu, Yan Yan

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Robot Navigation](<https://devfeed.tech/topics/robot-navigation.md>), [vlm](<https://devfeed.tech/topics/vlm.md>), [post-training](<https://devfeed.tech/topics/post-training.md>)

Tags: [camera](<https://devfeed.tech/tags/camera.md>), [coaching](<https://devfeed.tech/tags/coaching.md>), [completion](<https://devfeed.tech/tags/completion.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [horizon](<https://devfeed.tech/tags/horizon.md>), [improvement](<https://devfeed.tech/tags/improvement.md>), [instructions](<https://devfeed.tech/tags/instructions.md>), [language](<https://devfeed.tech/tags/language.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [model](<https://devfeed.tech/tags/model.md>), [naming](<https://devfeed.tech/tags/naming.md>), [next](<https://devfeed.tech/tags/next.md>), [policies](<https://devfeed.tech/tags/policies.md>), [policy](<https://devfeed.tech/tags/policy.md>), [robot](<https://devfeed.tech/tags/robot.md>), [self-improvement](<https://devfeed.tech/tags/self-improvement.md>), [vision](<https://devfeed.tech/tags/vision.md>), [vision-language-action](<https://devfeed.tech/tags/vision-language-action.md>), [vision-language-model](<https://devfeed.tech/tags/vision-language-model.md>), [vlm](<https://devfeed.tech/tags/vlm.md>)

### AI overview

CoachSI improves long-horizon robot task performance by having a frozen vision-language model watch the robot and name the next step in the policy's atomic-instruction format; it never supplies actions. Successful coached runs are relabeled with the original task instruction and used to fine-tune the policy, which is then deployed without a coach. The action backbone remains frozen. On stalled states, next-step coaching raises subgoal completion 4.6-fold over the unchanged task instruction. Unassisted success in unseen scenes rises from 47.4% to 59.3%, exceeding human-demonstration fine-tuning by 9.8 percentage points and equal-budget self-collected-success fine-tuning by 7.3 points. With a coach limited to camera images, success reaches 56.8%, above both baselines.

### Source excerpt

arXiv:2611.25529v1 Announce Type: cross Abstract: Generalist robot policies execute atomic skills reliably but fail when the same skills are composed into long-horizon tasks. Self-improvement from the policy's own experience does not close this gap: experience collected by the policy alone stays within its own distribution and reproduces what it can already do. We show that what the policy lacks is not a skill but the next step. We present CoachSI, a self-improvement method in which a frozen, off-the-shelf vision-language model (VLM) coaches the policy in language: it watches the robot and names the next step in the format of the policy's atomic instructions, and never outputs an action. Successful coached rollouts are relabeled in hindsight with the original task instruction and used to fine-tune the policy. The fine-tuned policy is then deployed on its own. On states where a World Action Model and a Vision-Language-Action model have stalled, naming the next step raises subgoal completion by 4.6x over the unchanged task instruction, and the frozen VLM supplies it from the same camera images the policy sees. After fine-tuning, unassisted success on long-horizon tasks in unseen scenes rises from 47.4% to 59.3%, 9.8 points above fine-tuning on human demonstrations and 7.3 points above fine-tuning on self-collected successes at an equal data budget. With a coach that sees only camera images, it reaches 56.8%, above both baselines. The gain is obtained with the action backbone frozen: language coaching teaches the policy what should happen next, and the skills it already has do the rest.

## Cross-Question Reliable Reinforcement Learning

DevFeed: [Cross-Question Reliable Reinforcement Learning](<https://devfeed.tech/articles/cross-question-reliable-reinforcement-learning-145036.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25292>)

Author: Hector G. Rodriguez, Marcus Rohrbach

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Uncertainty quantification LLMs](<https://devfeed.tech/topics/uncertainty-quantification-llms.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [capabilities](<https://devfeed.tech/tags/capabilities.md>), [confidence](<https://devfeed.tech/tags/confidence.md>), [coverage](<https://devfeed.tech/tags/coverage.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [error](<https://devfeed.tech/tags/error.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [large-language-models-llms](<https://devfeed.tech/tags/large-language-models-llms.md>), [learning](<https://devfeed.tech/tags/learning.md>), [math](<https://devfeed.tech/tags/math.md>), [math-500](<https://devfeed.tech/tags/math-500.md>), [qa](<https://devfeed.tech/tags/qa.md>), [questions](<https://devfeed.tech/tags/questions.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [reward](<https://devfeed.tech/tags/reward.md>), [risk](<https://devfeed.tech/tags/risk.md>), [solved](<https://devfeed.tech/tags/solved.md>), [utility](<https://devfeed.tech/tags/utility.md>), [wikipedia](<https://devfeed.tech/tags/wikipedia.md>)

### AI overview

CARE (Cross-question Abstention Reward Estimation) trains LLMs to rank confidence across a set of question-answer pairs. For several risk tolerances, the set's confidences determine an abstention threshold, and rewards favor confidence scores on the correct side of that threshold. This contrasts with pointwise rewards that push each answer toward confidence 1 if correct and 0 if incorrect, which the authors say loses useful distinctions for selective answering. In math benchmarks MATH-500, GSM8K, and Big-Math-Digits, CARE raises coverage at 5% risk (C@5) from 0.8% to 42.4% versus existing pointwise rewards. Training on HotpotQA also raises C@5 from 12.3% to 23.6% on out-of-distribution general-knowledge, reasoning, and math benchmarks, including CommonsenseQA, GPQA, SimpleQA, and TriviaQA. These findings are reported in the supplied abstract.

### Source excerpt

arXiv:2611.25292v1 Announce Type: new Abstract: AI systems operating near the limits of their capabilities can maximize utility while minimizing the risk of error by abstaining or deferring questions they are not confident about to a human or a stronger model. Recently, large language models (LLMs) have been post-trained to estimate confidence in their own answers. To achieve this, methods reinforce confidence scores based on the correctness signal to each question-answer pair individually, with rewards that encourage assigning confidence 1 to correct answers and 0 to incorrect answers. However, estimating confidence scores is useful in that it enables selecting only the most confident answers, to maximize the share of solved questions while accommodating the user's risk tolerance. Assigning confidence 1 to correct answers and 0 to incorrect ones is theoretically optimal, but unattainable in practice and detrimental for selective prediction with a non-zero risk tolerance: confidences pushed to 0 and 1 carry no information about how answers compare to one another. We therefore introduce CARE (Cross-question Abstention Reward Estimation), which computes rewards over a set of question-answer pairs: At each of several risk tolerances, the confidences across the set induce an abstention threshold, and each confidence is rewarded for lying on the correct side of it. Since the threshold itself comes from the set, every verbalized confidence is reinforced relative to other questions' confidence, not against its own binary correctness label. On math, CARE increases the coverage at 5% risk (C@5) across MATH-500, GSM8K, and Big-Math-Digits from 0.8% to 42.4%, compared to existing pointwise correctness-matching rewards. When trained on multi-hop Wikipedia QA from HotpotQA, CARE improves C@5 in out-of-distribution general-knowledge, reasoning, and math benchmarks (including CommonsenseQA, GPQA, SimpleQA, and TriviaQA) from 12.3% to 23.6%.

## Decision-Focused Reinforcement Learning for LLM Post-Training

DevFeed: [Decision-Focused Reinforcement Learning for LLM Post-Training](<https://devfeed.tech/articles/decision-focused-reinforcement-learning-for-llm-post-training-145075.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25597>)

Author: Haotian Zhai, Leqi Liu

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [decision](<https://devfeed.tech/tags/decision.md>), [learning](<https://devfeed.tech/tags/learning.md>), [llm](<https://devfeed.tech/tags/llm.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [planning](<https://devfeed.tech/tags/planning.md>), [post-training](<https://devfeed.tech/tags/post-training.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>)

### AI overview

Decision-focused reinforcement learning (DFRL) post-trains LLMs to predict optimization parameters using negative downstream decision regret as the reward, targeting decision costs rather than prediction accuracy alone. It does not require downstream problems to be convex or differentiable. To reduce costly reward evaluation, heuristic warm-starting begins with approximate rewards and later switches to exact rewards; reward correction mixes inexpensive estimates with randomized exact evaluations, preserving the expected policy gradient while reducing exact solves per update. The abstract reports gains over prediction-loss baselines on three synthetic problems and faster early improvement on a larger knapsack task under short training budgets. In H&M inventory planning, the best tested setup, where the LLM reasons before predicting, reduced regret by 31.4% versus SFT and 12.4% versus the strongest evaluated conventional forecasting baseline.

### Source excerpt

arXiv:2611.25597v1 Announce Type: new Abstract: Large language models (LLMs) have shown promise as predictors across various tasks, benefiting from their ability to use information from unstructured context. However, when predictions serve as parameters of downstream optimization problems, higher prediction accuracy does not necessarily translate into lower decision costs. We propose decision-focused reinforcement learning (DFRL), a framework for post-training LLMs to predict optimization parameters using negative downstream decision regret as reward. DFRL trains LLM predictors to minimize the costs of the decisions induced by their predictions, without requiring downstream convexity or differentiability, and integrates readily into existing RL post-training frameworks. To address expensive reward evaluation, we develop two approaches. Heuristic warm-starting uses inexpensive approximate rewards during an initial phase before switching to refinement with exact rewards. Reward correction combines inexpensive estimates with randomized exact evaluations, preserving the expected policy gradient while reducing the expected number of exact solves per update. Experiments on three synthetic optimization problems demonstrate improvements over prediction-loss baselines, while a larger knapsack experiment shows faster early improvement under short training budgets. On inventory planning with H&M transactions and product descriptions, our best configuration, in which the LLM reasons before predicting, reduces regret by 31.4% relative to the SFT baseline and 12.4% relative to the strongest evaluated conventional forecasting baseline.

## Cross-Behavior Policy Optimization: Unified Learning from Offline Demonstrations and On- and Off-Policy Rollouts via a Shared Regression Target

DevFeed: [Cross-Behavior Policy Optimization: Unified Learning from Offline Demonstrations and On- and Off-Policy Rollouts via a Shared Regression Target](<https://devfeed.tech/articles/cross-behavior-policy-optimization-unified-learning-from-offline-demonstrations-and-on-and-off-policy-rollouts-via-a-shared-regression-target-144960.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.12830>)

Author: Du Cheng, Junyu Lu, Jingfan Zhang, Haoxuan Jia, Shichao Weng

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [buffer](<https://devfeed.tech/tags/buffer.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [history](<https://devfeed.tech/tags/history.md>), [leading](<https://devfeed.tech/tags/leading.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [offline](<https://devfeed.tech/tags/offline.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [policy](<https://devfeed.tech/tags/policy.md>), [regression](<https://devfeed.tech/tags/regression.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [target](<https://devfeed.tech/tags/target.md>)

### AI overview

CBPO trains on offline demonstrations and current or historical policy rollouts with one regression loss using a fixed-reference target. Its reward-labeled replay buffer can include positive and negative offline completions and reference responses; the target does not depend on the generating policy, so the method does not require stored behavior-policy probabilities, importance weighting, or a learned critic. Source restrictions yield offline, on-policy, and off-policy variants, while CBPO-Unified combines them. Across two model scales and four mathematical reasoning benchmarks, the paper reports leading avg@32 and pass@32 results. On Qwen3-1.7B, CBPO-Unified exceeds SFT DAPO by 2.33 and 4.44 percentage points on benchmark-averaged avg@32 and pass@32; retaining historical rollouts adds 1.87 and 3.72 points over the no-history replay configuration.

### Source excerpt

arXiv:2610.12830v3 Announce Type: replace Abstract: Reasoning post-training draws on offline demonstrations and policy-generated rollouts, yet conventional supervised fine-tuning (SFT) followed by reinforcement learning (RL) separates their objectives and limits continued reuse of collected responses. We propose \textbf{Cross-Behavior Policy Optimization} (CBPO), which makes these sources compatible with one regression loss through a fixed-reference target. Because the generating policy does not enter this target, positive and negative offline completions, reference responses, and current and historical rollouts can share a common reward-labeled record format and replay buffer. As new responses enter the buffer, optimization retains the same regression objective without coordinating separate SFT and RL objectives or requiring stored behavior-policy probabilities, importance weighting, or a learned critic. Restricting the available sources gives offline, on-policy, and off-policy variants, while CBPO-Unified combines them. Across two model scales and four mathematical reasoning benchmarks, CBPO delivers leading \texttt{avg@32} and \texttt{pass@32} performance across benchmarks and data regimes. On Qwen3-1.7B, CBPO-Unified improves benchmark-averaged \texttt{avg@32} and \texttt{pass@32} over SFT DAPO by 2.33 and 4.44 percentage points, respectively. In the replay ablation, retaining historical rollouts improves these metrics by 1.87 and 3.72 percentage points over the no-history configuration.

## Duopra: From Full-Duplex Conversation to Full-Duplex Agency

DevFeed: [Duopra: From Full-Duplex Conversation to Full-Duplex Agency](<https://devfeed.tech/articles/duopra-from-full-duplex-conversation-to-full-duplex-agency-145586.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25773>)

Author: Chiyu Chen, Pengzhou Cheng, Yuezhao Ma, Xiaocheng Li, Can Liu, Haonan Hu, Yao Yao, Yifei Yang, Yijie Lu, Taolei Wang, Yansi Li, Muhuan Wu, Chenglu Sun, Gongshen Liu, Zhuosheng Zhang

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [AI Agent](<https://devfeed.tech/topics/ai-agent.md>), [function calling](<https://devfeed.tech/topics/function-calling.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [Multi-step AI processes](<https://devfeed.tech/topics/multi-step-ai-processes.md>)

Tags: [audio](<https://devfeed.tech/tags/audio.md>), [backend](<https://devfeed.tech/tags/backend.md>), [bench](<https://devfeed.tech/tags/bench.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [execution](<https://devfeed.tech/tags/execution.md>), [frontend](<https://devfeed.tech/tags/frontend.md>), [full-duplex-bench](<https://devfeed.tech/tags/full-duplex-bench.md>), [interaction](<https://devfeed.tech/tags/interaction.md>), [interface](<https://devfeed.tech/tags/interface.md>), [latency](<https://devfeed.tech/tags/latency.md>), [low-latency](<https://devfeed.tech/tags/low-latency.md>), [model](<https://devfeed.tech/tags/model.md>), [natural-language](<https://devfeed.tech/tags/natural-language.md>), [pass](<https://devfeed.tech/tags/pass.md>), [pipeline](<https://devfeed.tech/tags/pipeline.md>), [protocol](<https://devfeed.tech/tags/protocol.md>), [realtime](<https://devfeed.tech/tags/realtime.md>), [repair](<https://devfeed.tech/tags/repair.md>), [request](<https://devfeed.tech/tags/request.md>), [requires](<https://devfeed.tech/tags/requires.md>), [v3](<https://devfeed.tech/tags/v3.md>)

### AI overview

Duopra addresses a gap in full-duplex agents: fluent turn-taking does not ensure timely, complete tool requests or corrections. It treats the backend as a tool that receives a self-contained natural-language request, using one action interface for direct calls and backend delegation. Training combines protocol SFT, full-duplex reinforcement learning for timely and complete execution, and projector-only repair of talker conditioning after thinker updates. On FDB v3, the authors report Pass@1 rising from 4.7% to 44.7% with the same backend; Duopra reaches 47.7% with direct tool calls (26.0% after SFT) and 49.3% with a DeepSeek backend. It also improves three FDB v1.0 takeover rates over the base model. The excerpt supports the authors' claim that task agency can coexist with key conversational agency.

### Source excerpt

arXiv:2611.25773v1 Announce Type: new Abstract: Full-duplex interaction models exhibit strong conversational agency, enabling natural, low-latency turn-taking, backchannels, and interruptions. However, completing tasks further requires task agency: timely tool invocation and argument revision following user corrections. Frontend-backend agents separate initiation from execution: the frontend decides whether and what to request, while the backend determines how. Execution can be offloaded, but unsent requests cannot be executed, and incomplete ones can misdirect the backend. On Full-Duplex-Bench v3 (FDB v3), most failures of Realtime-Venus-Audio + Harness in our streaming pipeline occur before a domain tool executes. We introduce Duopra, a full-duplex interaction model that learns task execution alongside conversational control. Duopra treats the backend as a tool, accepting a self-contained natural-language request. Direct calls and backend delegation share an action interface and requirements for faithful initiation and correct execution. Its three-stage training combines protocol SFT, full-duplex RL rewarding timely and complete execution, and projector-only repair of talker conditioning after thinker updates. With the same backend, Pass@1 rises from 4.7% to 44.7%; Duopra achieves 47.7% calling tools directly (26.0% after SFT) and 49.3% with a DeepSeek backend. It improves three FDB v1.0 takeover rates over the base model. These results show that task agency can coexist with key conversational agency.

## Introducing harness adapters: adapt your favorite model to your harness to improve performance and cost

DevFeed: [Introducing harness adapters: adapt your favorite model to your harness to improve performance and cost](<https://devfeed.tech/articles/introducing-harness-adapters-adapt-your-favorite-model-to-your-harness-to-improve-performance-and-cost-146558.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/harness-adapters-at-scale>)

Author: Younes B; ben burtenshaw; Nathan Habib; Lysandre; Amine Dirhoussi; Adithya S K; shaun smith; Sergio Paniego; Joel Niklaus; Lewis Tunstall

Published: 2026-11-20T00:00:00Z

Content type: tutorial

Language: en

Sources: [Hugging Face](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [ai evaluation](<https://devfeed.tech/topics/ai-evaluation.md>), [Post-training optimization](<https://devfeed.tech/topics/post-training-optimization.md>)

Tags: [adapters](<https://devfeed.tech/tags/adapters.md>), [agent-harness](<https://devfeed.tech/tags/agent-harness.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-coding](<https://devfeed.tech/tags/agentic-coding.md>), [agents](<https://devfeed.tech/tags/agents.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [cost](<https://devfeed.tech/tags/cost.md>), [harness](<https://devfeed.tech/tags/harness.md>), [lora](<https://devfeed.tech/tags/lora.md>), [model](<https://devfeed.tech/tags/model.md>), [open](<https://devfeed.tech/tags/open.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [openenv](<https://devfeed.tech/tags/openenv.md>), [performance](<https://devfeed.tech/tags/performance.md>), [rl](<https://devfeed.tech/tags/rl.md>), [source](<https://devfeed.tech/tags/source.md>), [trl](<https://devfeed.tech/tags/trl.md>)

### AI overview

The article reports that reinforcement learning inside each target agent harness improved Qwen3.8-27B's coding benchmark results and reduced tool use. Separate LoRA adapters trained for Pi, Hermes Agent, and OpenCode each beat the base model under their own harness on Terminal-Bench 2.1 and hard SWE-bench Pro tasks. On tasks both models solved, specialists used 53-72% fewer tool calls. A merged adapter also beat the base under all three harnesses; with thinking off it averaged 39.5% to 49.0% pass@1, while cutting tool calls by 53-73%. The recipe uses validated Harbor tasks, difficulty filtering so GRPO receives tasks the base model solves inconsistently, and TRL's AsyncGRPOTrainer with the harness running as shipped. Results are not directly comparable across harnesses because their defaults differ. On 300 AutomationBench tasks outside the training domain and harnesses, adapters stayed near the base model, so transfer to that setting was not clear. The authors also note possible reasoning leakage in non-thinking outputs and call for further study.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Resolve AI's SRE-1 model finds root causes of production alerts as AI agents go on call

DevFeed: [Resolve AI's SRE-1 model finds root causes of production alerts as AI agents go on call](<https://devfeed.tech/articles/resolve-ai-s-sre-1-model-finds-root-causes-of-production-alerts-as-ai-agents-go-on-call-144569.md>)

Original publisher: [Read original article](<https://siliconangle.com/2026/11/19/resolve-ais-sre-1-model-finds-root-causes-of-production-alerts-as-ai-agents-go-on-call/>)

Author: Duncan Riley

Published: 2026-11-19T23:11:44Z

Content type: news

Language: en

Sources: [SiliconANGLE](<https://devfeed.tech/sources/siliconangle.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [site-reliability-engineering](<https://devfeed.tech/topics/site-reliability-engineering.md>), [agentic workflows](<https://devfeed.tech/topics/agentic-workflows.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [Monitoring](<https://devfeed.tech/topics/monitoring.md>), [incident](<https://devfeed.tech/topics/incident.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [bhupesh-patel](<https://devfeed.tech/tags/bhupesh-patel.md>), [coding-agents](<https://devfeed.tech/tags/coding-agents.md>), [coinbase](<https://devfeed.tech/tags/coinbase.md>), [cortex-training](<https://devfeed.tech/tags/cortex-training.md>), [crm-analytics](<https://devfeed.tech/tags/crm-analytics.md>), [doordash](<https://devfeed.tech/tags/doordash.md>), [dst-global](<https://devfeed.tech/tags/dst-global.md>), [figma](<https://devfeed.tech/tags/figma.md>), [frontier-model](<https://devfeed.tech/tags/frontier-model.md>), [graphics-processing-units](<https://devfeed.tech/tags/graphics-processing-units.md>), [incident-remediation](<https://devfeed.tech/tags/incident-remediation.md>), [mean-time-to-resolution](<https://devfeed.tech/tags/mean-time-to-resolution.md>), [msci](<https://devfeed.tech/tags/msci.md>), [nasdaq](<https://devfeed.tech/tags/nasdaq.md>), [news](<https://devfeed.tech/tags/news.md>), [on-call](<https://devfeed.tech/tags/on-call.md>), [open-weights](<https://devfeed.tech/tags/open-weights.md>), [primary-on-call](<https://devfeed.tech/tags/primary-on-call.md>), [production](<https://devfeed.tech/tags/production.md>), [production-alerts](<https://devfeed.tech/tags/production-alerts.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [resolve-ai](<https://devfeed.tech/tags/resolve-ai.md>), [root-cause](<https://devfeed.tech/tags/root-cause.md>), [salesforce](<https://devfeed.tech/tags/salesforce.md>), [salesforce-ventures](<https://devfeed.tech/tags/salesforce-ventures.md>), [sap](<https://devfeed.tech/tags/sap.md>), [series-a](<https://devfeed.tech/tags/series-a.md>), [shadow-mode](<https://devfeed.tech/tags/shadow-mode.md>), [snowflake](<https://devfeed.tech/tags/snowflake.md>), [spiros-xanthos](<https://devfeed.tech/tags/spiros-xanthos.md>), [sre-1](<https://devfeed.tech/tags/sre-1.md>), [the-latest](<https://devfeed.tech/tags/the-latest.md>)

### AI overview

Resolve AI has launched Primary On-call, which puts AI agents ahead of human engineers to investigate production alerts and, where team policies allow, close or mitigate them; otherwise it escalates to a person. The agent checks fixes against production baselines until recovery holds, links related investigations, and offers a shadow mode that records the call it would make. Resolve AI says up to 80% of paged alerts are handled without escalation. Its SRE-1 model is a 27-billion-parameter open-weight model, trained with reinforcement learning on private incident data. In the company's tests, a single-investigation pass rate rose from 6.9% to 38.8%; an ensemble matched the strongest frontier model tested on 49 difficult cases at lower cost, and the full system solved about twice as many incidents as general-purpose coding agents across 178 investigations. Primary On-call is generally available; SRE-1 access is limited to Resolve AI's platform. Reported performance figures are company claims.

### Source excerpt

Incident remediation company Resolve AI Inc. has announced the launch of SRE-1, a new model that has been trained to find the root cause of production alerts, and Primary On-call, a new service that puts its artificial intelligence agents on call ahead of human engineers. The new model reflects the company's argument that while AI [...] The post Resolve AI's SRE-1 model finds root causes of production alerts as AI agents go on call appeared first on SiliconANGLE.

## Palo Alto Networks bets $4.5B on enterprise AI with Applied Compute acquisition

DevFeed: [Palo Alto Networks bets $4.5B on enterprise AI with Applied Compute acquisition](<https://devfeed.tech/articles/palo-alto-networks-bets-4-5b-on-enterprise-ai-with-applied-compute-acquisition-143169.md>)

Original publisher: [Read original article](<https://www.networkworld.com/article/4250684/palo-alto-networks-bets-4-5b-on-enterprise-ai-with-applied-compute-acquisition-2.html>)

Author: Gyana Swain

Published: 2026-11-19T11:40:51Z

Content type: news

Language: en

Sources: [Network World](<https://devfeed.tech/sources/network-world.md>)

Topics: [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [LLM security](<https://devfeed.tech/topics/llm-security.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [AI Research](<https://devfeed.tech/topics/ai-research.md>)

Tags: [2027](<https://devfeed.tech/tags/2027.md>), [acquisition](<https://devfeed.tech/tags/acquisition.md>), [adoption](<https://devfeed.tech/tags/adoption.md>), [agent](<https://devfeed.tech/tags/agent.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-applications](<https://devfeed.tech/tags/ai-applications.md>), [ai-model](<https://devfeed.tech/tags/ai-model.md>), [ai-models](<https://devfeed.tech/tags/ai-models.md>), [ai-native](<https://devfeed.tech/tags/ai-native.md>), [ai-research](<https://devfeed.tech/tags/ai-research.md>), [analyst](<https://devfeed.tech/tags/analyst.md>), [applications](<https://devfeed.tech/tags/applications.md>), [applied-compute](<https://devfeed.tech/tags/applied-compute.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [artificial-intelligence-network-security-security](<https://devfeed.tech/tags/artificial-intelligence-network-security-security.md>), [building-ai-applications](<https://devfeed.tech/tags/building-ai-applications.md>), [call](<https://devfeed.tech/tags/call.md>), [ceo](<https://devfeed.tech/tags/ceo.md>), [compute](<https://devfeed.tech/tags/compute.md>), [edition-us](<https://devfeed.tech/tags/edition-us.md>), [enterprise](<https://devfeed.tech/tags/enterprise.md>), [enterprise-ai](<https://devfeed.tech/tags/enterprise-ai.md>), [language-en](<https://devfeed.tech/tags/language-en.md>), [network-security](<https://devfeed.tech/tags/network-security.md>), [palo-alto-networks](<https://devfeed.tech/tags/palo-alto-networks.md>), [security](<https://devfeed.tech/tags/security.md>), [storytype-news](<https://devfeed.tech/tags/storytype-news.md>)

### AI overview

Palo Alto Networks agreed to acquire Applied Compute for $4.5 billion in stock to add open-weight model post-training and inference capabilities, with the stated aim of training, deploying, and securing models in public clouds or customers' sovereign environments. The deal is expected to close in fiscal Q3 2027; Applied Compute CEO Yash Patil will become Palo Alto's co-chief AI officer. Palo Alto also plans to build dedicated GPU and server capacity while continuing to use cloud partners, targeting more than 100 megawatts within three to five years. Analyst Sanchit Vir Gogia cautions that a model supervisor built by the same team is not an independent check, and runtime risks can arise through prompts, tools, and permissions. He also says GPU sovereignty details remain unspecified and flags concentration risk from one vendor providing both models and security. His suggested safeguards include independent testing, documented compute regions and certification plans, written exit terms, and protections for customers' trained weights and data.

### Source excerpt

Palo Alto Networks has agreed to buy Applied Compute in an all-stock deal valued at $4.5 billion, a move the company said will let it train, deploy and secure open-weight AI models for enterprises in the public cloud or in their own sovereign environments. "The acquisition will bring Applied Compute's platform and world-class AI research team to Palo Alto Networks to help enterprises adopt AI securely, whether they choose open-weight models, closed frontier models, or both," the company said in a statement released alongside its fiscal first quarter 2027 results. The statement described Applied Compute as "a leading platform for post-training and inference of open-weight AI models" and said the startup will continue to serve AI-native companies, digital natives and large enterprises. The transaction is expected to close in Palo Alto's fiscal third quarter of 2027, which runs from February to April, the statement said. Applied Compute co-founder and CEO Yash Patil will become Palo Alto's co-chief AI officer, and the startup's research team will help build AI models across the company's offerings. A security company that wants to run models Nikesh Arora, Palo Alto's chairman and CEO, said on the company's earnings call that beyond building AI into its own security products, "we will help train and serve secure sovereign implementations of open-weight models to the enterprise." "Open-weight model implementations will allow us to deliver secure, open-weight models to our customers to enable adoption of secure AI in the enterprise," Arora said. He said open-weight models had grown from under 5% of token traffic to more than 60% over the past year, without saying where the figure came from. Asked by JP Morgan analyst Brian Essex about customer demand, Arora said deterministic security controls "cannot anticipate every scenario that an agent is going to conduct." Enterprises building AI applications "will have to train a supervisory AI model at the same time," he said, bec

[Next page](<https://devfeed.tech/topics/post-training.md?cursor=WyIyMDI2LTExLTE5VDExOjQwOjUxKzAwOjAwIiwgImY3NjY5ZWU3LTFkN2MtNDIyZC05NzhjLWRiNTMyY2ZhNmI5ZiJd>)