# experts

Published articles for experts.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## How Feedback-Free Self-Evolution Reshapes Agent Skills: A Systematic Study

DevFeed: [How Feedback-Free Self-Evolution Reshapes Agent Skills: A Systematic Study](<https://devfeed.tech/articles/how-feedback-free-self-evolution-reshapes-agent-skills-a-systematic-study-145532.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25534>)

Author: Litao Guo, Man Chen, Xinli Xu, Jinsong Zhou, Zixin Zhang, Ying-Cong Chen

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Agent Skills](<https://devfeed.tech/topics/agent-skills.md>), [AI Chat](<https://devfeed.tech/topics/ai-chat.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agent-skills](<https://devfeed.tech/tags/agent-skills.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [evolution](<https://devfeed.tech/tags/evolution.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [experts](<https://devfeed.tech/tags/experts.md>), [feedback](<https://devfeed.tech/tags/feedback.md>), [knowledge](<https://devfeed.tech/tags/knowledge.md>), [model](<https://devfeed.tech/tags/model.md>), [pass](<https://devfeed.tech/tags/pass.md>), [skills](<https://devfeed.tech/tags/skills.md>), [study](<https://devfeed.tech/tags/study.md>)

### AI overview

In experiments where models repeatedly rewrote expert skills without tasks, runs, or judgments, the skills became less useful: every tested model performed worse on every benchmark, with an average 15-point drop on SkillsBench after 30 rounds. The authors find that rewriting pulls skills toward what each model already tends to produce, discarding knowledge outside that prior first. Restoring this knowledge recovers much of the lost pass rate. Their SkillDiff method marks that knowledge and improves SkillsBench pass rate by an average 4.8 points, without feedback or changing the skill text. These findings are reported in the supplied abstract.

### Source excerpt

arXiv:2611.25534v1 Announce Type: new Abstract: Agent skills are documents in which experts write down procedural knowledge, and models increasingly maintain them by rewriting them without feedback. Such rewriting could adapt a skill to the model that uses it or erode the expert knowledge that makes it useful, yet how skills evolve under this feedback-free self-evolution has not been studied. We let models rewrite expert skills round after round without any task, run or judgment and run agents built on the same models with the rewritten skills. Our experiments show that the skills lose value rather than improve, as every model we test performs worse with them on every benchmark, by 15 points on average on SkillsBench after thirty rounds. Sampling each model's prior with skills it writes from the topic alone, we find that rewriting pulls skills toward this prior and gives up first the knowledge outside it. Putting this knowledge back recovers much of the lost pass rate, so an expert's skill is valuable largely for what the model does not write on its own. Building on this insight, SkillDiff marks this knowledge in place and raises the pass rate on SkillsBench by 4.8 points on average without feedback or any change to the skill's text. Project repository: https://github.com/LitaoGuo/How-Self-Evolution-Reshapes-Skills.

## SteerTrace: Tracing How Target Attacks Steer World Action Models

DevFeed: [SteerTrace: Tracing How Target Attacks Steer World Action Models](<https://devfeed.tech/articles/steertrace-tracing-how-target-attacks-steer-world-action-models-145575.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25719>)

Author: Yukun Dai, Tianshi Wang, Fengling Li, Jingjing Li, Xin Li, Lei Zhu

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [World models](<https://devfeed.tech/topics/world-models.md>), [Computer vision](<https://devfeed.tech/topics/computer-vision.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Physical AI](<https://devfeed.tech/topics/physical-ai.md>)

Tags: [a-safety](<https://devfeed.tech/tags/a-safety.md>), [actions](<https://devfeed.tech/tags/actions.md>), [attack](<https://devfeed.tech/tags/attack.md>), [attacks](<https://devfeed.tech/tags/attacks.md>), [bounded](<https://devfeed.tech/tags/bounded.md>), [budget](<https://devfeed.tech/tags/budget.md>), [camera](<https://devfeed.tech/tags/camera.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [effect](<https://devfeed.tech/tags/effect.md>), [experts](<https://devfeed.tech/tags/experts.md>), [ffn](<https://devfeed.tech/tags/ffn.md>), [frame](<https://devfeed.tech/tags/frame.md>), [future](<https://devfeed.tech/tags/future.md>), [inside](<https://devfeed.tech/tags/inside.md>), [layers](<https://devfeed.tech/tags/layers.md>), [model](<https://devfeed.tech/tags/model.md>), [models](<https://devfeed.tech/tags/models.md>), [noise](<https://devfeed.tech/tags/noise.md>), [target](<https://devfeed.tech/tags/target.md>), [tracing](<https://devfeed.tech/tags/tracing.md>), [world](<https://devfeed.tech/tags/world.md>)

### AI overview

SteerTrace reports that a single per-model perturbation bounded at 4/255, shared across frames, tasks, and camera views, steers actions toward an attacker-chosen target across eight world action models on LIBERO and RoboTwin, bringing task success close to zero. Same-budget noise and BadWAM cause little shift toward the target. Tracing the effect shows different dependencies: under the standard attack, restoring future tokens removes 61-94% of the targeted effect in five models whose action experts attend to them. When predicted futures are kept close to clean predictions, current-frame tokens become more important in three models: restoring them removes 87-99% of the effect, versus 9-15% for future tokens. In six models, restoring 0.36% of video-expert FFN neurons in layers 10-29 removes 72-96% of the effect. The abstract also reports that residual directions 2-16 contain only 11-18% of shift energy, yet removing them cancels most of the effect.

### Source excerpt

arXiv:2611.25719v1 Announce Type: cross Abstract: World action models (WAMs) predict future videos and actions with one backbone, and their imagined future is often seen as a safety signal. We introduce SteerTrace, a targeted universal attack on WAMs that also traces its effect inside the model. One $\ell_\infty$-bounded (4/255) perturbation per model, shared across frames, tasks, and camera views, pushes the executed action toward an attacker-chosen target. On eight WAMs from four families (FastWAM, Faster-WAM, OpenWAM and LingBot-VA) on LIBERO and RoboTwin, it drives task success close to zero and moves the attacked action toward the target. Same-budget noise and BadWAM produce little action shift in the target direction. We use the signed targeted effect to measure how much of the effect each intervention removes or reproduces. Under the standard attack, restoring future tokens removes 61-94% of the targeted effect in the five models whose action experts attend to them. When the attack is constrained to keep the predicted future close to the clean prediction, the targeted effect can become more dependent on current-frame tokens. In three of these five models, restoring current-frame tokens removes 87-99% of the effect, compared with 9-15% for future tokens. In six models, restoring just 0.36% of the FFN neurons in layers 10-29 of the video expert removes 72-96% of the targeted effect. Residual directions 2-16 hold only 11-18% of the shift energy, yet removing them cancels most of the targeted effect.

## Towards Scale-Up Networking: Challenges and Open Questions

DevFeed: [Towards Scale-Up Networking: Challenges and Open Questions](<https://devfeed.tech/articles/towards-scale-up-networking-challenges-and-open-questions-146131.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26156>)

Author: Tingting Xu, Zixun Su, Xinyang Huang, Bowen Liu, Kun Tan, Dong Lin, Hao Wang, Yinben Xia, Weifeng Zhang, Binzhang Fu, Shuihai Hu, Kai Chen

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>), [arXiv: Networking](<https://devfeed.tech/sources/arxiv-networking.md>)

Topics: [NVLink](<https://devfeed.tech/topics/nvlink.md>), [Distributed Systems & Parallel Computing](<https://devfeed.tech/topics/distributed-systems-parallel-computing.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [bandwidth](<https://devfeed.tech/tags/bandwidth.md>), [bottleneck](<https://devfeed.tech/tags/bottleneck.md>), [challenges](<https://devfeed.tech/tags/challenges.md>), [collective](<https://devfeed.tech/tags/collective.md>), [communication](<https://devfeed.tech/tags/communication.md>), [compatibility](<https://devfeed.tech/tags/compatibility.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cs-ni](<https://devfeed.tech/tags/cs-ni.md>), [design](<https://devfeed.tech/tags/design.md>), [device](<https://devfeed.tech/tags/device.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [distributed-training](<https://devfeed.tech/tags/distributed-training.md>), [domain](<https://devfeed.tech/tags/domain.md>), [experts](<https://devfeed.tech/tags/experts.md>), [foundation](<https://devfeed.tech/tags/foundation.md>), [framework](<https://devfeed.tech/tags/framework.md>), [networking](<https://devfeed.tech/tags/networking.md>), [open](<https://devfeed.tech/tags/open.md>), [open-questions](<https://devfeed.tech/tags/open-questions.md>), [scale-up](<https://devfeed.tech/tags/scale-up.md>)

### AI overview

The paper derives scale-up network requirements from critical-path communication in distributed training and inference, focusing on domain size, per-accelerator bandwidth, and collective completion latency. It uses those requirements to compare representative proprietary and open fabrics, describing trade-offs in performance, scalability, compatibility, and implementation cost across their interconnects, topologies, protocols, and communication semantics. It also identifies open challenges in switching and physical interconnects, efficient communication, semantics, resource sharing, and connecting heterogeneous devices. This is a requirement-driven analysis; the supplied excerpt does not provide specific fabric-by-fabric results.

### Source excerpt

arXiv:2611.26156v1 Announce Type: cross Abstract: The rapid growth of AI models and the widespread use of distributed training and inference have made inter-accelerator communication a critical system bottleneck. Scale-up fabrics address this bottleneck by combining high per-accelerator bandwidth with low and predictable latency over tightly coupled accelerator domains. However, the appropriate topology, protocol mechanisms, communication semantics, and system integration remain actively debated. We analyze the critical-path communication generated by parallelism strategies and derive the resulting requirements for domain scale, per-accelerator bandwidth, and collective completion latency. We then use these requirements as a framework to examine representative proprietary and open scale-up fabrics, revealing how their physical interconnects, topologies, protocols, and semantics make different trade-offs among performance, scalability, compatibility, and implementation cost. Moreover, based on this analysis and discussions with industry experts, we identify challenges and open problems in physical interconnects and switching, efficient communication, semantics, resource sharing and heterogeneous device interconnection. This paper provides a requirement-driven foundation for analyzing current scale-up fabrics and guiding the design of future systems.

## Agnostic Smoothed Online Regression with Adversarial Responses

DevFeed: [Agnostic Smoothed Online Regression with Adversarial Responses](<https://devfeed.tech/articles/agnostic-smoothed-online-regression-with-adversarial-responses-144936.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.32478>)

Author: Xuanyu Chen, Yue Yu

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [linear-regression](<https://devfeed.tech/topics/linear-regression.md>)

Tags: [adaptive](<https://devfeed.tech/tags/adaptive.md>), [adversarial](<https://devfeed.tech/tags/adversarial.md>), [algorithm](<https://devfeed.tech/tags/algorithm.md>), [bounded](<https://devfeed.tech/tags/bounded.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [distribution](<https://devfeed.tech/tags/distribution.md>), [experts](<https://devfeed.tech/tags/experts.md>), [hedge](<https://devfeed.tech/tags/hedge.md>), [linear](<https://devfeed.tech/tags/linear.md>), [lower-bound](<https://devfeed.tech/tags/lower-bound.md>), [matching](<https://devfeed.tech/tags/matching.md>), [math-st](<https://devfeed.tech/tags/math-st.md>), [minimax](<https://devfeed.tech/tags/minimax.md>), [noise](<https://devfeed.tech/tags/noise.md>), [online](<https://devfeed.tech/tags/online.md>), [open](<https://devfeed.tech/tags/open.md>), [parameter](<https://devfeed.tech/tags/parameter.md>), [prediction](<https://devfeed.tech/tags/prediction.md>), [regression](<https://devfeed.tech/tags/regression.md>), [response](<https://devfeed.tech/tags/response.md>), [responses](<https://devfeed.tech/tags/responses.md>), [sampling](<https://devfeed.tech/tags/sampling.md>), [stat-ml](<https://devfeed.tech/tags/stat-ml.md>), [stat-th](<https://devfeed.tech/tags/stat-th.md>)

### AI overview

Hedge-Cover achieves sublinear regret of Õ(√(Pdim(F) C_cov T)) for smoothed online regression with bounded adversarial responses and function classes of bounded pseudo-dimension. It combines a constructed family of experts with Hedge, using a prior tied to disagreements between a consistent selector and target; covariate smoothness bounds those disagreements. The paper reports a matching lower bound for linear predictors, resolving an open question about minimax adaptive regret. In the well-specified setting with conditionally ν²-subGaussian response noise, it gives a minimax expected regret lower bound Ω((1∨ν)√((C_cov−1)dT)) for VC dimension d, matched in its dependence on ν by an ERM upper bound.

### Source excerpt

arXiv:2609.32478v2 Announce Type: replace-cross Abstract: We study smoothed online prediction with bounded adversarial responses. This widely studied framework bridges i.i.d. sampling and adversarial covariate selection through a smoothness parameter $\textsf{C}_{\textsf{cov}}$, which bounds conditional covariate densities relative to a fixed, unknown base measure. We propose Hedge-Cover, an information-theoretic algorithm that achieves sublinear regret $\widetilde{O}(\sqrt{\text{Pdim}(\mathcal{F}) \textsf{C}_{\textsf{cov}} T})$ for function classes with bounded pseudo-dimension. The algorithm aggregates a carefully constructed family of experts using Hedge, with a prior that links regret to the number of disagreements between a consistent selector and a target function. We bound this number by exploiting covariate smoothness. This answers an open problem posed in \cite{blanchard2025agnostic} on the minimax optimal adaptive regret of the smoothed online regression problem. We establish a matching lower bound for the class of linear predictors. The main intricacy of the lower bound lies in explicitly constructing a challenging sequential covariate distribution supported on mutually orthogonal hyperplanes. This construction may be of independent technical interest. Finally, we revisit the well-specified setting and quantify the effect of response noise. For conditionally $\nu^2$-subGaussian responses, we extend the existing lower bound under realizable responses by showing that the minimax expected regret is $\Omega((1\vee \nu)\sqrt{(\textsf{C}_{\textsf{cov}}-1)dT})$ for a function class of VC dimension $d$. A corresponding upper bound for ERM matches this dependence on $\nu$.

## Relax Within, Balance Across: Load Balancing under Composition Shifts in Vision-Language Mixture-of-Experts

DevFeed: [Relax Within, Balance Across: Load Balancing under Composition Shifts in Vision-Language Mixture-of-Experts](<https://devfeed.tech/articles/relax-within-balance-across-load-balancing-under-composition-shifts-in-vision-language-mixture-of-experts-145359.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2608.00574>)

Author: Ziang Wu, Peng Jin, Qishen Yin, Munan Ning, Hao Li, Peizhen Zhang, Huihui Zhou, Li Yuan

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [vlm](<https://devfeed.tech/topics/vlm.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>)

Tags: [balance](<https://devfeed.tech/tags/balance.md>), [balancing](<https://devfeed.tech/tags/balancing.md>), [cancellation](<https://devfeed.tech/tags/cancellation.md>), [changes](<https://devfeed.tech/tags/changes.md>), [code](<https://devfeed.tech/tags/code.md>), [composition](<https://devfeed.tech/tags/composition.md>), [counts](<https://devfeed.tech/tags/counts.md>), [cross](<https://devfeed.tech/tags/cross.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [errors](<https://devfeed.tech/tags/errors.md>), [experts](<https://devfeed.tech/tags/experts.md>), [fast](<https://devfeed.tech/tags/fast.md>), [fixed](<https://devfeed.tech/tags/fixed.md>), [flat](<https://devfeed.tech/tags/flat.md>), [gap](<https://devfeed.tech/tags/gap.md>), [image](<https://devfeed.tech/tags/image.md>), [language](<https://devfeed.tech/tags/language.md>), [load](<https://devfeed.tech/tags/load.md>), [load-balancing](<https://devfeed.tech/tags/load-balancing.md>), [mixture-of-experts](<https://devfeed.tech/tags/mixture-of-experts.md>), [vision](<https://devfeed.tech/tags/vision.md>)

### AI overview

On a fixed vision-language MoE router, balancing image and text tokens in one pooled auxiliary loss can conceal opposing modality load errors: they cancel near the training image-to-text ratio, but load skews as resolution, tiling, or image count changes that mix. The paper derives a quadratic response law under fixed modality profiles, with the image-text load gap governing sensitivity. It also reports that patches from an image route similarly as patch counts vary, allowing large images to dominate flat pooling. ReBA separates modality losses by token fraction and weights images equally, allowing per-image specialization while balancing visual load overall. Across four split backbones, the authors report stable balance across resolutions, tilings, and image counts without hurting accuracy; large modality gaps in released native MoEs suggest potential for native post-training.

### Source excerpt

arXiv:2608.00574v2 Announce Type: replace-cross Abstract: Vision-language mixture-of-experts (MoE) models such as Qwen3-VL need balanced loads for fast parallel serving. Standard training pools image and text tokens into one auxiliary loss, assuming training balance holds at inference. Yet we find a surprising mismatch: on a fixed router, balance shifts sharply whenever resolution, dynamic tiling, or image count changes the image-to-text token ratio. Splitting load by modality reveals cross-modal load cancellation: opposing image and text errors cancel near the training ratio, hiding strong skew until the mix changes. To quantify this shift, we derive an exact quadratic law: under fixed modality profiles, one reference run predicts the full response curve, and the image-text gap governs sensitivity. Meanwhile, patches within an image route similarly while patch counts vary, letting large images dominate flat pooling. We propose ReBA (Relax Within, Balance Across) to separate modality losses by token fraction and weight images equally, letting each image specialize while keeping overall visual load even. Across four split backbones, ReBA turns fragile single-ratio balance into stable balance across resolutions, tilings, and image counts without hurting accuracy. Large modality gaps in released native MoEs also show its promise for native post-training. Code is available at https://github.com/ZiangWu-77/ReBA.

## Game-Theoretic Data Allocation for Expert Specialization and Routing

DevFeed: [Game-Theoretic Data Allocation for Expert Specialization and Routing](<https://devfeed.tech/articles/game-theoretic-data-allocation-for-expert-specialization-and-routing-145017.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25167>)

Author: Mark Lovett, Denis Lim, Soroush Vosoughi

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>)

Tags: [behavior](<https://devfeed.tech/tags/behavior.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ma](<https://devfeed.tech/tags/cs-ma.md>), [data](<https://devfeed.tech/tags/data.md>), [distribution](<https://devfeed.tech/tags/distribution.md>), [experts](<https://devfeed.tech/tags/experts.md>), [game](<https://devfeed.tech/tags/game.md>), [gate](<https://devfeed.tech/tags/gate.md>), [inference](<https://devfeed.tech/tags/inference.md>), [kernel](<https://devfeed.tech/tags/kernel.md>), [kernels](<https://devfeed.tech/tags/kernels.md>), [lora](<https://devfeed.tech/tags/lora.md>), [prompt](<https://devfeed.tech/tags/prompt.md>), [prompts](<https://devfeed.tech/tags/prompts.md>), [response](<https://devfeed.tech/tags/response.md>), [router](<https://devfeed.tech/tags/router.md>), [routing](<https://devfeed.tech/tags/routing.md>), [safety](<https://devfeed.tech/tags/safety.md>), [specialization](<https://devfeed.tech/tags/specialization.md>), [theoretical](<https://devfeed.tech/tags/theoretical.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

InflEns uses one location game in a fixed prompt-trait space to determine both training weights for LoRA experts and inference routing, without using expert losses or training an explicit gate. On a corpus from seven safety-related benchmarks, the highest-share expert had the lowest held-out response NLL on about 90% of prompts across four kernels with a sharpened simplex representation; permuting allocations across prompts reduced this to about 15%. The method reportedly performed comparably to published learned gates. Its fixed router also let the authors isolate and retrain one harmful expert, lowering HarmBench attack success from 0.51 to 0.02 while maintaining instruction following. Specialization can be controlled through kernel choice and parameters, trait-space geometry, and expert count. These findings are reported for the described evaluation; broader applicability is presented as a framework claim.

### Source excerpt

arXiv:2611.25167v1 Announce Type: new Abstract: Can the allocation used to train experts also predict which expert will perform best on a prompt? Such a connection would make the intended division of labor inspectable before training and allow the allocation itself to guide inference. We introduce InflEns, a mixture of LoRA experts whose training weights and inference routes come from the same location game in a fixed prompt-trait space. Players choose positions to maximize their expected shares of the prompt distribution, without using expert losses. We utilize the game's kernel-general theoretical criterion for when co-located positions become locally unstable to separation. We evaluate InflEns on a corpus assembled from seven safety-related benchmarks, which demonstrates that the highest-share expert has the lowest held-out response negative log-likelihood (NLL) on approximately 90% of prompts across four kernels under a sharpened simplex representation, compared with approximately 15% after permuting allocations across prompts. We demonstrate that the method performs comparably to published learned gates without having to train an explicit gate on the experts. Because the router is fixed, we can trace harmful behavior to a single expert and retrain only that expert, which lowers HarmBench attack success from 0.51 to 0.02 while keeping instruction following. The game also gives us several controls over how the experts specialize, including the kernel choice and its parameters, the trait-space geometry, and the number of experts. Importantly, InflEns is a general framework that can be extended to any encoder and data distribution.

## Understanding MoE Decisions Beyond Gradients

DevFeed: [Understanding MoE Decisions Beyond Gradients](<https://devfeed.tech/articles/understanding-moe-decisions-beyond-gradients-145136.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26075>)

Author: Zhen Xu, Zitao Zhang, Kevin Zhao, Xuefeng Liu

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>)

Tags: [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [error](<https://devfeed.tech/tags/error.md>), [experts](<https://devfeed.tech/tags/experts.md>), [gemma4](<https://devfeed.tech/tags/gemma4.md>), [gradient](<https://devfeed.tech/tags/gradient.md>), [influence](<https://devfeed.tech/tags/influence.md>), [language](<https://devfeed.tech/tags/language.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [layers](<https://devfeed.tech/tags/layers.md>), [mixture-of-experts](<https://devfeed.tech/tags/mixture-of-experts.md>), [mixture-of-experts-moe](<https://devfeed.tech/tags/mixture-of-experts-moe.md>), [model](<https://devfeed.tech/tags/model.md>), [moe](<https://devfeed.tech/tags/moe.md>), [prediction](<https://devfeed.tech/tags/prediction.md>), [questions](<https://devfeed.tech/tags/questions.md>), [qwen3-6](<https://devfeed.tech/tags/qwen3-6.md>), [score](<https://devfeed.tech/tags/score.md>), [selection](<https://devfeed.tech/tags/selection.md>), [switches](<https://devfeed.tech/tags/switches.md>)

### AI overview

In mixture-of-experts language models, replacing an expert can change which experts later layers select, producing answer-score effects that gradients and second derivatives may miss. The proposed predictor uses the model's expert outputs and selection rules to estimate these additional effects and adds them to a second-order prediction. Across questions from three benchmarks, it reduced answer-score-change prediction error by 60% on Qwen3.6 and 47% on Gemma4 compared with a multistep baseline using gradients and second derivatives.

### Source excerpt

arXiv:2611.26075v1 Announce Type: cross Abstract: Understanding a language model's decision requires knowing how its internal computations affect its preference between answers. Gradient-based methods estimate this influence by predicting the effect of changing those computations. In mixture-of-experts (MoE) models, replacing one expert can change which experts later layers select. Those switches can alter the answer scores in ways that derivatives do not capture. We introduce a predictor that uses the model's own expert outputs and selection rules to estimate these extra effects, then adds them to a second-order prediction. Across questions from three benchmarks, our method reduces the error in predicting answer-score changes by 60\% on Qwen3.6 and 47\% on Gemma4, relative to a multistep baseline using gradients and second derivatives.

## Meet the Zephyr Community at Open Source Summit Japan 2026

DevFeed: [Meet the Zephyr Community at Open Source Summit Japan 2026](<https://devfeed.tech/articles/meet-the-zephyr-community-at-open-source-summit-japan-2026-143048.md>)

Original publisher: [Read original article](<https://zephyrproject.org/meet-the-zephyr-community-at-open-source-summit-japan-2026/>)

Author: Susan Remmert

Published: 2026-11-19T05:00:16Z

Content type: news

Language: en

Sources: [Zephyr Project](<https://devfeed.tech/sources/zephyr-project-2.md>)

Topics: [Zephyr RTOS](<https://devfeed.tech/topics/zephyr-rtos.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [ai](<https://devfeed.tech/tags/ai.md>), [amd](<https://devfeed.tech/tags/amd.md>), [anniversary](<https://devfeed.tech/tags/anniversary.md>), [api](<https://devfeed.tech/tags/api.md>), [applications](<https://devfeed.tech/tags/applications.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [arm](<https://devfeed.tech/tags/arm.md>), [blog](<https://devfeed.tech/tags/blog.md>), [bluetooth](<https://devfeed.tech/tags/bluetooth.md>), [certification](<https://devfeed.tech/tags/certification.md>), [community](<https://devfeed.tech/tags/community.md>), [contributors](<https://devfeed.tech/tags/contributors.md>), [developers](<https://devfeed.tech/tags/developers.md>), [event](<https://devfeed.tech/tags/event.md>), [events](<https://devfeed.tech/tags/events.md>), [experts](<https://devfeed.tech/tags/experts.md>), [industry-conference](<https://devfeed.tech/tags/industry-conference.md>), [japan](<https://devfeed.tech/tags/japan.md>), [meetup](<https://devfeed.tech/tags/meetup.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [open-source-summit](<https://devfeed.tech/tags/open-source-summit.md>), [tokyo](<https://devfeed.tech/tags/tokyo.md>), [zephyr](<https://devfeed.tech/tags/zephyr.md>)

### AI overview

Open Source Summit Japan 2026 will host Zephyr technical sessions in Tokyo on December 8-9, covering topics such as TSN, safety certification, Edge AI, OTA development, hardware support, multimedia, and display architecture. The Zephyr Project will also have Booth 27, with planned demonstrations ranging from low-power keyword spotting and Bluetooth Channel Sounding to a Zephyr-based spacecraft computer and a Zaurus PDA. A local meetup is being planned, but its date, venue, agenda, and registration details have not yet been announced. The event also marks Zephyr's 10th anniversary.

### Source excerpt

The Zephyr community is heading to Tokyo this December for Open Source Summit Japan 2026! Taking place from December 7-9 at Toranomon Hills Forum in Tokyo, Japan, the event brings together open source developers, contributors, maintainers, technologists, community leaders, and industry experts to share knowledge, collaborate on solutions, and advance open source innovation.

## Meet the Zephyr Community at Open Source Summit Japan 2026

DevFeed: [Meet the Zephyr Community at Open Source Summit Japan 2026](<https://devfeed.tech/articles/meet-the-zephyr-community-at-open-source-summit-japan-2026-144769.md>)

Original publisher: [Read original article](<https://www.zephyrproject.org/meet-the-zephyr-community-at-open-source-summit-japan-2026/>)

Author: Susan Remmert

Published: 2026-11-19T05:00:16Z

Content type: news

Language: en

Sources: [Zephyr Project](<https://devfeed.tech/sources/zephyr-project.md>)

Topics: [Zephyr RTOS](<https://devfeed.tech/topics/zephyr-rtos.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>), [coding-community](<https://devfeed.tech/topics/coding-community.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [2027](<https://devfeed.tech/tags/2027.md>), [acceleration](<https://devfeed.tech/tags/acceleration.md>), [ai](<https://devfeed.tech/tags/ai.md>), [amd](<https://devfeed.tech/tags/amd.md>), [blog](<https://devfeed.tech/tags/blog.md>), [community](<https://devfeed.tech/tags/community.md>), [contributors](<https://devfeed.tech/tags/contributors.md>), [developers](<https://devfeed.tech/tags/developers.md>), [event](<https://devfeed.tech/tags/event.md>), [events](<https://devfeed.tech/tags/events.md>), [experts](<https://devfeed.tech/tags/experts.md>), [industry](<https://devfeed.tech/tags/industry.md>), [industry-conference](<https://devfeed.tech/tags/industry-conference.md>), [innovation](<https://devfeed.tech/tags/innovation.md>), [japan](<https://devfeed.tech/tags/japan.md>), [knowledge](<https://devfeed.tech/tags/knowledge.md>), [maintainers](<https://devfeed.tech/tags/maintainers.md>), [meetup](<https://devfeed.tech/tags/meetup.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [open-source-summit](<https://devfeed.tech/tags/open-source-summit.md>), [solutions](<https://devfeed.tech/tags/solutions.md>), [tokyo](<https://devfeed.tech/tags/tokyo.md>), [zephyr](<https://devfeed.tech/tags/zephyr.md>)

### AI overview

Open Source Summit Japan 2026 is scheduled for December 7-9 at Toranomon Hills Forum in Tokyo, with a Zephyr track on December 8-9 in Hall B3. Sessions are described on topics including TSN, PCIe, safety certification and testing, Edge AI, spacecraft OTA, API contracts, AMD Versal memory architecture, GPU acceleration, multimedia, and displays. The Zephyr Project plans to staff Booth 27 with live demonstrations, including Edge AI, Bluetooth Channel Sounding, space applications, and Zephyr on a Zaurus PDA; additional demos are still being planned. The event also marks Zephyr's 10th anniversary. A community meetup is planned, but its date, venue, agenda, and registration details have not yet been announced.

### Source excerpt

The Zephyr community is heading to Tokyo this December for Open Source Summit Japan 2026! Taking place from December 7-9 at Toranomon Hills Forum in Tokyo, Japan, the event brings together open source developers, contributors, maintainers, technologists, community leaders, and industry experts to share knowledge, collaborate on solutions, and advance open source innovation.

## Lodestone: Automated Semantic Discovery of Datacenter Hardware Specialization

DevFeed: [Lodestone: Automated Semantic Discovery of Datacenter Hardware Specialization](<https://devfeed.tech/articles/lodestone-automated-semantic-discovery-of-datacenter-hardware-specialization-142687.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23663>)

Author: Peizhe Liu, Akanksha Jain, Gerasimos Gerogiannis, Vidushi Dadu, Parthasarathy Ranganathan, Jovan Stojkovic

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [RISC-V](<https://devfeed.tech/topics/riscv.md>), [High-Performance Computing](<https://devfeed.tech/topics/high-performance-computing.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Reverse Engineering](<https://devfeed.tech/topics/reverse-engineering.md>)

Tags: [accelerators](<https://devfeed.tech/tags/accelerators.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [applications](<https://devfeed.tech/tags/applications.md>), [automated](<https://devfeed.tech/tags/automated.md>), [code](<https://devfeed.tech/tags/code.md>), [control](<https://devfeed.tech/tags/control.md>), [core](<https://devfeed.tech/tags/core.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-ar](<https://devfeed.tech/tags/cs-ar.md>), [datacenter](<https://devfeed.tech/tags/datacenter.md>), [discovery](<https://devfeed.tech/tags/discovery.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [execution](<https://devfeed.tech/tags/execution.md>), [experts](<https://devfeed.tech/tags/experts.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [semantic](<https://devfeed.tech/tags/semantic.md>), [specialization](<https://devfeed.tech/tags/specialization.md>)

### AI overview

Lodestone uses semantic reasoning to find hardware specialization opportunities that hot-function or instruction-pattern analysis may miss when similar computations are spread across applications and libraries. It extracts hot control-flow regions from execution traces, packages them for agent analysis, groups regions that compute the same thing, and checks semantic correctness iteratively. It then tests each family against reference inputs and outputs from recorded executions. For selected families, it proposes hardware such as ISA extensions, near-core units, or detached accelerators; human experts choose which proposals to pursue. Across 20 workloads in six datacenter categories, 30 discovered families were simulated and implemented in RTL. The reported optimizations sped workloads up by 12.6% on average, with a maximum of 80.2%.

### Source excerpt

arXiv:2611.23663v1 Announce Type: cross Abstract: Hardware specialization can improve datacenter efficiency. Unfortunately, identifying computations that justify dedicated hardware is difficult. Existing approaches analyze hot functions, instruction patterns, or overheads such as datacenter tax. They can miss fine-grained computations whose aggregate cost is spread across applications and libraries, especially when different pieces of code obscure the same underlying computation. We observe that seemingly unrelated code regions often perform the same computations, creating specialization opportunities that require semantic reasoning. The rise of agentic AI makes it possible to automate this. However, conventional AI agents cannot efficiently and reliably analyze large scale datacenter software. To unlock this opportunity, we present Lodestone, an agentic framework that automatically discovers hardware specialization opportunities by reasoning about what the executed code computes. First, Lodestone extracts hot control-flow regions from execution traces across workloads and organizes them into compact analysis packages that make large workloads tractable for agents. Second, agents describe what each region computes, merge those that perform the same computation into computational families, and check their semantic correctness in a loop. Third, to verify the discovered family, Lodestone runs every piece of code in a family against a set of reference inputs and outputs on recorded executions. Finally, for selected families, Lodestone proposes hardware, e.g., ISA extensions, near-core units, or detached accelerators, and human experts decide which proposals to develop further. We evaluate Lodestone on 20 workloads across six datacenter workload categories. We simulate 30 discovered families in a cycle-accurate simulator and implement them in RTL. These optimizations speed up the workloads by 12.6% on average and by up to 80.2%.

## Exploring multimodal AI--XR interactions for software development: From expert perspectives to semantic code outlines

DevFeed: [Exploring multimodal AI--XR interactions for software development: From expert perspectives to semantic code outlines](<https://devfeed.tech/articles/exploring-multimodal-ai-xr-interactions-for-software-development-from-expert-perspectives-to-semantic-code-outlines-142211.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24334>)

Author: Ilya Zakharov, Alexey Markin, Konstantin Ostroukhov, Alexey Perminov, Roham Koohestani, Ekaterina Koshchenko, Agnia Sergeyuk

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [human computer interaction](<https://devfeed.tech/topics/human-computer-interaction.md>), [User interface design](<https://devfeed.tech/topics/ui-design.md>), [Software Engineering](<https://devfeed.tech/topics/software-engineering.md>), [AI tools for developers](<https://devfeed.tech/topics/ai-tools-for-developers.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [attention](<https://devfeed.tech/tags/attention.md>), [code](<https://devfeed.tech/tags/code.md>), [cs-hc](<https://devfeed.tech/tags/cs-hc.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [development](<https://devfeed.tech/tags/development.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [experts](<https://devfeed.tech/tags/experts.md>), [extended-reality](<https://devfeed.tech/tags/extended-reality.md>), [headset](<https://devfeed.tech/tags/headset.md>), [ide](<https://devfeed.tech/tags/ide.md>), [input](<https://devfeed.tech/tags/input.md>), [inspection](<https://devfeed.tech/tags/inspection.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [multimodal-ai](<https://devfeed.tech/tags/multimodal-ai.md>), [navigation](<https://devfeed.tech/tags/navigation.md>), [original](<https://devfeed.tech/tags/original.md>), [overview](<https://devfeed.tech/tags/overview.md>), [prototype](<https://devfeed.tech/tags/prototype.md>), [software](<https://devfeed.tech/tags/software.md>), [software-development](<https://devfeed.tech/tags/software-development.md>), [xr](<https://devfeed.tech/tags/xr.md>)

### AI overview

The exploratory study examined whether XR context could help developers manage AI-generated code that still needs understanding and verification. Researchers interviewed 13 experts, then had 12 professional developers try two prototypes: one made viewed code fade away; the other replaced viewed sections with brief purpose descriptions. In the latter, head direction helped trigger an offer, while developers used manual input to apply it or reopen the original code. Seven software-engineering researchers also provided feedback in an open-ended demonstration. Participants said descriptions could orient them to unfamiliar code and act as reminders, but comparing descriptions with source code, headset limitations, and controls unlike familiar IDE routines could add effort. The findings call for summaries that are easy to find, direct access to original code, and familiar navigation and editing practices. These are exploratory findings, not evidence of comparative effectiveness.

### Source excerpt

arXiv:2611.24334v1 Announce Type: cross Abstract: AI tools can generate code quickly, but developers still need to understand and check the results. We explore how the enriched multimodal context derived from the extended reality (XR) workspace might help developers manage this information. We first interviewed 13 experts in AI, XR, and related fields to identify opportunities and design challenges (Study 1). These interviews informed two prototypes explored with 12 professional developers (Study 2): four tried a version where the code faded to become invisible after it was looked at, and eight tried a version that replaced viewed code sections with short descriptions of their purpose. In the second prototype, head direction helped determine when to offer a change, while manual input let developers apply it and reopen the original code. A separate open-ended demonstration with seven software-engineering researchers provided complementary feedback. Participants' accounts suggested that descriptions could provide an overview of unfamiliar code and serve as reminders when revisiting it. However, difficulty comparing descriptions with source code, headset limitations, and controls that deviates from regular IDE routines could introduce additional effort. These exploratory findings identify requirements for attention support: make summaries easy to recognize and access, preserve straightforward inspection of the original code, and retain familiar navigation and editing practices. The study contributes to human-centered multimodal systems by showing how the coordination of input channels and AI-generated representations must account for users' attention, verification needs, and established ways of working.

## Mixtures of In-Context Learners

DevFeed: [Mixtures of In-Context Learners](<https://devfeed.tech/articles/mixtures-of-in-context-learners-142491.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2411.02830>)

Author: Giwon Hong, Emile van Krieken, Edoardo Ponti, Esmeralda S. Whitammer, Pasquale Minervini

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [context window](<https://devfeed.tech/topics/context-window.md>), [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>)

Tags: [classification](<https://devfeed.tech/tags/classification.md>), [complexity](<https://devfeed.tech/tags/complexity.md>), [context-window](<https://devfeed.tech/tags/context-window.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [distributions](<https://devfeed.tech/tags/distributions.md>), [domain](<https://devfeed.tech/tags/domain.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [experts](<https://devfeed.tech/tags/experts.md>), [fewer](<https://devfeed.tech/tags/fewer.md>), [filter](<https://devfeed.tech/tags/filter.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [frontier](<https://devfeed.tech/tags/frontier.md>), [function](<https://devfeed.tech/tags/function.md>), [improvements](<https://devfeed.tech/tags/improvements.md>), [in-context-learning](<https://devfeed.tech/tags/in-context-learning.md>), [inference](<https://devfeed.tech/tags/inference.md>), [llms](<https://devfeed.tech/tags/llms.md>), [memory](<https://devfeed.tech/tags/memory.md>), [merge](<https://devfeed.tech/tags/merge.md>), [model](<https://devfeed.tech/tags/model.md>)

### AI overview

Mixtures of In-Context Learners (MoICL) treats subsets of demonstrations as experts and learns to weight and merge their output distributions using a training set. The authors report improved performance over strong baselines on 5 of 7 classification datasets, with gains up to 13% over ICL and LENS. They also report reaching comparable performance with fewer demonstrations and less inference time, and greater robustness to out-of-domain, imbalanced, or noisy demonstrations, with reported gains up to 11%, 49%, and 38%, respectively. These findings are from the reported experiments; the supplied text does not provide further methodological details.

### Source excerpt

arXiv:2411.02830v2 Announce Type: replace-cross Abstract: In-context learning (ICL) adapts LLMs by providing demonstrations without fine-tuning the model parameters; however, it does not differentiate between demonstrations and quadratically increases the complexity of Transformer LLMs, exhausting the memory. As a solution, we propose Mixtures of In-Context Learners (MoICL), a novel approach to treat subsets of demonstrations as experts and learn a weighting function to merge their output distributions based on a training set. In our experiments, we show performance improvements on 5 out of 7 classification datasets compared to a set of strong baselines (up to +13\% compared to ICL and LENS). Moreover, we enhance the Pareto frontier of ICL by reducing the inference time needed to achieve the same performance with fewer demonstrations. Finally, MoICL is more robust to out-of-domain (up to +11\%), imbalanced (up to +49\%), or noisy demonstrations (up to +38\%) or can filter these out from datasets. Overall, MoICL is a more expressive approach to learning from demonstrations without exhausting the context window or memory.

## MECI: Multi-Element Collaborative Interaction for Multimodal Entity Linking

DevFeed: [MECI: Multi-Element Collaborative Interaction for Multimodal Entity Linking](<https://devfeed.tech/articles/meci-multi-element-collaborative-interaction-for-multimodal-entity-linking-142806.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24522>)

Author: Jie Peng, Yongxue Shan, Yongfu Zha, Xiaodong Wang

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Entity resolution](<https://devfeed.tech/topics/entity-resolution.md>), [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>)

Tags: [collaboration](<https://devfeed.tech/tags/collaboration.md>), [consistency](<https://devfeed.tech/tags/consistency.md>), [cross](<https://devfeed.tech/tags/cross.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [divide-and-conquer](<https://devfeed.tech/tags/divide-and-conquer.md>), [element](<https://devfeed.tech/tags/element.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [experts](<https://devfeed.tech/tags/experts.md>), [feature](<https://devfeed.tech/tags/feature.md>), [framework](<https://devfeed.tech/tags/framework.md>), [generative](<https://devfeed.tech/tags/generative.md>), [global](<https://devfeed.tech/tags/global.md>), [grounding](<https://devfeed.tech/tags/grounding.md>), [interaction](<https://devfeed.tech/tags/interaction.md>), [knowledge](<https://devfeed.tech/tags/knowledge.md>), [knowledge-base](<https://devfeed.tech/tags/knowledge-base.md>), [language](<https://devfeed.tech/tags/language.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [large](<https://devfeed.tech/tags/large.md>), [linking](<https://devfeed.tech/tags/linking.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>)

### AI overview

MECI addresses element dominance in multimodal entity linking with decoupled feature learning and hierarchical cross-modal interaction. Its Multi-view Experts Network uses KL-guided routing to coordinate specialized experts, while a modality-aware weighting network combines evidence across semantic levels. A final generative refinement stage uses large language models for zero-shot re-ranking. On WikiDiverse, RichpediaMEL, and WikiMEL, the abstract reports Hits@1 improvements of 1.95%, 7.30%, and 2.31%, respectively, over state-of-the-art baselines.

### Source excerpt

arXiv:2611.24522v1 Announce Type: cross Abstract: Multimodal Entity Linking (MEL) aims to disambiguate mentions in multimodal contexts by grounding them to specific entities in a knowledge base. A pivotal challenge in MEL is capturing multi-level correspondences: the semantic consistency between mention-entity pairs and the complementary correlations across modalities. However, existing methods often suffer from element dominance due to their reliance on coupled interactions or coarse global aggregations. In response, we propose the Multi-Element Collaborative Interaction (MECI) framework. First, to capture multi-element mention-entity correspondences, we develop a Multi-view Experts Network that leverages a "divide-and-conquer" strategy for decoupled feature learning to mitigate element dominance, supported by a KL-guided routing mechanism that governs expert specialization and collaboration. Furthermore, to model cross-modal complementary correlations, we propose a Hierarchical Multimodal Interaction Module, where a dynamic modality-aware weighting network refines interactions across hierarchical semantic levels, thereby integrating multi-granular evidence to counteract element dominance. Finally, we incorporate a generative semantic refinement stage that utilizes large language models for zero-shot re-ranking. Extensive experiments on WikiDiverse, RichpediaMEL, and WikiMEL show that MECI consistently outperforms state-of-the-art baselines, improving Hits@1 by 1.95%, 7.30%, and 2.31%, respectively.

## Soft Mixture of Quantum Experts for Hybrid Quantum-Classical Learning

DevFeed: [Soft Mixture of Quantum Experts for Hybrid Quantum-Classical Learning](<https://devfeed.tech/articles/soft-mixture-of-quantum-experts-for-hybrid-quantum-classical-learning-142834.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24798>)

Author: Valeria Repetto, Romina D'Aurizio, Piero Farinelli

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>)

Tags: [benchmark](<https://devfeed.tech/tags/benchmark.md>), [circuits](<https://devfeed.tech/tags/circuits.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [experts](<https://devfeed.tech/tags/experts.md>), [feature](<https://devfeed.tech/tags/feature.md>), [fixed](<https://devfeed.tech/tags/fixed.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [hybrid](<https://devfeed.tech/tags/hybrid.md>), [learning](<https://devfeed.tech/tags/learning.md>), [mnist](<https://devfeed.tech/tags/mnist.md>), [qml](<https://devfeed.tech/tags/qml.md>), [quant-ph](<https://devfeed.tech/tags/quant-ph.md>), [quantum](<https://devfeed.tech/tags/quantum.md>), [quantum-circuits](<https://devfeed.tech/tags/quantum-circuits.md>), [quantum-classical](<https://devfeed.tech/tags/quantum-classical.md>), [quantum-machine-learning](<https://devfeed.tech/tags/quantum-machine-learning.md>), [results](<https://devfeed.tech/tags/results.md>), [routing](<https://devfeed.tech/tags/routing.md>), [selection](<https://devfeed.tech/tags/selection.md>), [training](<https://devfeed.tech/tags/training.md>), [weight](<https://devfeed.tech/tags/weight.md>)

### AI overview

Soft-MoQE avoids training quantum circuit parameters by combining fixed quantum feature maps through a classical gating network that selects weights for each input. On PCA-reduced MNIST parity, MNIST 10-class, and FashionMNIST tasks, it outperformed the best single expert, uniform expert averaging, and a linear baseline. Its performance was comparable, depending on the task, to a trainable quantum neural network, with quantum training cost reported as orders of magnitude lower. The learned routing combined a few experts per input and varied by class. These findings are limited to the three evaluated tasks.

### Source excerpt

arXiv:2611.24798v1 Announce Type: cross Abstract: Quantum Machine Learning (QML) has attracted increasing interest as a possible route to construct feature representations that are difficult to reproduce with classical models. However, near-term QML architectures based on trainable parametrized quantum circuits remain affected by limited hardware resources, noise, and severe optimization difficulties such as barren plateaus. In this work we propose Soft-MoQE, a Soft Mixture of Quantum Experts architecture that replaces variational quantum training with the input-dependent selection and combination of fixed quantum feature maps. The model is composed of a heterogeneous pool of non-trainable quantum circuits, each defining a distinct embedding, and a classical gating network that learns how to weight these embeddings for each data point. We benchmark Soft-MoQE on three PCA-reduced tasks: MNIST parity, MNIST 10-class, and FashionMNIST. Soft-MoQE outperforms the single best expert, the uniform average of all experts and a linear baseline, achieving task-dependent performance comparable to that of a trainable quantum neural network, without optimizing any quantum parameter and with a quantum training cost that is lower by orders of magnitude. The analysis of the class- conditioned expert usage shows that the learned routing combines a few experts for each input, with a selection that depends on the class. These results support the learned routing among fixed quantum circuits as a trainability-aware design principle for hybrid QML.

## FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment

DevFeed: [FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment](<https://devfeed.tech/articles/flexmoe-learning-one-for-all-multi-budget-expert-pruning-for-flexible-moe-deployment-143284.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2606.27866>)

Author: Fan Mo, Yuxuan Han, Geng Zhang, Wangbo Zhao, Yang You

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [Post-training optimization](<https://devfeed.tech/topics/post-training-optimization.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>)

Tags: [acceleration](<https://devfeed.tech/tags/acceleration.md>), [actions](<https://devfeed.tech/tags/actions.md>), [activation](<https://devfeed.tech/tags/activation.md>), [budget](<https://devfeed.tech/tags/budget.md>), [co-design](<https://devfeed.tech/tags/co-design.md>), [compression](<https://devfeed.tech/tags/compression.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [experts](<https://devfeed.tech/tags/experts.md>), [ffn](<https://devfeed.tech/tags/ffn.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [flexible](<https://devfeed.tech/tags/flexible.md>), [inference](<https://devfeed.tech/tags/inference.md>), [kernel](<https://devfeed.tech/tags/kernel.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [learning](<https://devfeed.tech/tags/learning.md>), [llms](<https://devfeed.tech/tags/llms.md>), [masks](<https://devfeed.tech/tags/masks.md>), [memory](<https://devfeed.tech/tags/memory.md>), [moe](<https://devfeed.tech/tags/moe.md>), [pruning](<https://devfeed.tech/tags/pruning.md>)

### AI overview

FlexMoE aims to produce deployable MoE subnetworks at multiple parameter budgets from one pretrained model, avoiding separate compression runs for each budget. It ranks expert FFN channels to create a shared sliceable weight space, then learns expert-wise discrete retention actions under increasing cost pressure. A single fine-tune at a mid-range budget can recover performance and transfer to unseen budgets, reducing repeated recovery and separate checkpoint storage. The abstract reports gains over tested compression baselines; on Qwen2-57B-A14B, pruning 50% of routed-expert parameters retained about 99.8% of base performance without recovery fine-tuning. It also reports memory reduction and throughput gains, with kernel co-design enabling runtime budget switching without reloading checkpoints.

### Source excerpt

arXiv:2606.27866v2 Announce Type: replace Abstract: Mixture-of-Experts (MoE) language models scale model capacity through sparse expert activation, yet deployment still requires storing and serving all experts. Moreover, deployment budgets can vary substantially across devices, users, and workloads. Existing MoE compression methods typically optimize a separate compressed model for each target budget. We introduce FlexMoE, a post-training one-for-all framework that converts a large pretrained MoE model into a multi-budget family of smaller deployable subnetworks. FlexMoE first ranks expert FFN channels by importance to construct a sliceable weight space shared across budgets, and then learns expert-wise discrete retention actions under gradually increasing cost pressure, allowing a single action-training run to produce reliable pruning masks across budgets. Moreover, only a single mid-budget fine-tune can further recover degraded performance and yields a shared recovered model that transfers to other unseen budgets, avoiding repeated per-budget recovery and separated checkpoint storage. Across tested MoE LLMs with diverse architectures and scales, FlexMoE consistently outperforms strong MoE compression baselines. In particular, on Qwen2-57B-A14B, it retains ~99.8% of base performance while pruning 50% of routed-expert parameters even without recovery fine-tuning. For deployment, pruned subnetworks deliver real memory reduction and end-to-end throughput gains, while our kernel co-design enables runtime budget switching without checkpoint reloading and delivers substantial inference acceleration.

## Measuring What Matters: Synthetic Benchmarks for Concept Bottleneck Models

DevFeed: [Measuring What Matters: Synthetic Benchmarks for Concept Bottleneck Models](<https://devfeed.tech/articles/measuring-what-matters-synthetic-benchmarks-for-concept-bottleneck-models-142528.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2606.04326>)

Author: Julian Skirzynski, Harry Cheon, Shreyas Kadekodi, Meredith Stewart, Berk Ustun

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Ground truth / benchmark quality](<https://devfeed.tech/topics/ground-truth-benchmark-quality.md>), [Model Development](<https://devfeed.tech/topics/model-development.md>), [AI Models](<https://devfeed.tech/topics/ai-models.md>), [human feedback](<https://devfeed.tech/topics/human-feedback.md>)

Tags: [annotation](<https://devfeed.tech/tags/annotation.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [automation](<https://devfeed.tech/tags/automation.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [bottleneck](<https://devfeed.tech/tags/bottleneck.md>), [concepts](<https://devfeed.tech/tags/concepts.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [decision](<https://devfeed.tech/tags/decision.md>), [experts](<https://devfeed.tech/tags/experts.md>), [failure](<https://devfeed.tech/tags/failure.md>), [failures](<https://devfeed.tech/tags/failures.md>), [interpretability](<https://devfeed.tech/tags/interpretability.md>), [models](<https://devfeed.tech/tags/models.md>), [performance](<https://devfeed.tech/tags/performance.md>), [quality](<https://devfeed.tech/tags/quality.md>), [real-world](<https://devfeed.tech/tags/real-world.md>), [supervision](<https://devfeed.tech/tags/supervision.md>), [support](<https://devfeed.tech/tags/support.md>), [synthetic](<https://devfeed.tech/tags/synthetic.md>)

### AI overview

The paper introduces synthetic benchmarks for concept bottleneck models, addressing the shortage of real-world datasets with concept labels. Researchers can generate labeled data and vary modality, concept selection, and annotation quality to study which problems suit these models and isolate causes of failure. The authors demonstrate evaluation for decision support, where models help human experts, and automation of routine tasks without supervision. They report that the benchmarks reveal failure modes and help attribute them to model architecture or data.

### Source excerpt

arXiv:2606.04326v2 Announce Type: replace-cross Abstract: Concept bottleneck models predict using high-level concepts detected in inputs. Although these models provide a simple way to reap benefits from interpretability, they are hard to train and evaluate because very few real-world datasets include concept labels. In this paper, we develop synthetic benchmarks for concept bottleneck models. With our benchmarks, researchers can generate labeled datasets and control properties that affect performance, such as data modality, the choice of concepts, or annotation quality. They can thus find problems that suit these models and isolate the factors behind their failures. We demonstrate how our benchmarks can be used to evaluate major use cases for concept bottleneck models: decision support, i.e., helping human experts make better decisions; and automation, i.e., handling routine tasks without supervision. Our demonstrations show how the benchmarks can identify new failure modes and attribute them to either the architecture or the data.

## A Fast, Programmable GPU-to-CPU Invocation Layer for GPU-Initiated Communication

DevFeed: [A Fast, Programmable GPU-to-CPU Invocation Layer for GPU-Initiated Communication](<https://devfeed.tech/articles/a-fast-programmable-gpu-to-cpu-invocation-layer-for-gpu-initiated-communication-142971.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23848>)

Author: Jiakun Yan, Shiksha Patel, David Sidler, Gengyu Rao, Brandon Potter, Bradford Beckmann

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Distributed Systems & Parallel Computing](<https://devfeed.tech/topics/distributed-systems-parallel-computing.md>), [gpu-initated](<https://devfeed.tech/topics/gpu-initated.md>)

Tags: [async](<https://devfeed.tech/tags/async.md>), [backend](<https://devfeed.tech/tags/backend.md>), [bandwidth](<https://devfeed.tech/tags/bandwidth.md>), [challenges](<https://devfeed.tech/tags/challenges.md>), [code](<https://devfeed.tech/tags/code.md>), [communication](<https://devfeed.tech/tags/communication.md>), [control](<https://devfeed.tech/tags/control.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [device](<https://devfeed.tech/tags/device.md>), [dispatch](<https://devfeed.tech/tags/dispatch.md>), [expert-parallel](<https://devfeed.tech/tags/expert-parallel.md>), [experts](<https://devfeed.tech/tags/experts.md>), [fallback](<https://devfeed.tech/tags/fallback.md>), [framework](<https://devfeed.tech/tags/framework.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [gpudirect](<https://devfeed.tech/tags/gpudirect.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [kernel](<https://devfeed.tech/tags/kernel.md>), [kernels](<https://devfeed.tech/tags/kernels.md>), [latency](<https://devfeed.tech/tags/latency.md>)

### AI overview

KIWI makes reverse offload to host proxy threads a programmable communication path for GPU workloads, rather than treating it as a slower fallback to GPUDirect Async (GDA). The framework targets shared-queue contention, polling overhead, and proxy load imbalance, and reports sub-microsecond one-way device-to-host latency and hundreds of millions of operations per second. Built on KIWI, KIWI-P2P outperforms state-of-the-art GDA implementations in latency and bandwidth. KIWI-EP uses workload-defined operations for mixture-of-experts dispatch and combine, reducing expert-parallel communication latency by up to 2.3x over the best existing backend on AMD GPUs and vLLM time per output token by up to 29%.

### Source excerpt

arXiv:2611.23848v1 Announce Type: new Abstract: Modern GPU workloads increasingly rely on device-initiated communication to let GPU kernels communicate directly without synchronizing with the host at kernel boundaries. Existing libraries commonly favor placing transport control on GPU threads, an approach known as GPUDirect Async (GDA), and treat reverse offload to host proxy threads as a slower fallback. We argue that reverse offload can outperform GDA and should be a first-class communication path. We further propose exposing reverse offload as a programming interface that lets users define workload-specific communication operations. We present KIWI, a programmable reverse-offload framework that achieves sub-microsecond one-way device-to-host latency and hundreds of millions of operations per second. We address practical challenges of scaling reverse offload, including contention on shared queues, polling overhead, and load imbalance across proxies. KIWI-P2P, a SHMEM-like library built on KIWI, outperforms state-of-the-art GDA implementations in both latency and bandwidth. More importantly, KIWI-EP redesigns low-latency mixture-of-experts dispatch and combine kernels around workload-defined operations, lowering expert-parallel communication latency by up to 2.3x over the best existing backend on AMD GPUs and time per output token in vLLM by up to 29%. The source code of KIWI is available at https://github.com/AMDResearch/kiwi.

## Knowing What You Know: A Trait Theory of Reviewer Value in Human-AI Cognitive Systems

DevFeed: [Knowing What You Know: A Trait Theory of Reviewer Value in Human-AI Cognitive Systems](<https://devfeed.tech/articles/knowing-what-you-know-a-trait-theory-of-reviewer-value-in-human-ai-cognitive-systems-142847.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24932>)

Author: Priyanka Sinha, Sugam Pandey

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Human-AI evaluation](<https://devfeed.tech/topics/human-ai-evaluation.md>), [human review](<https://devfeed.tech/topics/human-review.md>)

Tags: [accounting](<https://devfeed.tech/tags/accounting.md>), [agent](<https://devfeed.tech/tags/agent.md>), [aggregation](<https://devfeed.tech/tags/aggregation.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-assisted](<https://devfeed.tech/tags/ai-assisted.md>), [alignment](<https://devfeed.tech/tags/alignment.md>), [confidence](<https://devfeed.tech/tags/confidence.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-ce](<https://devfeed.tech/tags/cs-ce.md>), [cs-hc](<https://devfeed.tech/tags/cs-hc.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [decision](<https://devfeed.tech/tags/decision.md>), [defer](<https://devfeed.tech/tags/defer.md>), [effort](<https://devfeed.tech/tags/effort.md>), [errors](<https://devfeed.tech/tags/errors.md>), [events](<https://devfeed.tech/tags/events.md>), [experts](<https://devfeed.tech/tags/experts.md>), [good](<https://devfeed.tech/tags/good.md>), [job](<https://devfeed.tech/tags/job.md>), [model](<https://devfeed.tech/tags/model.md>), [observable](<https://devfeed.tech/tags/observable.md>), [systems](<https://devfeed.tech/tags/systems.md>), [theory](<https://devfeed.tech/tags/theory.md>), [value](<https://devfeed.tech/tags/value.md>)

### AI overview

The paper argues that a human reviewer's value in AI-assisted decisions depends on metacognitive calibration--the alignment between confidence and judgment correctness--not activity, speed, agreement, or tenure. In six months of an AI-assisted accounting system (541,621 events; 483 reviewers), behavior-only profiling ranked an overconfident cohort above genuine experts, while reviewer quality was flat across tenure. Applying the same calibration logic to the model concentrated 92.5% of human effort on genuine errors. The authors caution that behavioral traces alone cannot distinguish calibrated experts from confident novices without outcome data, and that majority-vote aggregation prevents identifying an individual reviewer's causal value.

### Source excerpt

arXiv:2611.24932v1 Announce Type: new Abstract: When a capable model and a human reviewer cooperate on the same decision, which human still adds value? We argue this is a question about a single cognitive trait, metacognitive calibration: the alignment between a reviewer's behavioral confidence and the actual correctness of their judgments. We develop a trait theory of reviewer value to make it precise. The theory makes five claims: value is governed by calibration rather than by activity, speed, agreement, or tenure; calibration leaves observable traces in behavioral telemetry; those traces are non-diagnostic on their own, because the fast, low-revision signature of a calibrated expert is indistinguishable from that of a confident novice until it is gated by outcomes; human calibration and a model's self-knowledge are the same construct, so the joint system should defer to whichever agent is better calibrated on each item; and individual calibration is causally identifiable only above the level of the single judgment. We ground each claim in a six-month production window of an AI-assisted accounting system (541621 reviewer events, 483 reviewers), and find that a purely behavioral expertise profile inverts the ground truth, scoring an overconfident cohort (5.2 percent correction survival) higher than genuine experts (99.2 percent); that reviewer quality is flat across tenure, so the trait is not learned on the job; and that the same calibration logic that separates good from bad reviewers, applied to the model, concentrates 92.5 percent of human effort on genuine errors. We close with the boundary the theory must respect: under majority-vote aggregation, individual causal value is unidentifiable.

## Architectural Design Decisions for Data Ingestion in Big Data Architectures

DevFeed: [Architectural Design Decisions for Data Ingestion in Big Data Architectures](<https://devfeed.tech/articles/architectural-design-decisions-for-data-ingestion-in-big-data-architectures-142161.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23297>)

Author: Yikang Huang, Stefano Fossati, Indika Kumara, Damian A. Tamburri, Willem-Jan van den Heuvel

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [software-architecture](<https://devfeed.tech/topics/software-architecture.md>), [Data pipelines](<https://devfeed.tech/topics/data-pipelines.md>), [big-data](<https://devfeed.tech/topics/big-data.md>), [Data science and engineering](<https://devfeed.tech/topics/data-science-and-engineering.md>)

Tags: [architecture](<https://devfeed.tech/tags/architecture.md>), [architectures](<https://devfeed.tech/tags/architectures.md>), [big-data](<https://devfeed.tech/tags/big-data.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [data](<https://devfeed.tech/tags/data.md>), [data-analysis](<https://devfeed.tech/tags/data-analysis.md>), [data-architecture](<https://devfeed.tech/tags/data-architecture.md>), [data-engineers](<https://devfeed.tech/tags/data-engineers.md>), [data-ingestion](<https://devfeed.tech/tags/data-ingestion.md>), [data-ingestion-pipeline](<https://devfeed.tech/tags/data-ingestion-pipeline.md>), [data-ingestion-pipelines](<https://devfeed.tech/tags/data-ingestion-pipelines.md>), [data-lakes](<https://devfeed.tech/tags/data-lakes.md>), [decision](<https://devfeed.tech/tags/decision.md>), [design](<https://devfeed.tech/tags/design.md>), [drivers](<https://devfeed.tech/tags/drivers.md>), [experts](<https://devfeed.tech/tags/experts.md>), [framework](<https://devfeed.tech/tags/framework.md>), [functional](<https://devfeed.tech/tags/functional.md>), [options](<https://devfeed.tech/tags/options.md>), [pipeline](<https://devfeed.tech/tags/pipeline.md>), [pipelines](<https://devfeed.tech/tags/pipelines.md>)

### AI overview

The paper presents a framework to help data engineers and architects choose designs for data ingestion pipelines based on functional and non-functional requirements. A systematic review of 42 gray literature articles and qualitative analysis produced a catalog of 8 architectural decisions, 59 options, and 36 selection drivers. Semi-structured expert interviews assessed the framework's completeness, simplicity, understandability, operational feasibility, and usefulness. The supplied text reports that these qualities were evaluated, but does not give the interview findings or name the decisions and options.

### Source excerpt

arXiv:2611.23297v1 Announce Type: new Abstract: Introduction: Data ingestion is the first critical step in a data architecture, in which data from various sources is moved to storage systems, such as data lakes and data warehouses, for analysis and insight generation. When developing a data ingestion pipeline, practitioners (e.g., data engineers and architects) must understand the data ingestion requirements (functional and non-functional) and select appropriate design options to support them. Objective: To help practitioners systematically develop data ingestion processes, this paper aims to compile a catalog of data ingestion design decisions, their options, and the selection drivers. Method: We conducted a systematic gray literature review on data ingestion, analyzed the selected literature using qualitative data analysis methods, and conducted semi-structured interviews with experts to assess the framework's practicality and utility. Results: From 42 gray literature articles, we identified 8 architectural design decisions, 59 decision options, and 36 decision drivers to guide practitioners in the design and implementation of data ingestion pipelines. The framework was evaluated through expert interviews, assessing its completeness, simplicity, understandability, operational feasibility, and usefulness.

## Clover-35B-A3B-Preview Technical Report

DevFeed: [Clover-35B-A3B-Preview Technical Report](<https://devfeed.tech/articles/clover-35b-a3b-preview-technical-report-142816.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24636>)

Author: DiffuSpace Technical Team

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [text-generation](<https://devfeed.tech/topics/text-generation.md>), [Performance Testing](<https://devfeed.tech/topics/performance-testing.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [attention](<https://devfeed.tech/tags/attention.md>), [batch](<https://devfeed.tech/tags/batch.md>), [bf16](<https://devfeed.tech/tags/bf16.md>), [block](<https://devfeed.tech/tags/block.md>), [clover](<https://devfeed.tech/tags/clover.md>), [context](<https://devfeed.tech/tags/context.md>), [convolution](<https://devfeed.tech/tags/convolution.md>), [costs](<https://devfeed.tech/tags/costs.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [decision](<https://devfeed.tech/tags/decision.md>), [decision-model](<https://devfeed.tech/tags/decision-model.md>), [denoising](<https://devfeed.tech/tags/denoising.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [experts](<https://devfeed.tech/tags/experts.md>), [gb300](<https://devfeed.tech/tags/gb300.md>), [generation](<https://devfeed.tech/tags/generation.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [technical](<https://devfeed.tech/tags/technical.md>)

### AI overview

Clover-35B-A3B-Preview is a 35-billion-parameter mixture-of-experts diffusion language model that activates 3 billion parameters and supports thinking-mode generation. Its block-diffusion architecture combines recurrent memory of earlier context with bidirectional softmax attention within the current block; both compete in a shared softmax, with learnable block-size calibration and a separate zero-initialized right-context convolution branch. The report says Clover leads seven evaluated agent metrics, scoring 73.6 on SWE-bench Verified and 51.7 on Terminal-Bench 2.1. On internal agent workloads, it reports 680 tokens/s on one NVIDIA GB300 at BF16 and batch size one. A decision-model variant predicts answers to related questions in one denoising pass; on 500 internal groups, it raises question accuracy from 90.37% to 93.88% and all-correct group accuracy from 65.4% to 77.0%. These are abstract-level reported results; the supplied text does not give detailed evaluation methods.

### Source excerpt

arXiv:2611.24636v1 Announce Type: cross Abstract: We introduce Clover-35B-A3B-Preview (Clover), a mixture-of-experts diffusion language model with 35 billion total parameters and 3 billion activated parameters, supporting thinking-mode generation. Clover adapts Gated Delta Networks to block diffusion by combining recurrent memory of previously generated context with bidirectional softmax attention inside the current block. Historical memory readout competes with current-block tokens in a shared softmax, with learnable block-size calibration and a separate, zero-initialized right-context convolution branch. Clover leads the seven evaluated agent metrics, including 73.6 on SWE-bench Verified and 51.7 on Terminal-Bench 2.1. On internal agent workloads, it reaches a reported generation speed of 680 tokens/s using one NVIDIA GB300 GPU at BF16 precision and batch size one. We organize the evaluation around model quality, drafting and verification costs, and deployment under constrained memory. We also describe a decision-model variant that generates answers to related questions in one denoising pass. On an internal set of 500 groups, joint prediction raises question accuracy from 90.37% to 93.88% and all-correct group accuracy from 65.4% to 77.0%.

## Inherited Adapter Experts for Replay-Free Domain-Incremental Learning

DevFeed: [Inherited Adapter Experts for Replay-Free Domain-Incremental Learning](<https://devfeed.tech/articles/inherited-adapter-experts-for-replay-free-domain-incremental-learning-142758.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24145>)

Author: Gyutae Oh, Hyunmin Kong, Jitae Shin

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>), [image recognition](<https://devfeed.tech/topics/image-recognition.md>)

Tags: [adapter](<https://devfeed.tech/tags/adapter.md>), [adapters](<https://devfeed.tech/tags/adapters.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [computer-vision](<https://devfeed.tech/tags/computer-vision.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [deepfake](<https://devfeed.tech/tags/deepfake.md>), [domain](<https://devfeed.tech/tags/domain.md>), [domains](<https://devfeed.tech/tags/domains.md>), [experts](<https://devfeed.tech/tags/experts.md>), [free](<https://devfeed.tech/tags/free.md>), [incremental](<https://devfeed.tech/tags/incremental.md>), [learning](<https://devfeed.tech/tags/learning.md>), [points](<https://devfeed.tech/tags/points.md>), [router](<https://devfeed.tech/tags/router.md>), [vit](<https://devfeed.tech/tags/vit.md>)

### AI overview

Lamarck addresses replay-free domain-incremental learning by strengthening per-domain experts rather than training a router: the abstract reports that a training-free Mahalanobis router costs the analyzed learners under 2.7 points versus always choosing the own-domain expert. Lamarck adds adapters after each deep ViT block, improving final accuracy by 4.4 points on CDDB-Hard, and initializes each new expert from its predecessor at no extra cost. This is intended to reduce the cost when an image is routed to a later expert, since domains share classes. Against twelve frozen-backbone baselines under one protocol, Lamarck ranks first on CDDB-Hard by 5.2 points, second on DomainNet and OfficeHome, and highest in mean accuracy.

### Source excerpt

arXiv:2611.24145v1 Announce Type: cross Abstract: Replay-free domain-incremental learning trains a model on the same classes in successive domains without keeping earlier images. Per-domain learners freeze a pre-trained ViT and train an additional expert for each new domain: the small set of parameters and the classification head learned on that domain. Test images carry no domain label, so a router selects one expert for each image. Recent work trains this router. However, a training-free Mahalanobis router costs the analyzed learners under 2.7 points of final accuracy relative to always selecting each image's own-domain expert. The experts, not this router, limit accuracy. Our learner, Lamarck, therefore uses this router and strengthens the experts. Its adapters, small trainable layers that correct every image token after each deep ViT block, add 4.4 points of final accuracy on the deepfake benchmark CDDB-Hard. In addition, each expert is initialized from its predecessor at no extra cost: the domains share their classes, so an image misrouted to a later expert loses less. Against twelve frozen-backbone baselines under one protocol, Lamarck is first on CDDB-Hard by 5.2 points, second on DomainNet and OfficeHome, and highest in mean accuracy.

## Training-Free Dynamic Upcycling of Expert Language Models

DevFeed: [Training-Free Dynamic Upcycling of Expert Language Models](<https://devfeed.tech/articles/training-free-dynamic-upcycling-of-expert-language-models-143249.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2603.29765>)

Author: Eros Fan\\\`i, O\\u{g}uzhan Ersoy

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>)

Tags: [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [domains](<https://devfeed.tech/tags/domains.md>), [experts](<https://devfeed.tech/tags/experts.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [performance](<https://devfeed.tech/tags/performance.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [training](<https://devfeed.tech/tags/training.md>), [upcycling](<https://devfeed.tech/tags/upcycling.md>)

### AI overview

DUME combines dense language models trained on different domains into a single Mixture of Experts model without additional training. It uses a closed-form ridge regression solution, avoiding further optimization and allowing experts to be added or removed dynamically. Across causal language modeling and reasoning experiments with models from 115M to 9B parameters, the authors report that DUME outperforms baseline approaches. In reasoning, it averages 104% of the domain expert's performance. The resulting model can also be fine-tuned to improve performance on complex reasoning tasks.

### Source excerpt

arXiv:2603.29765v2 Announce Type: replace Abstract: Large Language Models (LLMs) have achieved remarkable performance on a wide range of specialized tasks, exhibiting strong problem-solving capabilities. However, training these models is prohibitively expensive, and they often lack domain-specific expertise because they rely on general knowledge datasets. Expertise finetuning can address this issue; however, it often leads to overspecialization, and developing a single multi-domain expert remains difficult due to diverging objectives. Furthermore, multitask training is challenging due to interference and catastrophic forgetting. Existing work proposes combining the expertise of dense models within a Mixture of Experts (MoE) architecture, although this approach still requires multitask finetuning. To address these issues, we introduce Dynamic Upcycling MoE (DUME), a novel approach that reuses dense experts trained on different domains to construct a unified MoE model. Our method builds a single multitask model that preserves the capabilities of the original dense experts without requiring additional training. DUME is both cost-efficient and scalable: by leveraging the closed-form solution of ridge regression, it eliminates the need for further optimization and enables experts to be added or removed dynamically. Our extensive experiments on different tasks (in causal language modeling and reasoning settings) with several model sizes (between 115M and 9B parameters) and architectures demonstrate that DUME consistently outperforms baseline approaches. In particular, in a reasoning setting, DUME can also surpass the dense expert model's performance in its domain, achieving 104% of the dense expert's performance on average. Finally, we also show that the DUME model can be fine-tuned to further improve performance in complex reasoning scenarios. Our code is available at: github.com/gensyn-ai/dume.

## Beyond Routing Weights: Faithful Response-Level Interpretation of Mixture-of-Experts Reward Models via Contribution Contrast

DevFeed: [Beyond Routing Weights: Faithful Response-Level Interpretation of Mixture-of-Experts Reward Models via Contribution Contrast](<https://devfeed.tech/articles/beyond-routing-weights-faithful-response-level-interpretation-of-mixture-of-experts-reward-models-via-contribution-contrast-142551.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2608.06400>)

Author: Yifan Wang, Jinyi Mu, Mayank Jobanputra, Yu Wang, Soyoung Oh, Isabel Valera, Vera Demberg

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [human feedback](<https://devfeed.tech/topics/human-feedback.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [account](<https://devfeed.tech/tags/account.md>), [alternatives](<https://devfeed.tech/tags/alternatives.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [autoencoder](<https://devfeed.tech/tags/autoencoder.md>), [automatic](<https://devfeed.tech/tags/automatic.md>), [based](<https://devfeed.tech/tags/based.md>), [behavior](<https://devfeed.tech/tags/behavior.md>), [central](<https://devfeed.tech/tags/central.md>), [coco](<https://devfeed.tech/tags/coco.md>), [coherent](<https://devfeed.tech/tags/coherent.md>), [contribution](<https://devfeed.tech/tags/contribution.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [evaluations](<https://devfeed.tech/tags/evaluations.md>), [example](<https://devfeed.tech/tags/example.md>), [experts](<https://devfeed.tech/tags/experts.md>), [mixture-of-experts](<https://devfeed.tech/tags/mixture-of-experts.md>), [models](<https://devfeed.tech/tags/models.md>), [response](<https://devfeed.tech/tags/response.md>), [reward](<https://devfeed.tech/tags/reward.md>), [routing](<https://devfeed.tech/tags/routing.md>)

### AI overview

CoCo interprets Mixture-of-Experts reward models by selecting chosen-rejected response pairs with the largest contribution contrasts, combining evidence about which prompts an expert receives with how it judges responses. The abstract reports that automatic and human evaluations found CoCo's interpretations more coherent, faithful, and specialized than router-based, score-based, and sparse autoencoder-based alternatives, while reward modeling accuracy remained competitive. The supplied text is an abstract and does not provide evaluation details or quantitative results.

### Source excerpt

arXiv:2608.06400v2 Announce Type: replace Abstract: Reward models are central to learning from human preferences, yet identifying what drives their predictions remains challenging. Recent sparse Mixture-of-Experts (MoE) reward models seek to improve interpretability by routing prompts to specialized experts and characterizing experts through examples with high routing weights. However, routing weights only reveal which prompts an expert $\textit{receives}$, not how it $\textit{judges}$ responses, providing only a partial account of expert behavior. We therefore propose $\textbf{Co}$ntribution-$\textbf{Co}$ntrast ($\textbf{CoCo}$) response-level interpretation, which faithfully characterizes experts' roles using chosen-rejected response pairs with the largest contribution contrasts, jointly capturing routing and preference behavior. Across automatic and human evaluations, CoCo yields more coherent, faithful, and specialized interpretations than router-based, score-based, and sparse autoencoder-based alternatives while maintaining competitive reward modeling accuracy. Overall, we provide a controlled study of how different example-selection signals affect the interpretability of MoE reward models.

## Harvesting Expert-Parallel Bubbles for Fine-Tuning

DevFeed: [Harvesting Expert-Parallel Bubbles for Fine-Tuning](<https://devfeed.tech/articles/harvesting-expert-parallel-bubbles-for-fine-tuning-142803.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24496>)

Author: Oana Balmau, Saurav Banka, Jiaxuan Chen, Anne-Marie Kermarrec, Ramya Narasimha Prabhu, Rafael Pires, Mathis Randl, Martijn de Vos

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [GPU optimization](<https://devfeed.tech/topics/gpu-optimization.md>), [distributed-training](<https://devfeed.tech/topics/distributed-training.md>), [Multi-GPU](<https://devfeed.tech/topics/multi-gpu.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [a100](<https://devfeed.tech/tags/a100.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [balancing](<https://devfeed.tech/tags/balancing.md>), [bubble](<https://devfeed.tech/tags/bubble.md>), [capacity](<https://devfeed.tech/tags/capacity.md>), [colocation](<https://devfeed.tech/tags/colocation.md>), [compiler](<https://devfeed.tech/tags/compiler.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cs-os](<https://devfeed.tech/tags/cs-os.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [engine](<https://devfeed.tech/tags/engine.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [expert-parallel](<https://devfeed.tech/tags/expert-parallel.md>), [expert-parallelism](<https://devfeed.tech/tags/expert-parallelism.md>), [experts](<https://devfeed.tech/tags/experts.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [inference](<https://devfeed.tech/tags/inference.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [large-language-model](<https://devfeed.tech/tags/large-language-model.md>)

### AI overview

BubbleTea uses per-GPU idle periods created by uneven expert-parallel token routing to run fine-tuning alongside LLM serving. It predicts each GPU's available bubble at every layer and schedules short fine-tuning chunks that reuse serving weights; compiler passes derive the chunks from the served model without porting its architecture to a separate training engine. On three MoE models served across four A100 GPUs, BubbleTea delivered 1.9-6.2x the fine-tuning throughput of LLMStation and kept p90 inference latency within 10% of serving alone in 50 of 63 workloads, versus 44 for LLMStation. On bursty production traces, it trained 3.3x faster than LLMStation and 14x faster than a stock trainer under CUDA MPS, with p90 latency within 7% of serving alone.

### Source excerpt

arXiv:2611.24496v1 Announce Type: cross Abstract: GPUs are expensive, yet latency-critical large language model (LLM) inference leaves part of them idle. Colocation reclaims this capacity by running best-effort work, such as fine-tuning, alongside serving. Many leading LLMs now use a Mixture-of-Experts (MoE) architecture, which adds a new source of idle time. Under expert parallelism, MoE experts are spread across GPUs, and because the router sends uneven numbers of tokens to each GPU, lightly loaded GPUs wait for the most loaded one. Even after applying load balancing solutions, such bubbles still take 8--13% of prefill time. We present BubbleTea, the first colocation system that leverages expert-parallel bubbles for fine-tuning. Unlike existing colocation systems, which make one scheduling decision for all GPUs per forward pass, BubbleTea predicts each GPU's bubble at every layer from the router's token assignment, and fills it with short chunks of fine-tuning work that reuse the weights already loaded for serving. We also introduce compiler passes that automatically derive these chunks from the served LLM as-is, without porting each model architecture to a separate training engine. Our evaluation on three MoE models, served on four A100 GPUs, shows that fine-tuning throughput with BubbleTea is 1.9--6.2$\times$ faster than with LLMStation, a state-of-the-art colocation system. BubbleTea keeps p90 inference latency within 10% of serving alone in 50 of 63 workloads, compared to 44 for LLMStation. On bursty production traces, BubbleTea trains 3.3$\times$ faster than LLMStation and 14$\times$ faster than a stock trainer colocated under CUDA MPS, with p90 inference latency staying within 7% of serving alone.

[Next page](<https://devfeed.tech/tags/experts.md?cursor=WyIyMDI2LTExLTE5VDA1OjAwOjAwKzAwOjAwIiwgIjA3ZDE1ODY3LTUzMDktNDg2ZS04OTBhLWZkZDM1Mjc5NDhlNCJd>)