# Inference

The process of making predictions by applying a trained machine-learning model to unlabeled examples.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Inference pushes AI to smaller sites, making the data layer the deciding factor

DevFeed: [Inference pushes AI to smaller sites, making the data layer the deciding factor](<https://devfeed.tech/articles/inference-pushes-ai-to-smaller-sites-making-the-data-layer-the-deciding-factor-146669.md>)

Original publisher: [Read original article](<https://siliconangle.com/2026/11/20/inference-monolithic-data-center-ai-data-layer-sc26/>)

Author: Sloane Kali Faye

Published: 2026-11-20T20:09:20Z

Content type: news

Language: en

Sources: [SiliconANGLE](<https://devfeed.tech/sources/siliconangle.md>)

Topics: [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [AI Architecture](<https://devfeed.tech/topics/ai-architecture.md>), [data centers](<https://devfeed.tech/topics/data-centers.md>), [AI Factory](<https://devfeed.tech/topics/ai-factory.md>)

Tags: [accelerated-computing](<https://devfeed.tech/tags/accelerated-computing.md>), [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-data-centers](<https://devfeed.tech/tags/ai-data-centers.md>), [ai-data-layer](<https://devfeed.tech/tags/ai-data-layer.md>), [ai-inference](<https://devfeed.tech/tags/ai-inference.md>), [ai-infrastructure](<https://devfeed.tech/tags/ai-infrastructure.md>), [alex-bouzari](<https://devfeed.tech/tags/alex-bouzari.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [data-centers](<https://devfeed.tech/tags/data-centers.md>), [data-layer](<https://devfeed.tech/tags/data-layer.md>), [ddn](<https://devfeed.tech/tags/ddn.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [distributed-inference](<https://devfeed.tech/tags/distributed-inference.md>), [edge-data-centers](<https://devfeed.tech/tags/edge-data-centers.md>), [enterprise](<https://devfeed.tech/tags/enterprise.md>), [enterprise-ai](<https://devfeed.tech/tags/enterprise-ai.md>), [enterprise-ai-hyperpod](<https://devfeed.tech/tags/enterprise-ai-hyperpod.md>), [exascaler](<https://devfeed.tech/tags/exascaler.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [heterogeneous-computing](<https://devfeed.tech/tags/heterogeneous-computing.md>), [high-performance-computing](<https://devfeed.tech/tags/high-performance-computing.md>), [high-performance-storage](<https://devfeed.tech/tags/high-performance-storage.md>), [hpc](<https://devfeed.tech/tags/hpc.md>), [hpc-infrastructure](<https://devfeed.tech/tags/hpc-infrastructure.md>), [inference](<https://devfeed.tech/tags/inference.md>), [john-furrier](<https://devfeed.tech/tags/john-furrier.md>), [news](<https://devfeed.tech/tags/news.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [parallel-file-systems](<https://devfeed.tech/tags/parallel-file-systems.md>), [rebecca-knight](<https://devfeed.tech/tags/rebecca-knight.md>), [sc26](<https://devfeed.tech/tags/sc26.md>), [sc26eventpage](<https://devfeed.tech/tags/sc26eventpage.md>), [scientific-computing](<https://devfeed.tech/tags/scientific-computing.md>), [scientific-simulation](<https://devfeed.tech/tags/scientific-simulation.md>), [supercomputing](<https://devfeed.tech/tags/supercomputing.md>), [thecube](<https://devfeed.tech/tags/thecube.md>)

### AI overview

The excerpt argues that inference is shifting AI infrastructure toward smaller sites near users and data, increasing the importance of a flexible data layer as GPU, memory and SSD costs and shortages constrain budgets. DataDirect Networks CEO Alex Bouzari says software that spans different hardware can help maximize utilization across large AI facilities and smaller 1-5 MW sites. DDN's EXAScaler move to hardware independence reflects that approach; its Enterprise AI HyperPOD combines CPUs, GPUs, networking and storage in a rack for deployment at enterprise sites with available power. The excerpt presents on-premises pods as a way to keep tighter control of company data, and frames data infrastructure as increasingly important alongside accelerated compute. These are Bouzari's views and DDN examples, not an independent comparison of providers.

### Source excerpt

Inference is moving to the center of enterprise AI spending, and the AI data layer is emerging as the piece that decides whether those investments pay off. As GPU costs climb and component shortages persist, the architecture behind AI is shifting from a handful of massive facilities toward a distributed web of smaller sites. That [...] The post Inference pushes AI to smaller sites, making the data layer the deciding factor appeared first on SiliconANGLE.

## Optimal algorithmic complexity of inference in quantum kernel methods

DevFeed: [Optimal algorithmic complexity of inference in quantum kernel methods](<https://devfeed.tech/articles/optimal-algorithmic-complexity-of-inference-in-quantum-kernel-methods-144886.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2604.15214>)

Author: Elies Gil-Fuster, Seongwook Shin, Sofiene Jerbi, Jens Eisert, Maximilian J. Kramer

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Inference](<https://devfeed.tech/topics/inference.md>), [Algorithms](<https://devfeed.tech/topics/algorithms.md>), [Combinatorial optimization](<https://devfeed.tech/topics/combinatorial-optimization.md>)

Tags: [algorithm](<https://devfeed.tech/tags/algorithm.md>), [algorithms](<https://devfeed.tech/tags/algorithms.md>), [amplitude](<https://devfeed.tech/tags/amplitude.md>), [bottleneck](<https://devfeed.tech/tags/bottleneck.md>), [complexity](<https://devfeed.tech/tags/complexity.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [encoding](<https://devfeed.tech/tags/encoding.md>), [gate](<https://devfeed.tech/tags/gate.md>), [inference](<https://devfeed.tech/tags/inference.md>), [kernel](<https://devfeed.tech/tags/kernel.md>), [lower-bound](<https://devfeed.tech/tags/lower-bound.md>), [quant-ph](<https://devfeed.tech/tags/quant-ph.md>), [quantum](<https://devfeed.tech/tags/quantum.md>), [quantum-advantage](<https://devfeed.tech/tags/quantum-advantage.md>), [query](<https://devfeed.tech/tags/query.md>), [sampling](<https://devfeed.tech/tags/sampling.md>), [supervised-learning](<https://devfeed.tech/tags/supervised-learning.md>)

### AI overview

Quantum kernel inference estimates a weighted sum of kernel values, and independently sampling each term has query complexity O(N‖α‖₂²/ε²). The abstract identifies two choices--sampling or quantum amplitude estimation for kernel values, and term-by-term estimation or encoding the full sum as one observable. Combining the single-observable encoding with amplitude estimation achieves O(‖α‖₁/ε) queries, with a matching lower bound up to logarithmic factors. This removes N from the query count and improves dependence on ‖α‖₁ and ε. The paper also analyzes gate costs: the query-optimal method is not always best in gate complexity, so the practical strategy depends on hardware. All described algorithms use amplitude estimation as a subroutine.

### Source excerpt

arXiv:2604.15214v3 Announce Type: replace-cross Abstract: Quantum kernel methods are among the leading candidates for achieving quantum advantage in supervised learning. A key bottleneck is the cost of inference: evaluating a trained model on new data requires estimating a weighted sum $\sum_{i=1}^N \alpha_i k(x,x_i)$ of $N$ kernel values to additive precision $\varepsilon$, where $\alpha$ is the vector of trained coefficients. The standard approach estimates each term independently via sampling, yielding a query complexity of $O(N\lVert\alpha\rVert_2^2/\varepsilon^2)$. In this work, we identify two independent axes for improvement: (1) How individual kernel values are estimated (sampling versus quantum amplitude estimation), and (2) how the sum is approximated (term-by-term versus via a single observable), and systematically analyze all combinations thereof. The query-optimal combination, encoding the full inference sum as the expectation value of a single observable and applying quantum amplitude estimation, achieves a query complexity of $O(\lVert\alpha\rVert_1/\varepsilon)$, removing the dependence on $N$ from the query count and yielding a quadratic improvement in both $\lVert\alpha\rVert_1$ and $\varepsilon$. We prove a matching lower bound of $\Omega(\lVert\alpha\rVert_1/\varepsilon)$, establishing query-optimality of our approach up to logarithmic factors. Beyond query complexity, we also analyze how these improvements translate into gate costs and show that the query-optimal strategy is not always optimal in practice from the perspective of gate complexity. Our results provide both a query-optimal algorithm and a practically optimal choice of strategy depending on hardware capabilities, along with a complete landscape of intermediate methods to guide practitioners. All algorithms require only amplitude estimation as a subroutine and are thus natural candidates for early-fault-tolerant implementations.

## The ROI of Small Examples: Optimizing In-Context Learning Under Token Budgets

DevFeed: [The ROI of Small Examples: Optimizing In-Context Learning Under Token Budgets](<https://devfeed.tech/articles/the-roi-of-small-examples-optimizing-in-context-learning-under-token-budgets-145450.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24971>)

Author: Anxo Perez, Jorge Gab\\'in, Javier Parapar

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [few-shot](<https://devfeed.tech/topics/few-shot.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>)

Tags: [budget](<https://devfeed.tech/tags/budget.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [examples](<https://devfeed.tech/tags/examples.md>), [in-context-learning](<https://devfeed.tech/tags/in-context-learning.md>), [inference](<https://devfeed.tech/tags/inference.md>), [model-performance](<https://devfeed.tech/tags/model-performance.md>), [prompt](<https://devfeed.tech/tags/prompt.md>), [qa](<https://devfeed.tech/tags/qa.md>)

### AI overview

Under a fixed prompt token budget, the study finds that using more short demonstrations generally improves in-context learning for question answering compared with spending the budget on a few long demonstrations. The authors grouped examples by length and tested six instruction-tuned models on Beat the AI and Natural Questions. The effect was strongest on Beat the AI; it also appeared on Natural Questions, where gains were smaller and saturated earlier with long contexts. In an ANLI R3 inference experiment, one of the two shorter length groups performed best for five of six models, though longer examples remained competitive. The results suggest demonstration length is a relevant prompt design choice alongside example count.

### Source excerpt

arXiv:2611.24971v1 Announce Type: cross Abstract: In-context learning (ICL) improves large language model performance at inference time without updating model parameters. However, these gains are not free: in practical deployments, longer prompts directly increase inference cost and latency. Under a fixed token budget, prompt construction therefore becomes a resource-allocation problem. While prior work has often focused on how many examples to include in ICL, we study the complementary question of whether the available budget is better spent on a few long demonstrations or on a larger number of shorter ones. We study this aspect in question answering (QA), by organizing candidate demonstrations into length-based bins and constructing prompts from examples within the same length range under fixed budgets. Experiments on two QA benchmarks (\textit{Beat the AI} and \textit{Natural Questions}), using six instruction-tuned LLMs, show that shorter demonstrations are generally the most effective use of prompt space. This effect is especially strong on \textit{Beat the AI}, and remains visible on NQ, although gains are smaller and saturate earlier in the long-context setting. An additional natural language inference experiment on ANLI R3 finds that one of the two shorter bins performs best for five of six models, although longer demonstrations remain competitive. Overall, our results highlight demonstration granularity as an important factor in prompt design.

## Infer3D: Inverse Generative Modeling Enables Robust 3D Reconstruction

DevFeed: [Infer3D: Inverse Generative Modeling Enables Robust 3D Reconstruction](<https://devfeed.tech/articles/infer3d-inverse-generative-modeling-enables-robust-3d-reconstruction-145004.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25061>)

Author: Amani Kiruga, Mark Hamilton, Joshua B. Tenenbaum, William T. Freeman, Vincent Sitzmann, Yilun Du, Ayush Tewari

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [3D](<https://devfeed.tech/topics/3d.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [Image](<https://devfeed.tech/topics/image.md>)

Tags: [3d](<https://devfeed.tech/tags/3d.md>), [3d-reconstruction](<https://devfeed.tech/tags/3d-reconstruction.md>), [adaptive](<https://devfeed.tech/tags/adaptive.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [generative](<https://devfeed.tech/tags/generative.md>), [modeling](<https://devfeed.tech/tags/modeling.md>), [ood](<https://devfeed.tech/tags/ood.md>), [robust](<https://devfeed.tech/tags/robust.md>)

### AI overview

Infer3D reconstructs 3D scenes from a single image by optimizing the latent variables of a generative model to match the input, rather than relying only on fast feedforward inference. A frozen 3D lifting network constrains optimization to a valid, multi-view-consistent 3D manifold to prevent flat outputs. On controlled synthetic and real-world tasks, the authors report better out-of-distribution reconstruction accuracy than feedforward networks trained on equal or larger datasets. Its adaptive strategy spends more test-time compute on ambiguous out-of-distribution examples while routing in-distribution inputs efficiently; the abstract does not quantify this compute or accuracy tradeoff.

### Source excerpt

arXiv:2611.25061v1 Announce Type: cross Abstract: Humans can understand the 3D structure of scenes even from highly unfamiliar or extreme viewpoints. Cognitive science suggests this generalization stems from a top-down generative inference strategy. In contrast, state-of-the-art 3D reconstruction neural networks typically rely on amortized, feedforward inference. While fast, this pattern-matching approach degrades when faced with out-of-distribution (OOD) scene conditions unseen during training. Inspired by the generative nature of human vision, we propose Infer3D, an inverse generative modeling framework that recovers the latent variables of a generative model to best match the monocular input. To prevent degenerate flat outputs, we couple the generative image prior with a frozen 3D lifting network that constrains the optimization space to a valid, multi-view-consistent 3D manifold. Across a variety of controlled synthetic and real-world tasks, Infer3D demonstrates superior OOD reconstruction accuracy compared to feedforward networks trained on equal or larger datasets. Furthermore, mirroring how human perception requires more time to process confusing stimuli, we introduce an adaptive inference strategy that expends more test-time compute on ambiguous OOD examples while efficiently routing in-distribution inputs. Project website: https://amanikiruga.github.io/infer3d-site/

## When Does Depth Matter For In-Context Learning? Adaptive Inference in Deep Transformers

DevFeed: [When Does Depth Matter For In-Context Learning? Adaptive Inference in Deep Transformers](<https://devfeed.tech/articles/when-does-depth-matter-for-in-context-learning-adaptive-inference-in-deep-transformers-144908.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2606.16694>)

Author: Ravin Raj, Gautam Reddy

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Transformers](<https://devfeed.tech/topics/transformers.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Inference](<https://devfeed.tech/topics/inference.md>)

Tags: [adaptive](<https://devfeed.tech/tags/adaptive.md>), [context](<https://devfeed.tech/tags/context.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [distributed-inference](<https://devfeed.tech/tags/distributed-inference.md>), [gaussian](<https://devfeed.tech/tags/gaussian.md>), [in-context-learning](<https://devfeed.tech/tags/in-context-learning.md>), [inference](<https://devfeed.tech/tags/inference.md>), [learning](<https://devfeed.tech/tags/learning.md>), [linear-attention](<https://devfeed.tech/tags/linear-attention.md>), [linear-regression](<https://devfeed.tech/tags/linear-regression.md>), [patching](<https://devfeed.tech/tags/patching.md>), [physics-app-ph](<https://devfeed.tech/tags/physics-app-ph.md>), [q-bio-nc](<https://devfeed.tech/tags/q-bio-nc.md>), [routing](<https://devfeed.tech/tags/routing.md>), [structure](<https://devfeed.tech/tags/structure.md>), [transformers](<https://devfeed.tech/tags/transformers.md>)

### AI overview

For in-context linear regression, the theory predicts that a single transformer layer without feedforward blocks is optimal for Gaussian priors, while deeper transformers outperform it for non-Gaussian, tree-like priors. The authors propose that successive layers use intermediate "function vectors" to infer a latent context variable at finer scales, enabling adaptive information routing. Trained linear attention transformers reproduce the theory's quantitative predictions, and causal key-patching experiments support a role for intermediate function vectors in that routing. The results suggest depth and feedforward blocks are advantageous when latent variables have hierarchical structure.

### Source excerpt

arXiv:2606.16694v3 Announce Type: replace Abstract: Transformers perform computations through many successive attention and feedforward blocks, allowing them to learn complex correlations between a large collection of coupled variables. When does stacking successive attention-feedforward computations over many layers provide a computational advantage over a single transformer block? We address this question by examining in-context learning in generalized linear attention transformers. We first introduce a general theory of distributed inference in such transformers, subject to constraints on communication and depth. We show that such systems can exploit internal representations (`function vectors') to infer a latent context variable at increasingly finer scales over its layers. For an in-context linear regression task, the theory predicts that while one-layer transformers without feedforward blocks are optimal for Gaussian priors over the context variable, multi-layer transformers are superior for non-Gaussian, tree-like priors. Trained linear attention transformers reproduce quantitative predictions from the theory. Using causal key-patching experiments, we verify that function vectors in intermediate layers mediate adaptive routing of information. Our results suggest that depth and feedforward blocks enable transformers to implement adaptive inference, and this is advantageous when the distribution over latent variables has hierarchical structure.

## From Sequence Uncertainty to Phenotype Hypotheses: Uncertainty-Conditioned Inference over Biomedical Knowledge Graphs

DevFeed: [From Sequence Uncertainty to Phenotype Hypotheses: Uncertainty-Conditioned Inference over Biomedical Knowledge Graphs](<https://devfeed.tech/articles/from-sequence-uncertainty-to-phenotype-hypotheses-uncertainty-conditioned-inference-over-biomedical-knowledge-graphs-145053.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25422>)

Author: Huixin Zhan, Jason H. Moore

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [AI Foundation Models](<https://devfeed.tech/topics/ai-foundation-models.md>), [база знаний](<https://devfeed.tech/topics/tag-c51cc4bf76e5.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [graph theory](<https://devfeed.tech/topics/graph-theory.md>)

Tags: [biomedical](<https://devfeed.tech/tags/biomedical.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [distributions](<https://devfeed.tech/tags/distributions.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [energy](<https://devfeed.tech/tags/energy.md>), [entropy](<https://devfeed.tech/tags/entropy.md>), [foundation-models](<https://devfeed.tech/tags/foundation-models.md>), [framework](<https://devfeed.tech/tags/framework.md>), [hypothesis](<https://devfeed.tech/tags/hypothesis.md>), [inference](<https://devfeed.tech/tags/inference.md>), [knowledge-graphs](<https://devfeed.tech/tags/knowledge-graphs.md>), [masks](<https://devfeed.tech/tags/masks.md>), [probability](<https://devfeed.tech/tags/probability.md>), [q-bio-gn](<https://devfeed.tech/tags/q-bio-gn.md>), [query](<https://devfeed.tech/tags/query.md>), [recall](<https://devfeed.tech/tags/recall.md>), [recovery](<https://devfeed.tech/tags/recovery.md>), [sequence](<https://devfeed.tech/tags/sequence.md>), [supervision](<https://devfeed.tech/tags/supervision.md>), [support](<https://devfeed.tech/tags/support.md>), [temperature](<https://devfeed.tech/tags/temperature.md>), [uncertainty](<https://devfeed.tech/tags/uncertainty.md>)

### AI overview

The framework turns genomic sequence predictions and calibrated uncertainty into probabilities over graph-supported phenotype hypotheses, without patient-level phenotype supervision. It scores admissible gene-disease-phenotype paths with a learned energy, uses query uncertainty to set the target temperature, and aggregates path probabilities while preserving traceable support. On AlzKB and Hetionet, phenotype entropy tracked upstream uncertainty (Spearman 0.951 and 0.878). Uncertainty-dependent temperature training improved held-out recall@10 over a comparable constant temperature. In gene hold-out, recall@10 exceeded trained DistMult on 24 of 30 graph masks (0.183 vs. 0.123), but the difference was driven mainly by three genes with many variants and was not established for all genes.

### Source excerpt

arXiv:2611.25422v1 Announce Type: new Abstract: Genomic foundation models provide variant-level evidence, but connecting that evidence to phenotype hypotheses requires accounting for uncertainty throughout structured reasoning. We introduce an uncertainty-conditioned path-energy framework that converts sequence predictions and calibrated predictive uncertainty into explicit distributions over graph-supported phenotype hypotheses, without patient-level phenotype supervision. An interpretable learned energy scores admissible gene--disease--phenotype routes, query uncertainty controls the target temperature, and path probabilities are aggregated into phenotype hypotheses. This formulation makes evidence quality an explicit determinant of hypothesis breadth while retaining traceable multi-path support. Exact normalization over enumerable candidate sets provides a reference for evaluating a Hastings-corrected learned revision operator. On AlzKB and Hetionet, phenotype-level entropy tracks upstream uncertainty, with Spearman correlations of 0.951 and 0.878, respectively; ablations identify the contribution of uncertainty conditioning. In masked-edge recovery, the model exceeds chance and degree-matched probability-permutation controls. Training the energy under the uncertainty-dependent temperature yields higher held-out recall@10 than training under a constant temperature of comparable magnitude. Under gene hold-out, recall@10 exceeds that of a trained DistMult embedding on 24 of 30 graph masks (0.183 versus 0.123); this difference is attributable mainly to three genes with many variants and is not established for all genes. These results establish an uncertainty-aware interface from genomic predictions to graph-supported phenotype hypotheses that combines interpretable probability allocation with held-out association recovery of explicitly stated scope.

## Doing More by Storing Less: Speeding Up Real-Time Machine Learning

DevFeed: [Doing More by Storing Less: Speeding Up Real-Time Machine Learning](<https://devfeed.tech/articles/doing-more-by-storing-less-speeding-up-real-time-machine-learning-144308.md>)

Original publisher: [Read original article](<https://medium.com/feedzaitech/doing-more-by-storing-less-speeding-up-real-time-machine-learning-23639ae00e22?source=rss----e11168e7fe6b---4>)

Author: Augusto Peres

Published: 2026-11-19T15:40:42Z

Content type: article

Language: en

Sources: [Feedzai](<https://devfeed.tech/sources/feedzai.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Event-Streaming](<https://devfeed.tech/topics/event-streaming.md>), [streaming data analytics](<https://devfeed.tech/topics/streaming-data-analytics.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [rate-limiting](<https://devfeed.tech/topics/rate-limiting.md>)

Tags: [aggregate](<https://devfeed.tech/tags/aggregate.md>), [automatically](<https://devfeed.tech/tags/automatically.md>), [counts](<https://devfeed.tech/tags/counts.md>), [database](<https://devfeed.tech/tags/database.md>), [detection](<https://devfeed.tech/tags/detection.md>), [event](<https://devfeed.tech/tags/event.md>), [financial-applications](<https://devfeed.tech/tags/financial-applications.md>), [fraud-detection](<https://devfeed.tech/tags/fraud-detection.md>), [heavy-hitter](<https://devfeed.tech/tags/heavy-hitter.md>), [inference](<https://devfeed.tech/tags/inference.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [latency](<https://devfeed.tech/tags/latency.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [real-time](<https://devfeed.tech/tags/real-time.md>), [storage](<https://devfeed.tech/tags/storage.md>)

### AI overview

Feedzai's event filtering reduces costly read-modify-write database cycles for high-volume entities while aiming to preserve useful feature profiles. Below a chosen per-entity write-rate threshold, every event is stored; above it, events are randomly included with probability set by the threshold divided by the estimated arrival rate. Kept events are weighted by the inverse of that probability to compensate counts and sums for discarded updates. The system can estimate arrival rates from filtered persisted statistics, updating and scaling those estimates through the same sampling process rather than tracking every entity's rate in memory; the article describes this as self-correcting. It says the approach fits power-law transaction volumes and reports that model accuracy did not decline under heavy filtering, with small gains in some cases attributed to training noise. The excerpt provides no quantitative performance results.

### Source excerpt

Modern fraud detection systems rely heavily on tracking behavior over time. For example: How much money has a person spent this month? Or, how many purchases have they made in the last week? Any event-streaming system worth its salt tracks hundreds of such metrics. Every time a new transaction even arrives, these are retrieved from a database, updated, and then written back to persistent storage. This process is known as the read-modify-write cycle, and its final write stage is a major source of latency, database contention, and infrastructure cost. But here is the catch: In many financial applications, a small set of heavy-hitter entities triggers most of these cycles and accounts for a disproportionate number of transactions. Statistically, the more events we process for any entity, the less information each individual event adds to their overall profile. Thus, here at Feedzai we developed a process and system that automatically stops writing every single event for large financial entities, saving significant compute without losing downstream ML accuracy. A friendly introduction to feature profiles Before we go any deeper into discarding events, we briefly explain what feature profiles are. Essentially, feature profiles aggregate events over a period of time. Two common examples are: Counts: How many times have we seen an entity doing something? e.g., the number of transactions for a specific debit card in the last month. Sums: The running total of a quantity associated with an event, e.g., the total spend of a user in the last week. In a real-time fraud detection system, when a single event arrives, the feature profiles for its associated entity must be retrieved from persistent storage, (e.g., a relational database or key-value store). Then, the profiles must be updated with the information on the new event. Next, the updated profiles are sent to a Machine Learning inference service for classification (e.g., fraud or not fraud) and written back to persistent sto

## Monitoring Machine Learning Systems in Production: A Multivocal Literature Review

DevFeed: [Monitoring Machine Learning Systems in Production: A Multivocal Literature Review](<https://devfeed.tech/articles/monitoring-machine-learning-systems-in-production-a-multivocal-literature-review-142158.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23293>)

Author: Hira Naveed, Aleksandar Pasquini, Scott Barnett, Chetan Arora, John Grundy, Hourieh Khalajzadeh, Omar Haggag

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [Monitoring](<https://devfeed.tech/topics/monitoring.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Production Engineering](<https://devfeed.tech/topics/production-engineering.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [cs-se](<https://devfeed.tech/tags/cs-se.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [generative-ai](<https://devfeed.tech/tags/generative-ai.md>), [inference](<https://devfeed.tech/tags/inference.md>), [literature](<https://devfeed.tech/tags/literature.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [monitoring](<https://devfeed.tech/tags/monitoring.md>), [production](<https://devfeed.tech/tags/production.md>)

### AI overview

The review of 94 academic and gray-literature sources finds that production ML monitoring is uneven: tools broadly automate observing system behavior, detecting problems, and communicating findings, while configuration, investigation, and response still rely substantially on people. Monitoring focuses on inference, and no reviewed source covers the whole production system, limiting end-to-end diagnosis. Of 78 sources describing an approach or tool, 45 report no formal evaluation, 24 report offline or controlled evaluations, and nine evaluate live production. Evidence for generative AI monitoring is especially limited and relies heavily on gray literature and vendor claims. The authors propose research directions spanning monitor design, cross-layer diagnosis, proxy validation, corrective responses, production-oriented evaluation, and organizational practices.

### Source excerpt

arXiv:2611.23293v1 Announce Type: new Abstract: Monitoring is essential for maintaining reliable machine learning (ML) systems after deployment. However, knowledge about its practical implementation and effectiveness remains fragmented across academic and gray literature. We present a multivocal literature review of 94 sources (37 academic and 57 gray) covering both traditional ML and generative AI systems. We develop taxonomies covering monitored objects and properties, techniques, input signals, tools, automation, frequency, evaluation, and challenges. Our findings reveal an imbalanced monitoring process. Existing approaches provide broad and largely automated support for observing system behavior, detecting problems, and communicating findings, while monitor configuration, investigation, and response retain substantial human involvement. Monitoring is concentrated on inference, with no source covering the entire production ML system, limiting end-to-end diagnosis. Evidence of effectiveness is also limited: among 78 sources presenting an approach or tool, 45 provide no formal evaluation, 24 report offline or controlled evaluations, and only nine evaluate monitoring in live production. This gap is particularly evident for generative AI, where evidence is dominated by gray literature and vendor claims. We propose six directions for improving monitor design, cross-layer diagnosis, proxy validation, corrective responses, production-oriented evaluation, and organizational practices. Our review provides a structured basis for improving the detection, diagnosis, and resolution of production ML problems.

## ICICLE: In-Context Inference for Curve-Learning Extrapolation

DevFeed: [ICICLE: In-Context Inference for Curve-Learning Extrapolation](<https://devfeed.tech/articles/icicle-in-context-inference-for-curve-learning-extrapolation-143527.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24469>)

Author: Laurits Fredsgaard, Michael Riis Andersen, Mikkel N. Schmidt

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Hyperparameter optimization](<https://devfeed.tech/topics/hyperparameter-optimization.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [quantization](<https://devfeed.tech/topics/quantization.md>), [onnx](<https://devfeed.tech/topics/onnx.md>)

Tags: [bayesian](<https://devfeed.tech/tags/bayesian.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [budget](<https://devfeed.tech/tags/budget.md>), [configuration](<https://devfeed.tech/tags/configuration.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [encoder](<https://devfeed.tech/tags/encoder.md>), [extrapolation](<https://devfeed.tech/tags/extrapolation.md>), [freeze](<https://devfeed.tech/tags/freeze.md>), [inference](<https://devfeed.tech/tags/inference.md>), [jax](<https://devfeed.tech/tags/jax.md>), [learning](<https://devfeed.tech/tags/learning.md>), [meta](<https://devfeed.tech/tags/meta.md>), [model](<https://devfeed.tech/tags/model.md>), [noise](<https://devfeed.tech/tags/noise.md>), [number](<https://devfeed.tech/tags/number.md>), [onnx](<https://devfeed.tech/tags/onnx.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [pytorch](<https://devfeed.tech/tags/pytorch.md>), [quantization](<https://devfeed.tech/tags/quantization.md>), [ray](<https://devfeed.tech/tags/ray.md>), [release](<https://devfeed.tech/tags/release.md>)

### AI overview

ICICLE is an in-context surrogate for freeze-thaw Bayesian optimization that accepts a variable number of hyperparameters through a shared set encoder. Trained on synthetic tasks covering typed hyperparameters, heavy-tailed noise, quantization, and divergence, it uses 1.28 million parameters (8.7% of FT-PFN's count) and produces similar mean-regret trajectories to FT-PFN on LCBench, PD1, and TaskSet. On Meta-Album's 86 tasks with 33 hyperparameters, it achieves lower mean normalized regret than Quick-Tune overall and on extended, with no clear difference on mini or micro. ICICLE was not pre-trained or meta-trained on Meta-Album tasks, while Quick-Tune was meta-trained on related tasks. The release samples synthetic tasks on the fly and includes a Ray Tune freeze-thaw scheduler and an ONNX inference model that does not require JAX or PyTorch.

### Source excerpt

arXiv:2611.24469v1 Announce Type: new Abstract: Freeze-thaw Bayesian optimization allocates training budget by predicting how partially observed learning curves will continue. LC-PFN first applied prior-data fitted networks (PFNs) to learning-curve extrapolation, and FT-PFN extended this approach to configuration-aware freeze-thaw optimization with a ten-slot hyperparameter encoder. We introduce In-Context Inference for Curve-Learning Extrapolation (ICICLE), an in-context surrogate that accepts a variable number of hyperparameters through a shared set encoder and is trained on a synthetic prior covering typed hyperparameters, heavy-tailed noise, quantization, and divergence. ICICLE has 1.28 million parameters (8.7% of FT-PFN's count) and produces similar mean-regret trajectories to FT-PFN on LCBench, PD1, and TaskSet. On the 86-task Meta-Album benchmark with 33 hyperparameters, ICICLE reaches lower mean normalized regret than Quick-Tune overall and on extended, with no clear difference on mini or micro. It does so without pre-training or meta-training on Meta-Album tasks, whereas Quick-Tune is meta-trained on related Meta-Album tasks. Our release samples synthetic training tasks on the fly and provides a freeze-thaw scheduler for Ray Tune and an ONNX model for inference without JAX or PyTorch.

## STORM: A Lightweight Structure-Free Model for Binding Affinity Prediction

DevFeed: [STORM: A Lightweight Structure-Free Model for Binding Affinity Prediction](<https://devfeed.tech/articles/storm-a-lightweight-structure-free-model-for-binding-affinity-prediction-143445.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23777>)

Author: Ayush Pandit (Stanford University), Rohan V. Koodli (Stanford University), Wei Deng (Stanford University), Roxana Daneshjou (Stanford University), Ron O. Dror (Stanford University)

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Inference](<https://devfeed.tech/topics/inference.md>)

Tags: [affinity](<https://devfeed.tech/tags/affinity.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [drug-discovery](<https://devfeed.tech/tags/drug-discovery.md>), [fast](<https://devfeed.tech/tags/fast.md>), [inference](<https://devfeed.tech/tags/inference.md>), [lightweight](<https://devfeed.tech/tags/lightweight.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [machine-learning-model](<https://devfeed.tech/tags/machine-learning-model.md>), [prediction](<https://devfeed.tech/tags/prediction.md>), [protein](<https://devfeed.tech/tags/protein.md>), [q-bio-qm](<https://devfeed.tech/tags/q-bio-qm.md>), [small-molecule](<https://devfeed.tech/tags/small-molecule.md>), [storm](<https://devfeed.tech/tags/storm.md>), [structure](<https://devfeed.tech/tags/structure.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

STORM (Similarity Transfer from Observed Reference Molecules) predicts protein-small-molecule binding affinity by ensembling small model fusions of diverse protein and molecule representations. The abstract reports that, when trained on the same data, it matches or exceeds Boltz-2 accuracy on multiple evaluation datasets while providing over five orders of magnitude faster inference and training. The authors also suggest that combining historical experimental affinity data with structural and biophysical information could improve accuracy. These claims are limited to the reported evaluations; the excerpt gives no dataset details or per-dataset results.

### Source excerpt

arXiv:2611.23777v1 Announce Type: cross Abstract: Accurately predicting a small molecule's binding affinity to a protein, a critical task in drug discovery, remains challenging. Physics-based approaches like free-energy perturbation (FEP) have very high computational costs and require expert setup. Recent machine learning--based cofolding models have reported accuracy approaching that of FEP---and at least one of them, Boltz-2, has already been widely adopted for affinity prediction---but they remain expensive for large computational screens and even more expensive for retraining on newly collected experimental data. To address these challenges, we introduce STORM: Similarity Transfer from Observed Reference Molecules. STORM is a fast, simple machine learning model that ensembles many small model fusions of diverse protein and molecule representations for binding affinity prediction. Trained on the same data, STORM matches or exceeds the accuracy of Boltz-2 on multiple evaluation datasets, while delivering a speed improvement of over five orders of magnitude for both inference and training. Our results also suggest opportunities to substantially improve affinity prediction accuracy by integrating historical experimental affinity data with structural and biophysical information.

## DaisyRose-27B: Pushing the Frontier of Post-Training Efficiency

DevFeed: [DaisyRose-27B: Pushing the Frontier of Post-Training Efficiency](<https://devfeed.tech/articles/daisyrose-27b-pushing-the-frontier-of-post-training-efficiency-143008.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/RadixArk/daisyrose-27b-pushing-the-frontier-of-post-trainin>)

Author: Jason Chen

Published: 2026-11-18T19:54:56Z

Content type: release

Language: en

Sources: [Hugging Face](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [post-training](<https://devfeed.tech/topics/post-training.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [distributed-training](<https://devfeed.tech/topics/distributed-training.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Inference](<https://devfeed.tech/topics/inference.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agent-tasks](<https://devfeed.tech/tags/agent-tasks.md>), [apache-2-0](<https://devfeed.tech/tags/apache-2-0.md>), [asynchronous](<https://devfeed.tech/tags/asynchronous.md>), [backend](<https://devfeed.tech/tags/backend.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [coding](<https://devfeed.tech/tags/coding.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [post-training](<https://devfeed.tech/tags/post-training.md>), [qwen3-8](<https://devfeed.tech/tags/qwen3-8.md>), [qwen3-8-27b](<https://devfeed.tech/tags/qwen3-8-27b.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [sessions](<https://devfeed.tech/tags/sessions.md>), [tasks](<https://devfeed.tech/tags/tasks.md>), [terminal](<https://devfeed.tech/tags/terminal.md>), [terminal-bench](<https://devfeed.tech/tags/terminal-bench.md>), [tokens](<https://devfeed.tech/tags/tokens.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

RadixArk reports that DaisyRose-27B, built on Qwen3.8-27B and trained with Miles, raised Terminal-Bench 4.1 pass@1 from 5.6% to 12.0%. Training used 1,200 curated long-horizon agent tasks over 18 days on 144 H200 GPUs; the tasks targeted original-model errors, required multi-step work, and were checked to ensure successful completion was necessary. The model retains the base architecture, tokenizer, chat template, image understanding, and 256K context window; checkpoints are Apache 2.0. RadixArk says its internal backend achieved 37% MFU versus 16% for the open-source Megatron-based trainer on comparable runs, and roughly twice the end-to-end throughput. Its sandbox service reported 99% creation success over nine weeks, exceeding 99.95% since late October, and about 80% lower infrastructure costs than popular cloud sandbox providers. The article recommends SGLang with DSpark speculative decoding and points to the model card for inference settings and evaluation details.

### Source excerpt

Today we are releasing **DaisyRose-27B**, RadixArk's first model, built on Qwen3.8-27B and trained with reinforcement learning using Miles. We trained it on long-horizon agent tasks with sessions up to 1M tokens, with just 1,200 curated tasks in 18 days on 144 H200 GPUs. DaisyRose-27B improves Terminal-Bench 4.1 pass@1 from **5.6% to 12.0%**. Beyond Terminal-Bench 4.1, it also delivers gains on a variety of other reasoning, instruction following, and coding benchmarks.

## "Collaboration over competition": Hugging Face hands model inference traffic to OpenRouter

DevFeed: ["Collaboration over competition": Hugging Face hands model inference traffic to OpenRouter](<https://devfeed.tech/articles/collaboration-over-competition-hugging-face-hands-model-inference-traffic-to-openrouter-140641.md>)

Original publisher: [Read original article](<https://thenewstack.io/hugging-face-openrouter-inference/>)

Author: Paul Sawers

Published: 2026-11-18T13:39:13Z

Content type: news

Language: en

Sources: [The New Stack](<https://devfeed.tech/sources/the-new-stack.md>)

Topics: [inference-providers](<https://devfeed.tech/topics/inference-providers.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [hugging face](<https://devfeed.tech/topics/hugging-face.md>)

Tags: [accessible](<https://devfeed.tech/tags/accessible.md>), [acquisition](<https://devfeed.tech/tags/acquisition.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-companies](<https://devfeed.tech/tags/ai-companies.md>), [ai-infrastructure](<https://devfeed.tech/tags/ai-infrastructure.md>), [ai-model](<https://devfeed.tech/tags/ai-model.md>), [ai-models](<https://devfeed.tech/tags/ai-models.md>), [anthropic](<https://devfeed.tech/tags/anthropic.md>), [api](<https://devfeed.tech/tags/api.md>), [api-keys](<https://devfeed.tech/tags/api-keys.md>), [audio](<https://devfeed.tech/tags/audio.md>), [collaboration](<https://devfeed.tech/tags/collaboration.md>), [competition](<https://devfeed.tech/tags/competition.md>), [faster](<https://devfeed.tech/tags/faster.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [inference](<https://devfeed.tech/tags/inference.md>), [large-language-models](<https://devfeed.tech/tags/large-language-models.md>), [model](<https://devfeed.tech/tags/model.md>), [model-inference](<https://devfeed.tech/tags/model-inference.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [openrouter](<https://devfeed.tech/tags/openrouter.md>), [partnership](<https://devfeed.tech/tags/partnership.md>), [traffic](<https://devfeed.tech/tags/traffic.md>)

### AI overview

Hugging Face now routes language-model requests through OpenRouter, which can select among providers based on cost, response speed, and availability, then retry through another provider if one fails. The integration covers 160 models from 38 AI labs and lets developers compare provider metrics, prioritize price or speed, and disable providers; organizations can apply provider restrictions to billed requests. The October routing overhaul OpenRouter cites reduced capacity-related errors by 46% and improved latency and token generation speed in one test, while raising average request cost by 6%. Those figures concern that overhaul, not necessarily Hugging Face traffic. Requests must now be billed through Hugging Face; developers cannot use their own third-party provider API keys, though Hugging Face says it adds no markup. Image, video, and audio generation remain on Hugging Face's existing infrastructure. Plans for direct OpenRouter account connections and broader task coverage have no announced timeline.

### Source excerpt

Hugging Face and OpenRouter have announced a partnership that promises to make AI model inference faster and more reliable for The post "Collaboration over competition": Hugging Face hands model inference traffic to OpenRouter appeared first on The New Stack.

## When Qini Rewards the Wrong Uplift Model, Use Doubly Robust Scores

DevFeed: [When Qini Rewards the Wrong Uplift Model, Use Doubly Robust Scores](<https://devfeed.tech/articles/when-qini-rewards-the-wrong-uplift-model-use-doubly-robust-scores-140795.md>)

Original publisher: [Read original article](<https://towardsdatascience.com/when-qini-rewards-the-wrong-uplift-model-use-doubly-robust-scores/>)

Author: Ivan Gorban

Published: 2026-11-18T13:30:00Z

Content type: article

Language: en

Sources: [Towards Data Science](<https://devfeed.tech/sources/towards-data-science.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [AI Engineering](<https://devfeed.tech/topics/ai-engineering.md>), [Inference](<https://devfeed.tech/topics/inference.md>)

Tags: [ate](<https://devfeed.tech/tags/ate.md>), [bias](<https://devfeed.tech/tags/bias.md>), [bootstrap](<https://devfeed.tech/tags/bootstrap.md>), [causal-inference](<https://devfeed.tech/tags/causal-inference.md>), [confidence](<https://devfeed.tech/tags/confidence.md>), [data-science](<https://devfeed.tech/tags/data-science.md>), [dr](<https://devfeed.tech/tags/dr.md>), [editor-s-picks](<https://devfeed.tech/tags/editor-s-picks.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [marketing-analytics](<https://devfeed.tech/tags/marketing-analytics.md>), [model](<https://devfeed.tech/tags/model.md>), [ranking](<https://devfeed.tech/tags/ranking.md>), [robust](<https://devfeed.tech/tags/robust.md>), [uplift](<https://devfeed.tech/tags/uplift.md>), [uplift-modeling](<https://devfeed.tech/tags/uplift-modeling.md>)

### AI overview

In a simulation of one million customers with selective treatment assignment, treatment/control differences made calibrated Qini favor a Linear DR ranking over the known true-CATE ranking (0.0856 vs. 0.0826), even though the true ranking had greater causal gain. The authors explain that Qini uses observed outcomes and does not remove selection bias; calibration aligns scores at one reference but preserves each metric's model ordering. IPW and doubly robust (DR) gain scores were closer to known gains in the simulation. IPW requires a correct propensity model; DR requires a correct propensity model or both outcome models. Both require consistency, no unmeasured confounding, and overlap, and remain subject to estimation and finite-sample error. In this specific comparison, the authors favor a boosted S-learner as a simple starting point: its DR-gain advantage over DML was small and uncertain, while only its comparison with T showed a paired interval excluding zero. They caution that the bootstrap held fitted models fixed and that the findings come from simulations, not a guarantee for other datasets.

### Source excerpt

On 1 million simulated customers, calibrated Qini scored a Linear DR learner 0.0856 against 0.0826 for the true-CATE ranking that should have won. The post When Qini Rewards the Wrong Uplift Model, Use Doubly Robust Scores appeared first on Towards Data Science.

## In-Warehouse Machine Learning and the Modern Data Stack | Blog | Fivetran

DevFeed: [In-Warehouse Machine Learning and the Modern Data Stack | Blog | Fivetran](<https://devfeed.tech/articles/in-warehouse-machine-learning-and-the-modern-data-stack-blog-fivetran-141332.md>)

Original publisher: [Read original article](<https://wf.fivetran.com/blog/in-warehouse-machine-learning-and-the-modern-data-stack>)

Author: Fivetran

Published: 2026-11-18T06:13:15Z

Content type: article

Language: en

Sources: [Fivetran Blog](<https://devfeed.tech/sources/fivetran-blog.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [AI Development](<https://devfeed.tech/topics/ai-development.md>), [Hyperparameter optimization](<https://devfeed.tech/topics/hyperparameter-optimization.md>), [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [Amazon SageMaker](<https://devfeed.tech/topics/amazon-sagemaker.md>), [Tensorflow](<https://devfeed.tech/topics/tensorflow.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [airflow](<https://devfeed.tech/tags/airflow.md>), [analytics](<https://devfeed.tech/tags/analytics.md>), [automated-machine-learning](<https://devfeed.tech/tags/automated-machine-learning.md>), [aws](<https://devfeed.tech/tags/aws.md>), [bigquery](<https://devfeed.tech/tags/bigquery.md>), [data-engineering](<https://devfeed.tech/tags/data-engineering.md>), [data-ingestion](<https://devfeed.tech/tags/data-ingestion.md>), [data-lake](<https://devfeed.tech/tags/data-lake.md>), [data-lakehouse](<https://devfeed.tech/tags/data-lakehouse.md>), [data-lakehouse-architecture](<https://devfeed.tech/tags/data-lakehouse-architecture.md>), [data-pipelines](<https://devfeed.tech/tags/data-pipelines.md>), [data-science](<https://devfeed.tech/tags/data-science.md>), [data-warehouse](<https://devfeed.tech/tags/data-warehouse.md>), [data-warehousing](<https://devfeed.tech/tags/data-warehousing.md>), [databricks](<https://devfeed.tech/tags/databricks.md>), [delta-lake](<https://devfeed.tech/tags/delta-lake.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [modern-data-stack](<https://devfeed.tech/tags/modern-data-stack.md>)

### AI overview

A modern data science stack can bring analytics and machine learning workflows together around governed warehouse data, reducing duplicated processing and the risk of training-serving skew. BigQuery ML and Redshift ML extend SQL with CREATE MODEL workflows for automated or custom models; custom models must be TensorFlow models in BigQuery and deployed through SageMaker for Redshift ML. Both support invoking trained models in SELECT queries and storing predictions in warehouse tables. Snowflake describes an extensibility approach that integrates with tools such as SageMaker and Databricks. Delta Lake takes the reverse path, adding analytics features such as ACID transactions to data lakes as part of a lakehouse architecture. The excerpt presents these as different ways to consolidate data work and pipelines.

### Source excerpt

We evaluate various in-data warehouse machine learning services, and discuss each of their benefits and requirements.

## When the tail is earned by its companions: measuring label dependence in long-tailed multi-label classification

DevFeed: [When the tail is earned by its companions: measuring label dependence in long-tailed multi-label classification](<https://devfeed.tech/articles/when-the-tail-is-earned-by-its-companions-measuring-label-dependence-in-long-tailed-multi-label-classification-141814.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22859>)

Author: Christoffer L\\"offler

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [model architecture](<https://devfeed.tech/topics/model-architecture.md>), [Inference](<https://devfeed.tech/topics/inference.md>)

Tags: [architecture](<https://devfeed.tech/tags/architecture.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [classification](<https://devfeed.tech/tags/classification.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-ir](<https://devfeed.tech/tags/cs-ir.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [measuring](<https://devfeed.tech/tags/measuring.md>), [multi-label-classification](<https://devfeed.tech/tags/multi-label-classification.md>)

### AI overview

The study finds that long-tailed multi-label classifiers can appear to recover rare labels by relying on their frequent co-occurring labels. It proposes a within-label measure of this dependence, a rule that counts reduced dependence only when recall without companions improves, and a training-label statistic that bounds how much reported recall may exceed companion-free recall before training. Across 23 corpora, seven objectives, and five families of label-correlation mechanisms, companion-free recall was lower on all six corpora where the measure was defined; up to 18.4% of credited tail true positives would disappear at that rate, within the proposed bound on every corpus. No long-tail objective reduced dependence, but architecture affected it; models scoring tail labels from other labels' predictions were strongly dependent.

### Source excerpt

arXiv:2611.22859v1 Announce Type: new Abstract: A rare label seldom occurs alone, and a classifier can recover it from the frequent labels it co-occurs with, its companions, without having learned the label itself. Existing diagnostics of this need a trained model: none says before training whether a corpus admits the failure, or what the standard aggregate hides. Macro-F1, like any measure built from per-label confusion counts, is unchanged by how a label's recall divides between occurrences with and without a companion. We introduce a within-label estimand of that division, a reading rule under which a fall in dependence counts only if companion-free recall rises, and a statistic of training labels that bounds, before training, how far reported recall can exceed companion-free recall. Across 23 corpora, 7 training objectives and five families of label-correlation mechanisms, companion-free recall falls short on all six corpora where the estimand is defined. Up to 18.4% of the tail true positives macro-F1 credits would vanish at the companion-free rate, within the training-label bound on every corpus. No long-tail objective reduces the dependence, but architecture moves it. Models whose tail score is computed from other labels' predictions are strongly dependent.

## Residual-Guided Lightweight Reinforcement Learning for Risk-Aware Trajectory Selection with a Frozen World Model

DevFeed: [Residual-Guided Lightweight Reinforcement Learning for Risk-Aware Trajectory Selection with a Frozen World Model](<https://devfeed.tech/articles/residual-guided-lightweight-reinforcement-learning-for-risk-aware-trajectory-selection-with-a-frozen-world-model-140105.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22998>)

Author: Jiaqi Wu, Yongqi Dong, Yijie Zhong, Maple Shao, Yide Tao, Linsen Cai, Weize Cai, Hai L. Vu, Alvaro Garcia Hernandez

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [World models](<https://devfeed.tech/topics/world-models.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Inference](<https://devfeed.tech/topics/inference.md>)

Tags: [automated-driving](<https://devfeed.tech/tags/automated-driving.md>), [candidate](<https://devfeed.tech/tags/candidate.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-et](<https://devfeed.tech/tags/cs-et.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [cs-sy](<https://devfeed.tech/tags/cs-sy.md>), [eess-sy](<https://devfeed.tech/tags/eess-sy.md>), [error](<https://devfeed.tech/tags/error.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [exploration](<https://devfeed.tech/tags/exploration.md>), [features](<https://devfeed.tech/tags/features.md>), [feedback](<https://devfeed.tech/tags/feedback.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [fixed](<https://devfeed.tech/tags/fixed.md>), [framework](<https://devfeed.tech/tags/framework.md>), [inference](<https://devfeed.tech/tags/inference.md>), [learning](<https://devfeed.tech/tags/learning.md>), [lightweight](<https://devfeed.tech/tags/lightweight.md>), [model](<https://devfeed.tech/tags/model.md>), [network](<https://devfeed.tech/tags/network.md>), [ppo](<https://devfeed.tech/tags/ppo.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [risk](<https://devfeed.tech/tags/risk.md>), [selection](<https://devfeed.tech/tags/selection.md>), [world-model](<https://devfeed.tech/tags/world-model.md>)

### AI overview

The method improves trajectory selection while keeping the world model frozen: PPO trains a policy over fixed candidates and bounded residuals, while a context-conditioned ranking network learns candidate preferences from local perturbation feedback. At inference, fixed weights combine learned-preference and predicted-risk ranks, and the chosen candidate is returned unmodified. Training updates 16,106 actor-and-critic parameters; deployment keeps the 11,725-parameter ranking network, with the residual branch and critic used only in training. In open-loop nuScenes validation, L2 error falls 3.99% versus RiskWorld without RL selection. With matched feedback and update budgets, residual exploration improves L2 by 0.83% over categorical PPO without residuals. The method also has lower L2 than RiskWorld when risk features are removed or shuffled.

### Source excerpt

arXiv:2611.22998v1 Announce Type: new Abstract: In automated driving, world models provide future scene predictions for evaluating candidate trajectories, but fixed selection rules may not capture scene-dependent trade-offs. We propose a residual-guided reinforcement learning (RL) framework for lightweight trajectory selection with a frozen world model and a fixed candidate bank. The framework uses proximal policy optimization (PPO) to fine-tune a joint policy over candidate selection and bounded trajectory residuals. A context-conditioned ranking network learns candidate preferences from feedback obtained by evaluating local perturbations of the selected trajectories. At inference, the selector combines learned preference and predicted risk ranks using fixed weights and returns the selected candidate trajectory without modification. Fine-tuning updates 16,106 parameters across the actor and critic, while the deployed selector retains only the 11,725-parameter ranking network, with the residual branch and critic used exclusively during training. In open-loop evaluation on the nuScenes validation set, the method reduces average Euclidean trajectory error (L2) by 3.99% relative to the RiskWorld model baseline without RL-based selection. Under matched budgets for feedback accesses and actor updates, residual exploration reduces L2 by 0.83% relative to the categorical PPO variant trained without residuals. The method also maintains lower L2 than RiskWorld when risk features are removed or shuffled.

## NVIDIA Kumo Tabular: Pushing the Accuracy$\\unicode{8211}$Efficiency Frontier for Tabular Prediction

DevFeed: [NVIDIA Kumo Tabular: Pushing the Accuracy$\\unicode{8211}$Efficiency Frontier for Tabular Prediction](<https://devfeed.tech/articles/nvidia-kumo-tabular-pushing-the-accuracy-unicode-8211-efficiency-frontier-for-tabular-prediction-141830.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23001>)

Author: Jingang Qu, Valter Hudovernik, Martin Jurkovi\\v{c}, Cedric Lorenz, David Holzm\\"uller, Federico Lopez, Dmitry Gordeev, Akihiro Nitta, Ramona Bendias, Ardrian Wong, Vedaant Jain, Vignesh Kothapalli, Gi

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [Open Source Models & Datasets](<https://devfeed.tech/topics/open-source-models-datasets.md>)

Tags: [apache](<https://devfeed.tech/tags/apache.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [classification](<https://devfeed.tech/tags/classification.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [family](<https://devfeed.tech/tags/family.md>), [faster](<https://devfeed.tech/tags/faster.md>), [images](<https://devfeed.tech/tags/images.md>), [in-context-learning](<https://devfeed.tech/tags/in-context-learning.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [probabilistic](<https://devfeed.tech/tags/probabilistic.md>), [regression](<https://devfeed.tech/tags/regression.md>), [speed](<https://devfeed.tech/tags/speed.md>), [strings](<https://devfeed.tech/tags/strings.md>), [text](<https://devfeed.tech/tags/text.md>), [training-stability](<https://devfeed.tech/tags/training-stability.md>), [transformer](<https://devfeed.tech/tags/transformer.md>)

### AI overview

Kumo Tabular is a family of classification and regression models that predicts labels for new tables without task-specific training. It combines a row encoder with an in-context transformer and is pretrained on artificial tables from the Kumo prior. Its three sizes range from 27M to 214M parameters; accuracy improved with size in all tested comparisons. The abstract reports that Kumo Tabular-L ranked first on eight benchmarks, spanning probabilistic regression, very large or wide tables, and tables with strings, text, or images. All three sizes were on the accuracy-speed frontiers of TabArena and BeyondArena; the large model was reported 36 times faster than TabArena runner-up LimiX-2 and 1.7 times faster than BeyondArena runner-up TabPFN-3.5. The supplied text is an abstract, so it does not provide full methods or benchmark details. Weights are released under OpenMDW-1.1 and pretraining code under Apache 2.0.

### Source excerpt

arXiv:2611.23001v1 Announce Type: new Abstract: We introduce Kumo Tabular, a family of open tabular foundation models for classification and regression, in three sizes: Kumo Tabular-S (27M parameters), Kumo Tabular-M (61M), and Kumo Tabular-L (214M). Pretrained only on artificial tables from the Kumo prior, they can make predictions on new tables without any training. In our approach, a table encoder compresses each row into an embedding, and an in-context learning transformer predicts the unlabeled rows from the labeled ones. Kumo Tabular introduces architectural innovations that improve both training stability and predictive performance, enabling the model to scale up to 150M cells on modern GPUs. Kumo Tabular-L ranks first on eight benchmarks: TabArena, TALENT, BeyondArena, ScoringBench, STRABLE, MulTaBench, RamanBench, and TabBench-Bio. These cover probabilistic regression, tables with over 1M rows or 100K columns, and tables with strings, text, or images. Accuracy improves with model size in all tested comparisons. All three sizes lie on the accuracy-speed frontiers of TabArena and BeyondArena. Kumo Tabular-L predicts faster than the runners-up, LimiX-2 on TabArena (36 times) and TabPFN-3.5 on BeyondArena (1.7 times). Weights and pretraining code are released under OpenMDW-1.1 and Apache 2.0, respectively.

## KneeFS-Net: A Lightweight Deep Learning Model for Knee Osteoarthritis Severity Assessment with Cross-Domain External Validation

DevFeed: [KneeFS-Net: A Lightweight Deep Learning Model for Knee Osteoarthritis Severity Assessment with Cross-Domain External Validation](<https://devfeed.tech/articles/kneefs-net-a-lightweight-deep-learning-model-for-knee-osteoarthritis-severity-assessment-with-cross-domain-external-validation-138972.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.19988>)

Author: Shafqat Alam, Galib Anjum Talukder Mahi, Yeasir Rabbani Tanvir, Md. Ibrahim, Md. Shahed Hossain, Shashank Ghimire, Mahmud Hasan Mostofa Kamal, ASM Shahidul Hossain, Taufiq Hasan

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>), [Image processing](<https://devfeed.tech/topics/image-processing.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Inference](<https://devfeed.tech/topics/inference.md>)

Tags: [assessment](<https://devfeed.tech/tags/assessment.md>), [cross-validation](<https://devfeed.tech/tags/cross-validation.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [domain](<https://devfeed.tech/tags/domain.md>), [external](<https://devfeed.tech/tags/external.md>), [features](<https://devfeed.tech/tags/features.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [lightweight](<https://devfeed.tech/tags/lightweight.md>), [model](<https://devfeed.tech/tags/model.md>), [pretraining](<https://devfeed.tech/tags/pretraining.md>), [validation](<https://devfeed.tech/tags/validation.md>)

### AI overview

KneeFS-Net is a compact model for grading knee osteoarthritis on radiographs using the Kellgren-Lawrence scale. It combines spatial, Fourier and state-space features with landmark-guided attention, using 0.8 million parameters and 0.33 GFLOPs. On the OAI test set it achieved QWK 0.863, MAE 0.325 and 97.2% within-one-grade accuracy, comparable to models with 9-35 times more parameters. Cross-population performance varied: without adaptation, the OAI-trained model reached QWK 0.803 on a single-center Bangladeshi hospital cohort, but QWK 0.656 on multi-center teleradiology images. Self-supervised pretraining on 6,092 unlabeled radiographs followed by joint fine-tuning achieved QWK 0.823 and 99.7% within-one-grade accuracy in cross-validation. The abstract suggests adaptation with limited annotation may help, while the excerpt does not provide further methodological or validation details.

### Source excerpt

arXiv:2611.19988v1 Announce Type: cross Abstract: Knee osteoarthritis (OA) grading from radiographs using the Kellgren-Lawrence (KL) scale requires expertise that may be scarce. We present KneeFS-Net, a lightweight ordinal grader (0.8 million parameters; 0.33 GFLOPs) combining spatial, Fourier and state-space features with landmark-guided attention. On the Osteoarthritis Initiative (OAI) test set, KneeFS-Net achieved a quadratic weighted kappa (QWK) of 0.863, mean absolute error (MAE) of 0.325 and 97.2% within-one-grade accuracy, comparable to models with 9-35-fold more parameters. We evaluated cross-population generalization on two Bangladeshi cohorts, a single-center hospital cohort and a multi-center teleradiology cohort. Without further training or adaptation, the OAI-trained model achieved QWK 0.803, MAE 0.339 and 98.2% within-one-grade accuracy on its hospital cohort, declining to QWK 0.656 and MAE 0.522 on multi-center teleradiology images. Self-supervised pretraining on 6,092 unlabeled radiographs followed by joint fine-tuning achieved QWK 0.823, MAE 0.329 and 99.7% within-one-grade accuracy in cross-validation. These findings demonstrate accurate, efficient KL grading and potential adaptation to new populations with limited annotation.

## When Do Histology Foundation Models Need Geometric Test-Time Augmentation?

DevFeed: [When Do Histology Foundation Models Need Geometric Test-Time Augmentation?](<https://devfeed.tech/articles/when-do-histology-foundation-models-need-geometric-test-time-augmentation-139192.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.21565>)

Author: Lars Chen, Valentino F. Foit, Dorry Segev, Michal Mankowski

Published: 2026-11-17T05:00:00Z

Content type: news

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [AI Foundation Models](<https://devfeed.tech/topics/ai-foundation-models.md>), [Inference](<https://devfeed.tech/topics/inference.md>)

Tags: [benchmark](<https://devfeed.tech/tags/benchmark.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [entropy](<https://devfeed.tech/tags/entropy.md>), [foundation-models](<https://devfeed.tech/tags/foundation-models.md>), [models](<https://devfeed.tech/tags/models.md>), [tta](<https://devfeed.tech/tags/tta.md>)

### AI overview

Across the benchmark, D₄ test-time augmentation (pooling predictions across eight rotations and reflections) remained a post hoc improvement, but its benefit was smaller for histology foundation models than general-purpose backbones: on TCGA-UT, +0.50 versus +1.98 percentage points, and smaller still for the largest models. Entropy routing--applying augmentation only to uncertain predictions--recovered most of the gain while augmenting about 13% of samples. Patch-level improvements did not transfer to slide-level multiple instance learning. The abstract reports results across nine histology foundation models, six general-purpose models, four patch-level datasets and two whole-slide datasets.

### Source excerpt

arXiv:2611.21565v1 Announce Type: cross Abstract: Test-time augmentation (TTA) aggregates model predictions across multiple transformed views of an input during inference. Because histology images lack a canonical orientation, $D_4$ TTA---pooling across the 8 rotations and reflections of a square---is a standard default in computational pathology. However, the rise of self-supervised histology foundation models (FMs) with inherent rotational invariance calls its continued utility into question. We benchmark 9 histology FMs and 6 general-purpose models across four patch-level and two whole-slide image (WSI) datasets. We find that $D_4$ TTA remains a reliable post hoc improvement, but benefits diminish from $+1.98$ percentage points (pp) for general-purpose backbones to $+0.50$ pp for histology FMs on TCGA-UT, and shrink further for the largest models. To eliminate computational overhead, our experiments demonstrate that entropy routing---applying TTA selectively to uncertain predictions---recovers most of the gain while augmenting only ${\sim}13\%$ of samples. Finally, we show that patch-level TTA improvements do not transfer to slide-level multiple instance learning (MIL). Code: https://github.com/lars-chen/histology-tta

## CrypTorch: Unified PyTorch-based Auto-tuning Compiler for Private Machine Learning Inference

DevFeed: [CrypTorch: Unified PyTorch-based Auto-tuning Compiler for Private Machine Learning Inference](<https://devfeed.tech/articles/cryptorch-unified-pytorch-based-auto-tuning-compiler-for-private-machine-learning-inference-139225.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2511.19711>)

Author: Jinyu Liu, Gang Tan, Kiwan Maeng

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [Cryptography](<https://devfeed.tech/topics/cryptography.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [FHE](<https://devfeed.tech/topics/fhe.md>), [PyTorch](<https://devfeed.tech/topics/pytorch.md>), [Code generation](<https://devfeed.tech/topics/code-generation.md>)

Tags: [compiler](<https://devfeed.tech/tags/compiler.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cs-pl](<https://devfeed.tech/tags/cs-pl.md>), [fhe](<https://devfeed.tech/tags/fhe.md>), [inference](<https://devfeed.tech/tags/inference.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [mpc](<https://devfeed.tech/tags/mpc.md>), [performance](<https://devfeed.tech/tags/performance.md>), [private](<https://devfeed.tech/tags/private.md>), [pytorch](<https://devfeed.tech/tags/pytorch.md>), [share](<https://devfeed.tech/tags/share.md>)

### AI overview

CrypTorch addresses the cost and accuracy limits of globally fixed operator approximations in MPC-based machine learning. Its compiler estimates costs from the target MPC backend and quality through proxy PyTorch execution, then auto-selects approximations to maximize performance under a quality constraint. The abstract reports 1.14-1.60x performance improvement across multiple MPC backends with no accuracy loss, or 1.17-1.92x with less than 3% accuracy degradation. Its shared compiler design supports multiple backends, and its auto-tuner also reduces bootstrapping operations by 1.31-1.65x on a related FHE backend.

### Source excerpt

arXiv:2511.19711v3 Announce Type: replace Abstract: MPC-based ML uses secure multi-party computation (MPC) to execute machine learning (ML) workloads across multiple parties without requiring any party to reveal its private data or model parameters. Our characterization study finds that the performance of state-of-the-art MPC frameworks is often dominated by operator-approximation strategies, where ML operators not natively supported by MPC are replaced with iterative methods or polynomials. These approximations are typically chosen heuristically by each framework and applied globally. As a result, they can be unnecessarily expensive in some cases, yet insufficiently accurate in others. We find that the best approximation choice depends on the context, target framework, and application, making a single globally fixed approximation fundamentally suboptimal. To address this problem, we design CrypTorch, a multi-stage compiler for MPC-based ML that supports multiple MPC frameworks as backends. At the core of CrypTorch is its auto-tuning stage, which extracts cost estimates from the target MPC backend and quality estimates from a proxy PyTorch execution, and uses them to automatically select a set of approximations that maximizes performance while satisfying the given quality constraint. We show that CrypTorch's auto-tuner improves performance across multiple MPC backends by 1.14--1.60x without degrading accuracy, and by 1.17--1.92x with less than 3% accuracy degradation. CrypTorch's unified design also improves programmability and usability by allowing multiple backends to share other common compiler functionality and optimizations. We further show that the auto-tuner extends to a related fully homomorphic encryption (FHE) backend, reducing the number of bootstrapping operations by 1.31--1.65x.

## BrownoutMoE: Learning Structure-Aware Expert Grouping for Adaptive Mixture-of-Experts Model Serving

DevFeed: [BrownoutMoE: Learning Structure-Aware Expert Grouping for Adaptive Mixture-of-Experts Model Serving](<https://devfeed.tech/articles/brownoutmoe-learning-structure-aware-expert-grouping-for-adaptive-mixture-of-experts-model-serving-136963.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2607.04164>)

Author: Yi Ding, Minxian Xu, Zhengxin Fang, Kejiang Ye, Chengzhong Xu

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Post-training optimization](<https://devfeed.tech/topics/post-training-optimization.md>)

Tags: [a100](<https://devfeed.tech/tags/a100.md>), [adaptive](<https://devfeed.tech/tags/adaptive.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [clusters](<https://devfeed.tech/tags/clusters.md>), [controller](<https://devfeed.tech/tags/controller.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [decode](<https://devfeed.tech/tags/decode.md>), [demand](<https://devfeed.tech/tags/demand.md>), [distillation](<https://devfeed.tech/tags/distillation.md>), [execution](<https://devfeed.tech/tags/execution.md>), [experts](<https://devfeed.tech/tags/experts.md>), [framework](<https://devfeed.tech/tags/framework.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [gpu-clusters](<https://devfeed.tech/tags/gpu-clusters.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [inference](<https://devfeed.tech/tags/inference.md>), [learning](<https://devfeed.tech/tags/learning.md>), [mixture-of-experts](<https://devfeed.tech/tags/mixture-of-experts.md>), [model-serving](<https://devfeed.tech/tags/model-serving.md>), [serving](<https://devfeed.tech/tags/serving.md>), [structure](<https://devfeed.tech/tags/structure.md>)

### AI overview

BrownoutMoE targets underused GPU parallelism caused by imbalanced expert access in mixture-of-experts serving. It learns layer-wise expert groups offline with reinforcement learning, using post-distillation error as its reward; grouping-consistent distillation then trains one united expert per group. An online controller adjusts how often united experts replace cold experts to meet latency objectives. In tests on eight A100 GPUs, it reduced accuracy degradation by up to 71.4% versus index-order grouping, improved unfused throughput by up to 2.24x over vLLM, and lowered saturated decode latency by 38%.

### Source excerpt

arXiv:2607.04164v2 Announce Type: replace Abstract: Mixture-of-Experts (MoE) large language models (LLMs) are increasingly served on multi-GPU clusters in cloud data centers, where inference must stay accurate and responsive under bursty demand. Expert access in MoE inference is highly imbalanced: a few hot experts process most tokens, whereas many cold experts fragment kernels and leave GPU parallelism underused. Existing serving systems optimize runtime execution but keep the original expert organization. We propose BrownoutMoE, a structure-aware serving framework inspired by the brownout paradigm in cloud computing. It learns layer-wise expert grouping offline with reinforcement learning, using the post-distillation error as the reward. Grouping-consistent distillation trains one united expert per group, and an online controller adjusts how often united experts replace cold experts under latency objectives. On eight A100 GPUs, BrownoutMoE reduces accuracy degradation by up to 71.4% relative to index-order grouping, improves unfused throughput by up to 2.24x over vLLM, and lowers saturated decode latency by 38%.

## Microfitting: Continual Learning via Cached Test-Time Training

DevFeed: [Microfitting: Continual Learning via Cached Test-Time Training](<https://devfeed.tech/articles/microfitting-continual-learning-via-cached-test-time-training-137553.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.20258>)

Author: Nick Ryan, Peter A. Devine, Ben Sirb, Hannah Sansford, Alex Chiocchi

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Inference](<https://devfeed.tech/topics/inference.md>)

Tags: [adapter](<https://devfeed.tech/tags/adapter.md>), [continual-learning](<https://devfeed.tech/tags/continual-learning.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [private](<https://devfeed.tech/tags/private.md>), [test](<https://devfeed.tech/tags/test.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

Microfitting specializes a frozen language model to private documents by training and caching a small adapter for each document. Each adapter is trained offline once on synthetic questions aimed at demonstrated model shortcomings, then reused when that document is queried. Keeping adapters isolated is intended to avoid overwriting the base model or prior document adaptations; the process for generating training data also revises its recipe based on previous outcomes and fresh workload examples. On DeepSeek V4 Flash, the abstract reports a 38.2% relative accuracy improvement over the base model on Enterprise Document Operations, an open-book Q&A benchmark using real-world enterprise documents. It describes adapting overnight on local hardware, but the supplied abstract does not provide further implementation details or comparative results.

### Source excerpt

arXiv:2611.20258v1 Announce Type: cross Abstract: Continual learning is a promising approach to specialize language models to an organization's private data. However, current methods exhibit a number of drawbacks including catastrophic forgetting, where new training updates overwrite prior knowledge. We present microfitting, which trains and caches a small adapter for each document. Each adapter is trained once, offline, on synthetic questions targeted at demonstrated base-model shortcomings and reused whenever that document is queried. This method borrows from test-time learning literature and sidesteps the problem of continual learning through parameter isolation, allowing the system to specialize on new data without overwriting prior knowledge. The process that generates adapter training data also improves over time, revising its recipe from previous outcomes and fresh examples of the target workload. On DeepSeek V4 Flash, microfitting substantially improves accuracy over the base model. In Enterprise Document Operations (EDO), an open-book Q&A environment over real-world enterprise documents, our method achieves a 38.2% relative improvement over base-model performance. Microfitting makes continual specialization practical and cost-efficient: a frozen model can adapt to an evolving private corpus overnight on local hardware, without modifying the base model or previously cached adapters.

## Making Koopman Control Guarantees Hold on Robots

DevFeed: [Making Koopman Control Guarantees Hold on Robots](<https://devfeed.tech/articles/making-koopman-control-guarantees-hold-on-robots-137568.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.20402>)

Author: Abulikemu Abuduweili, Changliu Liu

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Robotics](<https://devfeed.tech/topics/robotics.md>), [Inference](<https://devfeed.tech/topics/inference.md>)

Tags: [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [constraint](<https://devfeed.tech/tags/constraint.md>), [constraint-satisfaction](<https://devfeed.tech/tags/constraint-satisfaction.md>), [control](<https://devfeed.tech/tags/control.md>), [controller](<https://devfeed.tech/tags/controller.md>), [controls](<https://devfeed.tech/tags/controls.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [cs-sy](<https://devfeed.tech/tags/cs-sy.md>), [eess-sy](<https://devfeed.tech/tags/eess-sy.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [error](<https://devfeed.tech/tags/error.md>), [guarantees](<https://devfeed.tech/tags/guarantees.md>), [invariant](<https://devfeed.tech/tags/invariant.md>), [linear](<https://devfeed.tech/tags/linear.md>), [local](<https://devfeed.tech/tags/local.md>), [margin](<https://devfeed.tech/tags/margin.md>), [matching](<https://devfeed.tech/tags/matching.md>), [model](<https://devfeed.tech/tags/model.md>), [noise](<https://devfeed.tech/tags/noise.md>), [nominal](<https://devfeed.tech/tags/nominal.md>), [offline](<https://devfeed.tech/tags/offline.md>), [open](<https://devfeed.tech/tags/open.md>), [robots](<https://devfeed.tech/tags/robots.md>)

### AI overview

A Koopman controller can appear stable in its learned latent model while destabilizing the nonlinear robot plant because deployment repeatedly re-encodes measured states. The paper gives a stability condition based on first-order agreement between learned and true closed-loop dynamics along the embedding's tangent space, plus a robustness margin for approximate matching; it also shows a stable open-loop latent model cannot exactly match an unstable equilibrium. Tangent-Invariant Koopman (TIK) estimates a local plant linearization from offline transitions and enforces that match. In four simulated unstable benchmarks, all 40 Standard Koopman controllers were locally unstable on the plant, while all 40 TIK controllers stabilized it locally. The empirical-curvature stability test was inconclusive under high noise. With explicit model-error and curvature bounds, an invariant terminal set guarantees constraint satisfaction after entry under local feedback. Reported results include 88% constrained cart-pole swing-up success at a 0.25 ms planning budget and 1-5 ms per control step on simulated seven-joint arms.

### Source excerpt

arXiv:2611.20402v1 Announce Type: cross Abstract: A Koopman controller can stabilize its learned latent-space linear model yet fail to stabilize the nonlinear plant. The learned model propagates predictions in latent space, while the deployed controller repeatedly re-encodes the measured physical state. We show that nominal closed-loop stability transfers to the nonlinear plant when the learned and true closed-loop dynamics agree to first order on the embedding's tangent space at the target. We derive a robustness margin for approximate matching and show that a stable open-loop latent model cannot exactly match an unstable equilibrium. We introduce Tangent-Invariant Koopman (TIK), which estimates a local plant linearization from offline transitions and enforces first-order matching to this estimate. TIK recovers the fitted LQR gain at the target, while a bilinear predictor captures state-dependent actuation during large maneuvers. Across four simulated unstable benchmarks, all 40 Standard Koopman controllers have stable latent closed loops, but their deployed nonlinear systems are locally unstable. In contrast, all 40 TIK controllers locally stabilize their respective nonlinear plants. An offline stability test using empirical curvature estimates identifies stable TIK controllers on two cart-pole systems, but is inconclusive under high noise. Under explicit model-error and curvature bounds, an invariant terminal set guarantees constraint satisfaction after entry under local feedback. TIK achieves 88\% constrained cart-pole swing-up success with a 0.25 ms planning budget and controls simulated seven-joint arms within 1-5 ms per step.

## A Temporal Convolutional Network with Multi-Head Attention Model: A Surrogate for Parallel DEVS Semiconductor Fab Simulation

DevFeed: [A Temporal Convolutional Network with Multi-Head Attention Model: A Surrogate for Parallel DEVS Semiconductor Fab Simulation](<https://devfeed.tech/articles/a-temporal-convolutional-network-with-multi-head-attention-model-a-surrogate-for-parallel-devs-semiconductor-fab-simulation-139063.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.20493>)

Author: Vamsi Krishna Pendyala, Hessam S. Sarjoughian, Edward J. Yellig, Bharadhwaj Balasubramaniam

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [model architecture](<https://devfeed.tech/topics/model-architecture.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [model-deployment](<https://devfeed.tech/topics/model-deployment.md>)

Tags: [architecture](<https://devfeed.tech/tags/architecture.md>), [architectures](<https://devfeed.tech/tags/architectures.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [attention](<https://devfeed.tech/tags/attention.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [capacity](<https://devfeed.tech/tags/capacity.md>), [capacity-planning](<https://devfeed.tech/tags/capacity-planning.md>), [cost](<https://devfeed.tech/tags/cost.md>), [counts](<https://devfeed.tech/tags/counts.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [days](<https://devfeed.tech/tags/days.md>), [event](<https://devfeed.tech/tags/event.md>), [execution](<https://devfeed.tech/tags/execution.md>), [factory](<https://devfeed.tech/tags/factory.md>), [model](<https://devfeed.tech/tags/model.md>), [network](<https://devfeed.tech/tags/network.md>), [parallel](<https://devfeed.tech/tags/parallel.md>), [semiconductor](<https://devfeed.tech/tags/semiconductor.md>), [simulation](<https://devfeed.tech/tags/simulation.md>), [temporal](<https://devfeed.tech/tags/temporal.md>)

### AI overview

The proposed Temporal Convolutional Network with Multi-head Attention (TCNA) predicts semiconductor-fab throughput and turnaround-time trajectories from simulated Parallel DEVS data. The study uses wafer lot size, arrival profile and maintenance policy as static covariates, testing configurable lots with uniform or sinusoidal arrivals and possible preventive maintenance. On selected trajectories, TCNA's MAPE, MSE and R² results are comparable to or better than other architectures with similar parameter counts and training epoch times; its size and execution time are also similar to the Temporal Fusion Transformer. Predictions are less accurate than high-resolution simulation, but the abstract reports orders-of-magnitude faster execution, which may support more frequent scheduling, capacity and maintenance planning.

### Source excerpt

arXiv:2611.20493v1 Announce Type: new Abstract: Semiconductor factories process many kinds of wafers organized into lots of various sizes and configurations. Different kinds of machines, each with its own operations, process wafers for hours to days, and the wafers undergo many incremental fabrication steps (with re-entry) and flows spanning days to weeks across connected machines. Fast and accurate predictions of throughput ($TH$) and turnaround time ($TA$) dynamics for individual machines and their combinations are essential for factory operations. To use machine learning methods for this purpose, this paper presents a Temporal Convolutional Network with Multi-head Attention (TCNA) architecture. It uses simulated datasets from a benchmark Parallel DEVS (PDEVS) semiconductor fabrication factory model. The factory processes configurable wafer lots with uniform and sinusoidal wafer arrival profiles and may undergo preventive maintenance. The wafer lot size, arrival profile, and factory maintenance policy are used as static covariates. TCNA is applied to a selected set of simulated discrete-event throughput and turnaround time trajectories. The $TH$ and $TA$ time series predicted by TCNA, evaluated with the common $MAPE$, $MSE$, and $R^2$ measures, show comparable or higher accuracy relative to other model architectures, with similar parameter counts and epoch times; the TCNA model size and execution time are similar to those of other architectures such as the Temporal Fusion Transformer. Although time series predictions are less accurate than high-resolution simulations, they can be deployed at a much lower computational cost. The TCNA achieves orders-of-magnitude speedup over PDEVS simulation with an acceptable loss of accuracy, enabling frequent product scheduling, capacity planning, and maintenance planning.

[Next page](<https://devfeed.tech/topics/inference.md?cursor=WyIyMDI2LTExLTE3VDA1OjAwOjAwKzAwOjAwIiwgIjY5NDA1YTkyLTliMzUtNDc5MC05ODA3LWNmNjY3YjZlZWI5ZCJd>)