# Data Science

Data science is a field that combines domain expertise, programming, mathematics, and statistics to extract meaningful insights from data.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Clairic Hit 98% Accuracy on 72.8% of Questions via Confidence Scores

DevFeed: [Clairic Hit 98% Accuracy on 72.8% of Questions via Confidence Scores](<https://devfeed.tech/articles/clairic-hit-98-accuracy-on-72-8-of-questions-via-confidence-scores-146441.md>)

Original publisher: [Read original article](<https://towardsdatascience.com/clairic-hit-98-accuracy-on-728-of-questions-via-confidence-scores/>)

Author: Mariya Mansurova

Published: 2026-11-20T15:00:00Z

Content type: tutorial

Language: en

Sources: [Towards Data Science](<https://devfeed.tech/sources/towards-data-science.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Data Science](<https://devfeed.tech/topics/data-science.md>), [ai-experiments](<https://devfeed.tech/topics/ai-experiments.md>), [Benchmark](<https://devfeed.tech/topics/benchmark.md>)

Tags: [analysts](<https://devfeed.tech/tags/analysts.md>), [arithmetic](<https://devfeed.tech/tags/arithmetic.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [autocomplete](<https://devfeed.tech/tags/autocomplete.md>), [based](<https://devfeed.tech/tags/based.md>), [bike](<https://devfeed.tech/tags/bike.md>), [charts](<https://devfeed.tech/tags/charts.md>), [chatgpt](<https://devfeed.tech/tags/chatgpt.md>), [classification](<https://devfeed.tech/tags/classification.md>), [cloudflare](<https://devfeed.tech/tags/cloudflare.md>), [complex](<https://devfeed.tech/tags/complex.md>), [confidence](<https://devfeed.tech/tags/confidence.md>), [confidence-interval](<https://devfeed.tech/tags/confidence-interval.md>), [control](<https://devfeed.tech/tags/control.md>), [data-science](<https://devfeed.tech/tags/data-science.md>), [decision-models](<https://devfeed.tech/tags/decision-models.md>), [easy](<https://devfeed.tech/tags/easy.md>), [editor-s-picks](<https://devfeed.tech/tags/editor-s-picks.md>), [generative-ai](<https://devfeed.tech/tags/generative-ai.md>), [jev](<https://devfeed.tech/tags/jev.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-applications](<https://devfeed.tech/tags/llm-applications.md>), [luna](<https://devfeed.tech/tags/luna.md>), [model](<https://devfeed.tech/tags/model.md>), [science](<https://devfeed.tech/tags/science.md>), [system-one](<https://devfeed.tech/tags/system-one.md>)

### AI overview

Clairic is a fast, low-cost extraction model that accepts text and images and can return typed values with confidence scores. In tests against Luna, Clairic was faster but less accurate overall, especially on calculations; examples also included confidently incorrect extractions. It handled some chart-based extraction and returned null for missing revenue. On 1,360 questions across four languages, a 0.9 confidence cutoff yielded 98% accuracy while Clairic answered 72.8% of questions. The author suggests using it for high-confidence extraction and routing the rest to a larger model, with validation because confidence is imperfect. The tests did not establish a reason to prefer it for forecasting over traditional methods.

### Source excerpt

With a 0.9 confidence cutoff, Cloudflare's System One model handles the easy cases and routes the rest to a bigger, slower LLM like Luna. The post Clairic Hit 98% Accuracy on 72.8% of Questions via Confidence Scores appeared first on Towards Data Science.

## DeepQuest: From Knowledge Gaps to Scientific Discovery in Alzheimer's Disease

DevFeed: [DeepQuest: From Knowledge Gaps to Scientific Discovery in Alzheimer's Disease](<https://devfeed.tech/articles/deepquest-from-knowledge-gaps-to-scientific-discovery-in-alzheimer-s-disease-145661.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26343>)

Author: Chunbo Jiang, Yifei Chen, Beining Wu, Fuyou Mao, Weihong Chen, Enyu Bao, Mingxuan Liu, Feiwei Qin, Qiyuan Tian

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [AI for science](<https://devfeed.tech/topics/ai-for-science.md>), [Data Science](<https://devfeed.tech/topics/data-science.md>)

Tags: [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [audit](<https://devfeed.tech/tags/audit.md>), [automated](<https://devfeed.tech/tags/automated.md>), [code](<https://devfeed.tech/tags/code.md>), [comparisons](<https://devfeed.tech/tags/comparisons.md>), [contribution](<https://devfeed.tech/tags/contribution.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [data](<https://devfeed.tech/tags/data.md>), [dimension](<https://devfeed.tech/tags/dimension.md>), [discovery](<https://devfeed.tech/tags/discovery.md>), [disease](<https://devfeed.tech/tags/disease.md>), [domain](<https://devfeed.tech/tags/domain.md>), [experimental](<https://devfeed.tech/tags/experimental.md>), [experimentation](<https://devfeed.tech/tags/experimentation.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [framework](<https://devfeed.tech/tags/framework.md>), [gap](<https://devfeed.tech/tags/gap.md>), [general](<https://devfeed.tech/tags/general.md>), [independent](<https://devfeed.tech/tags/independent.md>), [knowledge](<https://devfeed.tech/tags/knowledge.md>)

### AI overview

DeepQuest turns unresolved questions from Alzheimer's disease reviews into testable hypotheses, screens them for data feasibility, and prioritizes them with groupwise Elo ranking. Its iterative experiments are audited separately for reliability and contribution relative to two frozen literature corpora. On OASIS-3, it generated 42 feasible hypotheses and beat the strongest baselines in 14 of 15 evaluator-dimension comparisons across three language-model reviewers. The framework's audit-assessed scientific finding rate was 95.24%, exceeding general-purpose executors and domain-specific research agents and remaining stable across literature corpora and evaluator model families. Reported findings address neuropsychiatric symptoms and dementia risk, longitudinal amyloid spatial stability, and neocortical tau-associated clinical impairment; key associations were supported in independent ADNI and AIBL cohorts.

### Source excerpt

arXiv:2611.26343v1 Announce Type: new Abstract: Automating experiments does not necessarily advance scientific understanding. Research agents must identify meaningful knowledge gaps, determine what available data can resolve, and assess whether their findings advance existing knowledge. We introduce DeepQuest, a knowledge-gap-driven AI scientist framework for Alzheimer's disease research. DeepQuest transforms unresolved questions from domain reviews into traceable, testable hypotheses and prioritizes them through data feasibility screening and groupwise Elo ranking. It couples iterative experimentation with two-level auditing that separates experimental reliability from knowledge contribution relative to two frozen literature corpora. On OASIS-3, DeepQuest produces 42 feasible hypotheses and outperforms the strongest baselines in 14 of 15 evaluator-dimension comparisons across three language-model reviewers. The full framework achieves an audit-assessed scientific finding rate of 95.24%, outperforming both general-purpose executors and domain-specific research agents. This scientific finding rate remains stable across literature corpora and evaluator model families. Representative findings concern neuropsychiatric symptoms and dementia risk, longitudinal amyloid spatial stability, and neocortical tau-associated clinical impairment, with key associations supported in independent ADNI and AIBL cohorts. These results show how aligning knowledge gaps, data feasibility, and evidence auditing can turn automated research into traceable, evidence-supported scientific discovery. Our source code is available at https://anonymous.4open.science/r/DeepQuest-3066/

## DSFlow: A Neuro-Symbolic Data Workflow Agent with Statistical Diagnostic Guardrails

DevFeed: [DSFlow: A Neuro-Symbolic Data Workflow Agent with Statistical Diagnostic Guardrails](<https://devfeed.tech/articles/dsflow-a-neuro-symbolic-data-workflow-agent-with-statistical-diagnostic-guardrails-145469.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25111>)

Author: Junfeng Nie, Xiaohui Chen

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Data Science](<https://devfeed.tech/topics/data-science.md>), [Multi Agent Systems](<https://devfeed.tech/topics/multi-agent-systems.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [clustering](<https://devfeed.tech/topics/clustering.md>), [linear-regression](<https://devfeed.tech/topics/linear-regression.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [clustering](<https://devfeed.tech/tags/clustering.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [data](<https://devfeed.tech/tags/data.md>), [diagnostic](<https://devfeed.tech/tags/diagnostic.md>), [guardrails](<https://devfeed.tech/tags/guardrails.md>), [langgraph](<https://devfeed.tech/tags/langgraph.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [large-language-models-llms](<https://devfeed.tech/tags/large-language-models-llms.md>), [regression](<https://devfeed.tech/tags/regression.md>), [reports](<https://devfeed.tech/tags/reports.md>), [skills](<https://devfeed.tech/tags/skills.md>), [workflow](<https://devfeed.tech/tags/workflow.md>)

### AI overview

DSFlow combines reusable skills selected by an LLM with deterministic statistical checks to guide data-science workflows. Its symbolic validation can accept a proposed method, ask for clarification or replanning, override an unsuitable choice, or provide a safe fallback. Regression and clustering case studies show diagnostic evidence correcting method choices, with decisions recorded in auditable reports. The supplied text is an abstract and does not provide implementation details or case-study results.

### Source excerpt

arXiv:2611.25111v1 Announce Type: new Abstract: Data science workflows often require tools that presume non-trivial technical knowledge. Large language models (LLMs) reduce this barrier, but a gap remains between flexible general-purpose agents and statistically controlled pipelines. We introduce DSFlow, a LangGraph-based neuro-symbolic data-agent platform that combines reusable, LLM-selected skills with deterministic diagnostic guardrails. Skills provide extensible task knowledge. Symbolic validation can accept a proposed method, request clarification or replanning, override an unsuitable method, or supply a safe fallback. Regression and clustering case studies illustrate how DSFlow uses diagnostic evidence to correct method choices and documents those decisions in auditable reports.

## FACET - Facet Aware Contrastive Embeddings for Traits- Big Five Estimation from Life Narratives

DevFeed: [FACET - Facet Aware Contrastive Embeddings for Traits- Big Five Estimation from Life Narratives](<https://devfeed.tech/articles/facet-facet-aware-contrastive-embeddings-for-traits-big-five-estimation-from-life-narratives-142670.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23560>)

Author: Balakrishnan Arumugam, Priyen Parekh, Mehak Gupta, Joshua Oltmanns

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Data Science](<https://devfeed.tech/topics/data-science.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [vector embeddings in AI](<https://devfeed.tech/topics/vector-embeddings-in-ai.md>), [concatenation](<https://devfeed.tech/topics/concatenation.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [aging](<https://devfeed.tech/tags/aging.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [attention](<https://devfeed.tech/tags/attention.md>), [based](<https://devfeed.tech/tags/based.md>), [can](<https://devfeed.tech/tags/can.md>), [combines](<https://devfeed.tech/tags/combines.md>), [compare](<https://devfeed.tech/tags/compare.md>), [concatenation](<https://devfeed.tech/tags/concatenation.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [decision](<https://devfeed.tech/tags/decision.md>), [domain](<https://devfeed.tech/tags/domain.md>), [domains](<https://devfeed.tech/tags/domains.md>), [embeddings](<https://devfeed.tech/tags/embeddings.md>), [error](<https://devfeed.tech/tags/error.md>), [existing](<https://devfeed.tech/tags/existing.md>), [features](<https://devfeed.tech/tags/features.md>), [framework](<https://devfeed.tech/tags/framework.md>), [gives](<https://devfeed.tech/tags/gives.md>), [input](<https://devfeed.tech/tags/input.md>), [life](<https://devfeed.tech/tags/life.md>)

### AI overview

FACET estimates Big Five scores from life-narrative interviews by separately encoding text and psycholinguistic features, learning contrastive embeddings for personality facets, then using attention to pool them into domain predictions. In 1,332 SPAN participants, facet AUC reached 0.740 and domain R² reached 0.328, highest for Openness. End-to-end training had lower error than staged training in every domain, and attention pooling had the lowest error in every domain. Combining feature views reduced error versus text alone for Conscientiousness and Agreeableness; learned weighting showed no consistent advantage over concatenation. The method modestly outperformed raw-feature regression in four of five domains and provides facet-level predictions unavailable from a domain-only model.

### Source excerpt

arXiv:2611.23560v1 Announce Type: new Abstract: Personality is only weakly and inconsistently signaled by written language, and existing text-based methods typically combine language-model and psycholinguistic representations at the input or decision level. It remains open whether a more deliberate combination of the two views, and a decomposition of the prediction target, can recover more of the signal. We present a three-phase framework for estimating Big Five personality domain scores from life-narrative interview text. It encodes text and psycholinguistic features separately and combines them by concatenation or by a learned weighting, decomposes each domain into contrastively trained facet embeddings instead of predicting the domain directly, and pools the facet embeddings by attention into a domain-level prediction. We compare end-to-end with staged training and ablate the input modalities, their combination, and the facet-pooling strategy. On 1,332 participants from the St. Louis Personality and Aging Network (SPAN) study, facet embeddings reach AUC up to 0.740, and domain-level R2 reaches 0.328, highest for Openness. End-to-end training gives lower error than staged training in every domain, and attention pooling gives the lowest error in every domain. Combining the two views lowers error relative to text alone for Conscientiousness and Agreeableness, whereas a learned weighting of the views shows no consistent advantage over concatenation. The framework provides facet- level predictions that a domain-only model cannot, and it improves modestly on a regression from the raw features in four of the five domains.

## From "Fluff Metrics" to a 360-Degree Customer View | Blog | Fivetran

DevFeed: [From "Fluff Metrics" to a 360-Degree Customer View | Blog | Fivetran](<https://devfeed.tech/articles/from-fluff-metrics-to-a-360-degree-customer-view-blog-fivetran-141319.md>)

Original publisher: [Read original article](<https://wf.fivetran.com/blog/from-fluff-metrics-to-a-360-degree-customer-view>)

Author: Fivetran

Published: 2026-11-18T06:13:15Z

Content type: article

Language: en

Sources: [Fivetran Blog](<https://devfeed.tech/sources/fivetran-blog.md>)

Topics: [Data pipelines](<https://devfeed.tech/topics/data-pipelines.md>), [Data Science](<https://devfeed.tech/topics/data-science.md>), [Google Analytics](<https://devfeed.tech/topics/google-analytics.md>), [elt](<https://devfeed.tech/topics/elt.md>), [Grafana](<https://devfeed.tech/topics/grafana.md>)

Tags: [2](<https://devfeed.tech/tags/2.md>), [3](<https://devfeed.tech/tags/3.md>), [about](<https://devfeed.tech/tags/about.md>), [account](<https://devfeed.tech/tags/account.md>), [actions](<https://devfeed.tech/tags/actions.md>), [addition](<https://devfeed.tech/tags/addition.md>), [adopting](<https://devfeed.tech/tags/adopting.md>), [blog](<https://devfeed.tech/tags/blog.md>), [customer](<https://devfeed.tech/tags/customer.md>), [data](<https://devfeed.tech/tags/data.md>), [fivetran](<https://devfeed.tech/tags/fivetran.md>), [fluff](<https://devfeed.tech/tags/fluff.md>), [insights](<https://devfeed.tech/tags/insights.md>), [key](<https://devfeed.tech/tags/key.md>), [marketing](<https://devfeed.tech/tags/marketing.md>), [metrics](<https://devfeed.tech/tags/metrics.md>), [query](<https://devfeed.tech/tags/query.md>), [rapid](<https://devfeed.tech/tags/rapid.md>), [rich](<https://devfeed.tech/tags/rich.md>), [view](<https://devfeed.tech/tags/view.md>)

### AI overview

The article's central finding is that clicks, opens and conversions alone show campaign activity but may not establish whether marketing drove intended actions or revenue. Fivetran describes joining refreshed warehouse data from Marketo, Google Analytics, Pendo, PostgreSQL and Salesforce, alongside ad and other sources, to connect campaign touchpoints with product usage and downstream business outcomes. Its suggested workflow is to identify relevant sources, choose decision-linked KPIs, understand how tables relate, and test the analysis with an ad hoc query before building a monitoring dashboard. The author says prebuilt entity relationship diagrams helped with data modeling and reports setting up a Pendo integration in about 10 minutes. These are the author's company experience and recommendations; no quantified campaign results are provided.

### Source excerpt

We use Fivetran ourselves for rapid access to rich, query-ready marketing data -- the key to transformative insights.

## $\\beta$-number and Its Applications to Data Science

DevFeed: [$\\beta$-number and Its Applications to Data Science](<https://devfeed.tech/articles/beta-number-and-its-applications-to-data-science-138865.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.19379>)

Author: Pengxu Wen, Ziwei Nie, Liwen Zou, Ying Cao, Xiaoping Yang

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Data Science](<https://devfeed.tech/topics/data-science.md>), [Data science and engineering](<https://devfeed.tech/topics/data-science-and-engineering.md>)

Tags: [algorithms](<https://devfeed.tech/tags/algorithms.md>), [classification](<https://devfeed.tech/tags/classification.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data-analysis](<https://devfeed.tech/tags/data-analysis.md>), [data-science](<https://devfeed.tech/tags/data-science.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [image-classification](<https://devfeed.tech/tags/image-classification.md>), [lower-bounds](<https://devfeed.tech/tags/lower-bounds.md>), [metric](<https://devfeed.tech/tags/metric.md>), [principal-component-analysis](<https://devfeed.tech/tags/principal-component-analysis.md>), [reduction](<https://devfeed.tech/tags/reduction.md>), [scaling](<https://devfeed.tech/tags/scaling.md>), [theoretical](<https://devfeed.tech/tags/theoretical.md>)

### AI overview

The paper defines the β-number of a set as the minimum average p-th-power distance of its points to a d-dimensional plane, using it to quantify oscillation and identify a subspace for dimensionality reduction. In discrete settings with p=2, the selected plane coincides with the PCA plane, making β-number a generalization of PCA. The authors report bounds and a scaling law, describe algorithms for continuous and discrete settings, and test results numerically in low dimensions. They also demonstrate potential applications in dimensionality reduction and image classification; the abstract gives no performance details.

### Source excerpt

arXiv:2611.19379v1 Announce Type: new Abstract: In this paper, we introduce a novel metric, $\beta$-number, designed to measure the "$p$-th order $d$-dimensional oscillation degree" of a set. For a given set $K$ in the Euclidean space $\mathbb{R}^n$, $\beta$-number is computed by identifying an optimal $d$-dimensional plane $L$ that minimizes the average $p$-th order distance of all points in $K$ to $L$. This metric serves as a versatile tool in data science, functioning both as a dimensionality reduction technique to find the optimal $p$-th order $d$-dimensional subspace and as a measure to characterize data oscillation for classification. We conduct a thorough exploration of $\beta$-number through theoretical analysis and numerical experiments. Initially, we establish the upper and lower bounds of $\beta$-number for any set $K$, along with its scaling law. We then elucidate the relationships between $\beta$-number and its related parameters. A key finding of this study is that in discrete settings, when $p=2$, the $d$-dimensional plane determined by $\beta$-number coincides with that obtained via Principal Component Analysis (PCA), positioning $\beta$-number as a generalization of PCA. Furthermore, we propose two algorithms for calculating $\beta$-number in both continuous and discrete settings. We validate our theoretical results through a series of numerical experiments in low-dimensional cases. Finally, we demonstrate the potential of $\beta$-number in practical data analysis through a dimensionality reduction task and an image classification task.

## Advancing Cognitive Pairwise Comparisons for Human-Centric Artificial Intelligence and Data Science: A Human-in-the-Loop Framework

DevFeed: [Advancing Cognitive Pairwise Comparisons for Human-Centric Artificial Intelligence and Data Science: A Human-in-the-Loop Framework](<https://devfeed.tech/articles/advancing-cognitive-pairwise-comparisons-for-human-centric-artificial-intelligence-and-data-science-a-human-in-the-loop-framework-137632.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.20926>)

Author: Kevin Kam Fung Yuen

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Data Science](<https://devfeed.tech/topics/data-science.md>), [Interactive AI Systems](<https://devfeed.tech/topics/interactive-ai-systems.md>)

Tags: [algorithms](<https://devfeed.tech/tags/algorithms.md>), [applications](<https://devfeed.tech/tags/applications.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [automation-bias](<https://devfeed.tech/tags/automation-bias.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-hc](<https://devfeed.tech/tags/cs-hc.md>), [data-science](<https://devfeed.tech/tags/data-science.md>), [feature-selection](<https://devfeed.tech/tags/feature-selection.md>), [framework](<https://devfeed.tech/tags/framework.md>), [human-in-the-loop](<https://devfeed.tech/tags/human-in-the-loop.md>), [knowledge](<https://devfeed.tech/tags/knowledge.md>), [learning](<https://devfeed.tech/tags/learning.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [math-oc](<https://devfeed.tech/tags/math-oc.md>), [model-selection](<https://devfeed.tech/tags/model-selection.md>), [recommender-systems](<https://devfeed.tech/tags/recommender-systems.md>), [reduction](<https://devfeed.tech/tags/reduction.md>), [unsupervised-learning](<https://devfeed.tech/tags/unsupervised-learning.md>), [usability](<https://devfeed.tech/tags/usability.md>)

### AI overview

The extended Cognitive Pairwise Comparison (CPC) framework is presented as a structured way to elicit and quantify human preferences in AI and data science tasks that often require repeated human input, including data transformation, feature selection and weighting, and model selection. The abstract says it aims to address limits of fully automated machine learning, such as automation bias and failure to incorporate tacit domain knowledge. Applications using the MovieLens dataset illustrate interactions between human judgments and data across supervised learning, dimensionality reduction, and unsupervised learning. The supplied evidence does not provide details on implementation or comparative performance.

### Source excerpt

arXiv:2611.20926v1 Announce Type: new Abstract: Human-centric artificial intelligence and data science (HCAD) systems increasingly involve human-in-the-loop (HITL) interventions to overcome the limitations of fully automated machine learning algorithms, such as automation bias and the inability to incorporate tacit domain knowledge, and continuously improve predictive performance. Data transformations, feature selection, feature weighting, and model selection usually require laborious human involvement across iterations of artificial intelligence and data science (AI&DS) pipelines, making development a challenging and time-consuming process. The Cognitive Pairwise Comparison (CPC) framework is proposed as a human-centric methodology to elicit preferences and priorities in a structured format. To establish essential facets of human-centricity, this research advances the CPC methodology so that user preference elicitation can be systematically quantified in these challenging AI&DS tasks. To demonstrate the usability and validity of these facets, this study presents several applications within movie recommender systems using the MovieLens dataset. It exemplifies how the extended CPC framework can facilitate interactions between human judgments and data across supervised learning, dimensionality reduction, and unsupervised learning algorithms.

## Scentree: a framework for generating scenario trees for multistage stochastic programming

DevFeed: [Scentree: a framework for generating scenario trees for multistage stochastic programming](<https://devfeed.tech/articles/scentree-a-framework-for-generating-scenario-trees-for-multistage-stochastic-programming-131676.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.21495>)

Author: Cristian Pach\\'on-Garc\\'ia, Albert Sol\\\`a Vilalta, F. -Javier Heredia

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [Data Science](<https://devfeed.tech/topics/data-science.md>), [Predictive AI](<https://devfeed.tech/topics/predictive-ai.md>), [scikit-learn](<https://devfeed.tech/topics/scikit-learn.md>)

Tags: [automation](<https://devfeed.tech/tags/automation.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [data](<https://devfeed.tech/tags/data.md>), [dependencies](<https://devfeed.tech/tags/dependencies.md>), [design](<https://devfeed.tech/tags/design.md>), [distribution](<https://devfeed.tech/tags/distribution.md>), [forecasting](<https://devfeed.tech/tags/forecasting.md>), [framework](<https://devfeed.tech/tags/framework.md>), [generation](<https://devfeed.tech/tags/generation.md>), [learning](<https://devfeed.tech/tags/learning.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [model](<https://devfeed.tech/tags/model.md>), [modeling](<https://devfeed.tech/tags/modeling.md>), [models](<https://devfeed.tech/tags/models.md>), [modular](<https://devfeed.tech/tags/modular.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [package](<https://devfeed.tech/tags/package.md>), [parameter](<https://devfeed.tech/tags/parameter.md>), [probability](<https://devfeed.tech/tags/probability.md>), [programming](<https://devfeed.tech/tags/programming.md>), [stat-co](<https://devfeed.tech/tags/stat-co.md>), [trees](<https://devfeed.tech/tags/trees.md>)

### AI overview

Scentree is an open-source Python package that builds scenario trees for multistage stochastic programming from historical data or probability-distribution samples. It jointly models the stochastic processes with machine-learning and multivariate time-series methods, generates a scenario fan that captures dependencies, then reduces it to a representative tree. The package automates forecasting-model selection, parameter estimation, fan generation and tree construction, without assuming an underlying data distribution. Its modular design also allows external forecasts or partial workflow execution, and the framework can be applied to two-stage problems.

### Source excerpt

arXiv:2609.21495v2 Announce Type: replace-cross Abstract: We present scentree, an open-source Python package for constructing scenario trees for multistage stochastic programming models using either historical data or samples drawn from a probability distribution. It combines machine learning and multivariate time series models to obtain a scenario fan that captures dependencies in the stochastic processes. This scenario fan is subsequently reduced into a scenario tree that preserves representativeness of the stochastic process, providing a flexible and extensible framework for uncertainty modeling in stochastic programming problems. The framework can also be naturally applied to two-stage stochastic programming problems. Key contributions are the joint modeling of the stochastic process, preserving correlations between all random variables, and the automation of the complete workflow, including forecasting model selection, parameter estimation, scenario fan generation, and scenario tree construction. In addition, scentree does not assume any underlying data distribution, reducing the statistical expertise required to produce a scenario tree, and is modular by design, allowing external forecasts and other partial executions of the workflow.

## A Jev of All Trades: Build and Train a Jev-style Decision Model From Scratch

DevFeed: [A Jev of All Trades: Build and Train a Jev-style Decision Model From Scratch](<https://devfeed.tech/articles/a-jev-of-all-trades-build-and-train-a-jev-style-decision-model-from-scratch-128928.md>)

Original publisher: [Read original article](<https://towardsdatascience.com/a-jev-of-all-trades-build-and-train-a-jev-style-decision-model-from-scratch/>)

Author: Lambert Leong

Published: 2026-11-11T12:00:00Z

Content type: tutorial

Language: en

Sources: [Towards Data Science](<https://devfeed.tech/sources/towards-data-science.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Data Science](<https://devfeed.tech/topics/data-science.md>), [Learning](<https://devfeed.tech/topics/learning.md>), [Data analysis](<https://devfeed.tech/topics/data-analysis.md>)

Tags: [adapter](<https://devfeed.tech/tags/adapter.md>), [computer-vision](<https://devfeed.tech/tags/computer-vision.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [editor-s-picks](<https://devfeed.tech/tags/editor-s-picks.md>), [jev](<https://devfeed.tech/tags/jev.md>), [keep](<https://devfeed.tech/tags/keep.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [medical-imaging](<https://devfeed.tech/tags/medical-imaging.md>), [train](<https://devfeed.tech/tags/train.md>)

### AI overview

The tutorial builds a Jev-inspired pathology image model by freezing KEEP's image and text encoders and training a small adapter to score supplied tissue descriptions. On the inspected PathMNIST questions, the trained model made correct final decisions on about 66-67% of added-category cases, versus 40% for Qwen2.5-VL-3B-Instruct; unadapted KEEP reached 69%, so the adapter did not improve that measure. For familiar labels, a linear classifier reached 98% correct choices and 88% correct final decisions, while its fixed output head cannot represent added labels. The model's flexible options came with weak rejection of answer-absent cases and calibration transfer. Wider or longer adapter training did not improve added-category recall. Results are exploratory: the test set had prior exposure, the benchmark covers one Qwen model and 270 matched images, and pretraining overlap and patient-level independence remain unresolved.

### Source excerpt

Trained on six PathMNIST tissues with a small KEEP adapter, it handled added labels without retraining, though a linear classifier hit 88% on familiar ones. The post A Jev of All Trades: Build and Train a Jev-style Decision Model From Scratch appeared first on Towards Data Science.

## NFL and AWS Big Data Bowl will bridge the gap from college to the pros

DevFeed: [NFL and AWS Big Data Bowl will bridge the gap from college to the pros](<https://devfeed.tech/articles/nfl-and-aws-big-data-bowl-will-bridge-the-gap-from-college-to-the-pros-127783.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/media/nfl-and-aws-big-data-bowl-will-bridge-the-gap-from-college-to-the-pros/>)

Author: Ari Entin

Published: 2026-11-11T00:04:01Z

Content type: release

Language: en

Sources: [AWS for M&E Blog](<https://devfeed.tech/sources/aws-for-m-e-blog.md>)

Topics: [Sports](<https://devfeed.tech/topics/sports.md>), [Data Science](<https://devfeed.tech/topics/data-science.md>), [big-data](<https://devfeed.tech/topics/big-data.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [streaming data analytics](<https://devfeed.tech/topics/streaming-data-analytics.md>)

Tags: [analytics](<https://devfeed.tech/tags/analytics.md>), [aws](<https://devfeed.tech/tags/aws.md>), [big-data](<https://devfeed.tech/tags/big-data.md>), [college](<https://devfeed.tech/tags/college.md>), [competition](<https://devfeed.tech/tags/competition.md>), [data](<https://devfeed.tech/tags/data.md>), [industries](<https://devfeed.tech/tags/industries.md>), [media-entertainment](<https://devfeed.tech/tags/media-entertainment.md>), [sports](<https://devfeed.tech/tags/sports.md>), [tracking](<https://devfeed.tech/tags/tracking.md>)

### AI overview

The 2027 Big Data Bowl will test whether college production and NFL Combine tracking can help explain rookie NFL performance. Its dataset covers players from the 2023-2025 draft classes at five position groups, with x-y movement tracking from standardized Combine drills and NFL games plus drill and play context. The competition encourages position-specific analysis, such as comparing route breaks, agility, or first-step quickness with early in-game performance, while recognizing that some drills may show no predictive signal. Successful methods could inform the NFL's Draft IQ tools, with AWS helping scale competition code for production; prior Big Data Bowl work has also produced metrics used in broadcasts and team research. The article presents this as exploratory: predictive value has not been systematically established, and null findings are considered useful.

### Source excerpt

The ninth annual Big Data Bowl, the NFL's flagship analytics competition, is taking a new direction. For the first time, participants will work with data that spans the full arc of a player's journey into the NFL, combining college statistics, player tracking data from the NFL Scouting Combine, and Next Generation (Next Gen) Stats from their rookie seasons. The challenge is to find the connections that link what a player does before the league to what they do after they arrive. Join the 2027 Big Data Bowl and start exploring the data.

## redalpine Closes Oversubscribed $250M Fund to Back the Science Defining the Next Decade

DevFeed: [redalpine Closes Oversubscribed $250M Fund to Back the Science Defining the Next Decade](<https://devfeed.tech/articles/redalpine-closes-oversubscribed-250m-fund-to-back-the-science-defining-the-next-decade-127131.md>)

Original publisher: [Read original article](<https://hackernoon.com/redalpine-closes-oversubscribed-$250m-fund-to-back-the-science-defining-the-next-decade?source=rss>)

Author: Ishan Pandey

Published: 2026-11-10T16:07:32Z

Content type: news

Language: en

Sources: [HackerNoon](<https://devfeed.tech/sources/hackernoon.md>)

Topics: [Finance](<https://devfeed.tech/topics/finance.md>), [Robotics](<https://devfeed.tech/topics/robotics.md>), [AI Engineering](<https://devfeed.tech/topics/ai-engineering.md>), [Data Science](<https://devfeed.tech/topics/data-science.md>)

Tags: [0-1](<https://devfeed.tech/tags/0-1.md>), [1-4](<https://devfeed.tech/tags/1-4.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-and-ml](<https://devfeed.tech/tags/ai-and-ml.md>), [ai-architecture](<https://devfeed.tech/tags/ai-architecture.md>), [ai-companies](<https://devfeed.tech/tags/ai-companies.md>), [ai-for-drug-discovery](<https://devfeed.tech/tags/ai-for-drug-discovery.md>), [ai-model](<https://devfeed.tech/tags/ai-model.md>), [biotech](<https://devfeed.tech/tags/biotech.md>), [deeptech](<https://devfeed.tech/tags/deeptech.md>), [fusion](<https://devfeed.tech/tags/fusion.md>), [good-company](<https://devfeed.tech/tags/good-company.md>), [redalpine](<https://devfeed.tech/tags/redalpine.md>), [robotics](<https://devfeed.tech/tags/robotics.md>), [science](<https://devfeed.tech/tags/science.md>), [series](<https://devfeed.tech/tags/series.md>), [series-a](<https://devfeed.tech/tags/series-a.md>), [startups](<https://devfeed.tech/tags/startups.md>), [technology](<https://devfeed.tech/tags/technology.md>), [vc](<https://devfeed.tech/tags/vc.md>), [vc-funding](<https://devfeed.tech/tags/vc-funding.md>)

### AI overview

redalpine closed its oversubscribed $250 million RAC VIII fund, its largest early-stage fund, to invest in about 20 European companies, primarily leading or co-leading Seed and Series A rounds with initial checks of $1 million to $15 million. Half of the investment team are trained scientists, supporting evaluation of technologies before products or customer data exist. Its first three investments span robotics force sensing (Enact Intelligence), AI-supported drug discovery (Helical), and topological neural networks for critical systems (Arlequin AI). The fund starts with these three companies and room for about 17 more. redalpine says it screens more than 5,000 companies annually and makes five to ten new early-stage investments a year. The article cites prior portfolio outcomes including Mistral AI, Proxima Fusion and N26, while cautioning that past performance does not reliably predict future results.

### Source excerpt

redalpine closes its oversubscribed $250M RAC VIII fund to lead Seed and Series A rounds in European AI, robotics, fusion and biotech after 20 years.

## Adaptive Budgeted Data Acquisition for Robust Off-Policy Evaluation from Multiple Loggers

DevFeed: [Adaptive Budgeted Data Acquisition for Robust Off-Policy Evaluation from Multiple Loggers](<https://devfeed.tech/articles/adaptive-budgeted-data-acquisition-for-robust-off-policy-evaluation-from-multiple-loggers-125133.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.10095>)

Author: Amitakshar Biswas, Ruoqing Zhu

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Data Science](<https://devfeed.tech/topics/data-science.md>), [Combinatorial optimization](<https://devfeed.tech/topics/combinatorial-optimization.md>)

Tags: [acquisition](<https://devfeed.tech/tags/acquisition.md>), [adaptive](<https://devfeed.tech/tags/adaptive.md>), [algorithm](<https://devfeed.tech/tags/algorithm.md>), [application](<https://devfeed.tech/tags/application.md>), [budget](<https://devfeed.tech/tags/budget.md>), [class](<https://devfeed.tech/tags/class.md>), [clinical-trials](<https://devfeed.tech/tags/clinical-trials.md>), [conditional](<https://devfeed.tech/tags/conditional.md>), [cost](<https://devfeed.tech/tags/cost.md>), [costs](<https://devfeed.tech/tags/costs.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [design](<https://devfeed.tech/tags/design.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [estimator](<https://devfeed.tech/tags/estimator.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [gradient](<https://devfeed.tech/tags/gradient.md>), [greedy](<https://devfeed.tech/tags/greedy.md>), [multiple](<https://devfeed.tech/tags/multiple.md>), [policy](<https://devfeed.tech/tags/policy.md>), [robust](<https://devfeed.tech/tags/robust.md>), [stat-ml](<https://devfeed.tech/tags/stat-ml.md>)

### AI overview

Adaptive Greedy allocates a limited data-purchasing budget sequentially across existing logging policies with different per-observation costs. At each step, it selects an affordable logger with the greatest cost-normalized variance reduction across a finite class of target policies. The authors characterize each purchase as an approximate conditional-gradient step and report that, under mild assumptions, the allocation converges almost surely to the optimal design at an explicit rate. They also report asymptotic normality and semiparametric efficiency for the pooled estimator. Simulations and a semi-synthetic AIDS Clinical Trials Group Study 175 application indicate better performance than standard allocation strategies, useful accuracy when stopped early, and final accuracy matching repeated integer programming at lower computational cost.

### Source excerpt

arXiv:2611.10095v1 Announce Type: cross Abstract: We consider off-policy evaluation when additional data can be purchased from several existing logging policies (or loggers) with heterogeneous costs per observation. While existing work on multiple loggers presumes a fixed number of observations from each logger, we study how to allocate a limited budget across them sequentially to evaluate a finite class of target policies. We propose Adaptive Greedy, an algorithm that purchases observations sequentially from the affordable logger that has the most cost-normalized variance reduction across the target policies. It turns out that each such purchase is an 'approximate conditional-gradient step' for the corresponding design problem. Under mild assumptions, we establish that the empirical allocation converges almost surely to the optimal design at an explicit rate. We also derive the asymptotic normality of the final pooled estimator and show that it attains the semiparametric efficiency bound. Simulations and a semi-synthetic application based on the AIDS Clinical Trials Group Study 175 show that Adaptive Greedy outperforms standard allocation strategies, remains accurate if stopped early, and matches the final accuracy of repeated integer programming at a fraction of its computational cost.

## One Run Is Not Enough: A Monte Carlo Protocol for Evaluating LLM Reliability on Data Science Tasks

DevFeed: [One Run Is Not Enough: A Monte Carlo Protocol for Evaluating LLM Reliability on Data Science Tasks](<https://devfeed.tech/articles/one-run-is-not-enough-a-monte-carlo-protocol-for-evaluating-llm-reliability-on-data-science-tasks-123449.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.09729>)

Author: Wenqi Ni, Ozan Evkaya, Miguel de Carvalho

Published: 2026-11-09T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Data Science](<https://devfeed.tech/topics/data-science.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Uncertainty quantification LLMs](<https://devfeed.tech/topics/uncertainty-quantification-llms.md>), [Ground truth / benchmark quality](<https://devfeed.tech/topics/ground-truth-benchmark-quality.md>)

Tags: [benchmark](<https://devfeed.tech/tags/benchmark.md>), [consistency](<https://devfeed.tech/tags/consistency.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data-science](<https://devfeed.tech/tags/data-science.md>), [large-language-models-llms](<https://devfeed.tech/tags/large-language-models-llms.md>), [llm-reliability](<https://devfeed.tech/tags/llm-reliability.md>), [monte-carlo](<https://devfeed.tech/tags/monte-carlo.md>), [openai-models](<https://devfeed.tech/tags/openai-models.md>), [reliability](<https://devfeed.tech/tags/reliability.md>), [supervised-learning](<https://devfeed.tech/tags/supervised-learning.md>)

### AI overview

DS-QA evaluates LLM reliability on 100 data science and statistics questions from UK university courses, covering 16 datasets and three difficulty levels. Its three-layer framework measures general response properties (runtime, verbosity, text similarity), broad outcomes (accuracy, code executability, completeness), and task-specific criteria. In 100 repeated runs evaluating OpenAI models, responses varied substantially on data visualisation and supervised learning, especially for medium and hard questions. The results support assessing consistency across runs alongside response quality. The abstract also reports that a 0.1 increase in word verbosity ratio corresponds to about a 0.013 increase in token verbosity ratio.

### Source excerpt

arXiv:2611.09729v1 Announce Type: new Abstract: Large language models (LLMs) have shown promise for automating data science workflows. Benchmarks such as DataSciBench (2026) and StatQA (2024) have advanced the evaluation of their capabilities across data science and statistical tasks. Assessing their reliability requires examining both the quality of their responses and their consistency across repeated runs, task types, and difficulty levels. With this aim, we introduce DS-QA, a benchmark comprising 100 data science and statistics questions drawn from UK university courses, spanning 16 datasets and three difficulty levels. We develop a three-layer evaluation framework that combines general response properties (runtime, verbosity and text similarity), coarse-grained metrics (accuracy, code executability and completeness), and task-specific metrics. We evaluate OpenAI models using a Monte Carlo protocol with 100 repeated runs. The results reveal substantial variability in responses to data visualisation and supervised learning tasks, particularly at medium and hard difficulty levels, highlighting the importance of evaluating consistency alongside response quality. We also find that an increase of 0.1 in the word verbosity ratio corresponds to an increase of about 0.013 in the token verbosity ratio.

## Glyph: An Interpretability Engine for AI-Driven Scientific Discovery in Clinical Data

DevFeed: [Glyph: An Interpretability Engine for AI-Driven Scientific Discovery in Clinical Data](<https://devfeed.tech/articles/glyph-an-interpretability-engine-for-ai-driven-scientific-discovery-in-clinical-data-124592.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.08922>)

Author: Robbie Holland, Ashwin Kumar, Mateus A. Esmeraldo, David Le, Andrea Fisher, Jiazhen Pan, Louisa Fay, Magdalini Paschali, Vasiliki Sideri-Lampretsa, Ates Fettahoglu, Stefania Moroianu, Philipp Wesp, Iv

Published: 2026-11-09T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [AI for science](<https://devfeed.tech/topics/ai-for-science.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Data Science](<https://devfeed.tech/topics/data-science.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [autoencoders](<https://devfeed.tech/tags/autoencoders.md>), [automated](<https://devfeed.tech/tags/automated.md>), [concepts](<https://devfeed.tech/tags/concepts.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [discovery](<https://devfeed.tech/tags/discovery.md>), [disease](<https://devfeed.tech/tags/disease.md>), [embeddings](<https://devfeed.tech/tags/embeddings.md>), [features](<https://devfeed.tech/tags/features.md>), [foundation-models](<https://devfeed.tech/tags/foundation-models.md>), [framework](<https://devfeed.tech/tags/framework.md>), [interpretability](<https://devfeed.tech/tags/interpretability.md>), [literature](<https://devfeed.tech/tags/literature.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [neural-networks](<https://devfeed.tech/tags/neural-networks.md>), [novel](<https://devfeed.tech/tags/novel.md>), [power](<https://devfeed.tech/tags/power.md>), [query](<https://devfeed.tech/tags/query.md>), [release](<https://devfeed.tech/tags/release.md>), [reports](<https://devfeed.tech/tags/reports.md>), [scale](<https://devfeed.tech/tags/scale.md>)

### AI overview

Glyph uses foundation models and sparse autoencoders to extract features from clinical images, radiology reports, and lab data, then maps those features to coded clinical concepts for evaluation. Applied to data from 41,628 patients across institutions, it identified 397 predictive concepts with 4,463 Bonferroni-significant associations; most associations were automatically reproducible in held-out cohorts at different sites. Glyph concepts also yielded associations complementary to literature-derived markers and improved predictive power. The abstract presents this as a way to connect latent model features with clinical knowledge, while noting the challenge of interpreting those features.

### Source excerpt

arXiv:2611.08922v1 Announce Type: new Abstract: Automated scientific discovery has the potential to advance understanding of disease mechanisms and accelerate identification of early disease markers. Manual data annotation is infeasible at population scale, motivating methods that automatically extract pre-defined phenotypes in multimodal and unstructured clinical data. Such pre-defined feature sets determine and limit which associations these approaches can discover. This limitation extends to agentic AI systems that use these tools to query population-scale data. In contrast, foundation models (FMs) trained on large volumes of clinical data can perform tasks beyond their training objectives. This suggests their latent embeddings may represent novel or unexplored features that could yield insights into disease mechanisms, yet such latent features remain difficult to interpret. To address this challenge, we introduce Glyph, an automated multistage framework that extracts features from multimodal and unstructured clinical data using FMs and sparse autoencoders, interprets them as coded clinical concepts, and evaluates their value for clinical scientific discovery. We applied Glyph to abdominal CT volumes, CT radiology reports and laboratory measurements from 41,628 patients across different institutions. We identified 397 predictive concepts supported by 4,463 Bonferroni-significant associations. Moreover, a majority of discovered associations were automatically reproducible in held-out cohorts across different sites. Glyph concepts provided complementary associations to literature-derived markers, leading to gains in predictive power. We envision that interpretability can extend beyond a mechanistic understanding of neural networks to connecting implicitly learned features with clinical knowledge and establishing their significance for patient outcomes. We release Glyph at https://github.com/StanfordMIMI/Glyph.

## Timestral, t0, Tabby, and Morpheus-2.0 Compared on GIFT-Eval

DevFeed: [Timestral, t0, Tabby, and Morpheus-2.0 Compared on GIFT-Eval](<https://devfeed.tech/articles/timestral-t0-tabby-and-morpheus-2-0-compared-on-gift-eval-122205.md>)

Original publisher: [Read original article](<https://towardsdatascience.com/timestral-t0-tabby-and-morpheus-20-compared-on-gift-eval/>)

Author: Sean Moran

Published: 2026-11-07T15:00:00Z

Content type: comparison

Language: en

Sources: [Towards Data Science](<https://devfeed.tech/sources/towards-data-science.md>)

Topics: [Time Series](<https://devfeed.tech/topics/time-series.md>), [AI Foundation Models](<https://devfeed.tech/topics/ai-foundation-models.md>), [model architecture](<https://devfeed.tech/topics/model-architecture.md>), [Data Science](<https://devfeed.tech/topics/data-science.md>), [Data science and engineering](<https://devfeed.tech/topics/data-science-and-engineering.md>), [Open Source Models & Datasets](<https://devfeed.tech/topics/open-source-models-datasets.md>)

Tags: [2](<https://devfeed.tech/tags/2.md>), [all](<https://devfeed.tech/tags/all.md>), [artifical-intellegence](<https://devfeed.tech/tags/artifical-intellegence.md>), [data](<https://devfeed.tech/tags/data.md>), [data-science](<https://devfeed.tech/tags/data-science.md>), [editor-s-picks](<https://devfeed.tech/tags/editor-s-picks.md>), [eval](<https://devfeed.tech/tags/eval.md>), [first](<https://devfeed.tech/tags/first.md>), [foundation](<https://devfeed.tech/tags/foundation.md>), [foundation-models](<https://devfeed.tech/tags/foundation-models.md>), [information](<https://devfeed.tech/tags/information.md>), [large-language-model](<https://devfeed.tech/tags/large-language-model.md>), [models](<https://devfeed.tech/tags/models.md>), [patch](<https://devfeed.tech/tags/patch.md>), [post](<https://devfeed.tech/tags/post.md>), [scale](<https://devfeed.tech/tags/scale.md>), [science](<https://devfeed.tech/tags/science.md>), [series](<https://devfeed.tech/tags/series.md>), [split](<https://devfeed.tech/tags/split.md>), [time](<https://devfeed.tech/tags/time.md>), [time-series](<https://devfeed.tech/tags/time-series.md>), [time-series-foundation-models](<https://devfeed.tech/tags/time-series-foundation-models.md>), [timestral](<https://devfeed.tech/tags/timestral.md>)

### AI overview

The four models share a recipe--scaled time-series patches, transformers, quantile forecasts and substantial synthetic training--but differ in whether related series can inform a forecast. Tabby forecasts each series independently; Timestral and t0 exchange information in selected layers; Morpheus uses two-axis attention in every layer. On GIFT-Eval, Timestral leads on probabilistic CRPS and Morpheus on median-based MASE, but pairwise confidence intervals include ties. The headline results are not directly equivalent: Morpheus's entry is a five-model ensemble with a fitted mixing head, Tabby's is prompt-tuned on benchmark tasks, and Timestral includes forecast adjustments whose contribution was not measured. The author's reruns reproduce the reported scores, with Tabby requiring a longer masked forecast span for close agreement. The comparison suggests training data and procedures explain substantial gains, though this is not a controlled cross-lab finding; t0-gamma's training changes are undisclosed. For practical selection, covariate support, licensing, model size and performance on the intended data may matter more than small aggregate score differences.

### Source excerpt

All four time-series foundation models scale, patch, and predict quantiles; the real split is how series share information. The post Timestral, t0, Tabby, and Morpheus-2.0 Compared on GIFT-Eval appeared first on Towards Data Science.

## DART: Domain-Agnostic Residual Transfer for Generalist Anomaly Detection

DevFeed: [DART: Domain-Agnostic Residual Transfer for Generalist Anomaly Detection](<https://devfeed.tech/articles/dart-domain-agnostic-residual-transfer-for-generalist-anomaly-detection-120300.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.08271>)

Author: Muhammad Aqeel, Maham Nazir, Marco Cristani, Francesco Setti

Published: 2026-11-06T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Data Science](<https://devfeed.tech/topics/data-science.md>)

Tags: [anomaly-detection](<https://devfeed.tech/tags/anomaly-detection.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [cross](<https://devfeed.tech/tags/cross.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [dart](<https://devfeed.tech/tags/dart.md>), [detection](<https://devfeed.tech/tags/detection.md>), [domain](<https://devfeed.tech/tags/domain.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [generalist](<https://devfeed.tech/tags/generalist.md>)

### AI overview

DART targets a transfer problem in generalist anomaly detection: residuals from normal-abnormal reference guidance can mix transferable anomaly cues with domain-specific appearance, causing spurious activations on unseen domains. It adds a Residual Disentanglement Module to preserve anomaly-discriminative information while suppressing domain information, and Cross-Domain Meta-Training to teach that separation through episodes across source domains. The architecture-agnostic framework can be placed between residual computation and the detection head in existing pipelines. The abstract reports consistent improvements over multiple base methods on MVTec AD, VisA, and BraTS, with the largest gains under stronger domain shifts; it gives no numerical results in the supplied text.

### Source excerpt

arXiv:2611.08271v1 Announce Type: cross Abstract: Generalist Anomaly Detection (GAD) aims to detect anomalies in unseen domains using only a few reference samples. Recent normal-abnormal guided methods improve this setting by exploiting abnormal references, but their residuals often mix transferable anomaly cues with domain-specific appearance factors such as texture, geometry, and acquisition conditions. This residual domain entanglement limits cross-domain transfer and may produce spurious activations on novel domains. We propose DART, a plug-in framework for normal-abnormal guided GAD that learns to separate residuals into anomaly-relevant and domain-specific components. DART combines a Residual Disentanglement Module, which preserves anomaly-discriminative information while suppressing domain information, with Cross-Domain Meta-Training, which forms episodes across source domains to make the separation effective. The method is architecture-agnostic and can be inserted between residual computation and the detection head of existing GAD pipelines. Experiments on MVTec AD, VisA, and BraTS show consistent improvements over multiple base methods, with the largest gains under stronger domain shifts. https://github.com/aqeeelmirza/DART

## How We Built NASA's Zero-Shot Detector for Gas Leaks from Space

DevFeed: [How We Built NASA's Zero-Shot Detector for Gas Leaks from Space](<https://devfeed.tech/articles/how-we-built-nasa-s-zero-shot-detector-for-gas-leaks-from-space-119293.md>)

Original publisher: [Read original article](<https://towardsdatascience.com/how-we-built-nasas-zero-shot-detector-for-gas-leaks-from-space/>)

Author: Vít Růžička

Published: 2026-11-05T15:00:00Z

Content type: article

Language: en

Sources: [Towards Data Science](<https://devfeed.tech/sources/towards-data-science.md>)

Topics: [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Data Science](<https://devfeed.tech/topics/data-science.md>), [Data pipelines](<https://devfeed.tech/topics/data-pipelines.md>)

Tags: [ammonia](<https://devfeed.tech/tags/ammonia.md>), [automated](<https://devfeed.tech/tags/automated.md>), [carbon](<https://devfeed.tech/tags/carbon.md>), [climate-change](<https://devfeed.tech/tags/climate-change.md>), [computer-vision](<https://devfeed.tech/tags/computer-vision.md>), [editor-s-picks](<https://devfeed.tech/tags/editor-s-picks.md>), [leaks](<https://devfeed.tech/tags/leaks.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [nasa](<https://devfeed.tech/tags/nasa.md>), [remote-sensing](<https://devfeed.tech/tags/remote-sensing.md>), [space](<https://devfeed.tech/tags/space.md>), [zero-shot](<https://devfeed.tech/tags/zero-shot.md>)

### AI overview

NASA JPL's Daily Methane Digest automates EMIT satellite plume detection using three stages: matched-filter enhancement images, a U-Net ensemble trained on methane patterns, and a physics-based MODTRAN fit that rejects candidates with high residual error. This separates plume-shape recognition from spectral validation to reduce false alarms in a setting where plumes are rare. Gas-specific matched filters let the methane-trained model detect ammonia, nitrogen dioxide, and carbon monoxide without retraining; the post reports the first automated EMIT point-source CO detections. It says the pipeline went operational in January 2026, cuts alert latency to about 24 hours, and found 157 high-confidence methane plumes in 2024-25 archive scenes that prior review missed. The excerpt says code and weights are open source and that NASA Earthdata credentials are needed to run detections.

### Source excerpt

We trained one model on methane. Without retraining, it also maps carbon monoxide, ammonia and nitrogen dioxide plumes, including the first automated CO detections from NASA's EMIT. The post How We Built NASA's Zero-Shot Detector for Gas Leaks from Space appeared first on Towards Data Science.

## Iterated creation, grading, and revision of data science projects by language models

DevFeed: [Iterated creation, grading, and revision of data science projects by language models](<https://devfeed.tech/articles/iterated-creation-grading-and-revision-of-data-science-projects-by-language-models-118337.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.05548>)

Author: Rohan Alexander

Published: 2026-11-05T05:00:00Z

Content type: news

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Data Science](<https://devfeed.tech/topics/data-science.md>), [AI Chat](<https://devfeed.tech/topics/ai-chat.md>), [Multi-step AI processes](<https://devfeed.tech/topics/multi-step-ai-processes.md>)

Tags: [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [budget](<https://devfeed.tech/tags/budget.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [data-science](<https://devfeed.tech/tags/data-science.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [feedback](<https://devfeed.tech/tags/feedback.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [revision](<https://devfeed.tech/tags/revision.md>)

### AI overview

The study finds that feedback and revision can improve language models' end-to-end data science projects, though gains depend on which models write and grade. Seven models from five developers each served as writer and grader across 49 pairs, with three runs per pair. Of 147 initial submissions, 107 ran and rendered; average scores rose from 59% to 76% after three rounds. Writer-level gains ranged from 3 percentage points for Anthropic's Opus 5.5 to 46 for Haiku 4.5; grader-level gains ranged from 13 points for Google's Gemini 3.8 Flash to 32 for Haiku 4.5. Models tended to score their own work about 9 points higher than other graders did, and roughly half of round-to-round score differences fell within grader noise. Feedback generally produced the highest marks, but revision opportunities or extra time and tool calls without feedback helped some models and not others; weaker models generally did not improve stronger ones.

### Source excerpt

arXiv:2611.05548v1 Announce Type: new Abstract: We evaluate the extent to which agentic AI models can complete an end-to-end data science project, and how this improves with feedback. A 'writer' model with specific tools creates a complete data science project including downloading, cleaning, and analyzing a dataset, and writing a short paper about it. The project is executed and a 'grader' model evaluates it. The writer can then revise its submission, and this repeats for a total of three rounds. We consider seven models from five developers: Anthropic, OpenAI, Google, DeepSeek, and Z.AI. Each model is both a writer and a grader, so there are 49 writer-grader pairs and we run each pair three times. Of the 147 initial submissions, 107 run and render, and the average score increases from 59 per cent to 76 per cent after the final round of feedback. By writer the extent of the increase ranges from 3 percentage points for Anthropic's Opus 5.5 to 46 percentage points for Anthropic's Haiku 4.5, and by grader it ranges from 13 percentage points for Google's Gemini 3.8 Flash to 32 percentage points for Anthropic's Haiku 4.5. When a model grades its own writing the score tends to be about 9 percentage points higher than the average of what the other models give it. About half of the difference that we see from round to round is within the grader's own noise. Feedback generally results in the highest marks, but even just having the chance to revise, or having the additional budget, in terms of time or tool calls, without any feedback, results in considerable increases for some of the models but not for others. We find that how much such a loop helps depends on which model plays which role, but weaker models generally do not improve stronger models.

## BobFlow: Towards Human-Agent Collaborative Data Science Using Dataflows

DevFeed: [BobFlow: Towards Human-Agent Collaborative Data Science Using Dataflows](<https://devfeed.tech/articles/bobflow-towards-human-agent-collaborative-data-science-using-dataflows-118114.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.05761>)

Author: Jiadong Bai, Yicong Huang, Chen Li

Published: 2026-11-05T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Databases](<https://devfeed.tech/sources/arxiv-databases.md>)

Topics: [Data Science](<https://devfeed.tech/topics/data-science.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [collaboration](<https://devfeed.tech/tags/collaboration.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-db](<https://devfeed.tech/tags/cs-db.md>), [cs-hc](<https://devfeed.tech/tags/cs-hc.md>), [data-science](<https://devfeed.tech/tags/data-science.md>), [dataflow](<https://devfeed.tech/tags/dataflow.md>), [large-language-models-llms](<https://devfeed.tech/tags/large-language-models-llms.md>), [python](<https://devfeed.tech/tags/python.md>), [react](<https://devfeed.tech/tags/react.md>), [sql](<https://devfeed.tech/tags/sql.md>)

### AI overview

BobFlow uses dataflows as a shared representation for human-agent data science: an agent iteratively builds and executes a flow, while analysts can inspect it and provide feedback. The authors argue this makes processing logic easier for analysts to understand and refine than Python or SQL scripts, whose syntax and execution details can be difficult to modify. A user study found BobFlow helped analysts understand, inspect, and verify agent-generated analyses. In experiments with proprietary and open-weight LLMs, it achieved accuracy score gains of up to 18.6 points while using just over one-third the inference cost of a script-based ReAct agent. The supplied text does not specify evaluation tasks or how the accuracy scores were measured.

### Source excerpt

arXiv:2611.05761v1 Announce Type: new Abstract: Recent advances in large language models (LLMs) have established human-agent collaboration as a promising paradigm for data science. Effective collaboration requires a shared computational abstraction that enables agents to reason about processing logic while allowing human analysts to understand, guide, and refine the analysis. Although scripts written in Python or SQL provide precise control, their syntax and execution details can be difficult for analysts to understand and modify. In this paper, we argue that dataflows are an effective abstraction for human-agent collaboration in data science. We introduce BobFlow, a system that combines the dataflow abstraction with the ReAct paradigm. It enables agents to iteratively construct and execute dataflows for data science tasks while allowing analysts to inspect the analysis and provide feedback. Our user study shows that BobFlow helps analysts understand, inspect, and verify agent-generated analyses. A thorough experimental evaluation across proprietary and open-weight LLMs shows accuracy score gains of up to 18.6 points at just over 1/3 of a script-based ReAct agent's inference cost.

## Surrogate Models for Stochastic Al-Cu Phase-Field Solidification

DevFeed: [Surrogate Models for Stochastic Al-Cu Phase-Field Solidification](<https://devfeed.tech/articles/surrogate-models-for-stochastic-al-cu-phase-field-solidification-116973.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.04695>)

Author: Simon Savukoski, Tatu Pinomaa

Published: 2026-11-04T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Data Science](<https://devfeed.tech/topics/data-science.md>), [Complex Systems](<https://devfeed.tech/topics/complex-systems.md>), [computational complexity](<https://devfeed.tech/topics/computational-complexity.md>)

Tags: [alloy](<https://devfeed.tech/tags/alloy.md>), [bridge](<https://devfeed.tech/tags/bridge.md>), [cond-mat-mtrl-sci](<https://devfeed.tech/tags/cond-mat-mtrl-sci.md>), [condition](<https://devfeed.tech/tags/condition.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [denoising](<https://devfeed.tech/tags/denoising.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [fields](<https://devfeed.tech/tags/fields.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [process](<https://devfeed.tech/tags/process.md>)

### AI overview

VP-DDBM uses the current phase and concentration fields plus a prescribed temperature field to predict the next state in two-dimensional Al-Cu solidification, then rolls predictions forward autoregressively. On two held-out thermal gradients, it reports lower long-horizon errors than deterministic U-Net and SI-ConvNeXt baselines for mean phase, mean Cu concentration, and leading-tip position. It also retains fine dendritic structure over long rollouts, while the U-Net progressively smooths morphology. The model predicts phase and concentration evolution about 73 times faster than the phase-field solver, suggesting potential for process-condition screening and ensemble studies.

### Source excerpt

arXiv:2611.04695v1 Announce Type: new Abstract: Phase-field models resolve microstructural evolution during rapid alloy solidification, but their computational cost limits long trajectories, stochastic ensembles, and studies across process conditions. We present a variance-preserving denoising diffusion bridge model (VP-DDBM) for state-conditioned prediction of two-dimensional Al-Cu solidification. Given the current phase, concentration, and prescribed temperature fields, the model predicts the subsequent state and is applied autoregressively. Across two held-out thermal gradients, VP-DDBM produces lower long-horizon errors than deterministic U-Net and SI-ConvNeXt baselines in domain-mean phase, domain-mean Cu concentration, and leading-tip position. It also retains fine dendritic structure over long rollouts, whereas the deterministic U-Net progressively smooths the morphology. VP-DDBM predicts the phase and concentration evolution approximately $73\times$ faster than the phase-field solver, showing potential for faster process-condition screening and ensemble-based microstructure studies.

## A Good Shooter or Just a Good Shot Selector? Using Python to Find Out

DevFeed: [A Good Shooter or Just a Good Shot Selector? Using Python to Find Out](<https://devfeed.tech/articles/a-good-shooter-or-just-a-good-shot-selector-using-python-to-find-out-114934.md>)

Original publisher: [Read original article](<https://towardsdatascience.com/a-good-shooter-or-just-a-good-shot-selector-using-python-to-find-out/>)

Author: Pol Marin

Published: 2026-11-03T14:00:00Z

Content type: article

Language: en

Sources: [Towards Data Science](<https://devfeed.tech/sources/towards-data-science.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Data Science](<https://devfeed.tech/topics/data-science.md>), [Python](<https://devfeed.tech/topics/python.md>), [scikit-learn](<https://devfeed.tech/topics/scikit-learn.md>), [Sports](<https://devfeed.tech/topics/sports.md>)

Tags: [aggregate](<https://devfeed.tech/tags/aggregate.md>), [analyst](<https://devfeed.tech/tags/analyst.md>), [analytics](<https://devfeed.tech/tags/analytics.md>), [approach](<https://devfeed.tech/tags/approach.md>), [arc](<https://devfeed.tech/tags/arc.md>), [article](<https://devfeed.tech/tags/article.md>), [aspect](<https://devfeed.tech/tags/aspect.md>), [based](<https://devfeed.tech/tags/based.md>), [basic](<https://devfeed.tech/tags/basic.md>), [basketball](<https://devfeed.tech/tags/basketball.md>), [basketball-analytics](<https://devfeed.tech/tags/basketball-analytics.md>), [bayes](<https://devfeed.tech/tags/bayes.md>), [benefit](<https://devfeed.tech/tags/benefit.md>), [better](<https://devfeed.tech/tags/better.md>), [build](<https://devfeed.tech/tags/build.md>), [building](<https://devfeed.tech/tags/building.md>), [can](<https://devfeed.tech/tags/can.md>), [data](<https://devfeed.tech/tags/data.md>), [data-science](<https://devfeed.tech/tags/data-science.md>), [editor-s-picks](<https://devfeed.tech/tags/editor-s-picks.md>), [logistic-regression](<https://devfeed.tech/tags/logistic-regression.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [python](<https://devfeed.tech/tags/python.md>)

### AI overview

A logistic regression model can estimate the make probability for each shot from features such as distance, shot value, type, corner location, and late shot-clock status. Multiplying expected field-goal percentage by two or three points yields expected points per shot (xPPS), which helps distinguish the value of a player's shot selection from points scored above or below expectation. The article uses randomized synthetic data for 12 players and cautions that residual overperformance is not proof of repeatable skill. Its simple model omits context such as contests and movement, and does not capture free throws, turnovers, rebounds, or teammate opportunities; calibration checks and future-season tests or empirical-Bayes shrinkage are suggested as ways to assess reliability.

### Source excerpt

In basketball, field-goal percentage tells us how often a player makes a shot. But what about the shots' difficulty? Let's use Python and machine learning to separate shot selection from shot making. The post A Good Shooter or Just a Good Shot Selector? Using Python to Find Out appeared first on Towards Data Science.

## Grading the Unannotated: Insights from the WorkRB Skill Extraction Challenge at RecSys-HR 2026

DevFeed: [Grading the Unannotated: Insights from the WorkRB Skill Extraction Challenge at RecSys-HR 2026](<https://devfeed.tech/articles/grading-the-unannotated-insights-from-the-workrb-skill-extraction-challenge-at-recsys-hr-2026-115455.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.03525>)

Author: Warre Veys, Luis Gasco, Jens-Joris Decorte

Published: 2026-11-03T05:00:00Z

Content type: news

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [AI Chat](<https://devfeed.tech/topics/ai-chat.md>), [AI-driven information retrieval](<https://devfeed.tech/topics/ai-driven-information-retrieval.md>), [Data Science](<https://devfeed.tech/topics/data-science.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [annotation](<https://devfeed.tech/tags/annotation.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cascade](<https://devfeed.tech/tags/cascade.md>), [challenge](<https://devfeed.tech/tags/challenge.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [domain](<https://devfeed.tech/tags/domain.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [hr](<https://devfeed.tech/tags/hr.md>), [leaderboard](<https://devfeed.tech/tags/leaderboard.md>), [llm](<https://devfeed.tech/tags/llm.md>), [methodology](<https://devfeed.tech/tags/methodology.md>), [model](<https://devfeed.tech/tags/model.md>), [models](<https://devfeed.tech/tags/models.md>), [next](<https://devfeed.tech/tags/next.md>), [ordering](<https://devfeed.tech/tags/ordering.md>), [recsys](<https://devfeed.tech/tags/recsys.md>)

### AI overview

The WorkRB Challenge evaluates skill extraction with graded relevance, revealing errors that binary benchmarks can miss. It assigns each candidate in a ranking window a 0-4 grade and scores rankings with nDCG@100. Because ternary judgments were too coarse, annotation uses four binary questions about domain, whether the activity is mentioned, granularity, and replacement of a gold positive; labels were checked through blind human and frontier-model re-annotation. The five datasets include four using vacancy sentences as queries and one using lexically distant skill forms. Submissions converged on retrieve-and-rerank pipelines, with gains attributed to removing nonsensical candidates rather than finding more positives. At the leaderboard's top, graded ordering was uncorrelated with binary evaluation, and graded labels showed larger gaps over baselines. The abstract also reports a substantial efficiency cost: sized submissions used at least 42 times, often several hundred times, the parameters of the baselines they beat.

### Source excerpt

arXiv:2611.03525v1 Announce Type: new Abstract: Skill extraction benchmarks score models against a few human-annotated positives and treat every other skill in the vocabulary as equally wrong. At the scale of ESCO (13,939 skills) this binary view can reward models that rank the annotated positive high for the wrong reason, and hides semantically nonsensical predictions. The WorkRB Challenge at RecSys-HR 2026 replaces it with graded relevance: every candidate skill in a model's ranking window carries a 0 to 4 grade, and the leaderboard scores nDCG@100 against these grades. Since ternary judgements proved too coarse for objective annotation, relevance is decomposed into a cascade of four binary questions (same domain, activity mentioned, right granularity, replacement of a gold positive) that a smaller LLM judge can answer reliably; the labels are validated with blind human and frontier-model re-annotation. This paper documents the annotation methodology, the construction of five benchmark datasets (four with vacancy sentences as queries, one with lexically distant skill surface forms), the validation of the LLM cascade, and insights from the submissions. The field converged on multi-stage retrieve-and-rerank pipelines, whose gains come from clearing nonsense out of the ranking window rather than finding more positives: at the top of the leaderboard, the graded ordering is uncorrelated with what binary evaluation would have said. Graded labels expose much bigger gaps between top submissions and baselines than binary evaluation does, and show that semantic-understanding skill recommendation is still wide open. The clear next direction is bringing efficiency back: every submitted system we could size uses at least 42x, typically several hundred times, more parameters than the baselines it beats.

## BanglishDepYT: Depression Detection in Native Bengali Social Media with Emotion-Preserving Preprocessing and Domain-Adaptive Pretraining

DevFeed: [BanglishDepYT: Depression Detection in Native Bengali Social Media with Emotion-Preserving Preprocessing and Domain-Adaptive Pretraining](<https://devfeed.tech/articles/banglishdepyt-depression-detection-in-native-bengali-social-media-with-emotion-preserving-preprocessing-and-domain-adaptive-pretraining-114577.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.01633>)

Author: Ayon Roy, Sadman Sharif, Syed Shams Islam

Published: 2026-11-03T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Data Science](<https://devfeed.tech/topics/data-science.md>), [managing large-scale text data in AI](<https://devfeed.tech/topics/managing-large-scale-text-data-in-ai.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>)

Tags: [annotation](<https://devfeed.tech/tags/annotation.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-cy](<https://devfeed.tech/tags/cs-cy.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [depression](<https://devfeed.tech/tags/depression.md>), [detection](<https://devfeed.tech/tags/detection.md>), [domain](<https://devfeed.tech/tags/domain.md>), [emotion](<https://devfeed.tech/tags/emotion.md>), [llm](<https://devfeed.tech/tags/llm.md>), [native](<https://devfeed.tech/tags/native.md>), [pretraining](<https://devfeed.tech/tags/pretraining.md>), [social-media](<https://devfeed.tech/tags/social-media.md>), [youtube](<https://devfeed.tech/tags/youtube.md>)

### AI overview

BanglishDepYT introduces a native YouTube dataset for Bengali depression detection, with 191,000 unlabeled comments and 1,000 samples verified by bilingual annotators. Its EPH preprocessing retains emoticons, romanized Bengali, negation, and emotional elongation: standard cleaning reduced Banglish token survival to zero and yielded SVM F1 of 0.7062, while EPH raised SVM F1 to 0.8570. Continuing SahajBERT pretraining on in-domain comments produced mean F1 of 0.8161 and MCC of 0.6272, outperforming the baseline significantly and matching MuRIL in the reported comparison. Qwen2.5-7B annotation had human agreement κ=0.611; pseudo-labeling 96,071 comments improved semi-supervised AUC from 0.8499 to 0.8731. These results are reported in the supplied abstract.

### Source excerpt

arXiv:2611.01633v1 Announce Type: cross Abstract: Depression affects millions of Bengali speakers across South Asia, yet automated detection systems have relied almost entirely on datasets translated from English. Such datasets cannot capture the emoticons, romanized Bengali words, and elongated spellings that characterize real expressions of distress. We introduce BanglishDepYT, a Bengali depression dataset collected natively from YouTube, comprising 191,000 unlabeled comments and 1,000 samples verified through dual bilingual annotation. We propose EPH (Emotion-Preserving Hybrid preprocessing), which converts emoticons to semantic tags, preserves Banglish tokens, protects negation words, and normalizes emotional elongation. A key finding is that preprocessing choice substantially affects performance on informal text: standard cleaning reduces the Banglish Survival Rate to zero (SVM F1: 0.7062), while EPH preserves signals (CESS: 0.9996 vs. 0.6694) and raises SVM F1 to 0.8570, an improvement of 0.1508. We further apply domain-adaptive pretraining (DAPT), continuing SahajBERT pretraining on unlabeled in-domain comments. DAPT-SahajBERT achieves the highest mean F1 (0.8161) and MCC (0.6272), significantly outperforming the baseline (F1: 0.7805, p = 0.0013, d = 1.450) and matching MuRIL (p = 0.589). These results suggest that targeted domain pretraining is a competitive alternative to large-scale multilingual pretraining. LLM-assisted annotation using Qwen2.5-7B achieves substantial human agreement (kappa = 0.611), enabling pseudo-labeling of 96,071 comments. Semi-supervised training improves AUC from 0.8499 to 0.8731 (p = 0.044), offering a scalable path to expanding annotation. All code, models, and datasets are publicly released to support reproducible research.

## Benign population landscapes do not transfer to empirical risk minimization in nonsmooth optimization

DevFeed: [Benign population landscapes do not transfer to empirical risk minimization in nonsmooth optimization](<https://devfeed.tech/articles/benign-population-landscapes-do-not-transfer-to-empirical-risk-minimization-in-nonsmooth-optimization-114539.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.01384>)

Author: Penghao Wang, Mateo D\\'iaz, Liwei Jiang

Published: 2026-11-03T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Operations research and optimization](<https://devfeed.tech/topics/operations-research-and-optimization.md>), [Data Science](<https://devfeed.tech/topics/data-science.md>)

Tags: [arxiv](<https://devfeed.tech/tags/arxiv.md>), [cross](<https://devfeed.tech/tags/cross.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data-science](<https://devfeed.tech/tags/data-science.md>), [gaussian](<https://devfeed.tech/tags/gaussian.md>), [global](<https://devfeed.tech/tags/global.md>), [guarantees](<https://devfeed.tech/tags/guarantees.md>), [local](<https://devfeed.tech/tags/local.md>), [lower-bound](<https://devfeed.tech/tags/lower-bound.md>), [math-oc](<https://devfeed.tech/tags/math-oc.md>), [measurements](<https://devfeed.tech/tags/measurements.md>), [objectives](<https://devfeed.tech/tags/objectives.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [risk](<https://devfeed.tech/tags/risk.md>), [sharp](<https://devfeed.tech/tags/sharp.md>)

### AI overview

In robust phase retrieval with Gaussian measurements, benign geometry of the population objective does not ensure benign geometry for the empirical objective. The authors report that, despite the population objective being benign, the empirical objective has sharp spurious local minima with probability bounded away from zero as sample size tends to infinity; this lower bound is independent of dimension. The result challenges the population-to-empirical transfer strategy used for smooth optimization, where concentration of gradients and Hessians is used to transfer landscape properties, in the weakly convex nonsmooth setting.

### Source excerpt

arXiv:2611.01384v1 Announce Type: cross Abstract: Benign landscapes provide a geometric explanation for the success of iterative methods on nonconvex problems arising in data science. The landscape is benign if every local minimizer is global and every other critical point is a strict saddle. For methods that avoid strict saddles, convergence to a critical point therefore implies convergence to a global minimizer. For smooth stochastic optimization, a common strategy for establishing such landscapes proceeds in two steps: $(i)$ establish benign geometry for the population objective---which is often easier to analyze---and $(ii)$ transfer this geometry to the empirical objective using concentration of gradients and Hessians. Recent work has extended saddle-point avoidance guarantees to weakly convex nonsmooth optimization through the notion of active strict saddles. We show, however, that benign population geometry need not transfer to empirical objectives in this setting. Specifically, for robust phase retrieval with Gaussian measurements---a well-studied and highly structured problem---the population objective is benign. Nevertheless, the empirical objective has sharp spurious local minima with probability bounded away from zero as the sample size tends to infinity. This asymptotic probability lower bound is independent of the dimension.

[Next page](<https://devfeed.tech/topics/data-science.md?cursor=WyIyMDI2LTExLTAzVDA1OjAwOjAwKzAwOjAwIiwgIjhlYTQ2Mzc2LWRmYjMtNDUzMi1iNjcxLWEwMDk0M2JiZjRmYSJd>)