# Labeling Published articles for Labeling. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Cross-Asset Knowledge Transfer for Early Fault Detection of Data-Scarce Assets Without Target Fault Data DevFeed: [Cross-Asset Knowledge Transfer for Early Fault Detection of Data-Scarce Assets Without Target Fault Data]() Original publisher: [Read original article]() Author: Zekai Wang, Yuxuan Li, Zhangyue Shi Published: 2026-11-06T05:00:00Z Content type: article Language: en Sources: [arXiv: Machine Learning]() Topics: [reliability-engineering](), [Neural Network](), [outlier-detection](), [Feature Engineering]() Tags: [adversarial](), [classification](), [control](), [cs-lg](), [dataset](), [domain](), [fault-detection](), [features](), [horizon](), [labeling](), [network](), [neural](), [noise](), [precision](), [robustness](), [synthetic](), [target](), [theoretical](), [zero-shot]() ### AI overview The proposed ADANN framework transfers early fault detection from a data-rich asset to a target asset with normal data only and no observed target faults. It labels source run-to-failure data using an exponentially weighted moving average chart, then combines domain-adversarial training and a Maximum Mean Discrepancy penalty with an Adapted Classification-Domain loss that anchors the classifier's normal region to the target's normal data. An ahead-of-time labeling scheme aims to detect incipient faults. Theoretical analysis assumes exact classification and characterizes alignment that keeps source fault features separated from target normal features. On a public run-to-failure bearing dataset, ADANN had the highest mean F1-score among four transfer baselines and a nominally significant precision gain over the strongest baseline; robustness was also examined under synthetic noise. ### Source excerpt arXiv:2611.06194v2 Announce Type: replace Abstract: Reliability management of industrial fleets depends on early fault warnings, yet lifecycle data across similar assets are highly imbalanced. A few assets accumulate run-to-failure histories, while newly installed or rarely failing assets provide only normal-condition data. This paper asks how diagnostic knowledge can be transferred from a data-rich source asset to a data-scarce target asset when no fault data have been observed on the target. We formulate cross-asset early fault detection as a zero-shot domain adaptation problem, in which zero-shot refers to the absence of target fault samples, and propose a framework that requires only normal data from the target. The framework comprises (1) a statistical process control step that labels the source asset run-to-failure data with an exponentially weighted moving average chart, reducing manual annotation to verifying flagged segments, (2) an Adapted Domain-Adversarial Neural Network (ADANN) that combines domain-adversarial training and a Maximum Mean Discrepancy penalty with an Adapted Classification-Domain (ACD) loss, which anchors the classifier normal region to the target own normal data, and (3) an ahead-of-time labeling scheme with a prediction horizon that targets incipient faults. Under an exact-classification assumption, a theoretical analysis characterizes the domain-adversarial term for a normal-only target and the alignment state in which source fault features remain separated from target normal features. On a public run-to-failure bearing dataset, ADANN attains the highest mean F1-score among four transfer baselines, with a nominally significant precision gain over the strongest baseline, and its robustness is examined under synthetic noise. ## Volatility-Aware Extreme Event Detection in High-Frequency Financial Markets DevFeed: [Volatility-Aware Extreme Event Detection in High-Frequency Financial Markets]() Original publisher: [Read original article]() Author: Maorufa Zaman, Haris Md Sahed Published: 2026-11-04T05:00:00Z Content type: article Language: en Sources: [arXiv: Machine Learning]() Topics: [Finance](), [Temporal data](), [clustering](), [outlier-detection](), [Risk]() Tags: [bitcoin](), [clustering](), [cross-validation](), [cryptocurrency](), [cs-lg](), [detection](), [evaluation](), [event](), [financial](), [financial-markets](), [labeling](), [learning](), [limit-order-book](), [machine-learning](), [markets](), [model](), [time-series]() ### AI overview Using high-frequency Bitcoin limit order book data, the study reports that labeling events to include both large future returns and high-volatility regimes improved XGBoost detection of rare events: PR-AUC was approximately 0.40 versus 0.06 for the baseline, using time-series cross-validation and imbalance-aware evaluation. The authors attribute the gain to target design informed by volatility clustering and note that it can matter more than model complexity. The supplied abstract does not describe further methods or limitations. ### Source excerpt arXiv:2607.17555v2 Announce Type: replace Abstract: Predicting extreme price movements in high-frequency financial markets is a challenging task due to non-stationarity, heavy-tailed return distributions, and severe class imbalance. In particular, rare but impactful events are often difficult to detect using conventional modeling approaches, which typically treat extreme movements as isolated observations. This study proposes a volatility-aware approach for extreme event detection using high-frequency Bitcoin limit order book (LOB) data. Motivated by empirical evidence of volatility clustering, the target formulation is extended to incorporate both large future returns and high-volatility regimes. This redefinition increases the proportion of informative samples and aligns the learning objective with the underlying market dynamics. Using a tree-based model (XGBoost) with time-series cross-validation and imbalance-aware evaluation, the proposed method achieves a Precision-Recall AUC of approximately 0.40, significantly outperforming the baseline formulation with a PR-AUC of around 0.06. This represents more than a sixfold improvement in detecting rare events. The results highlight that target design plays a critical role in financial machine learning, often exceeding the impact of model complexity. By incorporating volatility structure into the labeling process, the proposed approach provides a more effective and realistic framework for extreme event detection in high-frequency cryptocurrency markets. ## When Ground Truth Is Not Ground Truth: Automatic Labeling for Hallucination Detection DevFeed: [When Ground Truth Is Not Ground Truth: Automatic Labeling for Hallucination Detection]() Original publisher: [Read original article]() Author: Rounak Sen, Divya Kumari, Deeksha Varshney Published: 2026-11-04T05:00:00Z Content type: article Language: en Sources: [arXiv: Artificial Intelligence]() Topics: [Hallucination detection](), [Ground truth / benchmark quality](), [LLM evaluation / benchmarking](), [Factuality scoring in AI]() Tags: [abstract](), [arxiv](), [automatic](), [automatically](), [based](), [better](), [can](), [capture](), [challenge](), [choice](), [construction](), [cs-ai](), [cs-cl](), [datasets](), [design](), [detection](), [english](), [experiments](), [generator](), [hallucination-detection](), [hallucinations](), [labeling]() ### AI overview Automatic labels for hallucination detection can be unreliable, and detector performance depends substantially on which judge creates the labels. In experiments spanning 15 automatic judges, seven detection baselines, two generators, two QA datasets, and five languages, human validation found QGen-9B had the lowest overall misclassification rate (12.86%); BLEURT and BLEURT-n were substantially higher at 47.23% and 47.32%. The authors argue that similarity scores are a weak proxy for factuality: BLEURT is English-centric, has an unbounded score range, and was designed to measure similarity rather than factual correctness. The abstract therefore points to a need for judges that better assess semantic and factual correctness. ### Source excerpt arXiv:2611.04697v1 Announce Type: cross Abstract: Hallucinations in Large Language Models (LLMs) remain a major challenge, as they can produce misleading, factually incorrect, or unverifiable information, thereby undermining their trustworthiness and reliability. However, reliable ground-truth labels remain a key challenge in evaluating hallucination detectors. Recent studies often construct labels automatically by comparing model generations with reference answers using similarity metrics such as BLEURT. This implicitly treats BLEURT similarity as a proxy for hallucination, despite its English-centric supervision, unbounded score range, and design as a similarity metric rather than a factuality measure, making threshold-based ground-truth construction potentially unreliable. Through comprehensive experiments using 15 automatic supervision judges, seven hallucination detection baselines, two generator models, two QA datasets, and five languages, we demonstrate that detector performance varies substantially with the choice of supervision judge. Human validation reveals substantial differences in labeling reliability, with QGen-9B achieving the lowest overall misclassification rate (12.86%), while BLEURT and BLEURT-n exhibit rates of 47.23% and 47.32%, respectively. Our findings highlight the limitations of automatic similarity-based labeling and the need for more reliable judging methods that better capture semantic and factual correctness. ## One Strategy Does Not Fit All: Rethinking Data Labeling for Just-in-Time Vulnerability Prediction DevFeed: [One Strategy Does Not Fit All: Rethinking Data Labeling for Just-in-Time Vulnerability Prediction]() Original publisher: [Read original article]() Author: Quan Tran, Duong Nguyen, Thanh Le-Cong, Triet Huynh Minh Le, M. Ali Babar, Hoang Dieu Vu, Quyet-Thang Huynh Published: 2026-11-04T05:00:00Z Content type: article Language: en Sources: [arXiv: Software Engineering]() Topics: [vulnerability management](), [Data Quality](), [Model Development]() Tags: [classification](), [cs-se](), [data](), [dataset](), [datasets](), [jit](), [kernel](), [labeling](), [linux](), [llm](), [llm-assisted](), [localization](), [model](), [models](), [openssl](), [performance](), [prediction](), [quality-assurance](), [security](), [strategy](), [vulnerability]() ### AI overview Labeling strategy affects JIT vulnerability prediction differently by task, so the study finds no single approach works best across the board. The researchers built 28 datasets from OpenSSL and the Linux kernel using four vulnerability-fixing commit (VFC) and four vulnerability-introducing commit (VIC) labeling strategies, then evaluated six models on classification, prioritization, and localization. Manual VFC labeling paired with vulnerability-specific VIC labeling performed most reliably for classification and prioritization, significantly outperforming 11 of 15 alternatives. For localization, precision-first VFC labeling with LLM-assisted VIC labeling performed best, significantly outperforming 7 of 15 alternatives. ### Source excerpt arXiv:2611.05013v1 Announce Type: new Abstract: Just-In-Time Vulnerability Prediction (JIT-VP) identifies security risks in code changes at commit time, enabling early feedback before vulnerabilities enter software systems. However, its practical reliability is challenged by the difficulty of constructing accurate labels. Multiple strategies with different trade-offs have been proposed to identify vulnerability-fixing commits (VFCs) and vulnerability-introducing commits (VICs) for dataset construction, leading to substantially different datasets. This makes it difficult to separate model performance from dataset construction. Moreover, real-world quality assurance requires not only classification but also prioritizing high-risk commits and localizing vulnerable causes. This paper presents an extensive empirical study of how labeling strategies affect these downstream tasks. We constructed 28 datasets using four VFC and four VIC labeling strategies across OpenSSL and Linux kernel, then evaluated six representative JIT-VP models for classification, prioritization, and localization. Our results show that labeling effectiveness is strongly task-dependent. Manual VFC labeling combined with vulnerability-specific VIC labeling provides the most reliable performance for classification and prioritization, significantly outperforming 11 of 15 alternatives. Meanwhile, precision-first VFC labeling combined with LLM-assisted VIC labeling performs best for localization, significantly outperforming 7 of 15 alternatives. These findings demonstrate that no single labeling strategy is universally optimal and highlight the importance of task-oriented dataset construction. ## How Far from Clinical Deployment? Evaluating the Complete Unsupervised Domain Adaptation Pipeline in Medical Imaging DevFeed: [How Far from Clinical Deployment? Evaluating the Complete Unsupervised Domain Adaptation Pipeline in Medical Imaging]() Original publisher: [Read original article]() Author: Yiheng Xiong, Luisa Gall\\'ee, Daniel Santak Wolf, Heiko Hillenhagen, Michael G\\"otz Published: 2026-11-04T05:00:00Z Content type: article Language: en Sources: [arXiv: Artificial Intelligence]() Topics: [Medical imaging](), [Machine learning](), [model-deployment]() Tags: [algorithm](), [cs-ai](), [cs-cv](), [datasets](), [deployment](), [imaging](), [labeling](), [medical](), [medical-imaging](), [performance](), [pipeline](), [selection]() ### AI overview The study finds that unsupervised domain adaptation (UDA) can produce capable medical imaging models, but choosing one without target labels remains a major deployment bottleneck. Across 11 cross-domain scenarios from nine datasets, the authors evaluated 10 UDA algorithms and 13 label-free validators on more than 80,000 trained models. Validator-selected models often fell substantially short of the best available target-domain model, and no validator was consistently reliable. Ensembling and using a small target-labeling budget narrowed this performance gap but did not eliminate it. The findings suggest that clinical readiness depends on improving model selection as well as adaptation. ### Source excerpt arXiv:2608.12035v3 Announce Type: replace-cross Abstract: Deploying unsupervised domain adaptation (UDA) in clinical practice requires choosing which algorithm to use and which of its trained models to ship. However, the deployment (target) domain is unlabeled, so models cannot be evaluated directly on it, leaving it unclear which to select. We address this by evaluating the complete UDA pipeline, considering both adaptation and label-free selection together. Our study covers eleven clinically relevant cross-domain scenarios from nine medical imaging datasets, with ten UDA algorithms and 13 label-free selection methods (validators), evaluating over 80,000 trained models in total. By this, we find that a capable adapted model usually exists, but identifying it without target labels is difficult: the validator-selected models leave a large and structural target performance gap to the best available one, with no evaluated validator consistently reliable. Towards closing it, we explore two strategies, ensembling and a small target-labeling budget; both narrow this gap but do not close it entirely. Overall, deployable UDA depends on the complete pipeline; addressing the less explored selection step could bring much of current UDA closer to clinical use. Our code is open sourced at \href{https://xiongyiheng.github.io/complete-uda-pipeline-site/}{Project Page}. ## Quantum Advantage for Distributed Symmetry Breaking DevFeed: [Quantum Advantage for Distributed Symmetry Breaking]() Original publisher: [Read original article]() Author: Maxime Flin, Longcheng Li, Jukka Suomela Published: 2026-11-04T05:00:00Z Content type: article Language: en Sources: [arXiv: Distributed Computing]() Topics: [distributed algorithm](), [Quantum Computing](), [quantum mechanics]() Tags: [3](), [abstract](), [algorithm](), [all](), [arxiv](), [bounded](), [can](), [colors](), [complexity](), [cs-dc](), [cycles](), [distributed](), [examples](), [first](), [graph](), [graphs](), [independent](), [labeling](), [quant-ph](), [quantum](), [quantum-advantage](), [symmetry]() ### AI overview The paper reports a distributed quantum algorithm that 3-colors cycles in O(1) rounds with high probability. As a consequence, it claims every locally checkable labeling problem solvable in O(log* n) rounds in the classical LOCAL model can be solved in O(1) rounds with high probability in the quantum-LOCAL model, including maximal independent set and maximal matching in bounded-degree graphs. The authors characterize these as the first natural graph-problem examples of an asymptotic quantum advantage in the LOCAL model; the supplied abstract gives no algorithmic details beyond these results. ### Source excerpt arXiv:2609.26788v2 Announce Type: replace-cross Abstract: We present a distributed quantum algorithm that $3$-colors cycles in $O(1)$ rounds, with high probability. It follows that all locally checkable labeling problems (LCLs) that have round complexity $O(\log^* n)$ in the classical LOCAL model can be solved in $O(1)$ rounds in the quantum-LOCAL model, with high probability; this includes problems such as maximal independent set and maximal matching in bounded-degree graphs. This presents the first natural examples of graph problems with an asymptotic distributed quantum advantage for the LOCAL model; all prior examples that separate LOCAL and quantum-LOCAL are artificial problems constructed merely for the sake of demonstrating quantum advantage. ## Pharos-ESG: A Framework for Multimodal Parsing, Contextual Narration, and Hierarchical Labeling of ESG Report DevFeed: [Pharos-ESG: A Framework for Multimodal Parsing, Contextual Narration, and Hierarchical Labeling of ESG Report]() Original publisher: [Read original article]() Author: Yan Chen, Yu Zou, Jialei Zeng, Haoran You, Xiaorui Zhou, Aixi Zhong Published: 2026-10-30T04:00:00Z Content type: article Language: en Sources: [arXiv: Artificial Intelligence]() Topics: [document ai](), [Artificial Intelligence](), [Data analysis]() Tags: [cs-ai](), [esg](), [framework](), [labeling](), [multimodal](), [narration](), [parsing](), [report]() ### AI overview Pharos-ESG converts ESG reports into structured representations to address disorderly reading order from irregular layouts and implicit hierarchies in lengthy content. It models reading order using layout flow, segments content with table-of-contents anchors, and turns visual elements into contextual natural-language descriptions through multimodal aggregation. Outputs include ESG, GRI, and sentiment labels. The abstract reports that it outperforms dedicated document-parsing systems and general-purpose multimodal models on annotated benchmarks, without providing scores or experimental details. The authors also introduce Aurora-ESG, a public dataset spanning Mainland China, Hong Kong, and U.S. markets, with structured multimodal content and fine-grained layout and semantic annotations. ### Source excerpt arXiv:2511.16417v4 Announce Type: replace Abstract: Environmental, Social, and Governance (ESG) principles are reshaping the foundations of global financial governance, transforming capital allocation architectures, regulatory frameworks, and systemic risk coordination mechanisms. However, as the core medium for assessing corporate ESG performance, the ESG reports present significant challenges for large-scale understanding, due to chaotic reading order from slide-like irregular layouts and implicit hierarchies arising from lengthy, weakly structured content. To address these challenges, we propose Pharos-ESG, a unified framework that transforms ESG reports into structured representations through multimodal parsing, contextual narration, and hierarchical labeling. It integrates a reading-order modeling module based on layout flow, hierarchy-aware segmentation guided by table-of-contents anchors, and a multi-modal aggregation pipeline that contextually transforms visual elements into coherent natural language. The framework further enriches its outputs with ESG, GRI, and sentiment labels, yielding annotations aligned with the analytical demands of financial research. Extensive experiments on annotated benchmarks demonstrate that Pharos-ESG consistently outperforms both dedicated document parsing systems and general-purpose multimodal models. In addition, we release Aurora-ESG, the first large-scale public dataset of ESG reports, spanning Mainland China, Hong Kong, and U.S. markets, featuring unified structured representations of multi-modal content, enriched with fine-grained layout and semantic annotations to better support ESG integration in financial governance and decision-making. ## A New AFIB Detection Method with Deep Neural Networks DevFeed: [A New AFIB Detection Method with Deep Neural Networks]() Original publisher: [Read original article]() Author: Sara Gjorgjieva, Ana Angjelevska, Dimitar Trajanov, Marjan Gusev Published: 2026-10-30T04:00:00Z Content type: article Language: en Sources: [arXiv: Machine Learning]() Topics: [Deep neural networks](), [Deep learning](), [Feature Engineering](), [Neural Network](), [Artificial Intelligence](), [generalization in machine learning]() Tags: [benchmark-dataset](), [cs-lg](), [deep-neural-networks](), [detection](), [ecg](), [feature-engineering](), [labeling](), [machine-learning](), [neural-networks](), [outliers](), [score]() ### AI overview The proposed AFIB detector uses a time series of successive heart-rate differences (dBPM) instead of traditional inter-beat intervals. It labels inputs into five classes, trains on intervals identified as clean, and treats the others as outliers. The study evaluates conventional machine-learning methods and neural networks using MITDB and LTAFDB; its best-performing model, a Deep Neural Network, reports an F1 score of 95.18% for binary AFIB classification on a benchmark dataset distinct from the training set. The supplied abstract does not specify the benchmark dataset or provide further evaluation details. ### Source excerpt arXiv:2610.39665v1 Announce Type: new Abstract: Early detection of atrial fibrillation (AFIB), characterized by irregular heart rhythms, can potentially save thousands of lives annually [20]. Numerous studies have attempted to address this challenge using various algorithms and methodologies. This paper introduces a novel and robust method that enhances the precision of AFIB detection. Neural Networks (NN) are widely recognized for their ability to identify complex electrocardiogram (ECG) patterns, facilitating the accurate detection of irregular heart rhythms. This study explores multiple conventional machine learning algorithms alongside neural network approaches to develop a robust AFIB detection system. To evaluate the effective- ness and robustness of the proposed method, the MIT BIH arrhythmia database (MITDB) and the Long-term Atrial Fibrillation Database (LTAFDB), both available on the Physionet site. Our approach presents a novel feature engineering technique, employing specific features such as a time series of the differences in successive heart rates (dBPM) rather than traditional inter-beat intervals. The model analyzes inputs labeled into five distinct classes, using only clean intervals identified through a new labeling method for training while treating others as outliers. This strategy ensures a more accurate and reliable AFIB detection process. The best-performing model, a Deep Neural Network (DNN), achieved an F1 score of 95.18% for binary atrial fibrillation classification on a benchmark dataset distinct from the training set. Models trained using our approach have demonstrated superior performance than previously published models. ## A Highly Efficient Diversity-based Input Selection for DNN Improvement Using VLMs DevFeed: [A Highly Efficient Diversity-based Input Selection for DNN Improvement Using VLMs]() Original publisher: [Read original article]() Author: Amin Abbasishahkoo, Mahboubeh Dadkhah, Lionel Briand Published: 2026-10-30T04:00:00Z Content type: article Language: en Sources: [arXiv: Software Engineering]() Topics: [Artificial Intelligence](), [ImageNet]() Tags: [cs-cv](), [cs-se](), [deep-neural-networks](), [diversity](), [dnn](), [fine-tuning](), [hybrid](), [image](), [improvement](), [input](), [labeling](), [margin](), [metric](), [model](), [models](), [neural](), [novel](), [performance](), [scalability](), [selection](), [time](), [vlms]() ### AI overview The paper proposes Concept-Based Diversity (CBD), a VLM-based metric for selecting informative image inputs to label for DNN fine-tuning. The abstract reports that CBD correlates strongly with Geometric Diversity while taking a fraction of its computation time. A hybrid selector combining CBD with the Margin uncertainty metric reportedly outperforms six state-of-the-art baselines across varied models, input sets, and selection budgets. Its selection time is described as close to simple uncertainty-based methods such as Margin, including on ImageNet-sized sets. The evidence is limited to the abstract and gives no implementation details or quantitative results. ### Source excerpt arXiv:2601.08024v3 Announce Type: replace-cross Abstract: Maintaining or improving the performance of Deep Neural Networks (DNNs) through fine-tuning requires labeling newly collected inputs, a process that is often costly and time-consuming. To alleviate this problem, input selection approaches have been developed in recent years to identify small, yet highly informative subsets for labeling. Diversity-based selection is one of the most effective approaches for this purpose. However, they are often computationally intensive and lack scalability for large input sets, limiting their practical applicability. To address this challenge, we introduce Concept-Based Diversity (CBD), a novel and highly efficient diversity metric for image inputs that leverages Vision-Language Models (VLMs). Our results show that CBD exhibits a strong correlation with Geometric Diversity (GD), an established diversity metric, while requiring only a fraction of its computation time. Building on this finding, we propose a hybrid input selection approach that combines CBD with Margin, a simple uncertainty metric. We conduct a comprehensive evaluation across a diverse set of DNN models, input sets, selection budgets, and six most effective state-of-the-art selection baselines. The results demonstrate that the CBD-based selection consistently outperforms all baselines at guiding input selection to improve the DNN model. Furthermore, the CBD-based selection approach remains highly efficient, requiring selection times close to those of simple uncertainty-based methods such as Margin, even on larger input sets like ImageNet. These results confirm not only the effectiveness and computational advantage of the CBD-based approach, particularly compared to hybrid baselines, but also its scalability in repetitive and extensive input selection scenarios. ## A Utility-Mining-Driven Active Learning Approach for Analyzing Clickstream Sequences DevFeed: [A Utility-Mining-Driven Active Learning Approach for Analyzing Clickstream Sequences]() Original publisher: [Read original article]() Author: Danny Y. C. Wang, Lars Arne Jordanger, Jerry Chun-Wei Lin Published: 2026-10-30T04:00:00Z Content type: article Language: en Sources: [arXiv: Machine Learning]() Topics: [Predictive AI](), [Data science and engineering](), [Data Mining & Modeling]() Tags: [behaviors](), [commerce](), [cost](), [cs-lg](), [data-processing](), [experiments](), [high-quality-data](), [labeling](), [learning](), [machine-learning](), [mining](), [model-training](), [modeling](), [performance](), [prediction](), [sequences](), [settings](), [shap](), [strategy](), [utility](), [values]() ### AI overview HUSPM-SHAP uses SHAP values with high-utility sequential pattern mining to select e-commerce clickstream data for model training, aiming to reduce labeling needs while retaining predictive performance for purchase and non-purchase behavior. The abstract reports strong results across diverse scenarios and notes that settings for positive and negative SHAP values affect which patterns are mined. It does not provide experiment details or quantify the labeling reduction. ### Source excerpt arXiv:2410.07282v2 Announce Type: replace Abstract: In rapidly evolving e-commerce industry, the capability of selecting high-quality data for model training is essential. This study introduces the High-Utility Sequential Pattern Mining using SHAP values (HUSPM-SHAP) model, a utility mining-based active learning strategy to tackle this challenge. We found that the parameter settings for positive and negative SHAP values impact the model's mining outcomes, introducing a key consideration into the active learning framework. Through extensive experiments aimed at predicting behaviors that do lead to purchases or not, the designed HUSPM-SHAP model demonstrates its superiority across diverse scenarios. The model's ability to mitigate labeling needs while maintaining high predictive performance is highlighted. Our findings demonstrate the model's capability to refine e-commerce data processing, steering towards more streamlined, cost-effective prediction modeling. ## From Generation to Scoring: Scaling LLM-Based Semantic Labeling at LinkedIn DevFeed: [From Generation to Scoring: Scaling LLM-Based Semantic Labeling at LinkedIn]() Original publisher: [Read original article]() Author: Jian Sheng, Sriram Vasudevan, Sundara Raman Ramachandran, Yang Hu, Junyu Zhou, Xiaojing Ma, Haolan Qin, Tiffany Zhou, Vedant Vipul Jhaveri, Ashwini Bhingare, Chinmay Kini, Qing Lan, Xiaoqing Wang, Sul Published: 2026-10-28T04:00:00Z Content type: article Language: en Sources: [arXiv: Distributed Computing](), [arXiv: Machine Learning]() Topics: [LLM optimization](), [Large Language Model](), [Machine learning](), [Enhancing AI model performance]() Tags: [api](), [attention](), [batch](), [cs-dc](), [cs-ir](), [cs-lg](), [fine-tuning](), [generation](), [inference](), [labeling](), [language-model](), [large-language-model](), [latency](), [linkedin](), [llm](), [model-agnostic](), [model-serving](), [multiple](), [offline](), [ray](), [scaling](), [score](), [semantic]() ### AI overview LinkedIn's profile qualification labeler replaces packed YES/NO generation with semantic scoring: each candidate gets an answer-token score from a prefill-only pass, and Multiple-Item Scoring (MIS) scores hundreds of candidates in one sequence with attention masked between them. Reordering candidates changes at most 0.01% of decisions; scoring matches packed generation's agreement with reference labels using about 4.7 times fewer training pairs. The MIS stack improves throughput over the deployed stack by 1.24x offline and 1.9x in model serving under a one-second p99 latency target. Offline batch scoring is implemented as a reusable, model-agnostic Ray Data API proposed for upstream adoption; serving uses MIS support contributed to SGLang. ### Source excerpt arXiv:2610.36383v1 Announce Type: new Abstract: For efficiency, industrial large language model (LLM) labelers often pack many candidate labels into one prompt and generate a YES/NO answer for each, even when each decision depends only on the entity and one candidate. Generating these independent decisions as one answer list couples them, spends decoding on the list, and relies on the list being well-formed. In this paper, we present an end-to-end recipe that reformulates the labeler as semantic scoring, fine-tuning it on single-candidate prompts to return each decision as one answer-token score from a prefill-only pass. We also adapt Multiple-Item Scoring (MIS) to score hundreds of candidates in one sequence with attention masked between them. Deployed for profile qualification labeling at LinkedIn, semantic scoring changes at most 0.01% of decisions when candidates are reordered and matches the agreement with reference labels that packed generation reaches only with about 4.7x as many training pairs. The MIS-based stack raises throughput over the deployed stack by 1.24x offline and 1.9x in model serving under a one-second p99 latency target. For offline batch inference, we implement MIS-based scoring as a reusable, model-agnostic Ray Data API proposed for upstream adoption. Model serving uses MIS support contributed to SGLang. ## ClickHouse-Powered Admin Analytics and Leaderboard Fixes DevFeed: [ClickHouse-Powered Admin Analytics and Leaderboard Fixes]() Original publisher: [Read original article]() Published: 2026-10-28T00:00:00Z Content type: release Language: en Sources: [devfeed.tech Changelog]() Topics: [Analytics Pipeline](), [bugfixes](), [Data pipelines]() Tags: [admin](), [administration](), [analytics](), [charts](), [clickhouse](), [labeling](), [leaderboard](), [leaderboards](), [migrations](), [operational](), [overview](), [pipelines](), [postgresql](), [processes](), [recovery](), [release](), [reporting](), [reports](), [required](), [responsive]() ### AI overview Administration pipelines, Overview charts, and analytical totals now use ClickHouse instead of PostgreSQL reporting tables, reducing reporting load and helping admin pages stay responsive as data scales. Charts can retry independently while reports prepare, and historical data is imported as part of the migration. Leaderboards now require a public display name or username, preventing repeated anonymous rows and improving top-ten and "You" rank matching. Recovery is safer when worker state is missing or incomplete. Self-hosted upgrades require migrations, deploying the new applications, draining older processes, and running the historical importer in the recommended order. ### Source excerpt This release focuses on faster, more reliable administration reporting, along with a couple of important correctness and stability fixes. Smoother admin reporting with ClickHouse analytics Administration pipelines, Overview charts, and analytical totals are now served from ClickHouse instead of PostgreSQL reporting tables. This reduces reporting load and helps keep admin pages responsive as data scales. Charts can now retry independently while reports are preparing, so you are less likely to hit a dead end while waiting for results. Historical data is imported automatically as part of the move. Fixes and stability updates Leaderboards now require a public display name or username to be included. This removes repeated anonymous rows, keeps the top ten accurate, and avoids incorrectly labeling someone else as "You" when matching your rank. Operational recovery is now safer when worker state is missing, reducing the chance of recovery actions failing due to absent or incomplete state. Upgrade notes If you self-host, review the analytics rollout guide before updating. You will need to complete the required migrations, deploy the new applications, drain older processes, and run the historical importer in the recommended order. ## Cycle-of-Thought: Self-Evaluative Iterative Reasoning for Table Visual Question Answering in Document Images DevFeed: [Cycle-of-Thought: Self-Evaluative Iterative Reasoning for Table Visual Question Answering in Document Images]() Original publisher: [Read original article]() Author: Somraj Gautam, Gaurav Harit Published: 2026-10-26T04:00:00Z Content type: article Language: en Sources: [arXiv: Artificial Intelligence]() Topics: [llm-reasoning](), [Enhancing AI model performance](), [computational complexity](), [Artificial Intelligence]() Tags: [abstract](), [analysis](), [annotation](), [benchmarks](), [cs-ai](), [cs-cl](), [cs-cv](), [data-efficiency](), [errors](), [feedback](), [fine-tuning](), [framework](), [generator](), [hallucinations](), [images](), [improvement](), [inference](), [labeling](), [language](), [language-models](), [lightweight](), [reasoning](), [table]() ### AI overview Self-RefineVQA improves table visual question answering through inference-time self-correction: one vision-language model iteratively generates an answer, critiques it, and refines it through prompt-guided re-evaluation. The abstract reports consistent accuracy gains on tabular VQA benchmarks for monolingual and cross-lingual tasks, with especially strong gains for smaller models (2B-7B parameters), which it says can rival larger counterparts. The method requires no labeled training data or model updates, avoiding the annotation burden of supervised fine-tuning, but increases inference cost linearly. These claims are based on the abstract; benchmark details and quantitative results are not provided. ### Source excerpt arXiv:2610.32423v1 Announce Type: cross Abstract: Recent advances in Vision-Language Models (VLMs) have significantly improved Visual Question Answering (VQA) performance, yet their reasoning over structured content such as tables remains inconsistent, often plagued by spatial misalignment and logical hallucinations. Existing approaches for improving table VQA typically rely on architectural modifications or expensive supervised fine-tuning (SFT), which impose high barriers to entry due to the need for large-scale, expert-annotated datasets and significant computational resources. In this work, we propose Self-RefineVQA, a training-free, inference-time self-correction framework that improves table VQA performance without additional labeled data or model updates. Inspired by human revision behavior, our method enables a single VLM to iteratively act as a generator, critic, and refiner, forming a closed self-feedback loop that corrects common reasoning errors through prompt-guided re-evaluation. Experiments on tabular VQA benchmarks demonstrate that Self-RefineVQA yields consistent accuracy gains in monolingual and cross-lingual understanding, with a particularly significant boost for small models, enabling lightweight models (e.g., 2B-7B parameters) to rival the performance of significantly larger counterparts. Furthermore, we provide a comparative complexity analysis demonstrating that while inference cost increases linearly, our framework offers superior data efficiency and a lower cumulative cost by eliminating the prohibitive annotation bottleneck of human labeling required for fine-tuning. Our results highlight the potential of prompt-based self-improvement as a scalable, accessible paradigm for reliable table reasoning in VLMs. ## Reliable AI-Assisted Decisions via Selective Auditing DevFeed: [Reliable AI-Assisted Decisions via Selective Auditing]() Original publisher: [Read original article]() Author: Isaac Gibbs, Elise L. Han, Ryan J. Tibshirani, Tijana Zrnic Published: 2026-10-23T04:00:00Z Content type: article Language: en Sources: [arXiv: Machine Learning]() Topics: [human review](), [Artificial Intelligence](), [Uncertainty quantification LLMs](), [Large Language Model]() Tags: [adversarial](), [agent](), [ai](), [ai-assisted](), [algorithms](), [annotation](), [auditing](), [confidence](), [confidence-score](), [control](), [cs-lg](), [data](), [error-control](), [experiments](), [labeling](), [language](), [language-models](), [large-language-models](), [model](), [monitoring](), [online](), [stat-me](), [stat-ml]() ### AI overview The proposed approach reduces expert audits while keeping the share of AI errors left unaudited below a user-set threshold. For each online decision, it uses a proposal and a confidence score--which may come from separate models--to choose between automatic acceptance and expert review. The paper formulates this as constrained optimization and reports algorithms with provable error-control guarantees without assumptions. One method asymptotically achieves the best possible auditing rate on both adversarial and i.i.d. streams. Experiments in harmful-request screening, qualitative text annotation, and data labeling reportedly maintained error control while reducing human auditing compared with baselines. ### Source excerpt arXiv:2610.31662v1 Announce Type: cross Abstract: We consider online monitoring of an AI system that produces a sequence of decisions. At each round, the system outputs a proposed decision together with a confidence score estimating the trustworthiness of the decision. The decision and confidence score need not be produced by the same model: for example, one agent may propose a decision, while a separate verifier agent evaluates it and outputs a correctness or safety score. Based on this information, we must decide whether to accept the AI decision automatically or consult an expert, such as a human auditor, who reviews the proposal and either accepts or corrects it. We formulate this task as an online constrained optimization problem in which the goal is to minimize the frequency of expert auditing while guaranteeing that the fraction of AI errors that go unaudited stays below a user-defined threshold. We derive a family of algorithms with provable error-control guarantees with no assumptions, including a method that is optimally efficient: it asymptotically attains the best achievable auditing rate for both adversarial and i.i.d. data streams. Experiments on screening harmful requests to large language models, qualitative text annotation, and data labeling show that our methods achieve valid error control in practice while substantially reducing the need for human auditing compared to baselines. ## Auditing Safety Preference Supervision with Recorded Derivations DevFeed: [Auditing Safety Preference Supervision with Recorded Derivations]() Original publisher: [Read original article]() Author: Chenhan Lyu, Amir Rahmani Published: 2026-10-21T04:00:00Z Content type: article Language: en Sources: [arXiv: Machine Learning]() Topics: [dpo](), [ai alignment](), [post-training]() Tags: [abstract](), [align](), [all](), [arxiv](), [auditing](), [but](), [can](), [cs-ai](), [cs-cl](), [cs-lg](), [cs-sc](), [derivations](), [fixed](), [inspect](), [inspection](), [intermediate](), [labeling](), [language](), [language-model](), [make](), [model](), [safety](), [supervision](), [with]() ### AI overview Symbolic Preference Labeling makes preference construction auditable by having a pretrained language model classify predefined prompt and response attributes, then applying explicit rules and a fixed verdict ranking. Stored attributes and rules let reviewers recompute verdicts and preference labels without rerunning the model; this supports reproducibility of label construction, not proof that the attributes are correct or that the responses are semantically safe. In an evaluation on 200 human-annotated prompts, structured outputs were valid for 192, with all eight parsing failures among harmful self-harm prompts. A policy trained by direct preference optimization on 465 pairs across five seeds had mean measured refusal rates of 40.82% on 1,319 benign prompts resembling unsafe ones and 84.98% on 655 toxic prompts. Attribute correctness and semantic safety require separate validation. ### Source excerpt arXiv:2610.27266v1 Announce Type: cross Abstract: Preference labels align model training, but the judgments behind them are difficult to inspect or verify. In this paper, we examine how preference construction can be checked when intermediate decisions are recorded, and which of them remain uncertain. We present Symbolic Preference Labeling, a method in which a pretrained language model classifies predefined attributes of prompts and responses. Explicit rules assign response verdicts, and a fixed verdict ranking determines the preference label. Each preference label has a recorded derivation: the response verdicts and preference label can be recomputed from the stored attribute assignments, rules, and verdict ranking without rerunning the language model. On 200 prompts annotated by human raters, the language model produces valid structured outputs for 192 prompts. All eight parsing failures occur in the subset of self-harm prompts marked as harmful. Using 465 preference pairs, we train one policy model with direct preference optimization across five training seeds. Mean measured benign refusal is 40.82 percent on 1,319 benign prompts that resemble unsafe ones, and mean measured toxic refusal is 84.98 percent on 655 toxic prompts. This study distinguishes the reproducibility of preference labels from attribute correctness. The recorded derivations make preference construction open to inspection, while attribute correctness and semantic safety require separate validation. ## Training with ML-labeled Data via Stochastic First- and Zeroth-Order Optimization DevFeed: [Training with ML-labeled Data via Stochastic First- and Zeroth-Order Optimization]() Original publisher: [Read original article]() Author: Maria-Eleni Sfyraki, Jun-Kun Wang Published: 2026-10-21T04:00:00Z Content type: article Language: en Sources: [arXiv: Machine Learning]() Topics: [Machine learning](), [Fine-tuning](), [Large Language Model]() Tags: [algorithms](), [cs-lg](), [datasets](), [fine-tuning](), [labeling](), [large-language-model](), [linear-regression](), [machine-learning](), [math-oc](), [optimization](), [training]() ### AI overview The paper proposes first-order and zeroth-order stochastic optimization methods for training when only some samples have ground-truth labels and the rest are labeled by an ML predictor, which may be wrong. The authors report empirical studies spanning linear regression and fine-tuning a large language model to demonstrate the methods' effectiveness. The supplied abstract does not provide algorithm details or specific results. ### Source excerpt arXiv:2610.26790v1 Announce Type: new Abstract: Training a machine learning (ML) model is an optimization process that typically requires a set of examples with ground-truth labels beforehand. Obtaining these ground truths usually requires laborious human effort and can be costly. The scalability of human labeling becomes a concern when there are a large number of data points that need to be labeled. To address this practical issue, we consider an optimization setting where a subset of samples has ground-truth labels, while the remaining data points were labeled by an ML predictor, which could potentially assign incorrect labels to those points. We propose both first-order and zeroth-order stochastic optimization methods for training on datasets consisting of a mix of true-labeled and ML-labeled samples. We further conduct comprehensive empirical studies from training a linear regression model to fine-tuning a large language model (LLM) to showcase the effectiveness of the proposed algorithms for training with ML-labeled data. ## Building an AI-powered multimodal compliance monitor: From training to tracking to chat DevFeed: [Building an AI-powered multimodal compliance monitor: From training to tracking to chat]() Original publisher: [Read original article]() Author: Ganesh Murthy Published: 2026-09-30T00:00:00Z Content type: tutorial Language: en Sources: [Red Hat Blog]() Topics: [ai security](), [real-time face tracking](), [AI Chat](), [Advanced AI workflows](), [ML model versioning]() Tags: [ai](), [computer-vision](), [database](), [database-schema](), [labeling](), [monitoring](), [tracking]() ### AI overview This Red Hat OpenShift AI quickstart walks through an AI monitoring application that combines computer vision and large language models. It covers training custom object detection models, real-time tracking of people and equipment, storing observations in a flexible JSONB database, and querying monitoring data through natural-language chat and dashboards. ### Source excerpt Organizations face a persistent challenge maintaining workplace safety, security, asset tracking, and operational compliance across facilities with multiple video feeds. Manual monitoring is resource-intensive, inherently reactive, and often misses critical events or patterns indicating safety risks. A single safety officer watching a bank of screens can't reliably track whether every worker on a busy factory floor is wearing their hardhat, vest, and mask--let alone recall compliance trends from last week.This project tackles that problem with a complete, end-to-end AI monitoring system combi ## When users don't click thumbs up: Inferring agent feedback from Datadog telemetry DevFeed: [When users don't click thumbs up: Inferring agent feedback from Datadog telemetry]() Original publisher: [Read original article]() Author: Michael Bevilacqua-Linn; Tanguy Renaudie Published: 2026-09-22T00:00:00Z Content type: tutorial Language: en Sources: [Datadog]() Topics: [agent observability](), [telemetry](), [Ground truth / benchmark quality]() Tags: [agent-observability](), [agentic-systems](), [bits-ai](), [don-t](), [feedback](), [labeling](), [telemetry]() ### AI overview Datadog explains how it used weak labeling on agent traces and telemetry to approximate user feedback when users rarely submit explicit ratings. The post covers creating, testing, and validating a public session-classification skill against a smaller golden dataset. ### Source excerpt Learn how we used weak labeling to infer user satisfaction from Datadog telemetry when agent users don't click thumbs up. ## Instagram limits the reach for AI influencers without 'AI labels' DevFeed: [Instagram limits the reach for AI influencers without 'AI labels']() Original publisher: [Read original article]() Author: Tech Desk Published: 2026-09-01T10:19:02Z Content type: news Language: en Sources: [Technology | The Indian Express]() Topics: [Instagram](), [Meta](), [Artificial Intelligence](), [Generative AI]() Tags: [ai](), [ai-influncers](), [ai-labeling-on-instagram](), [announcement](), [disclosure](), [fake-ai-influncers](), [generated-content](), [influencers](), [instagram](), [instagram-ai-label](), [instagram-news](), [labeling](), [meta](), [social](), [social-media](), [social-media-news](), [technology](), [technology-social](), [transparency]() ### AI overview Instagram is replacing its "AI creator" label with "AI-generated profile" and requiring clearer disclosure for AI-generated people in posts, Reels, Stories, and profile bios. Accounts that do not use the label may have their content recommended less often to non-followers. ### Source excerpt Meta-owned Instagram is tightening AI influencer rules with clearer labeling requirements. ## How to Auto-Label Image Data with Gemini 3.7 in Roboflow DevFeed: [How to Auto-Label Image Data with Gemini 3.7 in Roboflow]() Original publisher: [Read original article]() Author: Mostafa Ibrahim Published: 2026-08-28T17:51:00Z Content type: tutorial Language: en Sources: [Roboflow]() Topics: [object-detection](), [vlm](), [ImageNet]() Tags: [computer-vision](), [dataset](), [how-to](), [labeling](), [object-detection](), [roboflow-annotate](), [vision-language-model]() ### AI overview This tutorial explains how to use Roboflow Auto Label with Gemini 3.7 Flash to label object-detection datasets. It covers zero-shot bounding-box generation from class names, the model's limitations, and reviewing and correcting missed or inaccurate boxes. ### Source excerpt Roboflow Auto Label runs Gemini 3.7 across the batch and returns bounding boxes for the classes you define. ## Top Models for AI Data Labeling DevFeed: [Top Models for AI Data Labeling]() Original publisher: [Read original article]() Author: Aarnav Shah Published: 2026-08-07T21:05:41Z Content type: article Language: en Sources: [Roboflow]() Topics: [ImageNet](), [AI Foundation Models](), [AI Chat](), [Nemotron]() Tags: [ai-assisted](), [building](), [computer-vision](), [cost](), [evals](), [labeling]() ### AI overview The article explains how vision-language models can pre-label images for computer vision training, with human review correcting their outputs. It compares models using object-detection accuracy, speed, and cost benchmarks, and recommends testing leading options on representative project images because rankings vary by use case. ### Source excerpt Labeling is the slowest part of building a vision model. The fix is letting a foundation model take the first pass while a human reviews. We benchmarked every top vision model on object detection to find which ones you can trust with the job, and how to pick between them. ## LLMs for Relevance: Automating High-Quality Product Relevance Labeling in Flipkart Search DevFeed: [LLMs for Relevance: Automating High-Quality Product Relevance Labeling in Flipkart Search]() Original publisher: [Read original article]() Author: Amey Patil Published: 2026-08-03T12:26:43Z Content type: article Language: en Sources: [Flipkart Tech Blog]() Topics: [grpo](), [llm-reasoning](), [Reinforcement learning](), [Large Language Model](), [search-engines](), [Ground truth / benchmark quality](), [Fine-tuning]() Tags: [ai](), [data-science](), [deep-learning](), [flipkart](), [grpo](), [labeling](), [llm](), [llms](), [product](), [quality](), [relevance](), [search]() ### AI overview Flipkart's Product Analyser automates query-to-product relevance labeling with an LLM trained in two stages: supervised fine-tuning on human-majority gold labels and generated reasoning filtered against those labels, followed by GRPO alignment to improve consistency. Training examples deliberately cover Relevant, Partially Relevant, and Irrelevant products; a reasoning reward model favors correct labels and stronger explanations. The article reports gains over human annotators (+2.3% three-class and +2.9% binary accuracy), 12% better results than the best off-the-shelf proprietary LLMs at about 30% of their cost, and under 1% difference from manual NDCG reports across 22 weeks. The system scales labeling and provides explanations that can help diagnose result or catalog problems. The account says its roughly 8B-parameter model is too slow for live serving (>100 ms); planned work includes image input, distillation to smaller models, and adaptation to other domains. ### Source excerpt Amey Patil , Shreya Malani- November 2025 Every product you see on a Flipkart search results page is there because a system decided it was relevant to what you typed. But how do we know those decisions are good? Someone -- or something -- has to look at a query and a product and judge: is this a good match or not? That judgment is the raw material we use to train and evaluate every search model we build. If the judgments are slow, expensive, or noisy, everything downstream suffers. This article describes how we replaced a manual, human-driven labeling process with an LLM-based system -- the Product Analyser (PA) -- that now matches, and in places beats, human annotators, at a fraction of the cost and many times the scale. The key ingredient is a two-stage training recipe that first teaches the model how to reason about relevance, and then uses a Reinforcement Learning technique called Grouped Relative Policy Optimization (GRPO) to make that reasoning consistent and reliable. The Challenge: Why Manual Labelling Was a Bottleneck The core task sounds simple: take a user Query and a retrieved Product, and assign a relevance label -- Relevant, Partially Relevant, or Irrelevant. We call this Query-to-Product (Q<>P) relevance labelling, and it is the foundation for training our semantic retrievers and relevance models. For a long time, humans did this labelling. That created three problems: Limited bandwidth and high cost. A team of annotators can only get through so many queries a week. Scaling to the volumes we actually needed was simply too slow and too expensive. Inconsistent judgments (data noise). Relevance is subjective. Two people can look at the same query-product pair and disagree. That disagreement becomes noise, and noisy labels train weaker models. Coarse understanding. A human label tells you that a product is "Partially Relevant" -- but not why. Without the "why," it is hard to tell two products in the same bucket apart, or to diagnose what exactly is wrong with a ## Marc Andreessen Egg Game DevFeed: [Marc Andreessen Egg Game]() Original publisher: [Read original article]() Author: Nolen Royalty (eieiogames@gmail.com) Published: 2026-05-07T00:00:00Z Content type: article Language: en Sources: [Eieio Games]() Topics: [browser-game](), [Image]() Tags: [3](), [all](), [automatically](), [creative](), [demand](), [drawing](), [for-founders](), [gallery](), [game](), [global](), [gotten](), [great](), [groups](), [help](), [interface](), [labeling](), [light](), [lighting](), [manifesto](), [map]() ### AI overview Marc Andreessen Egg Game lets players draw eggs to resemble Marc Andreessen, either making one egg with unlimited time in Creative Mode or three quickly in Speedrun Mode. For lighting, it estimates a normal map for the egg's surface, then provides hand-chosen defaults that players can adjust in Creative Mode. To score speedruns, the creator made labeled reference drawings for each image, grouped them into colored regions, and uses those regions to prompt features such as beard or nose and suggest colors. Drawings are compared with the reference regions, with evaluation flexibility adjusted for the feature: sweater size and placement can be judged more broadly than the harder-to-place eyebrows. Players can submit drawings to the gallery. ### Source excerpt Marc Andreessen Egg Game helps you make eggs that look like Marc Andreessen. ## Product Evals in Three Simple Steps DevFeed: [Product Evals in Three Simple Steps]() Original publisher: [Read original article]() Author: Eugene Yan Published: 2025-11-23T00:00:00Z Content type: tutorial Language: en Sources: [Eugene Yan]() Topics: [Large Language Model](), [data](), [dataset]() Tags: [dataset](), [engineering](), [eval](), [evals](), [evaluation](), [harness](), [labeling](), [llm](), [pass](), [product](), [production]() ### AI overview A practical guide to product evaluations for LLM applications. It recommends labeling a small dataset, aligning human and LLM evaluators, and running an evaluation harness after configuration changes, with binary pass/fail labels preferred for objective criteria. ### Source excerpt Label some data, align LLM-evaluators, and run the eval harness with each change. [Next page]()