# managing large-scale text data in AI

Managing large-scale text data in AI concerns data quality assurance for large-scale textual datasets used in AI applications, including quality metrics and pipeline practices from collection to deployment.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## CounselTraj: Synthesizing Process-Aware Psychological Counseling Dialogues via Dialogue-Act Trajectory Planning

DevFeed: [CounselTraj: Synthesizing Process-Aware Psychological Counseling Dialogues via Dialogue-Act Trajectory Planning](<https://devfeed.tech/articles/counseltraj-synthesizing-process-aware-psychological-counseling-dialogues-via-dialogue-act-trajectory-planning-142762.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24168>)

Author: Zhiyang Qi, Michimasa Inaba

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Synthetic Data Generation](<https://devfeed.tech/topics/synthetic-data-generation.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [managing large-scale text data in AI](<https://devfeed.tech/topics/managing-large-scale-text-data-in-ai.md>), [AI-Ready Data](<https://devfeed.tech/topics/ai-ready-data.md>)

Tags: [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [llm](<https://devfeed.tech/tags/llm.md>), [planning](<https://devfeed.tech/tags/planning.md>), [process](<https://devfeed.tech/tags/process.md>), [response](<https://devfeed.tech/tags/response.md>), [session](<https://devfeed.tech/tags/session.md>), [sessions](<https://devfeed.tech/tags/sessions.md>), [synthetic](<https://devfeed.tech/tags/synthetic.md>), [trajectory-planning](<https://devfeed.tech/tags/trajectory-planning.md>)

### AI overview

CounselTraj expands brief synthetic counseling exchanges into session-length dialogues by learning dialogue-act patterns from counselor role-play dialogues, planning coordinated client and counselor trajectories, and using those plans to guide generation. Across three Chinese synthetic datasets, the authors report more complete, coherent sessions and higher-quality counselor interventions than direct expansion or expansion using randomly sampled trajectories. They also create CounselTrajD, with over 10K process-guided dialogues, and train CounselTrajLM; evaluations report stronger response quality, therapeutic understanding and intervention, and session-level coherence and progress than publicly available Chinese counseling models. These are reported results from the described evaluations; the supplied text does not provide their detailed methods or metrics.

### Source excerpt

arXiv:2611.24168v1 Announce Type: cross Abstract: The growing demand for psychological counseling has motivated the development of LLM-based systems that support human counselors, but training such systems requires large-scale, high-quality dialogue data. Authentic counseling transcripts are difficult to collect and share because of privacy concerns, making synthetic dialogues an important alternative. However, existing synthetic datasets, despite covering diverse scenarios and client profiles, often consist of brief exchanges and fail to capture session-level counseling processes. We propose CounselTraj, a dialogue-act trajectory planning framework for expanding brief synthetic dialogues into process-aware, session-level psychological counseling dialogues. CounselTraj learns dialogue-act patterns from role-play counseling dialogues produced by human counselors, plans coordinated trajectories for clients and counselors, and uses them as flexible guidance for long-form dialogue generation. Experiments across three Chinese synthetic counseling datasets show that CounselTraj produces more complete and coherent sessions with higher-quality counselor interventions than both direct expansion and expansion with randomly sampled trajectories. Based on CounselTraj, we construct CounselTrajD, a dataset containing over 10K process-guided Chinese psychological counseling dialogues, and use it to train CounselTrajLM for counseling response generation. Evaluations at both the response and session levels show that CounselTrajLM outperforms publicly available Chinese counseling models in response quality, therapeutic understanding and intervention, and counseling-process coherence and progress. Our code and model are publicly available at https://github.com/UEC-InabaLab/CounselTraj.

## LusoClin: A Synthetic Clinical Note Dataset for Medical Information Retrieval in European Portuguese

DevFeed: [LusoClin: A Synthetic Clinical Note Dataset for Medical Information Retrieval in European Portuguese](<https://devfeed.tech/articles/lusoclin-a-synthetic-clinical-note-dataset-for-medical-information-retrieval-in-european-portuguese-142657.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23454>)

Author: Carla Teixeira Lopes, Daniel F\\'elix

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [information retrieval](<https://devfeed.tech/topics/information-retrieval.md>), [AI-driven information retrieval](<https://devfeed.tech/topics/ai-driven-information-retrieval.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [lora](<https://devfeed.tech/topics/lora.md>), [llama](<https://devfeed.tech/topics/llama.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [managing large-scale text data in AI](<https://devfeed.tech/topics/managing-large-scale-text-data-in-ai.md>), [Open Source Models & Datasets](<https://devfeed.tech/topics/open-source-models-datasets.md>), [Internationalization (i18n)](<https://devfeed.tech/topics/i18n.md>)

Tags: [bert](<https://devfeed.tech/tags/bert.md>), [clinical-search](<https://devfeed.tech/tags/clinical-search.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-ir](<https://devfeed.tech/tags/cs-ir.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [document-retrieval](<https://devfeed.tech/tags/document-retrieval.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [information-retrieval](<https://devfeed.tech/tags/information-retrieval.md>), [languages](<https://devfeed.tech/tags/languages.md>), [llama](<https://devfeed.tech/tags/llama.md>), [low-resource-languages](<https://devfeed.tech/tags/low-resource-languages.md>), [medical](<https://devfeed.tech/tags/medical.md>), [note](<https://devfeed.tech/tags/note.md>), [resource](<https://devfeed.tech/tags/resource.md>), [synthetic](<https://devfeed.tech/tags/synthetic.md>)

### AI overview

LusoClin is a publicly released dataset of synthetic clinical notes in European Portuguese, intended to support privacy-preserving research on clinical information retrieval in a language with limited access to real clinical text. It was created by preprocessing real clinical data, fine-tuning Llama 3.2 (3B) with Quantized Low-Rank Adaptation, and generating notes from prompts; a BERT-based model automatically annotates clinical entities for search and indexing. Evaluations using automatic metrics and human review indicate that the dataset captures relevant clinical language and may support document retrieval and domain-specific model adaptation. However, annotators can often distinguish its notes from real ones, and some medical inconsistencies remain.

### Source excerpt

arXiv:2611.23454v1 Announce Type: new Abstract: Access to large-scale clinical text collections is crucial for advancing medical information retrieval research, including clinical note search, patient cohort identification, and domain-adapted retrieval models. However, real clinical notes are rarely available due to privacy regulations, which particularly limits progress for low-resource languages such as European Portuguese. We present LusoClin, a publicly released dataset of synthetic clinical notes in European Portuguese, created to support privacy-preserving research on clinical text retrieval. LusoClin was generated by pre-processing real clinical data, fine-tuning the open-source Llama 3.2 (3B) instruction-tuned model using Quantized Low-Rank Adaptation, and applying prompt-based clinical note generation. To facilitate semantic indexing and clinical search, the notes are automatically annotated with clinical entities using a BERT-based model. We evaluate LusoClin in terms of utility, fidelity, diversity, and privacy using automatic metrics and human evaluation. Results indicate that the dataset captures clinically relevant language patterns and can support downstream IR applications such as clinical document retrieval and domain-specific model adaptation, although annotators can often distinguish synthetic from real notes and some medical inconsistencies persist.

## De-identifying text at scale: How Airbnb protects personal data in AI training

DevFeed: [De-identifying text at scale: How Airbnb protects personal data in AI training](<https://devfeed.tech/articles/de-identifying-text-at-scale-how-airbnb-protects-personal-data-in-ai-training-141090.md>)

Original publisher: [Read original article](<https://airbnb.tech/infrastructure/de-identifying-text-at-scale-how-airbnb-protects-personal-data-in-ai-training/>)

Author: natalijafijacko

Published: 2026-11-18T16:33:01Z

Content type: article

Language: en

Sources: [Airbnb Engineering & Data Science](<https://devfeed.tech/sources/airbnb-engineering-data-science.md>)

Topics: [managing large-scale text data in AI](<https://devfeed.tech/topics/managing-large-scale-text-data-in-ai.md>), [data loss prevention](<https://devfeed.tech/topics/data-loss-prevention.md>), [Data Quality](<https://devfeed.tech/topics/data-quality.md>), [ner](<https://devfeed.tech/topics/ner.md>), [ML model versioning](<https://devfeed.tech/topics/ml-model-versioning.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-training](<https://devfeed.tech/tags/ai-training.md>), [airbnb](<https://devfeed.tech/tags/airbnb.md>), [data](<https://devfeed.tech/tags/data.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [personal](<https://devfeed.tech/tags/personal.md>), [personal-data](<https://devfeed.tech/tags/personal-data.md>), [scale](<https://devfeed.tech/tags/scale.md>), [technology](<https://devfeed.tech/tags/technology.md>), [text](<https://devfeed.tech/tags/text.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

Airbnb describes an in-process text de-identification library built on Presidio to remove personal data before ML training. Teams provide data schemas; Airbnb supplies recognizers, operators, language handling and runtime configuration, while dataset tags and recorded versions/configurations support production gating and traceability. Detection and replacement are separate stages: Airbnb adds phrase-aware context for identifiers such as listing IDs and gift cards, and uses GLiNER for person and street-address detection where spaCy's broad location tags were insufficient. Replacement usually uses realistic Faker values; run-specific secrets make outputs consistent within a run but different across runs, without storing a real-to-fake mapping. Other data receives tailored handling, including salted hashes, coordinate truncation and format-preserving encryption for internal IDs. Detection thresholds and replacement methods vary by use case. Quality is measured by entity and language for recall, precision and throughput, with recall favored for privacy; benchmark comparisons gate model and library changes against regressions. The account covers text deployment and says work on other-媒体

### Source excerpt

Here's how Airbnb detects and replaces personal data in messages, reviews, and support chats before it reaches a model.

## PersonaHate: A Scalable Persona-Based Data Synthesis Pipeline for Hate Speech Analysis

DevFeed: [PersonaHate: A Scalable Persona-Based Data Synthesis Pipeline for Hate Speech Analysis](<https://devfeed.tech/articles/personahate-a-scalable-persona-based-data-synthesis-pipeline-for-hate-speech-analysis-141476.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.21951>)

Author: Xinyu Zhang, Ziqing Yang, Michael Backes, Yang Zhang

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [managing large-scale text data in AI](<https://devfeed.tech/topics/managing-large-scale-text-data-in-ai.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [big-data](<https://devfeed.tech/topics/big-data.md>)

Tags: [annotation](<https://devfeed.tech/tags/annotation.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [data](<https://devfeed.tech/tags/data.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [extension](<https://devfeed.tech/tags/extension.md>), [generators](<https://devfeed.tech/tags/generators.md>), [identity](<https://devfeed.tech/tags/identity.md>), [language](<https://devfeed.tech/tags/language.md>), [large-language-models-llms](<https://devfeed.tech/tags/large-language-models-llms.md>), [llama](<https://devfeed.tech/tags/llama.md>), [llama-3-1](<https://devfeed.tech/tags/llama-3-1.md>), [llms](<https://devfeed.tech/tags/llms.md>), [meta](<https://devfeed.tech/tags/meta.md>), [openai](<https://devfeed.tech/tags/openai.md>), [performance](<https://devfeed.tech/tags/performance.md>), [persona](<https://devfeed.tech/tags/persona.md>), [pipeline](<https://devfeed.tech/tags/pipeline.md>), [speech](<https://devfeed.tech/tags/speech.md>), [synthesis](<https://devfeed.tech/tags/synthesis.md>)

### AI overview

PersonaHate uses 457K personas from two real-world sources to prompt eight LLMs to generate hate speech across 34 identity groups, then applies multi-judge annotation. The pipeline produces 791K samples and a 67K group-balanced training set, and is designed to extend to new groups, generators, and emerging targets. Training detectors from scratch or fine-tuning existing models yielded competitive results on four external benchmarks. A 184M-parameter DeBERTa-v3 model achieved 77.5% average F1 and 79.4% average accuracy, exceeding OpenAI's Moderation API on HateBenchSet and MHS. Fine-tuning Llama-3.1-8B raised average F1 from 70.7% to 78.4%; it surpassed LlamaGuard-3 on three of four benchmarks.

### Source excerpt

arXiv:2611.21951v1 Announce Type: new Abstract: Hate speech remains a persistent and evolving threat on online platforms. The widespread use of large language models (LLMs) further intensifies this problem by making hateful content easier and cheaper to generate at scale. Although stronger hate speech detectors continue to be proposed, their performance remains uneven across target identity groups and different forms of hateful expression, partly because existing training data provides only limited coverage of these cases. Thus, we introduce PersonaHate, a scalable persona-driven data synthesis pipeline for constructing diverse hate speech datasets. PersonaHate begins with a large persona pool of 457K personas drawn from two real-world sources. Conditioned on selected personas from this pool, it prompts eight LLMs to generate hate speech across 34 identity groups and performs multi-judge annotation to obtain reliable labels. The resulting corpus contains 791K samples, from which we curate a 67K group-balanced training set. Our modular design also supports extension to new identity groups, generators, and emerging hate targets. Built upon this dataset, we evaluate two detector training strategies: training from scratch and further fine-tuning. Results show that all trained models achieve competitive performance across four widely used external benchmarks. Notably, a 184M-parameter DeBERTa-v3 model trained on PersonaHate reaches 77.5% average F1 and 79.4% average accuracy, outperforming OpenAI's Moderation API on HateBenchSet and MHS. Furthermore, fine-tuning Llama-3.1-8B raises its average F1 from 70.7% to 78.4%, a gain of 7.7 percentage points, surpassing LlamaGuard-3, Meta's official safety model based on the same Llama-3.1-8B architecture, on three of four benchmarks. These results show that PersonaHate can substantially improve the robustness and cross-dataset generalization of hate speech detectors.

## De-identifying text at scale: How Airbnb protects personal data in AI training

DevFeed: [De-identifying text at scale: How Airbnb protects personal data in AI training](<https://devfeed.tech/articles/de-identifying-text-at-scale-how-airbnb-protects-personal-data-in-ai-training-138584.md>)

Original publisher: [Read original article](<https://medium.com/airbnb-engineering/de-identifying-text-at-scale-how-airbnb-protects-personal-data-in-ai-training-32f3ff8ff7d0?source=rss----53c7c27702d5---4>)

Author: Alexandr Climov

Published: 2026-11-17T18:01:03Z

Content type: article

Language: en

Sources: [Airbnb](<https://devfeed.tech/sources/the-airbnb-tech-blog-medium.md>)

Topics: [pii](<https://devfeed.tech/topics/pii.md>), [privacy-filter](<https://devfeed.tech/topics/privacy-filter.md>), [managing large-scale text data in AI](<https://devfeed.tech/topics/managing-large-scale-text-data-in-ai.md>), [Data Quality](<https://devfeed.tech/topics/data-quality.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [AI Integration Strategies](<https://devfeed.tech/topics/ai-integration-strategies.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-training](<https://devfeed.tech/tags/ai-training.md>), [airbnb](<https://devfeed.tech/tags/airbnb.md>), [data](<https://devfeed.tech/tags/data.md>), [de-identification](<https://devfeed.tech/tags/de-identification.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [extensibility](<https://devfeed.tech/tags/extensibility.md>), [hosting](<https://devfeed.tech/tags/hosting.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [messages](<https://devfeed.tech/tags/messages.md>), [modularity](<https://devfeed.tech/tags/modularity.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [personal-data](<https://devfeed.tech/tags/personal-data.md>), [pii](<https://devfeed.tech/tags/pii.md>), [presidio](<https://devfeed.tech/tags/presidio.md>), [privacy](<https://devfeed.tech/tags/privacy.md>), [reviews](<https://devfeed.tech/tags/reviews.md>), [support](<https://devfeed.tech/tags/support.md>), [technology](<https://devfeed.tech/tags/technology.md>), [text](<https://devfeed.tech/tags/text.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

Airbnb de-identifies text used for model training with an on-premises Presidio library integrated into its shared ML workflow. The Analyzer detects and labels PII; the Anonymizer resolves overlapping spans and replaces them. Airbnb added phrase-based context matching for company-specific identifiers and uses GLiNER for PERSON and LOCATION because spaCy's broad location tags did not reliably capture full street addresses in its benchmark. Detection thresholds and entity scope vary by consumer. Most text PII receives plausible Faker-generated replacements; per-run secret seeding keeps repeated values consistent within a run while changing fakes across runs, without storing a real-to-fake map. Structured data uses salted hashes for identifiers, truncated coordinates, and format-preserving encryption for internal IDs. Benchmarks report recall, precision, and throughput by entity type and language; releases must avoid regressions. The library's protection depends on pipelines using a current version and compliant configuration, and new languages require labeled data for evaluation.

### Source excerpt

Here's how Airbnb detects and replaces personal data in messages, reviews, and support chats before it reaches a model. By: Emelin Flores, Alex Klimov, Lee Formyduval, Bahador Nooraei Introduction Airbnb is built on trust. Every night that a guest books a stay, or a host welcomes someone into their home, that trust is backed by information they've shared with us, which includes personally identifiable information (PII) such as names, home location, current location, and payment details. We now use transaction information to train machine learning models. Before we do so, we remove PII to minimize the number of copies of this information and prevent leakage into and through our models. As we've written before, Airbnb maintains a shared taxonomy as to what constitutes PII, from commonly used fields such as names and card numbers to less frequently used fields such as precise location coordinates. We use de-identification to remove PII and replace it with simulated data that's as realistic as possible to maximize its usefulness for ML model training. Decisions relating to user privacy start as policy. These policies are set and maintained by Airbnb's Privacy Architecture team, with engagement from Legal, Policy, and Engineering. Engineering then implements policy decisions as code. This post focuses on text de-identification. We'll describe the OSS library we chose to start with, the changes we made to create a functional system for Airbnb, and how we confirm that the system is working. 1. Choosing Presidio as the foundation Many organizations are concerned with protecting PII, so there's a lot of existing best practice and tested code out there that we can use. We chose Presidio, an open-source library originally developed by Microsoft and recently transitioned to community governance. We needed to meet demanding requirements: On-premises hosting, to keep data on our infrastructure General-purpose functionality and high degree of modularity, for maximum extensibility

## MAPS: A Multimodal Agentic Framework for POI Data Production Based on Crowdsourced Street-view Imagery

DevFeed: [MAPS: A Multimodal Agentic Framework for POI Data Production Based on Crowdsourced Street-view Imagery](<https://devfeed.tech/articles/maps-a-multimodal-agentic-framework-for-poi-data-production-based-on-crowdsourced-street-view-imagery-137496.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.19807>)

Author: Sijie Ruan, Tingyu Xie, Mingfei Liu, Yicheng Tang, Haoyu Han, Jingnan Qi, Jingshu Zeng, Dongqi Xu, Jing Wang, Shuliang Wang

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Databases](<https://devfeed.tech/sources/arxiv-databases.md>)

Topics: [smolvlm](<https://devfeed.tech/topics/smolvlm.md>), [Entity resolution](<https://devfeed.tech/topics/entity-resolution.md>), [geospatial](<https://devfeed.tech/topics/geospatial.md>), [Image](<https://devfeed.tech/topics/image.md>), [managing large-scale text data in AI](<https://devfeed.tech/topics/managing-large-scale-text-data-in-ai.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [actions](<https://devfeed.tech/tags/actions.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [also](<https://devfeed.tech/tags/also.md>), [annotation](<https://devfeed.tech/tags/annotation.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [based](<https://devfeed.tech/tags/based.md>), [but](<https://devfeed.tech/tags/but.md>), [candidate](<https://devfeed.tech/tags/candidate.md>), [changes](<https://devfeed.tech/tags/changes.md>), [completeness](<https://devfeed.tech/tags/completeness.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-db](<https://devfeed.tech/tags/cs-db.md>), [data](<https://devfeed.tech/tags/data.md>), [database](<https://devfeed.tech/tags/database.md>), [databases](<https://devfeed.tech/tags/databases.md>), [framework](<https://devfeed.tech/tags/framework.md>), [imagery](<https://devfeed.tech/tags/imagery.md>), [maps](<https://devfeed.tech/tags/maps.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [production](<https://devfeed.tech/tags/production.md>), [view](<https://devfeed.tech/tags/view.md>)

### AI overview

MAPS turns crowdsourced street-view imagery into POI database updates through two cascaded modules. SPICE adapts a lightweight vision-language model to reconstruct and filter text, group entries by POI, and classify their meaning and completeness. RIMA retrieves candidate records, matches POIs using evidence and reasons about attributes, selectively gathering additional visual, geospatial, or external-source evidence when needed. In a human-reviewed full-task evaluation, MAPS outperformed the strongest baseline by 21.8%; deployment in Kuaishou's large-scale POI pipeline reduced average processing time by 76.9% compared with manual annotation.

### Source excerpt

arXiv:2611.19807v1 Announce Type: new Abstract: Point-of-interest (POI) databases support location-based services, yet urban changes quickly make their records stale and manual maintenance is costly. Crowdsourced street-view images provide timely visual evidence, but converting them into database actions requires separating multiple POIs and irrelevant text in each image, then resolving incomplete observations against ambiguous records. We study Street-view Image-based POI Data Production and propose MAPS, the first framework to integrate structured POI extraction with database maintenance from such imagery. MAPS contains two cascaded modules. SPICE progressively adapts a lightweight vision-language model to reconstruct and filter text, group entries belonging to the same POI, and classify their semantics and completeness. RIMA retrieves candidate records, performs evidence-grounded POI matching and attribute-level reasoning, and selectively acquires complementary visual, geospatial, or external-source evidence when necessary. In human-reviewed full-task evaluation, MAPS outperforms the strongest baseline by 21.8\%. MAPS has also been deployed in Kuaishou's large-scale POI production pipeline, reducing average processing time by 76.9\% relative to manual annotation. These results demonstrate the effectiveness and practical value of MAPS for scalable POI maintenance.

## OpenClassGen: A Large-Scale Corpus of Real-World Python Classes for LLM Research

DevFeed: [OpenClassGen: A Large-Scale Corpus of Real-World Python Classes for LLM Research](<https://devfeed.tech/articles/openclassgen-a-large-scale-corpus-of-real-world-python-classes-for-llm-research-137911.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2504.15564>)

Author: Musfiqur Rahman, SayedHassan Khatoonabadi, Emad Shihab

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [managing large-scale text data in AI](<https://devfeed.tech/topics/managing-large-scale-text-data-in-ai.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Vectors & LLMs](<https://devfeed.tech/topics/vectors-llms.md>), [qwen](<https://devfeed.tech/topics/qwen.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [branch](<https://devfeed.tech/tags/branch.md>), [but](<https://devfeed.tech/tags/but.md>), [capabilities](<https://devfeed.tech/tags/capabilities.md>), [cases](<https://devfeed.tech/tags/cases.md>), [class](<https://devfeed.tech/tags/class.md>), [classes](<https://devfeed.tech/tags/classes.md>), [claude](<https://devfeed.tech/tags/claude.md>), [code](<https://devfeed.tech/tags/code.md>), [code-generation](<https://devfeed.tech/tags/code-generation.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [large](<https://devfeed.tech/tags/large.md>), [llm](<https://devfeed.tech/tags/llm.md>), [python](<https://devfeed.tech/tags/python.md>), [real-world](<https://devfeed.tech/tags/real-world.md>), [research](<https://devfeed.tech/tags/research.md>), [scale](<https://devfeed.tech/tags/scale.md>), [world](<https://devfeed.tech/tags/world.md>)

### AI overview

OpenClassGen provides 324,843 human-written Python classes from 2,970 open-source projects, each paired with a self-contained generation skeleton of signatures and docstrings plus 27 static metrics. This addresses the small scale of prior class-level datasets and avoids requiring repository context to resolve specifications. In a curated executable subset of 300 classes with tests reaching 58% branch coverage, GPT-o4-mini, Claude-4-Sonnet, and Qwen-3-Coder achieved a CodeBERTScore-F3 of 0.89 but a functional pass rate of 0.33, with substantial variation across models. The corpus is positioned for training and analysis; these evaluation results apply to the curated subset, not necessarily the full dataset.

### Source excerpt

arXiv:2504.15564v4 Announce Type: replace Abstract: Existing class-level code generation datasets are either synthetic (ClassEval: 100 classes) or insufficient in scale for modern training needs (RealClassEval: 400 classes), hindering robust evaluation and empirical analysis. We present OpenClassGen, a large-scale corpus of 324,843 Python classes extracted from 2,970 engineered open-source projects. Each entry pairs a human-written class with its corresponding skeleton, which comprises class and method signatures with associated docstrings, and is enriched with 27 static code metrics covering complexity, coupling, cohesion, and inheritance properties. Unlike prior benchmarks that require repository-level context resolution, OpenClassGen provides self-contained class skeletons that serve as complete generation specifications. We demonstrate the corpus's utility by evaluating three LLMs (GPT-o4-mini, Claude-4-Sonnet, Qwen-3-Coder) on a curated, executable subset of 300 classes, enriched with test suites achieving 58% branch coverage. Results show strong semantic similarity (CodeBERTScore-F3: 0.89) but moderate functional correctness (pass rate: 0.33), with substantial variance across models. This variance, along with diverse class characteristics, confirms that OpenClassGen enables meaningful differentiation of LLM capabilities. The dataset supports diverse use cases, including fine-tuning, retrieval-augmented generation, difficulty modelling, and failure mode analysis. The complete dataset and curation scripts are publicly available at https://zenodo.org/records/18409150.

## ScrapeGraphAI-100k: Dataset for Schema-Constrained LLM Generation

DevFeed: [ScrapeGraphAI-100k: Dataset for Schema-Constrained LLM Generation](<https://devfeed.tech/articles/scrapegraphai-100k-dataset-for-schema-constrained-llm-generation-134960.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2602.15189>)

Author: William Brach, Francesco Zuppichini, Marco Vinciguerra, Lorenzo Padoan

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Large Language Model](<https://devfeed.tech/topics/llm.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [schema design](<https://devfeed.tech/topics/schema-design.md>), [managing large-scale text data in AI](<https://devfeed.tech/topics/managing-large-scale-text-data-in-ai.md>), [open-llm-leaderboard](<https://devfeed.tech/topics/open-llm-leaderboard.md>)

Tags: [2025](<https://devfeed.tech/tags/2025.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [case-study](<https://devfeed.tech/tags/case-study.md>), [complexity](<https://devfeed.tech/tags/complexity.md>), [compliance](<https://devfeed.tech/tags/compliance.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-ir](<https://devfeed.tech/tags/cs-ir.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [distillation](<https://devfeed.tech/tags/distillation.md>), [distribution](<https://devfeed.tech/tags/distribution.md>), [diversity](<https://devfeed.tech/tags/diversity.md>), [english](<https://devfeed.tech/tags/english.md>), [failure](<https://devfeed.tech/tags/failure.md>), [generation](<https://devfeed.tech/tags/generation.md>), [gpt](<https://devfeed.tech/tags/gpt.md>), [grounding](<https://devfeed.tech/tags/grounding.md>), [huggingface](<https://devfeed.tech/tags/huggingface.md>), [json](<https://devfeed.tech/tags/json.md>), [json-schema](<https://devfeed.tech/tags/json-schema.md>), [knowledge-base](<https://devfeed.tech/tags/knowledge-base.md>), [llm](<https://devfeed.tech/tags/llm.md>), [schema](<https://devfeed.tech/tags/schema.md>)

### AI overview

ScrapeGraphAI-100k contains 93,695 schema-constrained extraction events collected through opt-out ScrapeGraphAI telemetry in Q2-Q3 2025 and deduplicated and balanced by schema from 9 million raw events. Its 25,453 schemas cover 15 named languages, though English accounts for 89% of detected content. Each example pairs Markdown-converted page content with a prompt, schema, LLM response, and structural conformance labels. The authors report sharp failure thresholds as schema complexity increases. In a case study, a 1.7B fine-tuned student closely tracks its GPT-5-nano teacher's output distribution but trails a 30B-A3B reference (3.3B active parameters) on schema compliance; they describe this result as preliminary evidence for training and benchmarking on real practitioner workloads.

### Source excerpt

arXiv:2602.15189v3 Announce Type: replace-cross Abstract: Producing output that conforms to a specified JSON schema underlies tool use, structured extraction, and knowledge base construction in modern large language models. Despite this centrality, public datasets for the task remain small, synthetic, or text-only, and rarely pair real page content with the prompts and schemas used in practice. We introduce ScrapeGraphAI-100k, 93,695 schema-constrained extraction events collected via opt-out ScrapeGraphAI telemetry in Q2--Q3 2025, deduplicated and balanced by schema from 9M raw events. The corpus spans 25,453 unique schemas, across 15 named languages, with English covering 89% of detected content. Each instance pairs Markdown-converted page content with a prompt, schema, LLM response, and per-example jsonschema-rs structural conformance labels. We characterize structural diversity across the corpus and identify sharp failure thresholds as schema complexity grows. As a case study, a 1.7B student fine-tuned on this data closely tracks the output distribution of its GPT-5-nano teacher, though it still trails a 30B-A3B reference (3.3B active parameters) on schema compliance. We offer this distillation result as preliminary evidence that grounding schema-constrained generation in real practitioner workloads at scale enables training and benchmarking that prior synthetic or text-only corpora could not support. ScrapeGraphAI-100k is publicly available on HuggingFace.

## Fake News Detectors are Biased against Texts Generated by Large Language Models

DevFeed: [Fake News Detectors are Biased against Texts Generated by Large Language Models](<https://devfeed.tech/articles/fake-news-detectors-are-biased-against-texts-generated-by-large-language-models-134940.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2309.08674>)

Author: Jinyan Su, Terry Yue Zhuo, Jonibek Mansurov, Di Wang, Preslav Nakov

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [managing large-scale text data in AI](<https://devfeed.tech/topics/managing-large-scale-text-data-in-ai.md>)

Tags: [adversarial](<https://devfeed.tech/tags/adversarial.md>), [bias](<https://devfeed.tech/tags/bias.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [detection](<https://devfeed.tech/tags/detection.md>), [fake-news](<https://devfeed.tech/tags/fake-news.md>), [language-models](<https://devfeed.tech/tags/language-models.md>)

### AI overview

The abstract reports that many fake news detectors are more likely to flag LLM-generated content as fake while sometimes classifying human-written fake news as genuine, a bias the authors link to linguistic patterns in LLM output. They say adversarial training with LLM-paraphrased genuine news improved detection accuracy for both human- and LLM-generated news. The authors also release GossipCop++ and PolitiFact++, datasets combining human-validated articles with LLM-generated fake and real news.

### Source excerpt

arXiv:2309.08674v2 Announce Type: replace-cross Abstract: The spread of fake news has emerged as a critical challenge, undermining trust and posing threats to society. In the era of Large Language Models (LLMs), the capability to generate believable fake content has intensified these concerns. In this study, we present a novel paradigm to evaluate fake news detectors in scenarios involving both human-written and LLM-generated misinformation. Intriguingly, our findings reveal a significant bias in many existing detectors: they are more prone to flagging LLM-generated content as fake news while often misclassifying human-written fake news as genuine. This unexpected bias appears to arise from distinct linguistic patterns inherent to LLM outputs. To address this, we introduce a mitigation strategy that leverages adversarial training with LLM-paraphrased genuine news. The resulting model yielded marked improvements in detection accuracy for both human and LLM-generated news. To further catalyze research in this domain, we release two comprehensive datasets, \texttt{GossipCop++} and \texttt{PolitiFact++}, thus amalgamating human-validated articles with LLM-generated fake and real news.

## DesignSpecs: A DESIGN.md Dataset of Visual Design Specifications for AI Agents

DevFeed: [DesignSpecs: A DESIGN.md Dataset of Visual Design Specifications for AI Agents](<https://devfeed.tech/articles/designspecs-a-design-md-dataset-of-visual-design-specifications-for-ai-agents-136507.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.17713>)

Author: Jialiang Wei, Walid Maalej

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [User interface design](<https://devfeed.tech/topics/ui-design.md>), [Design system](<https://devfeed.tech/topics/design-system.md>), [AI in frontend development](<https://devfeed.tech/topics/ai-in-frontend-development.md>), [managing large-scale text data in AI](<https://devfeed.tech/topics/managing-large-scale-text-data-in-ai.md>), [json lines format](<https://devfeed.tech/topics/json-lines-format.md>)

Tags: [agents](<https://devfeed.tech/tags/agents.md>), [cs-hc](<https://devfeed.tech/tags/cs-hc.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [design-tokens](<https://devfeed.tech/tags/design-tokens.md>), [visual-design](<https://devfeed.tech/tags/visual-design.md>)

### AI overview

DesignSpecs provides a curated dataset of 36,906 DESIGN.md files from 15,794 public, non-fork GitHub repositories, containing 17,240 unique file contents. The authors collected 613,207 files with that name in October 2026, then filtered for YAML front matter, a design-system name, at least one recognized design token, and zero errors under the official linter to exclude unrelated design documents. The JSON Lines release offers file-level and deduplicated views of the original Markdown, links entries to repositories, paths, and commits, and includes related scripts. The format captures visual identity through design tokens and Markdown prose; the dataset is intended to support research on visual design knowledge and design-aware AI agents.

### Source excerpt

arXiv:2611.17713v1 Announce Type: new Abstract: While AI agents increasingly generate and modify user interfaces, they rarely rely on a visual design specification for a certain project. The DESIGN.md format, which originated from Google's AI design tool Stitch and which was open-sourced in April 2026, addresses this gap by describing a visual identity through design tokens (such as named colors, text styles) and Markdown prose. To our knowledge, no large-scale research dataset of such design specification files exists. We present DesignSpecs, a dataset of 36,906 DESIGN.md files from 15,794 public, non-fork GitHub repositories, representing 17,240 unique file contents. We collected 613,207 files named DESIGN.md in October 2026. Because the same file name also denotes architectural plans and other design documents, we filtered those with YAML front matter, a design-system name, at least one recognized design token, and zero errors under the official linter. Our JSON Lines dataset provides file-level and deduplicated views of the original Markdown, links files to repositories, paths, and commits, and is released with related scripts. We outline research opportunities on visual design knowledge and design-aware AI agents. Dataset is available at: https://huggingface.co/datasets/Jl-wei/DesignSpecs

## The WoW: A Large-Scale World Knowledge Corpus for Full-Lifecycle LLM Training

DevFeed: [The WoW: A Large-Scale World Knowledge Corpus for Full-Lifecycle LLM Training](<https://devfeed.tech/articles/the-wow-a-large-scale-world-knowledge-corpus-for-full-lifecycle-llm-training-128318.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.13487>)

Author: Tianze Sun (Harbin Institute of Technology), Yongyao Duan (OpenCSG), Sam Chen (OpenCSG)

Published: 2026-11-11T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Large Language Model](<https://devfeed.tech/topics/llm.md>), [AI-Ready Data](<https://devfeed.tech/topics/ai-ready-data.md>), [managing large-scale text data in AI](<https://devfeed.tech/topics/managing-large-scale-text-data-in-ai.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>)

Tags: [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [educational](<https://devfeed.tech/tags/educational.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

WoW is a large-scale Chinese educational dataset intended to support the full training lifecycle of foundation models. An Educational Knowledge Graph provides its taxonomy, guiding systematic collection and filling gaps across educational levels. The abstract says this version is larger and more systematically organized, with improved quality, multimodal signals, and coverage for every model-development stage. Experiments reportedly show gains across diverse benchmarks; an interactive platform supports graph-based data selection and retrieval. The excerpt does not specify dataset size, benchmark results, or how quality was assessed.

### Source excerpt

arXiv:2611.13487v1 Announce Type: new Abstract: Existing Chinese educational corpora often suffer from fragmented coverage and a lack of systematic organization. In this paper, we propose WoW (\textbf{Wo}rld Kno\textbf{w}ledge), a large-scale dataset designed for the full-cycle training of foundation models. The construction of this dataset is driven by an Educational Knowledge Graph, which serves as a taxonomic backbone for the systematic collection and completion of missing educational modules across all levels. Compared to previous versions, this iteration provides enhanced systematicity, superior quality, and a larger scale, while incorporating multimodal signals and supporting every stage of model development. Experiments demonstrate that WoW significantly improves model performance across diverse benchmarks. To facilitate community research, we further provide an interactive platform for graph-based data selection and retrieval.

## Building and Auditing Chemical Datasets with LLMs: A General LLM-as-a-Judge Framework Applied to Homogeneous Catalytic PET Depolymerisation

DevFeed: [Building and Auditing Chemical Datasets with LLMs: A General LLM-as-a-Judge Framework Applied to Homogeneous Catalytic PET Depolymerisation](<https://devfeed.tech/articles/building-and-auditing-chemical-datasets-with-llms-a-general-llm-as-a-judge-framework-applied-to-homogeneous-catalytic-pet-depolymerisation-126060.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.11030>)

Author: Muhammad bin Javaid, Karim K. Ben Hicham, Enkhnyam Battulga, Alexander Mitsos, Martin Grohe

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Large Language Model](<https://devfeed.tech/topics/llm.md>), [managing large-scale text data in AI](<https://devfeed.tech/topics/managing-large-scale-text-data-in-ai.md>)

Tags: [audit-logs](<https://devfeed.tech/tags/audit-logs.md>), [auditing](<https://devfeed.tech/tags/auditing.md>), [browser](<https://devfeed.tech/tags/browser.md>), [chemistry](<https://devfeed.tech/tags/chemistry.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-dl](<https://devfeed.tech/tags/cs-dl.md>), [customizable](<https://devfeed.tech/tags/customizable.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [framework](<https://devfeed.tech/tags/framework.md>), [interactive](<https://devfeed.tech/tags/interactive.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-as-a-judge](<https://devfeed.tech/tags/llm-as-a-judge.md>), [model-agnostic](<https://devfeed.tech/tags/model-agnostic.md>), [parsing](<https://devfeed.tech/tags/parsing.md>)

### AI overview

The framework pairs schema-enforced LLM extraction with an independent judge that checks each record against the full paper, flags unsupported fields, and proposes cited corrections. Requiring UUID-tagged, chunk-level citations improved extraction accuracy and provided paragraph- and table-level provenance. Against manual review of a curated set of 295 ionic-liquid-catalysed PET glycolysis experiments, the judge agreed on 85.7% of reviewed records and outperformed heuristic string matching at catching errors across related experiments. Across 3,942 records from 561 papers, it accepted 67.5% as written; proposed corrections focused on quantitative fields, where experiments were often conflated. The released dataset includes audit logs, and the browser-based, model-agnostic toolkit supports extraction, auditing, provenance review, and reproducible dataset bundling.

### Source excerpt

arXiv:2611.11030v1 Announce Type: cross Abstract: Large language models can rapidly extract structured multivariate experimental data from the chemistry literature, yet verifying thousands of generated records without ground truth remains difficult. We present a general, citation-grounded extraction and verification methodology centered on an independent "LLM-as-a-judge", applied to the literature on homogeneous catalytic poly(ethylene terephthalate) (PET) depolymerisation. In this pipeline, an extractor LLM generates schema-enforced records from research papers with citations to document chunks tagged with universally unique identifiers (UUIDs); an independent LLM judge then reads the full paper alongside each record, flags unsupported fields, and proposes cited corrections. Prompt ablations show that requiring chunk-level citations not only provides provenance down to specific paragraphs and tables but also improves extraction accuracy. Before deploying the judge at scale, we benchmarked it against manual evaluation on extractions from a hand-curated reference set of 295 ionic-liquid-catalysed PET glycolysis experiments. The judge agreed with human evaluation on 85.7% of reviewed records and was significantly more effective than a heuristic string-matching baseline at catching errors across related experiments. Deployed across 3,942 extracted records from 561 papers, the framework accepted 67.5% of them as written. Its proposed corrections concentrated on quantitative fields, where the extractor often conflated related experiments. We release the extracted dataset with the complete LLM judge audit logs, as well as the Chemistry Data Extractor Toolkit, an open-source application implementing this end-to-end pipeline in a code-free, browser-based environment for document parsing, customizable schema and prompt design, model-agnostic extraction and auditing, interactive provenance-highlighted review, and reproducible dataset bundling.

## Fact-Checking at Scale: Building a More Accurate and Efficient Synopsis Factuality System

DevFeed: [Fact-Checking at Scale: Building a More Accurate and Efficient Synopsis Factuality System](<https://devfeed.tech/articles/fact-checking-at-scale-building-a-more-accurate-and-efficient-synopsis-factuality-system-124798.md>)

Original publisher: [Read original article](<https://netflixtechblog.com/fact-checking-at-scale-building-a-more-accurate-and-efficient-synopsis-factuality-system-d7d85859d808?source=rss----2615bd06b42e---4>)

Author: Netflix Technology Blog

Published: 2026-11-09T17:01:02Z

Content type: article

Language: en

Sources: [Netflix](<https://devfeed.tech/sources/netflix-techblog-medium.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [human review](<https://devfeed.tech/topics/human-review.md>), [Parallel reasoning](<https://devfeed.tech/topics/parallel-reasoning.md>), [managing large-scale text data in AI](<https://devfeed.tech/topics/managing-large-scale-text-data-in-ai.md>)

Tags: [alignment](<https://devfeed.tech/tags/alignment.md>), [automated](<https://devfeed.tech/tags/automated.md>), [cost](<https://devfeed.tech/tags/cost.md>), [fact-checking](<https://devfeed.tech/tags/fact-checking.md>), [long-context](<https://devfeed.tech/tags/long-context.md>), [model](<https://devfeed.tech/tags/model.md>), [scale](<https://devfeed.tech/tags/scale.md>), [synopsis](<https://devfeed.tech/tags/synopsis.md>), [system](<https://devfeed.tech/tags/system.md>)

### AI overview

Netflix's synopsis fact-checking cascade reduces evaluation cost while catching more non-factual synopses and aligning its explanations more closely with human annotators. It first uses specialized literal-truth and misleading-implication judges; high-confidence decisions stand, while uncertain cases go to a router that turns points of doubt into verifiable claims for targeted source-material checks or web checks of external facts. On show-level synopses, the cascade cuts cost by 52.4%; on episode-level synopses, the summary-based cascade costs 15% less than the current holistic pipeline without summaries. Recall-FAIL rises, with limited impact on Recall-PASS, and rationale overlap is reported at about 82%, up from about 69%. Median latency increases but is described as reasonable. Episode checks benefit from full-show summaries, which reduce over-failing caused by missing broader context. The article describes evaluation on show- and episode-level datasets and notes future work on multimodal evidence and using rationales to correct faulty synopses.

### Source excerpt

By Shreya S Ramanujam, Yongce Li, Afshine Amidi, Sina Ghiassian, Benjamin Klein. Introduction Picture this -- you're home after a long day at work and can't wait to catch up on Bridgerton. You log on to Netflix, search for "Bridgerton" and land on this screen: If this sounds familiar, you already know what a synopsis is -- it's a short glimpse into what a viewer can expect to see in a show. We see two types of synopses here: a series-level synopsis describing the overarching plot, and multiple episode-level synopses describing each episode. A synopsis plays a very important role for us -- it helps viewers decide whether a title is worth watching or not. Every synopsis needs to strike a balance: sharing just enough to give viewers a sense of what to expect, while leaving enough intrigue to draw them in. But of course, we can't just write anything. "In the fog-drenched streets of Victorian London, Detective Daphne Bridgerton races to unmask a serial killer known only as the elusive Lady Whistledown." Intriguing? Definitely. But it is most certainly not the premise of Bridgerton. Which leads us to the first (and most important) rule of writing a good synopsis: it must be factual. At Netflix's scale, with thousands of titles, multiple episodes per season and multiple languages, fact-checking all of these by hand simply doesn't scale. This is where our factuality service comes in: an automated way to judge whether a synopsis is factual or not. Our current factuality checker feeds a reasoning model everything we know about a title (like metadata and text assets) and asks it to judge synopsis factuality according to a rigorous rubric. There are two main limitations of this approach that we want to optimize: High cost of long-context reasoning calls: Each decision requires expensive, long-context model calls to reasoning models, which doesn't scale well when we want to check multiple titles, episodes, languages, and style variants. Misaligned rationales: The model's explanatio

## Rethinking Data Selection as Distribution Matching on Semantic Manifolds

DevFeed: [Rethinking Data Selection as Distribution Matching on Semantic Manifolds](<https://devfeed.tech/articles/rethinking-data-selection-as-distribution-matching-on-semantic-manifolds-123261.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.08495>)

Author: Peng Sun, Yi Yang, Mingshuo Chen, Chunxiao Li, Yao Zhu, Huawen Shen, Tianfan Fu

Published: 2026-11-09T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [managing large-scale text data in AI](<https://devfeed.tech/topics/managing-large-scale-text-data-in-ai.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [vlm](<https://devfeed.tech/topics/vlm.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>)

Tags: [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [distribution](<https://devfeed.tech/tags/distribution.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [filter](<https://devfeed.tech/tags/filter.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [matching](<https://devfeed.tech/tags/matching.md>), [quality](<https://devfeed.tech/tags/quality.md>), [redundancy](<https://devfeed.tech/tags/redundancy.md>), [selection](<https://devfeed.tech/tags/selection.md>), [semantic](<https://devfeed.tech/tags/semantic.md>), [space](<https://devfeed.tech/tags/space.md>), [text](<https://devfeed.tech/tags/text.md>), [training](<https://devfeed.tech/tags/training.md>), [training-data](<https://devfeed.tech/tags/training-data.md>)

### AI overview

QMAC selects instruction-tuning data by matching the selected subset's empirical characteristic functions to those of the full dataset across multiple frequency scales, while using task-related quality scores to filter noisy or low-quality samples. The authors say this combines broad semantic coverage with sensitivity to local perturbations. Across four data types, eight target models, and 44 benchmarks, QMAC reportedly outperformed existing selection baselines on standard image-text instruction data, substantially beat random selection on text instruction and reasoning datasets, and exceeded full-data training in multiple model and budget combinations. The supplied abstract does not give effect sizes or implementation details.

### Source excerpt

arXiv:2611.08495v1 Announce Type: new Abstract: Instruction tuning of large language models and large vision language models relies on massive training data, which incurs substantial computational and storage costs. Meanwhile, training data often contains redundancy, noise, and low quality samples. Existing data selection methods mostly rely on diversity, importance, or a combination of both, without explicitly constraining the distributional consistency between the selected subset and the full data in the embedding space. We propose QMAC, a quality aware multiband characteristic function matching method that formulates data selection as a joint optimization problem of distribution matching and quality filtering. QMAC matches the empirical characteristic functions of the selected subset and the full data across multiple frequency scales, capturing both global semantic coverage and local perturbations, while using task related quality scores to filter noisy and low quality samples. We systematically evaluate QMAC on 4 data types, 8 target models, and 44 benchmarks. Experimental results show that QMAC outperforms existing data selection baselines on standard image-text instruction data, substantially surpasses random selection on standard text instruction, image-text reasoning, and text reasoning data, and exceeds full data training in multiple model and budget combinations. The code will be made publicly available upon acceptance of the paper.

## LongCounsel-8: A Benchmark Suite for Longitudinal Depression Tracking from Multi-Session Counseling Dialogues

DevFeed: [LongCounsel-8: A Benchmark Suite for Longitudinal Depression Tracking from Multi-Session Counseling Dialogues](<https://devfeed.tech/articles/longcounsel-8-a-benchmark-suite-for-longitudinal-depression-tracking-from-multi-session-counseling-dialogues-118261.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.03507>)

Author: Jiayi Li, Bingsheng He, Zhaomin Wu

Published: 2026-11-05T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [DeepFact-Bench](<https://devfeed.tech/topics/deepfact-bench.md>), [managing large-scale text data in AI](<https://devfeed.tech/topics/managing-large-scale-text-data-in-ai.md>)

Tags: [benchmark](<https://devfeed.tech/tags/benchmark.md>), [clinical-research](<https://devfeed.tech/tags/clinical-research.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [depression](<https://devfeed.tech/tags/depression.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [large-language-models-llms](<https://devfeed.tech/tags/large-language-models-llms.md>), [sessions](<https://devfeed.tech/tags/sessions.md>), [tracking](<https://devfeed.tech/tags/tracking.md>), [trends](<https://devfeed.tech/tags/trends.md>)

### AI overview

LongCounsel-8 evaluates whether models track depression changes across counseling sessions, rather than score sessions independently. It contains 7,749 five-session trajectories built from real counseling case profiles, longitudinal depression scores, and dialogue form; each generated session expresses a preset PHQ-8 score. Its trend-based protocol measures between-session changes, screening-cutoff crossings, trajectory types, and symptom-level changes. The abstract reports that all five tested methods performed poorly: none estimated changes better than a no-change baseline, at most 40% of cutoff crossings were detected, and adding earlier sessions reduced change error but increased missed improvements. The evidence provided is the abstract, not the full paper.

### Source excerpt

arXiv:2609.03507v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly used to assess depression in counseling conversations, where psychology evidence suggests that tracking change across sessions matters more than any single score. However, existing dialogue benchmarks score each session on its own, and an accurate score for every session does not guarantee that a method follows the changes between sessions. We introduce LongCounsel-8, a benchmark for tracking depression across counseling sessions. Its systematic pipeline takes client profiles from real counseling cases, depression scores from real longitudinal data, and turn length and form from real counseling dialogues, yielding 7,749 five-session trajectories from three LLMs. Every session is generated to express a preset score on the PHQ-8, a standard eight-symptom depression questionnaire. Validation shows that the labels are self-consistent and the dialogues close to real counseling. Following clinical research on treatment monitoring, we propose a trend-based evaluation protocol reflecting real-world tracking needs: changes between sessions, crossings of the screening cutoff, trajectory types, and symptom-level changes. Extensive experiments show all five existing methods tested fall short in tracking trends: none estimates changes better than a no-change baseline, at most 40% of cutoff crossings are detected, and adding earlier sessions lowers change error but causes more missed improvements.

## Mass Producing Training Data with Surprising Side Effects

DevFeed: [Mass Producing Training Data with Surprising Side Effects](<https://devfeed.tech/articles/mass-producing-training-data-with-surprising-side-effects-118452.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.06324>)

Author: Yaowen Ye, Jacob Steinhardt, Sarah Schwettmann, Cassidy Laidlaw

Published: 2026-11-05T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [post-training](<https://devfeed.tech/topics/post-training.md>), [ai alignment](<https://devfeed.tech/topics/ai-alignment.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [managing large-scale text data in AI](<https://devfeed.tech/topics/managing-large-scale-text-data-in-ai.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>)

Tags: [collection](<https://devfeed.tech/tags/collection.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [effects](<https://devfeed.tech/tags/effects.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [testbed](<https://devfeed.tech/tags/testbed.md>), [training](<https://devfeed.tech/tags/training.md>), [training-data](<https://devfeed.tech/tags/training-data.md>)

### AI overview

The paper presents an automated pipeline for finding datasets whose training can cause surprising behavior outside the data's apparent domain. It scores candidate datasets for eliciting a target behavior, looking like ordinary post-training data, and surprising human auditors, then searches using evolution, reinforcement learning, and subliminal learning. After rescoring and filtering with held-out metrics, the authors report 6,575 datasets spanning eight behaviors; many effects generalize across model families and sizes. Their automated metrics track ML researchers' judgments, while frontier models predict side effects better than humans. The datasets are offered as a testbed for studying generalization and developing ways to predict training side effects before deployment.

### Source excerpt

arXiv:2611.06324v1 Announce Type: cross Abstract: Training data can have unexpected side effects on language model behavior outside the training domain: insecure code data can induce broad misalignment, and outdated bird names can induce a 19th-century persona. Understanding such generalization is hard because only dozens of isolated examples exist, each constructed by hand or discovered by accident. We propose automatically discovering such datasets with surprising side effects (DSSEs). We define three objectives for DSSEs: they should (1) elicit a target behavior, (2) read like ordinary post-training data, and (3) be surprising to human auditors reading the data. We operationalize each with LLM judges, and search for high-scoring datasets via evolution, reinforcement learning, and subliminal learning approaches. After re-scoring and filtering with held-out metrics, our pipeline produces a collection of 6,575 DSSEs across 8 behaviors, many of which generalize across model families and sizes. We find our automated metrics track ML researchers' judgments, and frontier models outperform humans at predicting side effects. These datasets form a testbed for studying generalization and lay the groundwork for predicting training side effects before model deployment. Explore them at https://dsse.transluce.org.

## Maswao: An automatically collected, prepared, and benchmarked dataset for the South African Sign language

DevFeed: [Maswao: An automatically collected, prepared, and benchmarked dataset for the South African Sign language](<https://devfeed.tech/articles/maswao-an-automatically-collected-prepared-and-benchmarked-dataset-for-the-south-african-sign-language-115398.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.02903>)

Author: Thapelo Sindane, Unarine Netshifhefhe, Vukosi Marivate, Abiodun Modupe, Herkulaas Combrink, Ketro Sithole

Published: 2026-11-03T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Language Engineering](<https://devfeed.tech/topics/language-engineering.md>), [machine-translation](<https://devfeed.tech/topics/machine-translation.md>), [asr](<https://devfeed.tech/topics/asr.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [managing large-scale text data in AI](<https://devfeed.tech/topics/managing-large-scale-text-data-in-ai.md>)

Tags: [asr](<https://devfeed.tech/tags/asr.md>), [audio](<https://devfeed.tech/tags/audio.md>), [automatically](<https://devfeed.tech/tags/automatically.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [bleu](<https://devfeed.tech/tags/bleu.md>), [collect](<https://devfeed.tech/tags/collect.md>), [collection](<https://devfeed.tech/tags/collection.md>), [crop](<https://devfeed.tech/tags/crop.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [framework](<https://devfeed.tech/tags/framework.md>), [gradient](<https://devfeed.tech/tags/gradient.md>), [gradient-descent](<https://devfeed.tech/tags/gradient-descent.md>), [labour](<https://devfeed.tech/tags/labour.md>), [language](<https://devfeed.tech/tags/language.md>), [languages](<https://devfeed.tech/tags/languages.md>), [model](<https://devfeed.tech/tags/model.md>), [models](<https://devfeed.tech/tags/models.md>), [motion](<https://devfeed.tech/tags/motion.md>), [sign](<https://devfeed.tech/tags/sign.md>)

### AI overview

Maswao is a weakly sentence-aligned South African Sign Language dataset built with minimal human intervention. The framework collects public SASL videos, including news broadcasts, converts accompanying audio to text with available ASR tools, crops the signing area, and uses gradient descent to detect non-motion valleys as sentence boundaries. On sign-to-text translation, Transformer models outperformed the recurrent baselines tested. The best Transformer scored BLEU-1 through BLEU-4 of 13.78, 5.21, 2.52, and 1.54, with ChrF 18.09 and ROUGE-L 10.51. The reported scores suggest the task remains open for further investigation.

### Source excerpt

arXiv:2611.02903v1 Announce Type: cross Abstract: Recent advancements in sign language processing can be attributed to developments in both large-scale dataset collection and algorithmic complexity in areas such as pose estimation techniques, signal processing through Transformer models, and model optimization. However, only a handful of sign languages, such as the American, British, and German, from over 200 globally recognized sign languages, reap the benefits of such advancements, while the rest lag behind. In this study, we aim to address this technological disparity by proposing a framework for the automatic collection and preparation of sign language datasets, with very minimal human-labour intervention, using the South African Sign Language (SASL) as a testbed. Specifically, we automatically collect SASL video content using public resources such as news broadcasts. Additionally, we convert their accompanying audio streams into text using available Automatic Speech Recognition (ASR) tools. Furthermore, we crop the sign language area in the videos and use gradient descent to locate valleys of non-motion to determine end-of-sentence points within long videos to create our weakly sentence-aligned parallel corpora -- the Maswao (Signs) dataset. Finally, we benchmark the weakly aligned dataset on the task of Sign-to-text Translation using Recurrent Neural Networks and Transformer models to compare their individual intricacies. Our results show that Transformer models proposed in the literature outperform recurrent baselines, with the best-performing Transformer achieving BLEU-1 to BLEU-4 scores of 13.78, 5.21, 2.52, and 1.54, respectively, as well as ChrF = 18.09 and ROUGE-L = 10.51. These results indicate that the task remains promising and warrants further investigations.

## BanglishDepYT: Depression Detection in Native Bengali Social Media with Emotion-Preserving Preprocessing and Domain-Adaptive Pretraining

DevFeed: [BanglishDepYT: Depression Detection in Native Bengali Social Media with Emotion-Preserving Preprocessing and Domain-Adaptive Pretraining](<https://devfeed.tech/articles/banglishdepyt-depression-detection-in-native-bengali-social-media-with-emotion-preserving-preprocessing-and-domain-adaptive-pretraining-114577.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.01633>)

Author: Ayon Roy, Sadman Sharif, Syed Shams Islam

Published: 2026-11-03T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Data Science](<https://devfeed.tech/topics/data-science.md>), [managing large-scale text data in AI](<https://devfeed.tech/topics/managing-large-scale-text-data-in-ai.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>)

Tags: [annotation](<https://devfeed.tech/tags/annotation.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-cy](<https://devfeed.tech/tags/cs-cy.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [depression](<https://devfeed.tech/tags/depression.md>), [detection](<https://devfeed.tech/tags/detection.md>), [domain](<https://devfeed.tech/tags/domain.md>), [emotion](<https://devfeed.tech/tags/emotion.md>), [llm](<https://devfeed.tech/tags/llm.md>), [native](<https://devfeed.tech/tags/native.md>), [pretraining](<https://devfeed.tech/tags/pretraining.md>), [social-media](<https://devfeed.tech/tags/social-media.md>), [youtube](<https://devfeed.tech/tags/youtube.md>)

### AI overview

BanglishDepYT introduces a native YouTube dataset for Bengali depression detection, with 191,000 unlabeled comments and 1,000 samples verified by bilingual annotators. Its EPH preprocessing retains emoticons, romanized Bengali, negation, and emotional elongation: standard cleaning reduced Banglish token survival to zero and yielded SVM F1 of 0.7062, while EPH raised SVM F1 to 0.8570. Continuing SahajBERT pretraining on in-domain comments produced mean F1 of 0.8161 and MCC of 0.6272, outperforming the baseline significantly and matching MuRIL in the reported comparison. Qwen2.5-7B annotation had human agreement κ=0.611; pseudo-labeling 96,071 comments improved semi-supervised AUC from 0.8499 to 0.8731. These results are reported in the supplied abstract.

### Source excerpt

arXiv:2611.01633v1 Announce Type: cross Abstract: Depression affects millions of Bengali speakers across South Asia, yet automated detection systems have relied almost entirely on datasets translated from English. Such datasets cannot capture the emoticons, romanized Bengali words, and elongated spellings that characterize real expressions of distress. We introduce BanglishDepYT, a Bengali depression dataset collected natively from YouTube, comprising 191,000 unlabeled comments and 1,000 samples verified through dual bilingual annotation. We propose EPH (Emotion-Preserving Hybrid preprocessing), which converts emoticons to semantic tags, preserves Banglish tokens, protects negation words, and normalizes emotional elongation. A key finding is that preprocessing choice substantially affects performance on informal text: standard cleaning reduces the Banglish Survival Rate to zero (SVM F1: 0.7062), while EPH preserves signals (CESS: 0.9996 vs. 0.6694) and raises SVM F1 to 0.8570, an improvement of 0.1508. We further apply domain-adaptive pretraining (DAPT), continuing SahajBERT pretraining on unlabeled in-domain comments. DAPT-SahajBERT achieves the highest mean F1 (0.8161) and MCC (0.6272), significantly outperforming the baseline (F1: 0.7805, p = 0.0013, d = 1.450) and matching MuRIL (p = 0.589). These results suggest that targeted domain pretraining is a competitive alternative to large-scale multilingual pretraining. LLM-assisted annotation using Qwen2.5-7B achieves substantial human agreement (kappa = 0.611), enabling pseudo-labeling of 96,071 comments. Semi-supervised training improves AUC from 0.8499 to 0.8731 (p = 0.044), offering a scalable path to expanding annotation. All code, models, and datasets are publicly released to support reproducible research.

## TRAIL: Towards Effective Log Parsing for Rare Logs through Template-Preserving Augmentation

DevFeed: [TRAIL: Towards Effective Log Parsing for Rare Logs through Template-Preserving Augmentation](<https://devfeed.tech/articles/trail-towards-effective-log-parsing-for-rare-logs-through-template-preserving-augmentation-113478.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.25261>)

Author: Sepideh Hodaeian, Zhenhao Li, An Ran Chen

Published: 2026-11-03T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [data augmentation](<https://devfeed.tech/topics/data-augmentation.md>), [ai observability](<https://devfeed.tech/topics/ai-observability.md>), [managing large-scale text data in AI](<https://devfeed.tech/topics/managing-large-scale-text-data-in-ai.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [account](<https://devfeed.tech/tags/account.md>), [all](<https://devfeed.tech/tags/all.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [anomaly-detection](<https://devfeed.tech/tags/anomaly-detection.md>), [architectures](<https://devfeed.tech/tags/architectures.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [based](<https://devfeed.tech/tags/based.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [but](<https://devfeed.tech/tags/but.md>), [challenge](<https://devfeed.tech/tags/challenge.md>), [complete](<https://devfeed.tech/tags/complete.md>), [contain](<https://devfeed.tech/tags/contain.md>), [core](<https://devfeed.tech/tags/core.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [debugging](<https://devfeed.tech/tags/debugging.md>), [effective](<https://devfeed.tech/tags/effective.md>), [logs](<https://devfeed.tech/tags/logs.md>), [parsing](<https://devfeed.tech/tags/parsing.md>), [template](<https://devfeed.tech/tags/template.md>)

### AI overview

On Loghub-2.0, log groups with fewer than five instances make up nearly 20% of templates but less than 0.01% of log messages, and evaluated parsers perform substantially worse on them. The paper proposes TAILOR, which augments rare groups with template-consistent messages before inference; the added structural evidence helps separate static tokens from dynamic variables. The authors report a 19% parsing-accuracy improvement on rare groups over the strongest baseline while maintaining competitive performance on complete datasets. The augmentation generalizes across LLM backbones and improves existing LLM-based parsers without changing their core architectures.

### Source excerpt

arXiv:2609.25261v2 Announce Type: replace Abstract: Log parsing is essential for system log analysis because it supports tasks such as debugging, monitoring, and anomaly detection by transforming unstructured log messages into structured log templates. However, real-world log datasets exhibit highly imbalanced, long-tailed distributions, where a small number of frequent templates dominate while many rare templates appear only a few times. This imbalance causes evaluation results to be overly optimistic by allowing frequent templates to dominate benchmark metrics, while poor performance on rare yet operationally important events remains largely hidden. In this paper, we investigate the prevalence and impact of rare log groups, defined as log groups with fewer than five instances. Our empirical study on the widely used Loghub-2.0 benchmark shows that rare log groups account for nearly 20% of all templates but less than 0.01% of log messages. Because they contain only a handful of instances, all evaluated parsers experience substantial performance degradation on these groups. To address this challenge, we propose TAILOR, a log parsing framework that improves template inference for rare log groups through template-preserving augmentation. TAILOR enriches rare log groups with template- consistent log messages before template inference. The additional structural evidence helps distinguish static tokens from dynamic variables. Our experimental results show that TAILOR improves parsing accuracy on rare log groups by 19% over the strongest baseline while maintaining competitive performance on complete datasets. We further show that the proposed augmentation strategy generalizes across different LLM backbones and consistently improves existing LLM-based parsers without modifying their core architectures.

## PlagiaCorpus: A Bilingual, Adversarial, and Cross-Lingual Dataset for Forensic Plagiarism Detection

DevFeed: [PlagiaCorpus: A Bilingual, Adversarial, and Cross-Lingual Dataset for Forensic Plagiarism Detection](<https://devfeed.tech/articles/plagiacorpus-a-bilingual-adversarial-and-cross-lingual-dataset-for-forensic-plagiarism-detection-115404.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.02940>)

Author: Adel Ammar, Serry Sibaee, Wadii Boulila

Published: 2026-11-03T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [managing large-scale text data in AI](<https://devfeed.tech/topics/managing-large-scale-text-data-in-ai.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Internationalization (i18n)](<https://devfeed.tech/topics/i18n.md>), [Phoronix Test Suite](<https://devfeed.tech/topics/phoronix-test-suite.md>)

Tags: [adversarial](<https://devfeed.tech/tags/adversarial.md>), [arabic](<https://devfeed.tech/tags/arabic.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [book](<https://devfeed.tech/tags/book.md>), [cc-by-sa](<https://devfeed.tech/tags/cc-by-sa.md>), [collections](<https://devfeed.tech/tags/collections.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-cy](<https://devfeed.tech/tags/cs-cy.md>), [cs-dl](<https://devfeed.tech/tags/cs-dl.md>), [cs-ir](<https://devfeed.tech/tags/cs-ir.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [detection](<https://devfeed.tech/tags/detection.md>), [english](<https://devfeed.tech/tags/english.md>), [forensic](<https://devfeed.tech/tags/forensic.md>), [formats](<https://devfeed.tech/tags/formats.md>), [json](<https://devfeed.tech/tags/json.md>), [language](<https://devfeed.tech/tags/language.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [large-language-models](<https://devfeed.tech/tags/large-language-models.md>), [large-language-models-llms](<https://devfeed.tech/tags/large-language-models-llms.md>), [license](<https://devfeed.tech/tags/license.md>), [llm](<https://devfeed.tech/tags/llm.md>), [parquet](<https://devfeed.tech/tags/parquet.md>), [plagiarism](<https://devfeed.tech/tags/plagiarism.md>)

### AI overview

PlagiaCorpus provides 198,643 Arabic-English records for benchmarking plagiarism and text-reuse detection, including LLM paraphrasing and cross-language reuse. Its 34,885 source passages come from Wikipedia and openly licensed books; 163,758 queries include training and test sets across copying, character-level obfuscation, same-language LLM paraphrasing, English-Arabic translation with structural paraphrasing, and non-reused text. Positive queries link to source passages, and source-grouped splitting keeps each passage's queries within one split. Curation included language identification, removal of LLM generation artefacts, and exact and cross-lingual deduplication. Reported checks include class-specific length statistics, split validation, and inter-annotator agreement on a manually rated sample. The dataset is available in JSON and Parquet under CC BY-SA 4.0.

### Source excerpt

arXiv:2611.02940v1 Announce Type: cross Abstract: PlagiaCorpus is a bilingual Arabic-English dataset for benchmarking plagiarism and text-reuse detection, including reuse produced by large language models (LLMs) and reuse across languages. It contains 198,643 text records: a source corpus of 34,885 passages drawn from English and Arabic Wikipedia and two openly licensed book collections, and 163,758 queries (147,381 for training and 16,377 for testing). The queries cover five reuse classes of increasing difficulty, namely verbatim copying, character-level obfuscation, same-language paraphrasing by two different LLMs, and English--Arabic translation with structural paraphrasing, together with a Safe class of non-reused passages. Every positive query is linked to its source passage, and the train/test split is grouped by source passage so that no passage contributes queries to both splits. The release was curated through language identification, removal of LLM generation artefacts, and exact and cross-lingual de-duplication. Technical validation reports per-class length statistics, automatic checks of the train/test split, and inter-annotator agreement on a manually rated sample. The dataset is distributed in JSON and Parquet formats under a CC BY-SA 4.0 license.

## RECAST: REconstruction of Chain-of-Thought via Auditable Structured Traces

DevFeed: [RECAST: REconstruction of Chain-of-Thought via Auditable Structured Traces](<https://devfeed.tech/articles/recast-reconstruction-of-chain-of-thought-via-auditable-structured-traces-112113.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.41813>)

Author: Qi Wang, Kang Xu, Chengcheng Wan, Jiangtao Wang

Published: 2026-11-02T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Chain-of-thought](<https://devfeed.tech/topics/chain-of-thought.md>), [managing large-scale text data in AI](<https://devfeed.tech/topics/managing-large-scale-text-data-in-ai.md>), [Agent Skills](<https://devfeed.tech/topics/agent-skills.md>)

Tags: [agent-skill](<https://devfeed.tech/tags/agent-skill.md>), [api](<https://devfeed.tech/tags/api.md>), [chain-of-thought](<https://devfeed.tech/tags/chain-of-thought.md>), [claude](<https://devfeed.tech/tags/claude.md>), [claude-sonnet-5-5](<https://devfeed.tech/tags/claude-sonnet-5-5.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [data](<https://devfeed.tech/tags/data.md>), [decoder](<https://devfeed.tech/tags/decoder.md>), [framework](<https://devfeed.tech/tags/framework.md>), [frontier](<https://devfeed.tech/tags/frontier.md>), [gemini](<https://devfeed.tech/tags/gemini.md>), [gpt-6-1](<https://devfeed.tech/tags/gpt-6-1.md>), [hidden](<https://devfeed.tech/tags/hidden.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [pipeline](<https://devfeed.tech/tags/pipeline.md>), [prompt](<https://devfeed.tech/tags/prompt.md>), [protocol](<https://devfeed.tech/tags/protocol.md>), [question](<https://devfeed.tech/tags/question.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [structured](<https://devfeed.tech/tags/structured.md>), [traces](<https://devfeed.tech/tags/traces.md>)

### AI overview

RECAST reconstructs structured rationales from a closed model's question and final answer, then cleans them into supervised fine-tuning (SFT) data without relying on hidden reasoning channels. The authors report that three existing CoT extraction methods yielded zero tokens on Claude-Sonnet-5.5 through the standard chat-completions interface because the expected reasoning fields and replay paths were unavailable or filtered. They instantiate RECAST on Claude-Sonnet-5.5 and report qualitative reconstructions on GPT-6.1 and Gemini 3 using the same answer-only protocol. The preprint describes a controlled artifact validation, not large-scale benchmark gains.

### Source excerpt

arXiv:2610.41813v1 Announce Type: new Abstract: Large closed-source reasoning models expose only their final answers. The intermediate reasoning that produces those answers is hidden behind API surfaces, tool-call policies, and safety filters. Existing chain-of-thought (CoT) extraction methods assume a side channel: a visible reasoning field, a scratchpad carried through tool calls, or a signed/encrypted reasoning block. We reproduce three representative state-of-the-art extraction methods and find that, on Claude-Sonnet-5.5 accessed through the standard chat-completions interface, all three return zero CoT tokens. The failure is structural rather than a bug: the reasoning field is absent, named tool selection is rejected, free-text scratchpad arguments are filtered, and no replayable reasoning block is exposed. We therefore take a different route. We present RECAST, a reconstruction-based framework that conditions a decoder on the question and the target model's final answer, reconstructs a first-person structured rationale, cleans it, and builds SFT-ready data. To the best of our knowledge, RECAST is the first system to reconstruct CoT from frontier closed models such as Claude-Sonnet-5.5, GPT-6.1, and Gemini 3 without using hidden reasoning channels. We instantiate RECAST on Claude-Sonnet-5.5 and additionally obtain successful qualitative reconstructions on GPT-6.1 and Gemini 3 with the same answer-only protocol. The system is open-source and packaged as an agent skill: one prompt deploys it and runs the complete pipeline to produce SFT-ready CoT data. This preprint describes the system, its threat model, and a controlled artifact validation; it does not claim large-scale benchmark improvements.

## From synthetic enterprise data evaluations to real Microsoft Copilot AI confidence

DevFeed: [From synthetic enterprise data evaluations to real Microsoft Copilot AI confidence](<https://devfeed.tech/articles/from-synthetic-enterprise-data-evaluations-to-real-microsoft-copilot-ai-confidence-109787.md>)

Original publisher: [Read original article](<https://www.microsoft.com/en-us/research/blog/from-synthetic-enterprise-data-evaluations-to-real-microsoft-copilot-ai-confidence/>)

Author: Somya Chatterjee, Harshita Sahijwani, Min Chen, Sushmitha Rajeswari Muppa, Dayquan Julienne, Irene Shaffer, Chandra Inguva, Jenny Chen, Minha Hwang, Sam Ade Jacobs

Published: 2026-10-30T13:05:28Z

Content type: article

Language: en

Sources: [Microsoft Research](<https://devfeed.tech/sources/microsoft-research.md>)

Topics: [synthetic-data](<https://devfeed.tech/topics/synthetic-data.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [Data pipelines](<https://devfeed.tech/topics/data-pipelines.md>), [managing large-scale text data in AI](<https://devfeed.tech/topics/managing-large-scale-text-data-in-ai.md>), [enterprise data protection](<https://devfeed.tech/topics/enterprise-data-protection.md>), [trust](<https://devfeed.tech/topics/trust.md>), [data-governance](<https://devfeed.tech/topics/data-governance.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [complexity](<https://devfeed.tech/tags/complexity.md>), [confidence](<https://devfeed.tech/tags/confidence.md>), [copilot](<https://devfeed.tech/tags/copilot.md>), [data](<https://devfeed.tech/tags/data.md>), [enterprise](<https://devfeed.tech/tags/enterprise.md>), [enterprise-data](<https://devfeed.tech/tags/enterprise-data.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [evaluations](<https://devfeed.tech/tags/evaluations.md>), [first](<https://devfeed.tech/tags/first.md>), [full](<https://devfeed.tech/tags/full.md>), [information](<https://devfeed.tech/tags/information.md>), [microsoft](<https://devfeed.tech/tags/microsoft.md>), [microsoft-copilot](<https://devfeed.tech/tags/microsoft-copilot.md>), [organizations](<https://devfeed.tech/tags/organizations.md>), [post](<https://devfeed.tech/tags/post.md>), [requires](<https://devfeed.tech/tags/requires.md>), [research-blog](<https://devfeed.tech/tags/research-blog.md>), [scale](<https://devfeed.tech/tags/scale.md>), [synthetic](<https://devfeed.tech/tags/synthetic.md>), [test](<https://devfeed.tech/tags/test.md>)

### AI overview

Microsoft describes an Industry Evaluation Data Generation Platform for testing Copilot with synthetic enterprise tenants rather than customer data. Its environments model evolving organizations and workflows across pharmaceutical, capital markets, and IT/software industries, using public O*NET data, job postings, and SEC filings to guide realism. A six-stage automated pipeline adapts workplace tasks into industry-specific queries grounded in those environments, then checks accuracy and uniqueness. Internal quality measures cover structural fidelity, contextual relevance, behavioral realism, and content safety. Microsoft reports a 39% improvement in its composite synthetic-data quality score, creation time reduced from multiple days to automated runs, and estimated savings of about $100,000 per dataset and $300,000 overall. The text says the datasets use no customer data and are screened for harmful or identifying content; public sources receive legal and policy review.

### Source excerpt

Evaluating Microsoft Copilot at enterprise scale requires test data that captures the full complexity of real organizations, yet real enterprise data contains sensitive information that cannot be used for evaluation under any circumstances. The post From synthetic enterprise data evaluations to real Microsoft Copilot AI confidence appeared first on Microsoft Research.

## Building a Human Review Pipeline for AI Search: A Practical Developer's Guide

DevFeed: [Building a Human Review Pipeline for AI Search: A Practical Developer's Guide](<https://devfeed.tech/articles/building-a-human-review-pipeline-for-ai-search-a-practical-developer-s-guide-109278.md>)

Original publisher: [Read original article](<https://www.sitepoint.com/building-a-human-review-pipeline-for-ai-search-a-practical-developer-s-guide/>)

Author: Saifullah Adenwalla

Published: 2026-10-30T09:57:36Z

Content type: tutorial

Language: en

Sources: [SitePoint](<https://devfeed.tech/sources/sitepoint.md>)

Topics: [human review](<https://devfeed.tech/topics/human-review.md>), [responsible-ai](<https://devfeed.tech/topics/responsible-ai.md>), [managing large-scale text data in AI](<https://devfeed.tech/topics/managing-large-scale-text-data-in-ai.md>), [AI Chat](<https://devfeed.tech/topics/ai-chat.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-search](<https://devfeed.tech/tags/ai-search.md>), [api](<https://devfeed.tech/tags/api.md>), [api-documentation](<https://devfeed.tech/tags/api-documentation.md>), [apis](<https://devfeed.tech/tags/apis.md>), [authentication](<https://devfeed.tech/tags/authentication.md>), [authorization](<https://devfeed.tech/tags/authorization.md>), [automated](<https://devfeed.tech/tags/automated.md>), [batch](<https://devfeed.tech/tags/batch.md>), [bearer](<https://devfeed.tech/tags/bearer.md>), [bearer-token](<https://devfeed.tech/tags/bearer-token.md>), [behavior](<https://devfeed.tech/tags/behavior.md>), [browser](<https://devfeed.tech/tags/browser.md>), [browser-tools](<https://devfeed.tech/tags/browser-tools.md>), [build](<https://devfeed.tech/tags/build.md>), [developer](<https://devfeed.tech/tags/developer.md>), [developer-tools](<https://devfeed.tech/tags/developer-tools.md>), [development-environment](<https://devfeed.tech/tags/development-environment.md>), [guide](<https://devfeed.tech/tags/guide.md>), [human-review](<https://devfeed.tech/tags/human-review.md>), [pipeline](<https://devfeed.tech/tags/pipeline.md>), [practical](<https://devfeed.tech/tags/practical.md>), [review](<https://devfeed.tech/tags/review.md>), [search](<https://devfeed.tech/tags/search.md>)

### AI overview

A useful human-review pipeline separates whether an AI search answer is supported by its evidence, completes the user's task, and relies on suitable, current sources. The guide recommends preserving each run's answer, retrieved passages, document versions, and configuration; applying a versioned rubric; and validating structured reviews with Pydantic 2. It stresses that schema checks do not establish judgment correctness, and that reviewer identity and evidence references should be verified server-side. Independent reviews and adjudication can reveal rubric ambiguity; Cohen's kappa measures agreement, not correctness, and needs context such as raw agreement and label frequencies. Adjudicated cases can become regression tests, with DVC tracking dataset metadata while an access-controlled remote stores the data. Label Studio is an interface option whose exports need an adapter; a custom interface may better support API-version navigation. The guide also recommends testing retrieval separately from generation and reviewing usability alongside answer quality. These are practical steps for a hypothetical API documentation assistant, not reported production results.

### Source excerpt

null Continue reading Building a Human Review Pipeline for AI Search: A Practical Developer's Guide on SitePoint.

## De-identification Doesn't End at Deployment: How to Know What Is Safe to Release

DevFeed: [De-identification Doesn't End at Deployment: How to Know What Is Safe to Release](<https://devfeed.tech/articles/de-identification-doesn-t-end-at-deployment-how-to-know-what-is-safe-to-release-104407.md>)

Original publisher: [Read original article](<https://www.johnsnowlabs.com/de-identification-doesnt-end-at-deployment-how-to-know-what-is-safe-to-release/>)

Author: Veysel Kocaman

Published: 2026-10-27T20:31:07Z

Content type: article

Language: en

Sources: [John Snow Labs](<https://devfeed.tech/sources/john-snow-labs.md>)

Topics: [Data Quality](<https://devfeed.tech/topics/data-quality.md>), [managing large-scale text data in AI](<https://devfeed.tech/topics/managing-large-scale-text-data-in-ai.md>)

Tags: [about](<https://devfeed.tech/tags/about.md>), [alert](<https://devfeed.tech/tags/alert.md>), [all](<https://devfeed.tech/tags/all.md>), [allows](<https://devfeed.tech/tags/allows.md>), [also](<https://devfeed.tech/tags/also.md>), [audit](<https://devfeed.tech/tags/audit.md>), [audits](<https://devfeed.tech/tags/audits.md>), [batch](<https://devfeed.tech/tags/batch.md>), [business](<https://devfeed.tech/tags/business.md>), [de-identification](<https://devfeed.tech/tags/de-identification.md>), [first](<https://devfeed.tech/tags/first.md>), [hospital](<https://devfeed.tech/tags/hospital.md>), [john](<https://devfeed.tech/tags/john.md>), [labs](<https://devfeed.tech/tags/labs.md>), [note](<https://devfeed.tech/tags/note.md>), [notes](<https://devfeed.tech/tags/notes.md>), [post](<https://devfeed.tech/tags/post.md>), [program](<https://devfeed.tech/tags/program.md>), [records](<https://devfeed.tech/tags/records.md>), [release](<https://devfeed.tech/tags/release.md>), [type](<https://devfeed.tech/tags/type.md>)

### AI overview

A de-identification pipeline's validation score applies only to the data it was tested on, so each release needs fresh evidence. The article recommends tracking data drift against a frozen, source-specific baseline; screening every document with pinned PHI decision models; and auditing a truly random sample from the screened release to estimate residual PHI with a stated confidence bound. Its example says 300 clean documents support a claim that fewer than 1% of a release contains PHI at 95% confidence, while audits must account for patient clustering and reviewer misses. Models help select and rank documents but do not certify safety, since they may share the pipeline's blind spots. The article describes calibrating drift thresholds to normal corpus variation, treating unavailable signals as unknown, and setting release thresholds to balance leak detection against withholding clean records. For large releases, it recommends separate risk buckets and audits, with claims limited to the batch, pipeline, model version, and threshold assessed; expert determination remains the expert's decision.

### Source excerpt

Every de-identification program reaches the same moment. A new batch arrives -- a million notes from a new hospital, a new note type, a year of fresh records. Someone asks:... The post De-identification Doesn't End at Deployment: How to Know What Is Safe to Release appeared first on John Snow Labs.

[Next page](<https://devfeed.tech/topics/managing-large-scale-text-data-in-ai.md?cursor=WyIyMDI2LTEwLTI3VDIwOjMxOjA3KzAwOjAwIiwgImE0Y2JhZWYwLTI5ODUtNDIyNC1iZGU2LTZhNjE5NDI1NmUxYyJd>)