# Hallucination detection A technical task for identifying hallucinations: plausible but incorrect information produced by natural-language-generation models, particularly large language models. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Putting Captions to the Test: Evaluating Video Caption Quality through Multiple-Choice Question Answering DevFeed: [Putting Captions to the Test: Evaluating Video Caption Quality through Multiple-Choice Question Answering]() Original publisher: [Read original article]() Published: 2026-09-11T00:00:00Z Content type: article Language: en Sources: [Apple Machine Learning Research]() Topics: [LLM evaluation / benchmarking](), [Ground truth / benchmark quality](), [Hallucination detection]() Tags: [benchmark](), [computer-vision](), [evaluation](), [metrics](), [multimodal](), [research](), [video]() ### AI overview The article introduces CapQuiz, a reference-free benchmark for evaluating video-caption quality through human-verified multiple-choice questions. It also proposes CapF1, combining factuality and visual-information coverage, and reports stronger correlation with human judgments than existing metrics. ### Source excerpt Evaluating video captioning remains a critical challenge for Visual Large Language Models (VLLMs). Existing metrics primarily rely on matching generated text against ground-truth references. This paradigm suffers from the "one-to-many" nature of video description, where high-quality captions are often penalized for lexical mismatches or valid shifts in visual focus. Furthermore, such assessments are typically one-dimensional, failing to provide a fine-grained analysis of caption quality. To address this, we redefine caption quality via information fidelity: A caption must maximize the coverage... ## Empty shelves or lost keys? Recall is the bottleneck for parametric factuality DevFeed: [Empty shelves or lost keys? Recall is the bottleneck for parametric factuality]() Original publisher: [Read original article]() Published: 2026-08-12T09:51:00Z Content type: article Language: en Sources: [The latest research from Google]() Topics: [Google](), [LLM evaluation / benchmarking](), [Large Language Model](), [Benchmark](), [Hallucination detection](), [AI Inference]() Tags: [benchmark](), [classification](), [errors](), [generative-ai](), [google](), [inference](), [large-language-models-llms](), [natural-language-processing](), [post-training](), [research]() ### AI overview This Google Research article argues that many factual errors in frontier large language models arise from recall failures rather than missing encoded knowledge. It presents knowledge profiling, which separates encoding, recall, and recognition, and introduces WikiProfile, a benchmark of 2,150 Wikipedia-derived facts tested through questions targeting these abilities. ### Source excerpt Generative AI ## Science One Framework: A verifiable autonomous research framework via Chain-of-Evidence DevFeed: [Science One Framework: A verifiable autonomous research framework via Chain-of-Evidence]() Original publisher: [Read original article]() Published: 2026-07-30T20:36:36Z Content type: article Language: en Sources: [The latest research from Google]() Topics: [Artificial Intelligence](), [Google](), [AI-generated research reports](), [Large language models (LLMs)](), [Hallucination detection](), [LLM evaluation / benchmarking]() Tags: [agents](), [ai](), [autonomous-agents](), [benchmarks](), [evaluation](), [general-science](), [google](), [hallucinations](), [language-models](), [machine-intelligence](), [natural-language-processing](), [research](), [research-prototype]() ### AI overview Google Research introduces the Science One Framework, an experimental autonomous research prototype built around Chain-of-Evidence. It is designed to make AI-generated research verifiable by linking claims to supporting evidence and by auditing papers against their code and evidence. The article reports that the framework eliminates phantom references and produces fully verifiable scores in the described evaluations. ### Source excerpt General Science ## Doximity Ask DevFeed: [Doximity Ask]() Original publisher: [Read original article]() Author: Doximity Published: 2026-06-23T09:30:00Z Content type: article Language: en Sources: [Doximity]() Topics: [Artificial Intelligence](), [Retrieval-Augmented Generation](), [Hallucination detection]() Tags: [agentic](), [ai](), [grounding](), [hallucinations](), [retrieval-augmented-generation](), [review](), [verification]() ### AI overview The article explains how Doximity Ask approaches reliability in clinical AI. It describes a HIPAA-compliant clinical reference platform that combines retrieval-augmented generation, validated tools, a reasoning model, citation-grounded responses, automated verification, physician-authored evaluation, and production monitoring. It argues that reliability is a system property spanning retrieval, ranking, reasoning, tool use, verification, and evaluation. ### Source excerpt Medical AI has attracted confident claims: perfect scores, no hallucinations, a system that gracefully declines whenever the evidence runs out. But are these dimensions the right way to think about a tool that physicians use to make real-world decisions? For clinical AI, reliability is not just a question of whether an answer is correct. It is also a question of whether a physician can understand where the answer came from, evaluate the supporting evidence, and identify situations where uncertainty remains. How Doximity Ask Answers Clinical Questions Few domains face stakes as high as medical AI, where system performance can directly influence patient outcomes, clinical workflows, and healthcare professionals' decisions. Clinical reference requires traceable grounding, numerically exact doses and scores, and explicit handling of situations in which the available evidence is insufficient to support an answer. At the same time, healthcare providers often work under significant time constraints and have exceptionally high expectations for the performance and reliability of AI systems. Doximity Ask is a HIPAA compliant composite clinical reference platform for question answering at the point of care. It combines retrieval-augmented generation over a physician-governed evidence library, multi-turn agentic tool use through a fixed registry of validated calculators and structured lookups, and a post-trained reasoning model that orchestrates retrieval, tool calls, and citation-grounded generation. Automated verifiers, physician-authored rubrics, and in-product physician review form one evaluation loop that grades releases, defines training rewards, and monitors production. Reliability Is A System Property Over 150 health systems have purchased the Doximity Clinical AI Suite including 8 of the top 20 hospitals in the country, therefore reliability is paramount. The reliability of a clinical AI system depends on more than the capabilities of its underlying language model. Pro ## Open challenges in LLM research DevFeed: [Open challenges in LLM research]() Original publisher: [Read original article]() Author: Chip Huyen Published: 2023-08-16T00:00:00Z Content type: opinion Language: en Sources: [Chip Huyen]() Topics: [Large Language Model](), [Hallucination detection](), [context](), [LLM evaluation / benchmarking](), [Chain-of-thought](), [multimodal]() Tags: [ai](), [chain-of-thought](), [challenges](), [context](), [evaluation](), [hallucination-detection](), [llm](), [multimodal](), [research]() ### AI overview This article surveys ten research directions aimed at improving large language models. The supplied excerpt focuses on reducing and measuring hallucinations, optimizing context length and construction, and briefly identifies multimodality, new architectures, and GPU alternatives as areas of interest. ### Source excerpt [LinkedIn discussion, Twitter thread] Never before in my life had I seen so many smart people working on the same goal: making LLMs better. After talking to many people working in both industry and academia, I noticed the 10 major research directions that emerged. The first two directions, hallucinations and context learning, are probably the most talked about today. I'm the most excited about numbers 3 (multimodality), 5 (new architecture), and 6 (GPU alternatives). 1. Reduce and measure hallucinations Hallucination is a heavily discussed topic already so I'll be quick. Hallucination happens when an AI model makes stuff up. For many creative use cases, hallucination is a feature. However, for most other use cases, hallucination is a bug. I was at a panel on LLM with Dropbox, Langchain, Elastics, and Anthropic recently, and the #1 roadblock they see for companies to adopt LLMs in production is hallucination. Mitigating hallucination and developing metrics to measure hallucination is a blossoming research topic, and I've seen many startups focus on this problem. There are also ad-hoc tips to reduce hallucination, such as adding more context to the prompt, chain-of-thought, self-consistency, or asking your model to be concise in its response. To learn more about hallucination: Survey of Hallucination in Natural Language Generation (Ji et al., 2022) How Language Model Hallucinations Can Snowball (Zhang et al., 2023) A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity (Bang et al., 2023) Contrastive Learning Reduces Hallucination in Conversations (Sun et al., 2022) Self-Consistency Improves Chain of Thought Reasoning in Language Models (Wang et al., 2022) SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models (Manakul et al., 2023) A simple example of fact-checking and hallucination by NVIDIA's NeMo-Guardrails 2. Optimize context length and context construction A vast ma