# Evaluating GPT-5 for hallucination handling in RAG applications

DevFeed: [Evaluating GPT-5 for hallucination handling in RAG applications](<https://devfeed.tech/articles/gpt-5-will-it-rag-21739.md>)

Original publisher: [Read original article](<http://blog.pamelafox.org/2025/08/gpt-5-will-it-rag.html>)

Author: Pamela Fox (noreply@blogger.com)

Published: 2025-08-11T18:40:00Z

Content type: opinion

Language: en

Sources: [Pamela Fox](<https://devfeed.tech/sources/pamela-fox.md>)

Topics: [Retrieval-Augmented Generation](<https://devfeed.tech/topics/retrieval-augmented-generation.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [OpenAI](<https://devfeed.tech/topics/openai.md>), [SDKs](<https://devfeed.tech/topics/sdks.md>), [Azure](<https://devfeed.tech/topics/azure.md>), [Ground truth / benchmark quality](<https://devfeed.tech/topics/ground-truth-benchmark-quality.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [azure](<https://devfeed.tech/tags/azure.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [gpt](<https://devfeed.tech/tags/gpt.md>), [hallucinations](<https://devfeed.tech/tags/hallucinations.md>), [llm](<https://devfeed.tech/tags/llm.md>), [models](<https://devfeed.tech/tags/models.md>), [openai](<https://devfeed.tech/tags/openai.md>), [rag](<https://devfeed.tech/tags/rag.md>), [sdk](<https://devfeed.tech/tags/sdk.md>)

## AI overview

The article evaluates GPT-5 models in an Azure AI Foundry RAG template. It reports that GPT-5 identified when the source documents lacked enough information to answer a question, and describes broader bulk evaluations using the azure-ai-evaluations SDK across 50 question-and-answer pairs.

## Source excerpt

OpenAI released the GPT-5 model family today, with an emphasis on accurate tool calling and reduced hallucinations. For those of us working on RAG (Retrieval-Augmented Generation), it's particularly exciting to see a model specifically trained to reduce hallucination. There are five variants in the family: gpt-5 gpt-5-mini gpt-5-nano gpt-5-chat: Not a reasoning model, optimized for chat applications gpt-5-pro: Only available in ChatGPT, not via the API As soon as GPT-5 models were available in Azure AI Foundry, I deployed them and evaluated them inside our popular open source RAG template. I was immediately impressed - not by the model's ability to answer a question, but by it's ability to admit it could not answer a question! You see, we have one test question for our sample data (HR documents for a fictional company's) that sounds like it should be an easy question: "What does a Product Manager do?" But, if you actually look at the company documents, there's no job description for "Product Manager", only related jobs like "Senior Manager of Product Management". Every other model, including the reasoning models, has still pretended that it could answer that question. For example, here's a response from o4-mini: However, the gpt-5 model realizes that it doesn't have the information necessary, and responds that it cannot answer the question: As I always say: I would much rather have an LLM admit that it doesn't have enough information instead of making up an answer. Bulk evaluation But that's just a single question! What we really need to know is whether the GPT-5 models will generally do a better job across the board, on a wide range of questions. So I ran bulk evaluations using the azure-ai-evaluations SDK, checking my favorite metrics: groundedness (LLM-judged), relevance (LLM-judged), and citation_match (regex based off ground truth citations). I didn't bother evaluating gpt-5-nano, as I did some quick manual tests and wasn't impressed enough - plus, we've never