# Evaluating gpt-4o-mini vs. gpt-3.5-turbo for RAG applications

DevFeed: [Evaluating gpt-4o-mini vs. gpt-3.5-turbo for RAG applications](<https://devfeed.tech/articles/evaluating-gpt-4o-mini-vs-gpt-3-5-turbo-for-rag-applications-21728.md>)

Original publisher: [Read original article](<http://blog.pamelafox.org/2025/03/gpt-4o-mini-vs-gpt-35-turbo-for-rag.html>)

Author: Pamela Fox (noreply@blogger.com)

Published: 2025-03-06T08:22:00Z

Content type: comparison

Language: en

Sources: [Pamela Fox](<https://devfeed.tech/sources/pamela-fox.md>)

Topics: [Retrieval Augmented Generation (RAG)](<https://devfeed.tech/topics/retrieval-augmented-generation-rag.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Azure](<https://devfeed.tech/topics/azure.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [azure](<https://devfeed.tech/tags/azure.md>), [cost](<https://devfeed.tech/tags/cost.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [gpt](<https://devfeed.tech/tags/gpt.md>), [gpt-3](<https://devfeed.tech/tags/gpt-3.md>), [latency](<https://devfeed.tech/tags/latency.md>), [openai](<https://devfeed.tech/tags/openai.md>), [rag](<https://devfeed.tech/tags/rag.md>), [vs](<https://devfeed.tech/tags/vs.md>)

## AI overview

The article evaluates gpt-4o-mini against gpt-35-turbo as the default model for the Azure RAG sample application. Across evaluations using HR documents and the author's blog, gpt-4o-mini had comparable groundedness and relevance, produced longer answers, took more generation time, and had lower overall cost because of its lower per-token pricing.

## Source excerpt

The azure-search-openai-demo repository was first created in March 2023 and is now the most popular RAG sample solution for Azure. Since the world of generative AI changes so rapidly, we've made many upgrades to its underlying packages and technologies over the past two years. But we've never changed the default GPT model used for the RAG flow: gpt-35-turbo. Why, when there are new models that are cheaper and reportedly better, such as gpt-4o-mini? Well, changing the model is one of the most significant changes you can make to impact RAG answer quality, and I did not want to make the change without thorough evaluation. Good news! I have now run several bulk evaluations on different RAG knowledge bases, and I feel fairly confident that a switch to gpt-4o-mini is a positive overall change, with some caveats. In my evaluations, gpt-4o-mini generates answers with comparable groundedness and relevance. The time-per-token is slightly less, but the answers are 50% longer on average, thus they take 45% more time for generation. The additional answer length often provides additional details based off the context, especially for questions where the answer is a list or a sequential process. The gpt-4o-mini per-token pricing is about 1/3 of gpt-35-turbo pricing, which works out to a lower overall cost. Let's dig into the results more in this post. Evaluation results I ran bulk evaluations on two knowledge bases, starting with the sample data that we include in the repository, a bunch of invented HR documents for a fictitious company. Then, since I always like to evaluate knowledge that I know deeply, I also ran evaluations on a search index composed entirely of my own blog posts from this very blog. Here are the results for the HR documents, for 50 Q/A pairs: metric stat gpt-35-turbo gpt-4o-mini gpt_groundedness pass_rate 0.98 0.98 mean_rating 4.94 4.9 gpt_relevance pass_rate 0.98 0.96 mean_rating 4.42 4.54 answer_length mean 667.7 934.36 latency mean 2.96 3.8 citations_matched