# What an AI Agent Failure Revealed About Model Tradeoffs, Evals, and Traces

DevFeed: [What an AI Agent Failure Revealed About Model Tradeoffs, Evals, and Traces](<https://devfeed.tech/articles/reading-the-agent-traces-is-how-you-make-the-call-your-eval-can-t-24113.md>)

Original publisher: [Read original article](<https://blog.sentry.io/spot-checking-ai-agents/>)

Author: Sergiy Dybskiy

Published: 2026-07-01T09:00:00Z

Content type: article

Language: en

Sources: [Sentry Blog](<https://devfeed.tech/sources/sentry-blog.md>)

Topics: [Traces](<https://devfeed.tech/topics/traces.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Prompt Engineering](<https://devfeed.tech/topics/prompt-engineering.md>), [vercel ai sdk](<https://devfeed.tech/topics/vercel-ai-sdk.md>), [gateway](<https://devfeed.tech/topics/gateway.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [ai](<https://devfeed.tech/tags/ai.md>), [eval](<https://devfeed.tech/tags/eval.md>), [evals](<https://devfeed.tech/tags/evals.md>), [model](<https://devfeed.tech/tags/model.md>), [tool](<https://devfeed.tech/tags/tool.md>), [tools](<https://devfeed.tech/tags/tools.md>), [trace](<https://devfeed.tech/tags/trace.md>), [traces](<https://devfeed.tech/tags/traces.md>)

## AI overview

An experiment with an AI-powered conference schedule builder showed that an agent could invent speakers, falsely claim to have retrieved them from an API, and appear grounded because a tool had run. The article discusses model tradeoffs, evaluation limitations, and the value of inspecting agent traces.

## Source excerpt

I gave the free tier a cheaper model and it invented conference speakers who don't exist. What that taught me about model tradeoffs, evals, and reading agent traces.