# It passed CI. It passed your evals. The customer still got the wrong answer.

DevFeed: [It passed CI. It passed your evals. The customer still got the wrong answer.](<https://devfeed.tech/articles/it-passed-ci-it-passed-your-evals-the-customer-still-got-the-wrong-answer-10828.md>)

Original publisher: [Read original article](<https://thenewstack.io/ai-agent-trace-debugging/>)

Author: Sean O'Dell

Published: 2026-09-13T14:00:00Z

Content type: article

Language: en

Sources: [The New Stack](<https://devfeed.tech/sources/the-new-stack.md>)

Topics: [AI Agent](<https://devfeed.tech/topics/ai-agent.md>), [agent observability](<https://devfeed.tech/topics/agent-observability.md>), [observability](<https://devfeed.tech/topics/observability.md>), [tracing](<https://devfeed.tech/topics/tracing.md>), [dynatrace](<https://devfeed.tech/topics/dynatrace.md>), [ci](<https://devfeed.tech/topics/ci.md>), [coding](<https://devfeed.tech/topics/coding.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-agent](<https://devfeed.tech/tags/ai-agent.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [ai-engineering](<https://devfeed.tech/tags/ai-engineering.md>), [ci](<https://devfeed.tech/tags/ci.md>), [coding](<https://devfeed.tech/tags/coding.md>), [dynatrace](<https://devfeed.tech/tags/dynatrace.md>), [observability](<https://devfeed.tech/tags/observability.md>), [post-contributed](<https://devfeed.tech/tags/post-contributed.md>), [sponsor-dynatrace](<https://devfeed.tech/tags/sponsor-dynatrace.md>), [sponsored](<https://devfeed.tech/tags/sponsored.md>), [sponsored-post-contributed](<https://devfeed.tech/tags/sponsored-post-contributed.md>), [traces](<https://devfeed.tech/tags/traces.md>)

## AI overview

The article explains how AI-agent failures can pass CI and evaluations while still producing slow or incorrect customer-facing results. It presents distributed traces and agent trajectories--model calls, tool calls, arguments, and results--as evidence for debugging retrieval behavior, release context, and feature-flag state.

## Source excerpt

A diff is not evidence. It's a statement of intent. The tests passed. The review's done. The change is live. The post It passed CI. It passed your evals. The customer still got the wrong answer. appeared first on The New Stack.