# The agent evaluation playbook: Built-in, DeepEval, and AutoEval on Amazon Bedrock AgentCore Evaluations

DevFeed: [The agent evaluation playbook: Built-in, DeepEval, and AutoEval on Amazon Bedrock AgentCore Evaluations](<https://devfeed.tech/articles/the-agent-evaluation-playbook-built-in-deepeval-and-autoeval-on-amazon-bedrock-agentcore-evaluations-127755.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/the-agent-evaluation-playbook-built-in-deepeval-and-autoeval-on-agentcore/>)

Author: Bharathi Srinivasan

Published: 2026-11-10T22:35:03Z

Content type: article

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [Amazon Bedrock AgentCore](<https://devfeed.tech/topics/amazon-bedrock-agentcore.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [agent observability](<https://devfeed.tech/topics/agent-observability.md>)

Tags: [advanced-300](<https://devfeed.tech/tags/advanced-300.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agent-evaluation](<https://devfeed.tech/tags/agent-evaluation.md>), [agentcore](<https://devfeed.tech/tags/agentcore.md>), [amazon-bedrock](<https://devfeed.tech/tags/amazon-bedrock.md>), [amazon-bedrock-agentcore](<https://devfeed.tech/tags/amazon-bedrock-agentcore.md>), [announcements](<https://devfeed.tech/tags/announcements.md>), [api](<https://devfeed.tech/tags/api.md>), [arguments](<https://devfeed.tech/tags/arguments.md>), [aws](<https://devfeed.tech/tags/aws.md>), [backend](<https://devfeed.tech/tags/backend.md>), [batch](<https://devfeed.tech/tags/batch.md>), [behavior](<https://devfeed.tech/tags/behavior.md>), [behaviors](<https://devfeed.tech/tags/behaviors.md>), [bias](<https://devfeed.tech/tags/bias.md>), [built-in](<https://devfeed.tech/tags/built-in.md>), [business](<https://devfeed.tech/tags/business.md>), [data](<https://devfeed.tech/tags/data.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [evaluations](<https://devfeed.tech/tags/evaluations.md>), [leak](<https://devfeed.tech/tags/leak.md>), [skip](<https://devfeed.tech/tags/skip.md>)

## AI overview

AgentCore evaluations should start from the agent's business outcome, translate it into observable behaviors, then select evaluators at the trace or session level. AWS built-ins cover response quality, goal completion, tool and skill use, safety, and expected tool trajectories. Managed DeepEval adds multi-turn, tool-use, and safety signals such as conversation completeness, knowledge retention, and PII leakage; AutoEval offers specialized security, impossibility, and humor checks. Multi-dimension custom judges score independently failing criteria in one call when they share the same evidence, level, and model; keep criteria separate when established metrics, different levels or models, or ground truth requirements apply. On-demand evaluation supports all evaluators; online sampling supports built-in and third-party metrics; batch evaluation runs fixed session sets and is required for evaluators needing ground truth. The same evaluator IDs work across modes.

## Source excerpt

When you deploy an agent, whether it is working is really several questions: did it finish the task, stay on topic, leak data, or skip steps? This post shows how to evaluate agents on Amazon Bedrock AgentCore with built-in, DeepEval, and AutoEval evaluators, and how to match each behavior that matters to the right evaluator at the right level.