# How sandbox boundaries affect AI coding agent evaluations

DevFeed: [How sandbox boundaries affect AI coding agent evaluations](<https://devfeed.tech/articles/your-ai-coding-agent-evaluation-is-only-as-good-as-its-sandbox-30939.md>)

Original publisher: [Read original article](<https://devblogs.microsoft.com/blog/your-ai-coding-agent-evaluation-is-only-as-good-as-its-sandbox/>)

Author: Waldek Mastykarz

Published: 2026-09-16T09:09:51Z

Content type: opinion

Language: en

Sources: [Developer Blogs](<https://devfeed.tech/sources/developer-blogs.md>)

Topics: [AI-assisted coding](<https://devfeed.tech/topics/ai-assisted-coding.md>), [Testing](<https://devfeed.tech/topics/testing.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [coding](<https://devfeed.tech/topics/coding.md>)

Tags: [agent-experience](<https://devfeed.tech/tags/agent-experience.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-coding](<https://devfeed.tech/tags/ai-coding.md>), [ax](<https://devfeed.tech/tags/ax.md>), [coding](<https://devfeed.tech/tags/coding.md>), [developers](<https://devfeed.tech/tags/developers.md>), [eval](<https://devfeed.tech/tags/eval.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [measurement](<https://devfeed.tech/tags/measurement.md>), [sandbox](<https://devfeed.tech/tags/sandbox.md>)

## AI overview

The article explains that an AI coding agent evaluation can produce misleading scores when the agent retrieves answers from its prompt, environment, or other accessible resources. It argues that evaluators should define the capability being tested and set information boundaries accordingly, including restricting access to evidence when measuring internal model knowledge.

## Source excerpt

Your AI coding agent passed the eval. But did the model know the answer, or did it find it somewhere on your machine? A correct answer can still invalidate your measurement. The post Your AI coding agent evaluation is only as good as its sandbox appeared first on Microsoft for Developers.