# AX Published articles for AX. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## How sandbox boundaries affect AI coding agent evaluations DevFeed: [How sandbox boundaries affect AI coding agent evaluations]() Original publisher: [Read original article]() Author: Waldek Mastykarz Published: 2026-09-16T09:09:51Z Content type: opinion Language: en Sources: [Developer Blogs]() Topics: [AI-assisted coding](), [Testing](), [Artificial Intelligence](), [coding]() Tags: [agent-experience](), [ai](), [ai-coding](), [ax](), [coding](), [developers](), [eval](), [evaluation](), [measurement](), [sandbox]() ### AI overview The article explains that an AI coding agent evaluation can produce misleading scores when the agent retrieves answers from its prompt, environment, or other accessible resources. It argues that evaluators should define the capability being tested and set information boundaries accordingly, including restricting access to evidence when measuring internal model knowledge. ### Source excerpt Your AI coding agent passed the eval. But did the model know the answer, or did it find it somewhere on your machine? A correct answer can still invalidate your measurement. The post Your AI coding agent evaluation is only as good as its sandbox appeared first on Microsoft for Developers. ## Evaluating Model Cost and Quality for Developer Workflows DevFeed: [Evaluating Model Cost and Quality for Developer Workflows]() Original publisher: [Read original article]() Author: Waldek Mastykarz Published: 2026-09-09T09:14:04Z Content type: comparison Language: en Sources: [Developer Blogs]() Topics: [Claude](), [GitHub Copilot](), [Visual Studio Code](), [Testing](), [upgrade](), [Code]() Tags: [agent-experience](), [ai](), [ax](), [claude](), [comparison](), [cost](), [evaluation](), [github-copilot](), [model](), [testing](), [upgrade](), [visual-studio-code](), [windows]() ### AI overview This article compares GPT-6 Astra with Claude Sonnet 4.6 across code upgrade scenarios in GitHub Copilot Chat for Visual Studio Code on Windows. It argues that teams should evaluate models on their own repositories and workflows because higher cost did not consistently produce better results in the reported tests. ### Source excerpt The smartest model can cost five times more and deliver the same result, or even a worse one. See how evaluating your own work helps you get more value from your agent budget. The post Your work might not need the smartest model appeared first on Microsoft for Developers. ## How to test agent experience changes without shipping them DevFeed: [How to test agent experience changes without shipping them]() Original publisher: [Read original article]() Author: Waldek Mastykarz, Garry Trinder Published: 2026-07-21T07:15:12Z Content type: tutorial Language: en Sources: [Developer Blogs]() Topics: [AI Agent](), [Documentation](), [API](), [MCP](), [Command-line interface](), [upgrade]() Tags: [agent-experience](), [ai](), [ai-agent](), [api](), [ax](), [cli](), [developers](), [documentation](), [how-to](), [mcp](), [mcp-server](), [upgrade]() ### AI overview This Microsoft for Developers article explains how to test changes intended to influence AI agent behavior before shipping them. It describes validating documentation, API, and MCP server changes locally, using evaluations and controlled hypotheses, and reports that a warning stating an existing plan would fail changed the agent's behavior in five out of five runs. ### Source excerpt Most changes you think will improve AI agent behavior won't. We tested a dozen hypotheses on a real project upgrade scenario and the majority failed. Learn how to emulate documentation, API, and MCP server changes locally so you can validate what works before shipping anything to production. The post How to test agent experience changes without shipping them appeared first on Microsoft for Developers. ## How to test agent skills without hitting real APIs DevFeed: [How to test agent skills without hitting real APIs]() Original publisher: [Read original article]() Author: Waldek Mastykarz Published: 2026-07-17T09:27:50Z Content type: tutorial Language: en Sources: [Developer Blogs]() Topics: [Agent Skills](), [API](), [Testing]() Tags: [agent-experience](), [agent-skills](), [ai](), [apis](), [ax](), [evals](), [how-to](), [microsoft](), [testing]() ### AI overview This tutorial explains how to evaluate agent skills that call APIs without incurring external API costs or mutating live data. It introduces transparent API mocking to support isolated, repeatable evaluation runs without changing the skill or contacting real endpoints. ### Source excerpt Your agent skill calls an API. The moment you start evaluating it, every run either costs money or mutates production data. Learn how to mock APIs transparently so you can run evals without changing your skill or hitting real endpoints. The post How to test agent skills without hitting real APIs appeared first on Microsoft for Developers. ## Building AX evals that actually work DevFeed: [Building AX evals that actually work]() Original publisher: [Read original article]() Author: Waldek Mastykarz Published: 2026-07-15T12:53:13Z Content type: tutorial Language: en Sources: [Developer Blogs]() Topics: [AI-assisted coding](), [Loop Engineering](), [Prompt Engineering]() Tags: [agent-experience](), [ai](), [ai-coding-agents](), [article](), [ax](), [coding-agents](), [evals](), [evaluation](), [how-to](), [microsoft](), [microsoft-for-developers](), [quality]() ### AI overview This eighth and final article in a series about Agent Experience explains how to build meaningful evaluations for AI coding agents. It identifies representative prompts, accurate and unambiguous criteria, and other structural decisions needed to produce useful signal rather than misleading scores. ### Source excerpt This is the eighth and final article in a series about Agent Experience (AX): the practice of making AI coding agents work correctly with your technology. The series covers what you can and can't control in the agent stack, how to measure whether your extensions are helping or hurting, and how to iterate toward better [...] The post Building AX evals that actually work appeared first on Microsoft for Developers. ## The hidden variables in your agent eval DevFeed: [The hidden variables in your agent eval]() Original publisher: [Read original article]() Author: Waldek Mastykarz Published: 2026-07-08T12:11:17Z Content type: article Language: en Sources: [Developer Blogs]() Topics: [AI-assisted coding](), [context](), [Operating system](), [Shell]() Tags: [agent](), [agent-experience](), [ai](), [ai-coding-agents](), [ax](), [benchmarks](), [context](), [reproducibility]() ### AI overview This seventh article in Microsoft's Agent Experience series examines hidden environmental variables that can make the same AI coding-agent evaluation produce different results. It discusses context such as operating system, shell, diagnostics, build errors, file paths, and workspace details. ### Source excerpt This is the seventh article in a series about Agent Experience (AX): the practice of making AI coding agents work correctly with your technology. The series covers what you can and can't control in the agent stack, how to measure whether your extensions are helping or hurting, and how to iterate toward better outcomes. You [...] The post The hidden variables in your agent eval appeared first on Microsoft for Developers. ## Don't rewrite your CLI for agents DevFeed: [Don't rewrite your CLI for agents]() Original publisher: [Read original article]() Author: Waldek Mastykarz Published: 2026-07-07T13:52:06Z Content type: article Language: en Sources: [Developer Blogs]() Topics: [Command-line interface](), [JSON](), [test](), [GitHub]() Tags: [agent-experience](), [ai](), [ax](), [cli](), [developers](), [github-copilot](), [json](), [microsoft]() ### AI overview This article examines whether developer CLIs should replace conventional arguments with a single JSON payload for use by coding agents. It describes a controlled comparison using synthetic multi-service deployment CLIs, identical prompts, and several coding models; in the supplied results, every tested args profile achieved perfect correctness across five runs. ### Source excerpt There's advice making the rounds: replace your CLI args with a single --json payload so agents can use your tool more effectively. The thinking being, that agents already think in structured formats, and nested data maps cleanly to JSON. Flat args on the other hand, force awkward conventions like repeating --service-name to delimit multi-value groups, [...] The post Don't rewrite your CLI for agents appeared first on Microsoft for Developers.