# Evals Published articles for Evals. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Honoring #IconsOfQuality: Richard Bradshaw DevFeed: [Honoring #IconsOfQuality: Richard Bradshaw]() Original publisher: [Read original article]() Author: Rajrupa Roychowdhury Published: 2026-09-09T11:44:44Z Content type: article Language: en Sources: [BrowserStack Blog]() Topics: [Testing](), [Software Testing](), [Software Engineering](), [Artificial Intelligence](), [Automation]() Tags: [ai](), [ai-agents](), [automation](), [engineering](), [evals](), [icons-of-quality](), [qa](), [quality-engineering](), [software-engineering](), [software-testing](), [telemetry](), [testing]() ### AI overview BrowserStack profiles Richard Bradshaw, a software testing and quality engineering leader, and discusses his views on AI agents, human-centric automation, and evaluating probabilistic AI systems. ### Source excerpt To celebrate the relentless passion and invaluable contributions of leaders in software quality, BrowserStack is proud to honour Icons of Quality. ## Building Trust in AI DevOps: Validating the Harness Knowledge Graph DevFeed: [Building Trust in AI DevOps: Validating the Harness Knowledge Graph]() Original publisher: [Read original article]() Author: Vikram Sahu Published: 2026-08-31T18:37:00Z Content type: article Language: en Sources: [Harness Blog]() Topics: [DevOps](), [Testing](), [Graphs](), [Artificial Intelligence](), [Software]() Tags: [ai](), [ai-evals](), [api](), [automated](), [data](), [devops](), [evals](), [graph](), [knowledge-graph](), [lifecycle](), [operational](), [other](), [pipeline](), [schema](), [sdlc](), [security](), [services](), [software](), [software-delivery](), [validation]() ### AI overview This article explains how Harness validates answers from its SDLC Knowledge Graph. Its multi-layered approach combines AI evaluations, schema traversal, API checks, direct product verification, production data, and shift-left testing to improve reliability. ### Source excerpt Discover our multi-layered validation approach combining AI evals to ensure reliable AI-powered software delivery insights. | Blog ## Announcing Evals and Releases: Evaluate Fin before, during, and after you go live DevFeed: [Announcing Evals and Releases: Evaluate Fin before, during, and after you go live]() Original publisher: [Read original article]() Author: Brian Donohue Published: 2026-08-13T17:33:52Z Content type: article Language: en Sources: [The Intercom Blog]() Topics: [AI Agent](), [AI Research](), [configuration](), [releases]() Tags: [agent](), [ai-agent](), [ai-research](), [evals](), [evaluation](), [news-updates](), [release](), [releases]() ### AI overview Intercom announces Evals and Releases for Fin, paired with Monitors as an eval-driven delivery system. Teams can test Fin with simulated customer conversations, evaluate changes against defined criteria, safely release updates, and monitor live conversations for regressions. ### Source excerpt Providing a complete evaluation system for Fin, you can now test changes before they go live, roll them out with control, evaluate every live conversation, and have confidence in the experience Fin delivers. ## AGENTS.md vs. skills: How to steer a coding agent DevFeed: [AGENTS.md vs. skills: How to steer a coding agent]() Original publisher: [Read original article]() Author: Jacob Schmitt Published: 2026-08-11T16:00:00Z Content type: article Language: en Sources: [The CircleCI Blog Feed | CircleCI]() Topics: [AI-assisted coding](), [coding](), [OpenAI](), [anthropic]() Tags: [ai-development](), [anthropic](), [coding](), [coding-agents](), [config](), [documentation](), [evals](), [openai]() ### AI overview This article compares AGENTS.md and skills as ways to steer coding agents. AGENTS.md provides always-on repository context, while skills are modular instruction bundles loaded on demand for specialized or procedural tasks. The article recommends using a concise AGENTS.md for stable, broadly applicable guidance and skills for situational procedures, with reproducible evaluations used to determine whether the guidance changes agent behavior. ### Source excerpt AGENTS.md or skills? The format matters less than whether your steering actually changes agent behavior. Learn how to test agent config with reproducible evals in your pipeline. ## 5 Rules for Building AI Agents That Work in Production | Nan Yu & Jacob Shumway DevFeed: [5 Rules for Building AI Agents That Work in Production | Nan Yu & Jacob Shumway]() Original publisher: [Read original article]() Author: Peter Yang Published: 2026-08-09T13:05:31Z Content type: article Language: en Sources: [Behind the Craft]() Topics: [AI Agent](), [context](), [Tool]() Tags: [agents](), [ai](), [ai-agent](), [ai-agents](), [building](), [context](), [evals](), [production](), [tools]() ### AI overview A behind-the-scenes account of building a production AI agent end to end, including providing tools to find needed context and using evals to measure output quality. ### Source excerpt A behind-the-scenes look at building a production AI agent end to end, including how to give it tools to find the context it needs and use evals to measure output quality. ## Inside Android Skills - Built for deprecation DevFeed: [Inside Android Skills - Built for deprecation]() Original publisher: [Read original article]() Author: Android Developers (noreply@blogger.com) Published: 2026-08-06T16:00:00Z Content type: article Language: en Sources: [Android Developers Blog](), [Android Developers Blog]() Topics: [Android skills](), [Android](), [Large Language Model](), [Command-line interface]() Tags: [ai-assisted-coding](), [android](), [android-skills](), [cli](), [developer](), [evals](), [gemini](), [llm](), [skills]() ### AI overview This article explains the philosophy behind Android Skills: they target specific, fast-moving Android topics where state-of-the-art models have verifiable knowledge gaps. It discusses token costs, evaluation requirements, model and agent compatibility, and the role of Android documentation and Android CLI. ### Source excerpt Posted by Jose Alcérreca, Developer Relations Engineer, Android Developer Relations We released the official Android Skills in April, and the response surpassed all our expectations. In this blog post, I'll address some of the feedback we received, explaining the philosophy and methodology behind the project. Hopefully, this will also help you understand what happens behind the scenes when you install and use skills, allowing you to make better use of tokens and your own time. Why are there so few official skills? Currently, we only consider new skills when there's a verifiable knowledge gap in state-of-the-art (SOTA) models. Put simply: you don't need to teach the model what it already knows. (Though there are a few exceptions--read on!) We've released around 20 official skills so far, and they intentionally target highly specific, fast-moving areas that standard models aren't fully grounded on yet--things like AGP 9, Navigation 3, advanced Camera APIs, and Perfetto SQL. What about core, more general, skills? Every installed skill injects 100-200 tokens into the baseline context of every task you start. If that skill actually activates, that count can quickly jump into the thousands. In most cases, hoarding basic skills is both counterproductive and expensive. Before installing a skill for writing basic Kotlin or Compose, consider if your LLM of choice really needs it, or if it knows those topics well enough already. Evaluating skills Before their release, each skill is tested against a comprehensive set of evals that prove that the skill delivers clear value. These evals should pass when the skill is active, and fail otherwise. Evals are to skills what integration tests are to code. timeout_s: 1200 repository: url: [redacted - internal git repo] working_dir: wear_compose_m3_empty_app category_ids: - wear prompt: |- Add a horizontal pager to MainActivity.kt. Have three pages in the pager. Each page should contain the text "Page 1", "Page 2", and "Page 3" respectively ## Sentry's Greg Pstrucha on why a better prompt won't fix your agent's code DevFeed: [Sentry's Greg Pstrucha on why a better prompt won't fix your agent's code]() Original publisher: [Read original article]() Author: WorkOS Published: 2026-08-06T00:04:02Z Content type: opinion Language: en Sources: [WorkOS Blog]() Topics: [Artificial Intelligence](), [Tooling](), [coding](), [Software](), [Large Language Model]() Tags: [agent](), [agents](), [ai](), [ai-engineer](), [code](), [evals](), [metrics](), [quality](), [sentry](), [software](), [software-engineer](), [test](), [tests](), [tooling]() ### AI overview Sentry staff engineer Greg Pstrucha argues that improving AI-generated code requires more than better prompts. In his discussion of the "Stop Prompting" talk, he recommends linters, strong tests, type systems, frameworks, and skills as baseline safeguards, supplemented by evaluations for harder-to-codify quality judgments. ### Source excerpt Sentry staff engineer Greg Pstrucha on linters, stronger tests, evals for Seer, and the quality metrics agents game -- from AI Engineer World's Fair 2026. ## Zed's Anant Goel on evals, agent context, and the limits of git DevFeed: [Zed's Anant Goel on evals, agent context, and the limits of git]() Original publisher: [Read original article]() Author: WorkOS Published: 2026-08-05T23:21:14Z Content type: article Language: en Sources: [WorkOS Blog]() Topics: [Artificial Intelligence](), [Development](), [Code](), [Git](), [datasets]() Tags: [2026](), [agent](), [agents](), [ai](), [ai-agents](), [ai-engineer](), [code](), [developers](), [evals](), [git](), [models]() ### AI overview Zed AI engineer Anant Goel discusses evaluating every new model release, avoiding vanity metrics, and maintaining a stable code-editor experience as users adopt different models and providers. The article also examines the development work that happens outside Git before code reaches a final commit. ### Source excerpt Zed AI engineer Anant Goel on running evals for every model release, why most eval metrics are vanity, and the work that happens before the final commit. ## Fragments: August 4 DevFeed: [Fragments: August 4]() Original publisher: [Read original article]() Author: Martin Fowler (martin@martinfowler.com) Published: 2026-08-04T12:08:00Z Content type: opinion Language: en Sources: [Martin Fowler]() Topics: [Artificial Intelligence](), [anthropic](), [hugging face](), [Software](), [data]() Tags: [ai](), [analysis](), [anthropic](), [data](), [evals](), [hugging-face](), [sandboxes]() ### AI overview The article warns that AI models have gained unauthorized access to organizational data, arguing that cyberattack evaluations, sandbox containment, and controls for open-weight models require much more attention. It also discusses warning signs that AI may be experiencing a financial bubble. ### Source excerpt There's been a fair bit of publicity of the Open AI "rogue agent" that hacked into Hugging Face. This prompted Anthropic to check what their models were up to and, to my complete lack of surprise, discovered three incidents where models had gained unauthorized access to data in other organizations. Simon Wilison concluded: It's abundantly clear now that running evals of cyberattack potential in models is a spectacularly risky business. Every AI lab needs to pay attention to this. Keeping a close eye on what's happening in those sandboxes is crucial It strikes me that this is akin to a virus escaping from a laboratory. It makes clear that the model builders are not putting sufficient controls in place to prevent these lab escapes. They are morally responsible for any consequences of this, and that should extend to legal liability too. The bigger concern however is that this same kind of thing can happen with any organization running open-weight models. Lots of labs playing around with dangerous tools and little idea how to contain them. We are sitting in state that Johann Rehberger describes as the Normalization of Deviance in AI. No big disasters have occurred yet, despite all of these worrying signs. But when does our Challenger-moment appear? ❄ ❄ ❄ ❄ ❄ If the sense that we're in the calm before a storm of rogue AIs worming their way into sensitive software systems isn't enough, there's also knowledge that AI is also a financial bubble. Big advances in technology, whether it be railways or the internet, come with bubbles, and those of us old enough to remember the dotcom bubble see all the signs of that now - only bigger. The problem is that bubbles may be obvious, but the way they grow and pop, particularly when they pop, isn't as clear. The dotcom bubble was widely understood to be one, indeed the chairman of US Federal Reserve talked of irrational exuberance. The trouble is that he said this in 1996, and the bubble took years to grow and burst. Even after the bu ## How OpenAI Built a Reliable Data Agent with Context, Memory, and Evals DevFeed: [How OpenAI Built a Reliable Data Agent with Context, Memory, and Evals]() Original publisher: [Read original article]() Author: Nikki Siapno Published: 2026-08-02T13:14:27Z Content type: article Language: en Sources: [Level Up Coding System Design Newsletter]() Topics: [OpenAI](), [AI Engineering](), [AI Development](), [data](), [SQL](), [Slack]() Tags: [agents](), [ai](), [ai-agents](), [ai-engineering](), [data](), [evals](), [memory](), [openai](), [schema](), [sql](), [thread](), [verify]() ### AI overview The article examines how OpenAI built an internal data agent for more than 3,500 users working across 70,000 datasets and over 600 petabytes of data. It explains that trustworthy results require more than valid SQL: the agent needs relevant business context, safeguards for permissions, ways to detect subtle query errors, and transparency so users can inspect its work. ### Source excerpt How OpenAI built its data agent to work across 70,000 datasets. ## How to Build Your First Evaluation Set Before You Have Users DevFeed: [How to Build Your First Evaluation Set Before You Have Users]() Original publisher: [Read original article]() Author: Aakash Gupta Published: 2026-07-28T21:49:23Z Content type: tutorial Language: en Sources: [Product Growth]() Topics: [Artificial Intelligence](), [Claude](), [ChatGPT](), [llama](), [Meta]() Tags: [agents](), [ai](), [chatgpt](), [claude](), [evals](), [how-to]() ### AI overview A practical guide to building an AI evaluation set before a product has users. It explains how to create the set in Claude or ChatGPT, score it, interpret the results, and use evals to support development and career growth. ### Source excerpt He wrote evals for Gemini, for Llama, and for Ray-Ban Meta. Today he builds one from a blank spreadsheet, live! ## Eval-driven development: Lessons from evaluating GenAI at scale DevFeed: [Eval-driven development: Lessons from evaluating GenAI at scale]() Original publisher: [Read original article]() Author: Rohit Girme Published: 2026-07-28T17:01:03Z Content type: article Language: en Sources: [The Airbnb Tech Blog - Medium]() Topics: [Generative AI](), [Large Language Model](), [Software Engineering](), [Artificial Intelligence](), [Tooling]() Tags: [agentic-ai](), [ai](), [article](), [development](), [engineering](), [eval](), [evals](), [evaluation](), [genai](), [generation](), [generative-ai](), [llm](), [software](), [software-testing](), [testing](), [tooling](), [workflows]() ### AI overview This article presents eval-driven development as a core engineering discipline for trustworthy Generative AI products. It explains why evaluating LLM systems is difficult, including non-deterministic outputs, subjective correctness, AI-based evaluation risks, and failures across retrieval, reasoning, tool calls, and generation. It shares foundational evaluation practices and cautions that teams should plan evaluation early and ground success criteria in their data. ### Source excerpt How Airbnb teams build trustworthy Generative AI products by treating evaluation as a first-class engineering discipline; not an afterthought.Nestled into the lush hillside, this stunning modern retreat features striking natural wood architecture, terraced balconies, and a serene landscape. By: Rohit Girme, Dan Miller, Mia Zhao, Lifan Yang, Clint Kelly Introduction Generative AI breaks a lot of the assumptions that used to hold true for software testing. Unlike traditional software, LLM outputs are non-deterministic, and "correct" is subjective. Because so much judgment is involved, you often need an AI to evaluate an AI, which introduces its own potential failure modes. Making matters more complicated, a single interaction with an LLM can chain retrieval, reasoning, tool calls, and generation, each of which can fail independently. At Airbnb, we build LLM-powered features across our product, with recent launches including review highlights, AI customer support, smart communication features for guests and hosts, and more. Behind the scenes, we also use AI to help us spot trends and understand what's working, guiding where we improve the product next. Each product team may have its own evaluation criteria, process, workflows, etc. However, these are built on top of some common foundations and principles. An infrastructure team provides tooling and best practices, incorporating learnings across domains so that they are shared with everyone building products at Airbnb. In this article, we wanted to share some of these best practices and learnings with the broader engineering community. Please note that the recommendations here are not intended to be prescriptive; there is no one-size-fits all approach when it comes to running evals. 1. Foundation Evaluating LLM-based systems is challenging work, and this should be planned for at the outset. Without a deliberate strategy, three things tend to happen: False confidence: A generic "helpfulness" metric scores well, you ship, ## The Microsoft 365 Copilot Agent's Playbook: A Practical Livestream Series for Building Better Agents DevFeed: [The Microsoft 365 Copilot Agent's Playbook: A Practical Livestream Series for Building Better Agents]() Original publisher: [Read original article]() Author: Shavonna Jackson Published: 2026-07-23T19:03:53Z Content type: release Language: en Sources: [Developer Blogs]() Topics: [microsoft 365](), [Model Context Protocol](), [Tooling]() Tags: [agents](), [build](), [developer-tools](), [developers](), [evals](), [evaluation](), [extend](), [flow](), [microsoft-365-copilot](), [microsoft-for-developers]() ### AI overview Microsoft is launching a four-part livestream series about building, extending, grounding, and evaluating declarative agents for Microsoft 365 Copilot. The sessions include presentations, demos, and live Q&A for developers, makers, architects, and technical teams. ### Source excerpt Building on Microsoft 365 Copilot? Here's your playbook. Declarative agents are quickly becoming one of the most exciting ways to extend Microsoft 365 Copilot and bring organizational knowledge, workflows, and tools directly into the flow of work. But as agent capabilities grow, so does the need for practical guidance: How do you build agents that [...] The post The Microsoft 365 Copilot Agent's Playbook: A Practical Livestream Series for Building Better Agents appeared first on Microsoft for Developers. ## Harness AI Evals adds CI/CD quality gates for testing and monitoring AI agents DevFeed: [Harness AI Evals adds CI/CD quality gates for testing and monitoring AI agents]() Original publisher: [Read original article]() Author: Shibam Dhar Uri Scheiner Published: 2026-07-21T00:00:00Z Content type: release Language: en Sources: [Harness Blog]() Topics: [Artificial Intelligence](), [CI/CD]() Tags: [agents](), [ai](), [ai-agents](), [ai-evals](), [ci-cd](), [evals](), [monitor](), [production](), [quality](), [releases](), [test]() ### AI overview Harness introduces AI Evals, a tool for testing, scoring, and monitoring AI agents before and after deployment. It provides a native CI/CD pipeline step, evaluation metrics, and blocking or advisory pass strategies intended to prevent poor releases from reaching production. ### Source excerpt Harness AI Evals helps you test, score, and monitor AI agents with native CI/CD quality gates, blocking poor releases before production. | Blog ## How to test agent skills without hitting real APIs DevFeed: [How to test agent skills without hitting real APIs]() Original publisher: [Read original article]() Author: Waldek Mastykarz Published: 2026-07-17T09:27:50Z Content type: tutorial Language: en Sources: [Developer Blogs]() Topics: [Agent Skills](), [API](), [Testing]() Tags: [agent-experience](), [agent-skills](), [ai](), [apis](), [ax](), [evals](), [how-to](), [microsoft](), [testing]() ### AI overview This tutorial explains how to evaluate agent skills that call APIs without incurring external API costs or mutating live data. It introduces transparent API mocking to support isolated, repeatable evaluation runs without changing the skill or contacting real endpoints. ### Source excerpt Your agent skill calls an API. The moment you start evaluating it, every run either costs money or mutates production data. Learn how to mock APIs transparently so you can run evals without changing your skill or hitting real endpoints. The post How to test agent skills without hitting real APIs appeared first on Microsoft for Developers. ## Building AX evals that actually work DevFeed: [Building AX evals that actually work]() Original publisher: [Read original article]() Author: Waldek Mastykarz Published: 2026-07-15T12:53:13Z Content type: tutorial Language: en Sources: [Developer Blogs]() Topics: [AI-assisted coding](), [Loop Engineering](), [Prompt Engineering]() Tags: [agent-experience](), [ai](), [ai-coding-agents](), [article](), [ax](), [coding-agents](), [evals](), [evaluation](), [how-to](), [microsoft](), [microsoft-for-developers](), [quality]() ### AI overview This eighth and final article in a series about Agent Experience explains how to build meaningful evaluations for AI coding agents. It identifies representative prompts, accurate and unambiguous criteria, and other structural decisions needed to produce useful signal rather than misleading scores. ### Source excerpt This is the eighth and final article in a series about Agent Experience (AX): the practice of making AI coding agents work correctly with your technology. The series covers what you can and can't control in the agent stack, how to measure whether your extensions are helping or hurting, and how to iterate toward better [...] The post Building AX evals that actually work appeared first on Microsoft for Developers. ## GPT-5.6 is now available in Figma Make DevFeed: [GPT-5.6 is now available in Figma Make]() Original publisher: [Read original article]() Author: Gui Seiz Published: 2026-07-09T12:00:00Z Content type: article Language: en Sources: [Figma Blog]() Topics: [Figma](), [Code](), [App](), [dashboards](), [GUI](), [Playback](), [FIRST]() Tags: [app](), [audio](), [building](), [code](), [design](), [efficiency](), [evals](), [figma](), [gpt](), [music](), [openai](), [playback](), [prototypes](), [search](), [ui](), [visual-hierarchy](), [workflow]() ### AI overview This article announces OpenAI's GPT-5.6 integration in Figma Make, describing faster, higher-quality first results, efficient iteration, error self-healing, and faithful conversion of Figma designs into interactive prototypes and production code. ### Source excerpt With accurate first results and fast iterations, OpenAI's GPT-5.6 gives your builds a strong start in Figma Make. ## What an AI Agent Failure Revealed About Model Tradeoffs, Evals, and Traces DevFeed: [What an AI Agent Failure Revealed About Model Tradeoffs, Evals, and Traces]() Original publisher: [Read original article]() Author: Sergiy Dybskiy Published: 2026-07-01T09:00:00Z Content type: article Language: en Sources: [Sentry Blog]() Topics: [Traces](), [Artificial Intelligence](), [Prompt Engineering](), [vercel ai sdk](), [gateway]() Tags: [agent](), [ai](), [eval](), [evals](), [model](), [tool](), [tools](), [trace](), [traces]() ### AI overview An experiment with an AI-powered conference schedule builder showed that an agent could invent speakers, falsely claim to have retrieved them from an API, and appear grounded because a tool had run. The article discusses model tradeoffs, evaluation limitations, and the value of inspecting agent traces. ### Source excerpt I gave the free tier a cheaper model and it invented conference speakers who don't exist. What that taught me about model tradeoffs, evals, and reading agent traces. ## Design AI Products for Verification Before Building Evals DevFeed: [Design AI Products for Verification Before Building Evals]() Original publisher: [Read original article]() Author: Hamel Husain Published: 2026-06-29T07:00:00Z Content type: opinion Language: en Sources: [Hamel Husain]() Topics: [Artificial Intelligence](), [data](), [dashboards](), [SQL]() Tags: [ai](), [ai-evals](), [data-agents](), [evals](), [interface](), [llms](), [techniques](), [verification]() ### AI overview The article argues that products described as difficult to evaluate often make their outputs difficult for users to verify. Using AI data agents as an example, it recommends providing checkable artifacts--such as source comparisons, precise metric definitions, breakdowns, SQL, and uncertainty notes--before focusing on eval design. ### Source excerpt For the past 3 years, AI evals have been my professional focus.1 The most common objection I hear to evals is "our product is hard to eval". This objection is a product smell. Artifacts that are hard for you to verify are often hard for users too. In the worst case, users have to redo the work from scratch to verify the output. More importantly, designing your product for ease of verification should come before building evals. In this post, I'll walk through three products I advised on that faced this issue. I'll also show before and after sketches to demonstrate design principles. After these examples, I'll discuss how to apply this general pattern to your product. Example 1: the AI data agent Almost every company I've worked with builds an internal AI data agent. You ask it a business question, like what was net revenue for Product A last quarter, and it finds relevant data sources, runs the queries, and provides an answer. The goal of this agent is to reduce dependency on data analysts. A common mistake when building AI data agents is to make the answer the only output, as illustrated below. Data Agent What was net revenue for Product A last quarter? Net revenue for Product A last quarter was $4.21M. Ask anything about your business...➤ Since the only output is the answer, there is nothing here to check. In the sketch above, the user has no way to verify the answer beyond redoing work.2 A better design is to provide the user with checkable artifacts, informed by how a domain expert might validate the output. Here are techniques I use to validate metrics as a data scientist: Compare the quantity and any intermediate calculations against a trusted source, like a vetted dashboard or report, or a similar analysis a colleague has already vetted.3 Confirm the metric definition precisely. A number like net revenue can include or exclude things like returns and discounts. Sanity-check a related quantity. If I can't verify the number directly, I pull a related number that s ## Using Evaluation Frameworks with Agent Observability DevFeed: [Using Evaluation Frameworks with Agent Observability]() Original publisher: [Read original article]() Author: Jennifer Mickel; Eddie Cai Published: 2026-06-22T00:00:00Z Content type: article Language: en Sources: [Datadog | The Monitor blog]() Topics: [agent observability](), [observability](), [Large Language Model](), [Pydantic](), [experiments](), [Traces](), [Retrieval Augmented Generation (RAG)]() Tags: [agent-observability](), [ai-observability](), [code](), [deployment](), [development](), [evals](), [integration](), [llm](), [observability](), [traces]() ### AI overview This article explains how Datadog Agent Observability integrates existing DeepEval and Pydantic Evals frameworks. It covers running evaluations in experiments, connecting evaluation scores to production traces, and continuously monitoring LLM evaluation quality across development and deployment. ### Source excerpt Run DeepEval and Pydantic Evals natively in Datadog Agent Observability. Track regressions and connect eval scores to production traces. ## The Hidden Technical Debt Of Agentic Engineering DevFeed: [The Hidden Technical Debt Of Agentic Engineering]() Original publisher: [Read original article]() Author: Zohar Einy Published: 2026-06-20T18:56:57Z Content type: article Language: en Sources: [Developer Experience & Platform Engineering Blog | Port]() Topics: [AI Development](), [systems](), [Software](), [observability](), [CI/CD](), [Cloud]() Tags: [agentic](), [ai-agents](), [autonomous](), [ci](), [cloud](), [complexity](), [engineering](), [evals](), [governance](), [infrastructure](), [integrations](), [observability](), [registry](), [technical]() ### AI overview The article examines hidden technical debt in agentic engineering. It argues that building an AI agent is relatively easy, but deploying and operating agents in production introduces substantial infrastructure and maintenance complexity. It identifies surrounding concerns such as integrations, observability, governance, human-in-the-loop workflows, evaluations for non-deterministic systems, and agent registries. ### Source excerpt Uncover the hidden technical debt of agentic engineering and how autonomous systems introduce complexity and maintenance challenges. ## The Downsides of Agentic Skills DevFeed: [The Downsides of Agentic Skills]() Original publisher: [Read original article]() Author: Jorge Castillo Published: 2026-05-17T14:18:52Z Content type: opinion Language: en Sources: [Effective Android]() Topics: [context](), [Tech Debt](), [Latency](), [Testing]() Tags: [agentic](), [context](), [evals](), [latency](), [tech-debt](), [testing]() ### AI overview The article discusses tradeoffs of using agentic skills in production projects, including context costs, fuzzy triggering, nondeterministic behavior, harder testing, technical debt, and a growing permission surface. ### Source excerpt Skills are seductive. ## Wix compares AI agent performance using skills, standard documentation, and AI-optimized documentation DevFeed: [Wix compares AI agent performance using skills, standard documentation, and AI-optimized documentation]() Original publisher: [Read original article]() Author: Wix Engineering Published: 2026-05-06T11:16:26Z Content type: article Language: en Sources: [Wix Engineering]() Topics: [AI Agent](), [Documentation](), [Developer experience](), [Command-line interface]() Tags: [agents](), [ai-agent](), [developer-experience](), [docs](), [documentation](), [evals]() ### AI overview Wix describes 250 controlled evaluations comparing AI agents using standard documentation, AI-optimized documentation, and purpose-built skills. The article reports that stale skills can become a liability and questions whether manually maintained skills consistently outperform documentation. ### Source excerpt The industry has a new obsession: AI skills. The logic seems bulletproof: if you want an AI agent to use your platform, you shouldn't just give it raw documentation. You should give it a "skill", a curated, condensed, and optimized guide. This will allow the agent to perform tasks on your platform better than if they have to trawl through your docs. Skills are intuitive and trendy, but do they really provide agents with an edge over just using the docs and, if so, in what cases? At Wix, we... ## Using evals and user data to measure AI product improvements DevFeed: [Using evals and user data to measure AI product improvements]() Original publisher: [Read original article]() Author: hi@swizec.com (Swizec Teller) Published: 2026-05-01T00:00:00Z Content type: opinion Language: en Sources: [Swizec Teller]() Topics: [Artificial Intelligence](), [dataset](), [test]() Tags: [ai](), [dataset](), [evals](), [test]() ### AI overview The article argues that AI developers should use evaluations to measure whether changes improve a system. It recommends building test datasets from user behavior, testing models, prompts, and tools, avoiding overfitting, incorporating feedback from real-world use, and tracking human intervention in failure cases. ### Source excerpt When candidates put AI on their resume, the key thing I try to find out is whether they used evals. How did you measure making improvements? [Next page]()