# data-agents Published articles for data-agents. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Design AI Products for Verification Before Building Evals DevFeed: [Design AI Products for Verification Before Building Evals]() Original publisher: [Read original article]() Author: Hamel Husain Published: 2026-06-29T07:00:00Z Content type: opinion Language: en Sources: [Hamel Husain]() Topics: [Artificial Intelligence](), [data](), [dashboards](), [SQL]() Tags: [ai](), [ai-evals](), [data-agents](), [evals](), [interface](), [llms](), [techniques](), [verification]() ### AI overview The article argues that products described as difficult to evaluate often make their outputs difficult for users to verify. Using AI data agents as an example, it recommends providing checkable artifacts--such as source comparisons, precise metric definitions, breakdowns, SQL, and uncertainty notes--before focusing on eval design. ### Source excerpt For the past 3 years, AI evals have been my professional focus.1 The most common objection I hear to evals is "our product is hard to eval". This objection is a product smell. Artifacts that are hard for you to verify are often hard for users too. In the worst case, users have to redo the work from scratch to verify the output. More importantly, designing your product for ease of verification should come before building evals. In this post, I'll walk through three products I advised on that faced this issue. I'll also show before and after sketches to demonstrate design principles. After these examples, I'll discuss how to apply this general pattern to your product. Example 1: the AI data agent Almost every company I've worked with builds an internal AI data agent. You ask it a business question, like what was net revenue for Product A last quarter, and it finds relevant data sources, runs the queries, and provides an answer. The goal of this agent is to reduce dependency on data analysts. A common mistake when building AI data agents is to make the answer the only output, as illustrated below. Data Agent What was net revenue for Product A last quarter? Net revenue for Product A last quarter was $4.21M. Ask anything about your business...➤ Since the only output is the answer, there is nothing here to check. In the sketch above, the user has no way to verify the answer beyond redoing work.2 A better design is to provide the user with checkable artifacts, informed by how a domain expert might validate the output. Here are techniques I use to validate metrics as a data scientist: Compare the quantity and any intermediate calculations against a trusted source, like a vetted dashboard or report, or a similar analysis a colleague has already vetted.3 Confirm the metric definition precisely. A number like net revenue can include or exclude things like returns and discounts. Sanity-check a related quantity. If I can't verify the number directly, I pull a related number that s ## Data Agent Benchmark for Multi-step Reasoning (DABstep) DevFeed: [Data Agent Benchmark for Multi-step Reasoning (DABstep)]() Original publisher: [Read original article]() Author: Adyen Published: 2025-02-05T09:19:27Z Content type: article Language: en Sources: [Adyen Tech]() Topics: [Benchmark](), [Data analysis](), [LLMs](), [AI Bots]() Tags: [adyen](), [agents](), [ai-models](), [benchmark](), [data](), [data-agents](), [data-analysis](), [data-science](), [genaimodels](), [hugging-face]() ### AI overview Adyen and Hugging Face introduce DABstep, a benchmark with more than 450 data analysis tasks for evaluating language models and AI agents on multi-step reasoning. The article reports that the strongest reasoning-based agents achieved 16% accuracy, indicating that complex real-world data analysis remains difficult for current models. ### Source excerpt Alex Egg (Adyen), Martin Iglesias (Adyen), Friso Kingma (Adyen), Andreu Mora (Adyen), Leandro Von Werra (HuggingFace), Thomas Wolf (HuggingFace) https://huggingface.co/spaces/adyen/DABstep 👉🏽 You can access DABStep with this link here: https://huggingface.co/spaces/adyen/DABstep Language models are becoming increasingly capable and can solve tasks autonomously as agents. There are many exciting use cases, especially at the intersection of reasoning, code, and data. However, proper evaluation benchmarks on real-world problems are lacking and hinder progress in the field. To tackle this challenge, Adyen and Hugging Face built the Data Agent Benchmark for Multi-step Reasoning (DABstep) together. DABstep consists of over 450 data analysis tasks designed to evaluate the capabilities of state-of-the-art LLMs and AI agents. Our findings reveal that DABstep presents a significant challenge for current AI models, with the most capable Reasoning-based agents achieving only 16% accuracy, highlighting significant progress to be made in the field. DABStep requires AI models to: dive in details of data and be rigorous (no hallucinations) reason over free form text and databases connect with real life use-cases (not just math or code) In this blog post, we'll cover the design and construction of the benchmark, explore evaluation results, and discuss the significant gap between current models and the ability to solve complex data analysis tasks effectively.. Motivation Data analysis is both an art and a science that requires technical skill, domain knowledge and creativity, and thus, it's rarely straightforward. Even seasoned data analysts face challenges like: Simple but time-consuming tasks: The sheer volume of even simple tasks often turns straightforward analysis into hours of repetitive work. Complex context and high cognitive load: Some tasks require analysis to juggle intricate domain-specific knowledge, making them both time-intensive and mentally draining. For example, (1)