# Using evals and user data to measure AI product improvements

DevFeed: [Using evals and user data to measure AI product improvements](<https://devfeed.tech/articles/quick-note-on-evals-and-putting-ai-in-your-resume-37636.md>)

Original publisher: [Read original article](<https://swizec.com/blog/quick-note-on-evals-and-putting-ai-in-your-resume>)

Author: hi@swizec.com (Swizec Teller)

Published: 2026-05-01T00:00:00Z

Content type: opinion

Language: en

Sources: [Swizec Teller](<https://devfeed.tech/sources/swizec-teller.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [dataset](<https://devfeed.tech/topics/dataset.md>), [test](<https://devfeed.tech/topics/test.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [evals](<https://devfeed.tech/tags/evals.md>), [test](<https://devfeed.tech/tags/test.md>)

## AI overview

The article argues that AI developers should use evaluations to measure whether changes improve a system. It recommends building test datasets from user behavior, testing models, prompts, and tools, avoiding overfitting, incorporating feedback from real-world use, and tracking human intervention in failure cases.

## Source excerpt

When candidates put AI on their resume, the key thing I try to find out is whether they used evals. How did you measure making improvements?