# AlignEval: Building an App for Data-Driven LLM Evaluator Development

DevFeed: [AlignEval: Building an App for Data-Driven LLM Evaluator Development](<https://devfeed.tech/articles/aligneval-building-an-app-to-make-evals-easy-fun-and-automated-46409.md>)

Original publisher: [Read original article](<https://eugeneyan.com//writing/aligneval/>)

Author: Eugene Yan

Published: 2024-10-27T00:00:00Z

Content type: article

Language: en

Sources: [Eugene Yan](<https://devfeed.tech/sources/eugene-yan.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Software Testing](<https://devfeed.tech/topics/software-testing.md>), [dataset](<https://devfeed.tech/topics/dataset.md>), [CSV](<https://devfeed.tech/topics/csv.md>), [Optimization](<https://devfeed.tech/topics/optimization.md>), [App](<https://devfeed.tech/topics/app.md>), [data](<https://devfeed.tech/topics/data.md>)

Tags: [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-evals](<https://devfeed.tech/tags/ai-evals.md>), [code](<https://devfeed.tech/tags/code.md>), [csv](<https://devfeed.tech/tags/csv.md>), [data](<https://devfeed.tech/tags/data.md>), [eval](<https://devfeed.tech/tags/eval.md>), [learning](<https://devfeed.tech/tags/learning.md>), [llm](<https://devfeed.tech/tags/llm.md>), [optimize](<https://devfeed.tech/tags/optimize.md>), [product](<https://devfeed.tech/tags/product.md>), [testing](<https://devfeed.tech/tags/testing.md>)

## AI overview

The article introduces AlignEval, an app for building and automating evaluators for LLM-powered products. It argues that effective evaluations should begin with observed data and defects, then use human criteria and metrics to assess and improve evaluator performance.

## Source excerpt

Look at and label your data, build and evaluate your LLM-evaluator, and optimize it against your labels.