# Can Jev Be a Better Agent Evaluator?

DevFeed: [Can Jev Be a Better Agent Evaluator?](<https://devfeed.tech/articles/can-jev-be-a-better-agent-evaluator-78315.md>)

Original publisher: [Read original article](<https://www.langchain.com/blog/jev-agent-evals-langsmith>)

Author: LangChain Accounts

Published: 2026-09-21T15:29:03Z

Content type: article

Language: en

Sources: [LangChain Blog](<https://devfeed.tech/sources/langchain-blog.md>)

Topics: [AI Context Switching](<https://devfeed.tech/topics/ai-context-switching.md>), [LLM observability](<https://devfeed.tech/topics/llm-observability.md>), [agent observability](<https://devfeed.tech/topics/agent-observability.md>)

Tags: [agent-evals](<https://devfeed.tech/tags/agent-evals.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [llm](<https://devfeed.tech/tags/llm.md>), [models](<https://devfeed.tech/tags/models.md>), [system-one-models](<https://devfeed.tech/tags/system-one-models.md>)

## AI overview

LangChain compares Jev, a System One decision model, with code-based evaluators and LLM judges for agent evaluation. In a test of five weather-agent runs repeated 100 times, Jev matched human labels on all 500 binary decisions and showed substantially lower score variance than the compared LLM judges. The authors also report lower latency and cost, while emphasizing that the results are early and may not generalize to other agents or production workflows.

## Source excerpt

We tested using Jev-as-a-Judge against LLM judges on accuracy, repeatability, latency, and cost to see whether System One models could offer a new approach to agent evaluation.