# Eugene Yan

Eugene Yan works at the intersection of consumer data & tech to build machine learning products, and writes about effective data science, learning & career.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## AI Engineer 2026 - Using LLMs to Secure Source Code

DevFeed: [AI Engineer 2026 - Using LLMs to Secure Source Code](<https://devfeed.tech/articles/ai-engineer-2026-using-llms-to-secure-source-code-134126.md>)

Original publisher: [Read original article](<https://eugeneyan.com/speaking/aie-2026>)

Author: Eugene Yan

Published: 2026-06-30T00:00:00Z

Content type: article

Language: en

Sources: [Eugene Yan](<https://devfeed.tech/sources/eugene-yan.md>)

Topics: [code security](<https://devfeed.tech/topics/code-security.md>), [ai code review](<https://devfeed.tech/topics/ai-code-review.md>), [snyk-platform](<https://devfeed.tech/topics/snyk-platform.md>), [Claude Code](<https://devfeed.tech/topics/claude-code.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-engineer](<https://devfeed.tech/tags/ai-engineer.md>), [ai-security](<https://devfeed.tech/tags/ai-security.md>), [bottlenecks](<https://devfeed.tech/tags/bottlenecks.md>), [building](<https://devfeed.tech/tags/building.md>), [claude](<https://devfeed.tech/tags/claude.md>), [code-security](<https://devfeed.tech/tags/code-security.md>), [field](<https://devfeed.tech/tags/field.md>), [github-action](<https://devfeed.tech/tags/github-action.md>), [glasswing](<https://devfeed.tech/tags/glasswing.md>), [harness](<https://devfeed.tech/tags/harness.md>), [interactive](<https://devfeed.tech/tags/interactive.md>), [llms](<https://devfeed.tech/tags/llms.md>), [org](<https://devfeed.tech/tags/org.md>), [patches](<https://devfeed.tech/tags/patches.md>), [production](<https://devfeed.tech/tags/production.md>), [source](<https://devfeed.tech/tags/source.md>), [source-code](<https://devfeed.tech/tags/source-code.md>)

### AI overview

The talk covers using LLMs to secure source code, including why the approach is timely, six steps for building an agentic security harness, and organizational bottlenecks that can emerge when vulnerability discovery becomes easier. The excerpt points to related resources: Claude Security for scanning code, validating findings, and suggesting patches; an interactive reference harness; a GitHub Action for pull request security review; and a write-up of production field lessons. It does not provide the six steps or elaborate on the lessons.

### Source excerpt

Sharing what we learned from Project Glasswing.

## Patterns for Building Cybersecurity Evals

DevFeed: [Patterns for Building Cybersecurity Evals](<https://devfeed.tech/articles/patterns-for-building-cybersecurity-evals-46423.md>)

Original publisher: [Read original article](<https://eugeneyan.com//writing/cybersecurity-evals/>)

Author: Eugene Yan

Published: 2026-06-21T00:00:00Z

Content type: article

Language: en

Sources: [Eugene Yan](<https://devfeed.tech/sources/eugene-yan.md>)

Topics: [Cybersecurity](<https://devfeed.tech/topics/cybersecurity.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Security Attacks](<https://devfeed.tech/topics/security-attacks.md>), [Vulnerabilities](<https://devfeed.tech/topics/vulnerabilities.md>), [Exploit](<https://devfeed.tech/topics/exploit.md>), [Hacking](<https://devfeed.tech/topics/hacking.md>), [Docker](<https://devfeed.tech/topics/docker.md>), [c/c++](<https://devfeed.tech/topics/c-c-plus-plus.md>), [Bash](<https://devfeed.tech/topics/bash.md>)

Tags: [bash](<https://devfeed.tech/tags/bash.md>), [building](<https://devfeed.tech/tags/building.md>), [c](<https://devfeed.tech/tags/c.md>), [c-plus-plus](<https://devfeed.tech/tags/c-plus-plus.md>), [code](<https://devfeed.tech/tags/code.md>), [cybersecurity](<https://devfeed.tech/tags/cybersecurity.md>), [docker](<https://devfeed.tech/tags/docker.md>), [eval](<https://devfeed.tech/tags/eval.md>), [evals](<https://devfeed.tech/tags/evals.md>), [exploit](<https://devfeed.tech/tags/exploit.md>), [learning](<https://devfeed.tech/tags/learning.md>), [tools](<https://devfeed.tech/tags/tools.md>)

### AI overview

The article describes how to evaluate models and agents on cybersecurity tasks, including capture-the-flag exercises, data exfiltration, and vulnerability exploitation. It outlines a benchmark pattern built around sandboxed targets, difficulty-adjusting inputs, tools, and deterministic graders, with partial credit and transcript audits for more granular evaluation.

### Source excerpt

A sandboxed target, inputs that influence task difficulty, tools, and a grader.

## Using LLMs to Secure Source Code

DevFeed: [Using LLMs to Secure Source Code](<https://devfeed.tech/articles/using-llms-to-secure-source-code-46546.md>)

Original publisher: [Read original article](<https://eugeneyan.com//writing/secure-source-code/>)

Published: 2026-05-27T00:00:00Z

Content type: tutorial

Language: en

Sources: [Eugene Yan](<https://devfeed.tech/sources/eugene-yan.md>)

Topics: [code security](<https://devfeed.tech/topics/code-security.md>), [Vulnerabilities](<https://devfeed.tech/topics/vulnerabilities.md>), [Static code analysis](<https://devfeed.tech/topics/static-code-analysis.md>), [Test-driven development](<https://devfeed.tech/topics/tdd.md>), [alert triage](<https://devfeed.tech/topics/alert-triage.md>)

Tags: [2](<https://devfeed.tech/tags/2.md>), [2026](<https://devfeed.tech/tags/2026.md>), [22](<https://devfeed.tech/tags/22.md>), [3](<https://devfeed.tech/tags/3.md>), [about](<https://devfeed.tech/tags/about.md>), [admin](<https://devfeed.tech/tags/admin.md>), [adversarial](<https://devfeed.tech/tags/adversarial.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [build](<https://devfeed.tech/tags/build.md>), [code](<https://devfeed.tech/tags/code.md>), [llms](<https://devfeed.tech/tags/llms.md>), [model](<https://devfeed.tech/tags/model.md>), [patch](<https://devfeed.tech/tags/patch.md>), [source](<https://devfeed.tech/tags/source.md>), [source-code](<https://devfeed.tech/tags/source-code.md>), [triage](<https://devfeed.tech/tags/triage.md>), [using](<https://devfeed.tech/tags/using.md>), [verify](<https://devfeed.tech/tags/verify.md>), [vulnerabilities](<https://devfeed.tech/tags/vulnerabilities.md>)

### AI overview

The guide's main finding is that vulnerability discovery with LLMs is increasingly easy to parallelize, shifting the workload to verification, triage, and patching. It recommends a six-step cycle: define a threat model, create an isolated sandbox, scan code, independently verify exploitability, deduplicate and prioritize findings, then patch and re-scan. Threat models clarify trust boundaries and severity criteria; sandboxes can test proof-of-concept exploits but should reflect production defenses and dependencies. If a representative sandbox is impractical, code-only discovery can still work, with more verification effort. Keep discovery focused on recall and use a fresh verifier to challenge findings, since failed PoC attempts do not prove a report false. Triage by root cause, reachability, attacker control, preconditions, authentication, and impact. Before shipping fixes, add regression tests, check for variants, and have an independent agent probe the patch; human review remains necessary.

### Source excerpt

Build a threat model, discover vulnerabilities, verify, triage, and patch.

## How to Work and Compound with AI

DevFeed: [How to Work and Compound with AI](<https://devfeed.tech/articles/how-to-work-and-compound-with-ai-46574.md>)

Original publisher: [Read original article](<https://eugeneyan.com//writing/working-with-ai/>)

Author: Eugene Yan

Published: 2026-05-03T00:00:00Z

Content type: article

Language: en

Sources: [Eugene Yan](<https://devfeed.tech/sources/eugene-yan.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [context](<https://devfeed.tech/topics/context.md>), [Claude Code](<https://devfeed.tech/topics/claude-code.md>), [Documentation](<https://devfeed.tech/topics/documentation.md>), [Code](<https://devfeed.tech/topics/code.md>), [systems](<https://devfeed.tech/topics/systems.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [autonomy](<https://devfeed.tech/tags/autonomy.md>), [claude](<https://devfeed.tech/tags/claude.md>), [claude-code](<https://devfeed.tech/tags/claude-code.md>), [code](<https://devfeed.tech/tags/code.md>), [config](<https://devfeed.tech/tags/config.md>), [context](<https://devfeed.tech/tags/context.md>), [delegation](<https://devfeed.tech/tags/delegation.md>), [docs](<https://devfeed.tech/tags/docs.md>), [how-to](<https://devfeed.tech/tags/how-to.md>), [infra](<https://devfeed.tech/tags/infra.md>), [mechanism](<https://devfeed.tech/tags/mechanism.md>), [memory](<https://devfeed.tech/tags/memory.md>), [model](<https://devfeed.tech/tags/model.md>), [onboarding](<https://devfeed.tech/tags/onboarding.md>), [organization](<https://devfeed.tech/tags/organization.md>), [practices](<https://devfeed.tech/tags/practices.md>), [productivity](<https://devfeed.tech/tags/productivity.md>), [verification](<https://devfeed.tech/tags/verification.md>), [work](<https://devfeed.tech/tags/work.md>)

### AI overview

An article presents a workflow for working effectively with AI by providing context, encoding preferences in configuration, making verification easy, delegating larger tasks, and continuously improving the system. It discusses organized project files, organizational knowledge sources, annotated indexes, onboarding each session, and building a memory layer.

### Source excerpt

Context as infra, taste as config, verification for autonomy, scale via delegation, closing the loop.

## 2025 Year in Review

DevFeed: [2025 Year in Review](<https://devfeed.tech/articles/2025-year-in-review-46404.md>)

Original publisher: [Read original article](<https://eugeneyan.com//writing/2025-review/>)

Author: Eugene Yan

Published: 2025-12-14T00:00:00Z

Content type: opinion

Language: en

Sources: [Eugene Yan](<https://devfeed.tech/sources/eugene-yan.md>)

Topics: [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [App](<https://devfeed.tech/topics/app.md>), [recommendation systems](<https://devfeed.tech/topics/recommendation-systems.md>), [Chat Bot](<https://devfeed.tech/topics/chatbot.md>)

Tags: [2025](<https://devfeed.tech/tags/2025.md>), [ai](<https://devfeed.tech/tags/ai.md>), [evals](<https://devfeed.tech/tags/evals.md>), [leadership](<https://devfeed.tech/tags/leadership.md>), [led](<https://devfeed.tech/tags/led.md>), [life](<https://devfeed.tech/tags/life.md>), [prototype](<https://devfeed.tech/tags/prototype.md>), [year-in-review](<https://devfeed.tech/tags/year-in-review.md>)

### AI overview

A personal review of 2025 covering technical writing on recommender systems, LLMs, evaluations, leadership, and writing advice, along with several software prototypes and personal goals.

### Source excerpt

An eventful year of progress in health and career, while making time for travel and reflection.

## Product Evals in Three Simple Steps

DevFeed: [Product Evals in Three Simple Steps](<https://devfeed.tech/articles/product-evals-in-three-simple-steps-46524.md>)

Original publisher: [Read original article](<https://eugeneyan.com//writing/product-evals/>)

Author: Eugene Yan

Published: 2025-11-23T00:00:00Z

Content type: tutorial

Language: en

Sources: [Eugene Yan](<https://devfeed.tech/sources/eugene-yan.md>)

Topics: [Large Language Model](<https://devfeed.tech/topics/llm.md>), [data](<https://devfeed.tech/topics/data.md>), [dataset](<https://devfeed.tech/topics/dataset.md>)

Tags: [dataset](<https://devfeed.tech/tags/dataset.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [eval](<https://devfeed.tech/tags/eval.md>), [evals](<https://devfeed.tech/tags/evals.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [harness](<https://devfeed.tech/tags/harness.md>), [labeling](<https://devfeed.tech/tags/labeling.md>), [llm](<https://devfeed.tech/tags/llm.md>), [pass](<https://devfeed.tech/tags/pass.md>), [product](<https://devfeed.tech/tags/product.md>), [production](<https://devfeed.tech/tags/production.md>)

### AI overview

A practical guide to product evaluations for LLM applications. It recommends labeling a small dataset, aligning human and LLM evaluators, and running an evaluation harness after configuration changes, with binary pass/fail labels preferred for objective criteria.

### Source excerpt

Label some data, align LLM-evaluators, and run the eval harness with each change.

## Advice for New Principal Tech ICs (i.e., Notes to Myself)

DevFeed: [Advice for New Principal Tech ICs (i.e., Notes to Myself)](<https://devfeed.tech/articles/advice-for-new-principal-tech-ics-i-e-notes-to-myself-46520.md>)

Original publisher: [Read original article](<https://eugeneyan.com//writing/principal/>)

Author: Eugene Yan

Published: 2025-10-19T00:00:00Z

Content type: opinion

Language: en

Sources: [Eugene Yan](<https://devfeed.tech/sources/eugene-yan.md>)

Topics: [Job](<https://devfeed.tech/topics/job.md>), [Programming](<https://devfeed.tech/topics/programming.md>), [Repository](<https://devfeed.tech/topics/repository.md>), [context](<https://devfeed.tech/topics/context.md>)

Tags: [advice](<https://devfeed.tech/tags/advice.md>), [career](<https://devfeed.tech/tags/career.md>), [coding](<https://devfeed.tech/tags/coding.md>), [communication](<https://devfeed.tech/tags/communication.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [leadership](<https://devfeed.tech/tags/leadership.md>), [principal-engineer](<https://devfeed.tech/tags/principal-engineer.md>), [product](<https://devfeed.tech/tags/product.md>), [repository](<https://devfeed.tech/tags/repository.md>)

### AI overview

The article distills observations and advice about effective principal engineers and scientists, emphasizing different leadership strengths, continued hands-on technical work, broader influence, technical vision, communication, and cross-functional responsibility.

### Source excerpt

Based on what I've learned from role models and mentors in Amazon

## Training an LLM-RecSys Hybrid for Steerable Recs with Semantic IDs

DevFeed: [Training an LLM-RecSys Hybrid for Steerable Recs with Semantic IDs](<https://devfeed.tech/articles/training-an-llm-recsys-hybrid-for-steerable-recs-with-semantic-ids-46547.md>)

Original publisher: [Read original article](<https://eugeneyan.com//writing/semantic-ids/>)

Author: Eugene Yan

Published: 2025-09-14T00:00:00Z

Content type: article

Language: en

Sources: [Eugene Yan](<https://devfeed.tech/sources/eugene-yan.md>)

Topics: [Large Language Model](<https://devfeed.tech/topics/llm.md>), [recommendation systems](<https://devfeed.tech/topics/recommendation-systems.md>), [Code](<https://devfeed.tech/topics/code.md>), [Prompt Engineering](<https://devfeed.tech/topics/prompt-engineering.md>), [dataset](<https://devfeed.tech/topics/dataset.md>), [text-generation](<https://devfeed.tech/topics/text-generation.md>)

Tags: [code](<https://devfeed.tech/tags/code.md>), [data](<https://devfeed.tech/tags/data.md>), [learning](<https://devfeed.tech/tags/learning.md>), [llm](<https://devfeed.tech/tags/llm.md>), [model](<https://devfeed.tech/tags/model.md>), [prompting](<https://devfeed.tech/tags/prompting.md>), [recsys](<https://devfeed.tech/tags/recsys.md>), [retrieval](<https://devfeed.tech/tags/retrieval.md>), [semantic](<https://devfeed.tech/tags/semantic.md>)

### AI overview

The article describes training an LLM-recommender hybrid that uses semantically meaningful item ID tokens to represent a catalog. The model learns from catalog relationships and user-behavior sequences, enabling recommendations that can be steered through natural-language conversation and accompanied by explanations.

### Source excerpt

An LLM that can converse in English & item IDs, and make recommendations w/o retrieval or tools.

## Evaluating Long-Context Question & Answer Systems

DevFeed: [Evaluating Long-Context Question & Answer Systems](<https://devfeed.tech/articles/evaluating-long-context-question-answer-systems-46530.md>)

Original publisher: [Read original article](<https://eugeneyan.com//writing/qa-evals/>)

Author: Eugene Yan

Published: 2025-06-22T00:00:00Z

Content type: article

Language: en

Sources: [Eugene Yan](<https://devfeed.tech/sources/eugene-yan.md>)

Topics: [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [long-context](<https://devfeed.tech/topics/long-context.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [dataset](<https://devfeed.tech/topics/dataset.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [eval](<https://devfeed.tech/tags/eval.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [llm](<https://devfeed.tech/tags/llm.md>), [long-context](<https://devfeed.tech/tags/long-context.md>), [survey](<https://devfeed.tech/tags/survey.md>)

### AI overview

This article explains how to evaluate long-context question-and-answer systems. It discusses evaluation metrics, evaluation datasets, human annotations, LLM evaluators, and benchmarks across narrative, technical, academic, and multi-document contexts.

### Source excerpt

Evaluation metrics, how to build eval datasets, eval methodology, and a review of several benchmarks.

## AI Engineer 2025 - Improving RecSys & Search with LLM techniques

DevFeed: [AI Engineer 2025 - Improving RecSys & Search with LLM techniques](<https://devfeed.tech/articles/ai-engineer-2025-improving-recsys-search-with-llm-techniques-46369.md>)

Original publisher: [Read original article](<https://eugeneyan.com//speaking/aie-2025/>)

Author: Eugene Yan

Published: 2025-06-04T00:00:00Z

Content type: article

Language: en

Sources: [Eugene Yan](<https://devfeed.tech/sources/eugene-yan.md>)

Topics: [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [recommendation systems](<https://devfeed.tech/topics/recommendation-systems.md>), [dataset](<https://devfeed.tech/topics/dataset.md>), [synthetic-data](<https://devfeed.tech/topics/synthetic-data.md>), [AI Foundation Models](<https://devfeed.tech/topics/ai-foundation-models.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [data](<https://devfeed.tech/tags/data.md>), [data-augmentation](<https://devfeed.tech/tags/data-augmentation.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [foundation-models](<https://devfeed.tech/tags/foundation-models.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-techniques](<https://devfeed.tech/tags/llm-techniques.md>), [llms](<https://devfeed.tech/tags/llms.md>), [production](<https://devfeed.tech/tags/production.md>), [recsys](<https://devfeed.tech/tags/recsys.md>), [search](<https://devfeed.tech/tags/search.md>), [semantic](<https://devfeed.tech/tags/semantic.md>)

### AI overview

A write-up from AI Engineer World's Fair 2025 about the convergence of recommender systems and search with large language models, including semantic IDs, data augmentation, and unified foundation models.

### Source excerpt

Recsys & search are converging with LLMs via semantic IDs, data augmentation, and unified foundation models.

## Exceptional Leadership: Some Qualities, Behaviors, and Styles

DevFeed: [Exceptional Leadership: Some Qualities, Behaviors, and Styles](<https://devfeed.tech/articles/exceptional-leadership-some-qualities-behaviors-and-styles-46470.md>)

Original publisher: [Read original article](<https://eugeneyan.com//writing/leadership/>)

Author: Eugene Yan

Published: 2025-05-18T00:00:00Z

Content type: opinion

Language: en

Sources: [Eugene Yan](<https://devfeed.tech/sources/eugene-yan.md>)

Topics: [execution](<https://devfeed.tech/topics/execution.md>), [context](<https://devfeed.tech/topics/context.md>), [intel](<https://devfeed.tech/topics/intel.md>)

Tags: [coaching](<https://devfeed.tech/tags/coaching.md>), [execution](<https://devfeed.tech/tags/execution.md>), [leadership](<https://devfeed.tech/tags/leadership.md>), [organizational](<https://devfeed.tech/tags/organizational.md>), [strategy](<https://devfeed.tech/tags/strategy.md>), [team](<https://devfeed.tech/tags/team.md>), [vision](<https://devfeed.tech/tags/vision.md>)

### AI overview

The article describes qualities of exceptional leaders, focusing on vision, execution, empathy, support, and removing obstacles. It connects leadership with team growth, customer focus, organizational coordination, and navigating uncertainty.

### Source excerpt

What makes a good leader? What do good leaders do? And commando, soldier, and police leadership.

## Building News Agents for Daily News Recaps with MCP, Q, and tmux

DevFeed: [Building News Agents for Daily News Recaps with MCP, Q, and tmux](<https://devfeed.tech/articles/building-news-agents-for-daily-news-recaps-with-mcp-q-and-tmux-46495.md>)

Original publisher: [Read original article](<https://eugeneyan.com//writing/news-agents/>)

Author: Eugene Yan

Published: 2025-05-04T00:00:00Z

Content type: tutorial

Language: en

Sources: [Eugene Yan](<https://devfeed.tech/sources/eugene-yan.md>)

Topics: [Model Context Protocol](<https://devfeed.tech/topics/model-context-protocol.md>), [agentic workflows](<https://devfeed.tech/topics/agentic-workflows.md>), [Multi Agent Systems](<https://devfeed.tech/topics/multi-agent-systems.md>), [Command-line interface](<https://devfeed.tech/topics/cli.md>), [RSS](<https://devfeed.tech/topics/rss.md>), [Parsing](<https://devfeed.tech/topics/parsing.md>), [Tool](<https://devfeed.tech/topics/tool.md>), [amazon](<https://devfeed.tech/topics/amazon.md>), [anthropic](<https://devfeed.tech/topics/anthropic.md>)

Tags: [agentic-workflows](<https://devfeed.tech/tags/agentic-workflows.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [amazon](<https://devfeed.tech/tags/amazon.md>), [anthropic](<https://devfeed.tech/tags/anthropic.md>), [cli](<https://devfeed.tech/tags/cli.md>), [code](<https://devfeed.tech/tags/code.md>), [demo](<https://devfeed.tech/tags/demo.md>), [learning](<https://devfeed.tech/tags/learning.md>), [llm](<https://devfeed.tech/tags/llm.md>), [mcp](<https://devfeed.tech/tags/mcp.md>), [mcp-server](<https://devfeed.tech/tags/mcp-server.md>), [multi-agent](<https://devfeed.tech/tags/multi-agent.md>), [parsing](<https://devfeed.tech/tags/parsing.md>), [rss](<https://devfeed.tech/tags/rss.md>), [server](<https://devfeed.tech/tags/server.md>), [tmux](<https://devfeed.tech/tags/tmux.md>)

### AI overview

A tutorial on building news-agents, a multi-agent workflow that uses Amazon Q CLI, MCP tools, RSS feed parsers, tmux, and separate sub-agents to collect news and combine feed summaries into a daily recap.

### Source excerpt

Learning to automate simple agentic workflows with Amazon Q CLI, Anthropic MCP, and tmux.

## An LLM-as-Judge Won't Save The Product--Fixing Your Process Will

DevFeed: [An LLM-as-Judge Won't Save The Product--Fixing Your Process Will](<https://devfeed.tech/articles/an-llm-as-judge-won-t-save-the-product-fixing-your-process-will-46433.md>)

Original publisher: [Read original article](<https://eugeneyan.com//writing/eval-process/>)

Author: Eugene Yan

Published: 2025-04-20T00:00:00Z

Content type: opinion

Language: en

Sources: [Eugene Yan](<https://devfeed.tech/sources/eugene-yan.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [software-development](<https://devfeed.tech/topics/software-development.md>), [Prompt Engineering](<https://devfeed.tech/topics/prompt-engineering.md>), [Retrieval Augmented Generation (RAG)](<https://devfeed.tech/topics/retrieval-augmented-generation-rag.md>), [dataset](<https://devfeed.tech/topics/dataset.md>), [experiments](<https://devfeed.tech/topics/experiments.md>), [Monitoring](<https://devfeed.tech/topics/monitoring.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [development](<https://devfeed.tech/tags/development.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [eval](<https://devfeed.tech/tags/eval.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [llm](<https://devfeed.tech/tags/llm.md>), [metric](<https://devfeed.tech/tags/metric.md>), [model](<https://devfeed.tech/tags/model.md>), [monitoring](<https://devfeed.tech/tags/monitoring.md>), [rag](<https://devfeed.tech/tags/rag.md>)

### AI overview

The article argues that effective product evaluations for AI systems require a repeatable process based on the scientific method, eval-driven development, and monitoring AI outputs. It recommends examining data and user interactions, annotating representative successes and failures, forming hypotheses, running controlled experiments, measuring outcomes, and iterating based on error analysis.

### Source excerpt

Applying the scientific method, building via eval-driven development, and monitoring AI output.

## Frequently Asked Questions about My Writing Process

DevFeed: [Frequently Asked Questions about My Writing Process](<https://devfeed.tech/articles/frequently-asked-questions-about-my-writing-process-46577.md>)

Original publisher: [Read original article](<https://eugeneyan.com//writing/writing-faq/>)

Author: Eugene Yan

Published: 2025-03-30T00:00:00Z

Content type: article

Language: en

Sources: [Eugene Yan](<https://devfeed.tech/sources/eugene-yan.md>)

Topics: [Data Science](<https://devfeed.tech/topics/data-science.md>), [Requirements](<https://devfeed.tech/topics/requirements.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [career-growth](<https://devfeed.tech/tags/career-growth.md>), [communication](<https://devfeed.tech/tags/communication.md>), [data-science](<https://devfeed.tech/tags/data-science.md>), [technical](<https://devfeed.tech/tags/technical.md>), [writing](<https://devfeed.tech/tags/writing.md>)

### AI overview

An FAQ explains how the author began writing, why writing supports learning and career growth, and how communication helps data scientists understand stakeholders, explain machine learning, write clear requirements, gain buy-in, and earn trust.

### Source excerpt

How I started, why I write, who I write for, how I write, and more.

## NVIDIA GTC 2025 - Building LLM-Powered Applications

DevFeed: [NVIDIA GTC 2025 - Building LLM-Powered Applications](<https://devfeed.tech/articles/nvidia-gtc-2025-building-llm-powered-applications-46388.md>)

Original publisher: [Read original article](<https://eugeneyan.com//speaking/nvidia-gtc-2025/>)

Author: Eugene Yan

Published: 2025-03-18T00:00:00Z

Content type: article

Language: en

Sources: [Eugene Yan](<https://devfeed.tech/sources/eugene-yan.md>)

Topics: [AI Development](<https://devfeed.tech/topics/ai-development.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Nvidia](<https://devfeed.tech/topics/nvidia.md>)

Tags: [2025](<https://devfeed.tech/tags/2025.md>), [applications](<https://devfeed.tech/tags/applications.md>), [best-practices](<https://devfeed.tech/tags/best-practices.md>), [building](<https://devfeed.tech/tags/building.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [insights](<https://devfeed.tech/tags/insights.md>), [llm](<https://devfeed.tech/tags/llm.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [production](<https://devfeed.tech/tags/production.md>)

### AI overview

A write-up of a panel at NVIDIA GTC 2025 where Chip Huyen and Ziyou Yan shared insights, lessons learned, and best practices for building applications powered by large language models.

### Source excerpt

Chip Huyen and I share what we've learned, best practices, and insights at NVIDIA GTC 2025.

## Improving Recommendation Systems & Search in the Age of LLMs

DevFeed: [Improving Recommendation Systems & Search in the Age of LLMs](<https://devfeed.tech/articles/improving-recommendation-systems-search-in-the-age-of-llms-46536.md>)

Original publisher: [Read original article](<https://eugeneyan.com//writing/recsys-llm/>)

Author: Eugene Yan

Published: 2025-03-16T00:00:00Z

Content type: article

Language: en

Sources: [Eugene Yan](<https://devfeed.tech/sources/eugene-yan.md>)

Topics: [recommendation systems](<https://devfeed.tech/topics/recommendation-systems.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [Transformer](<https://devfeed.tech/topics/transformer.md>), [VAE](<https://devfeed.tech/topics/vae.md>), [quantization](<https://devfeed.tech/topics/quantization.md>), [multimodal-ai](<https://devfeed.tech/topics/multimodal-ai.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>)

Tags: [architectures](<https://devfeed.tech/tags/architectures.md>), [data](<https://devfeed.tech/tags/data.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llms](<https://devfeed.tech/tags/llms.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [quantization](<https://devfeed.tech/tags/quantization.md>), [ranking](<https://devfeed.tech/tags/ranking.md>), [recommendation-systems](<https://devfeed.tech/tags/recommendation-systems.md>), [recsys](<https://devfeed.tech/tags/recsys.md>), [retrieval](<https://devfeed.tech/tags/retrieval.md>), [search](<https://devfeed.tech/tags/search.md>), [teardown](<https://devfeed.tech/tags/teardown.md>), [training](<https://devfeed.tech/tags/training.md>), [transformer](<https://devfeed.tech/tags/transformer.md>)

### AI overview

This article examines how industrial search and recommendation systems are adopting language-model and multimodal techniques. It discusses hybrid architectures, semantic IDs, content embeddings, RQ-VAE compression, and ranking models aimed at improving cold-start and long-tail recommendations while increasing efficiency.

### Source excerpt

Model architectures, data generation, training paradigms, and unified frameworks inspired by LLMs.

## Building AI Reading Club: Features & Behind the Scenes

DevFeed: [Building AI Reading Club: Features & Behind the Scenes](<https://devfeed.tech/articles/building-ai-reading-club-features-behind-the-scenes-46408.md>)

Original publisher: [Read original article](<https://eugeneyan.com//writing/aireadingclub/>)

Author: Eugene Yan

Published: 2025-01-12T00:00:00Z

Content type: article

Language: en

Sources: [Eugene Yan](<https://devfeed.tech/sources/eugene-yan.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [context](<https://devfeed.tech/topics/context.md>), [User Interfaces](<https://devfeed.tech/topics/user-interfaces.md>), [test](<https://devfeed.tech/topics/test.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [behind-the-scenes](<https://devfeed.tech/tags/behind-the-scenes.md>), [building](<https://devfeed.tech/tags/building.md>), [design](<https://devfeed.tech/tags/design.md>), [features](<https://devfeed.tech/tags/features.md>), [learning](<https://devfeed.tech/tags/learning.md>), [llm](<https://devfeed.tech/tags/llm.md>), [prototype](<https://devfeed.tech/tags/prototype.md>), [ui](<https://devfeed.tech/tags/ui.md>)

### AI overview

This write-up describes AI Reading Club, a prototype built to explore AI-assisted reading. Its companion Dewey provides contextual explanations, book recaps, lookups, quizzes, and discussion history through an intuitive reading interface.

### Source excerpt

Exploring how an AI-powered reading experience could look like.

## 2024 Year in Review

DevFeed: [2024 Year in Review](<https://devfeed.tech/articles/2024-year-in-review-46403.md>)

Original publisher: [Read original article](<https://eugeneyan.com//writing/2024-review/>)

Author: Eugene Yan

Published: 2024-12-22T00:00:00Z

Content type: article

Language: en

Sources: [Eugene Yan](<https://devfeed.tech/sources/eugene-yan.md>)

Topics: [AI Development](<https://devfeed.tech/topics/ai-development.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Deployment](<https://devfeed.tech/topics/deployment.md>), [synthetic-data](<https://devfeed.tech/topics/synthetic-data.md>), [Framework](<https://devfeed.tech/topics/framework.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [deploy](<https://devfeed.tech/tags/deploy.md>), [life](<https://devfeed.tech/tags/life.md>), [llms](<https://devfeed.tech/tags/llms.md>), [production](<https://devfeed.tech/tags/production.md>), [synthetic-data](<https://devfeed.tech/tags/synthetic-data.md>), [year-in-review](<https://devfeed.tech/tags/year-in-review.md>)

### AI overview

A 2024 personal and professional review covering the productionization of ML and LLM systems, reliable deployment at scale, writing and speaking activities, learning about synthetic data and application-specific evaluations, and personal health, travel, and snowboarding goals.

### Source excerpt

A peaceful year of steady progress on my craft and health.

## A Spark of the Anti-AI Backlash (on Bluesky)

DevFeed: [A Spark of the Anti-AI Backlash (on Bluesky)](<https://devfeed.tech/articles/a-spark-of-the-anti-ai-backlash-on-bluesky-134127.md>)

Original publisher: [Read original article](<https://eugeneyan.com/writing/anti>)

Author: Eugene Yan

Published: 2024-12-08T00:00:00Z

Content type: article

Language: en

Sources: [Eugene Yan](<https://devfeed.tech/sources/eugene-yan.md>)

Topics: [Bluesky](<https://devfeed.tech/topics/bluesky-social.md>), [hugging face](<https://devfeed.tech/topics/hugging-face.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [human review](<https://devfeed.tech/topics/human-review.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-sentiment](<https://devfeed.tech/tags/ai-sentiment.md>), [bluesky](<https://devfeed.tech/tags/bluesky.md>), [consent](<https://devfeed.tech/tags/consent.md>), [creative](<https://devfeed.tech/tags/creative.md>), [data-collection](<https://devfeed.tech/tags/data-collection.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [posts](<https://devfeed.tech/tags/posts.md>), [sharing](<https://devfeed.tech/tags/sharing.md>)

### AI overview

A public Bluesky firehose dataset of one million posts--including text, metadata, and media alt text--was taken down within 12 hours after users objected, with some alleging privacy violations and others fearing their posts could be used for AI training or expose sensitive content. The author says the firehose is publicly accessible and describes the collection as neither scraping nor theft, while acknowledging that the release lacked transparency and consent. Their review of critics found anti-AI creatives and users posting adult content among two clusters, together representing only an estimated 10-20% of critical accounts; most had generic profiles. The backlash included blocklists, threats, and reports of AI-associated users feeling unwelcome. Further large datasets appeared soon after, illustrating how readily bulk posts could be collected through the API. The author frames the conflict as rooted partly in concern and fear, and calls for empathetic dialogue.

### Source excerpt

How the sharing of 1M Bluesky posts uncovered the strong anti-AI sentiment on Bluesky.

## How to Run a Weekly Paper Club (and Build a Learning Community)

DevFeed: [How to Run a Weekly Paper Club (and Build a Learning Community)](<https://devfeed.tech/articles/how-to-run-a-weekly-paper-club-and-build-a-learning-community-46514.md>)

Original publisher: [Read original article](<https://eugeneyan.com//writing/paper-club/>)

Author: Eugene Yan

Published: 2024-11-24T00:00:00Z

Content type: tutorial

Language: en

Sources: [Eugene Yan](<https://devfeed.tech/sources/eugene-yan.md>)

Topics: [Learning](<https://devfeed.tech/topics/learning.md>)

Tags: [community](<https://devfeed.tech/tags/community.md>), [guides](<https://devfeed.tech/tags/guides.md>), [how-to](<https://devfeed.tech/tags/how-to.md>), [learning](<https://devfeed.tech/tags/learning.md>), [mechanism](<https://devfeed.tech/tags/mechanism.md>), [productivity](<https://devfeed.tech/tags/productivity.md>)

### AI overview

A practical guide to starting and running a weekly paper club. It covers scheduling, pre-reading, facilitation, paper selection, and discussion, with the goal of helping peers learn together and apply research ideas to their work and projects.

### Source excerpt

Benefits of running a weekly paper club, how to start one, and how to read and facilitate papers.

## My Minimal MacBook Pro Setup Guide

DevFeed: [My Minimal MacBook Pro Setup Guide](<https://devfeed.tech/articles/my-minimal-macbook-pro-setup-guide-46480.md>)

Original publisher: [Read original article](<https://eugeneyan.com//writing/mac-setup/>)

Author: Eugene Yan

Published: 2024-11-17T00:00:00Z

Content type: tutorial

Language: en

Sources: [Eugene Yan](<https://devfeed.tech/sources/eugene-yan.md>)

Topics: [macOS](<https://devfeed.tech/topics/macos.md>), [Homebrew](<https://devfeed.tech/topics/homebrew.md>), [Terminal](<https://devfeed.tech/topics/terminal.md>), [friendly interactive shell](<https://devfeed.tech/topics/fish.md>), [cursor](<https://devfeed.tech/topics/cursor.md>), [Ollama](<https://devfeed.tech/topics/ollama.md>), [Command-line interface](<https://devfeed.tech/topics/cli.md>), [open webui](<https://devfeed.tech/topics/open-webui.md>), [Web Development](<https://devfeed.tech/topics/web-development.md>), [ssh](<https://devfeed.tech/topics/ssh.md>)

Tags: [apple](<https://devfeed.tech/tags/apple.md>), [chrome](<https://devfeed.tech/tags/chrome.md>), [cursor](<https://devfeed.tech/tags/cursor.md>), [development](<https://devfeed.tech/tags/development.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [guide](<https://devfeed.tech/tags/guide.md>), [homebrew](<https://devfeed.tech/tags/homebrew.md>), [local-models](<https://devfeed.tech/tags/local-models.md>), [macos](<https://devfeed.tech/tags/macos.md>), [misc](<https://devfeed.tech/tags/misc.md>), [setup](<https://devfeed.tech/tags/setup.md>)

### AI overview

A practical guide to setting up a new MacBook Pro from scratch, covering system settings, developer tools, research and writing applications, local model interfaces, and productivity utilities.

### Source excerpt

Setting up my new MacBook Pro from scratch

## 39 Lessons on Building ML Systems, Scaling, Execution, and More

DevFeed: [39 Lessons on Building ML Systems, Scaling, Execution, and More](<https://devfeed.tech/articles/39-lessons-on-building-ml-systems-scaling-execution-and-more-46420.md>)

Original publisher: [Read original article](<https://eugeneyan.com//writing/conf-lessons/>)

Author: Eugene Yan

Published: 2024-11-03T00:00:00Z

Content type: article

Language: en

Sources: [Eugene Yan](<https://devfeed.tech/sources/eugene-yan.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [systems](<https://devfeed.tech/topics/systems.md>), [Scalability](<https://devfeed.tech/topics/scalability.md>), [execution](<https://devfeed.tech/topics/execution.md>), [data](<https://devfeed.tech/topics/data.md>), [Users](<https://devfeed.tech/topics/users.md>), [recommendation systems](<https://devfeed.tech/topics/recommendation-systems.md>), [Database](<https://devfeed.tech/topics/database.md>)

Tags: [engineering](<https://devfeed.tech/tags/engineering.md>), [evals](<https://devfeed.tech/tags/evals.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [execution](<https://devfeed.tech/tags/execution.md>), [leadership](<https://devfeed.tech/tags/leadership.md>), [machinelearning](<https://devfeed.tech/tags/machinelearning.md>), [ml](<https://devfeed.tech/tags/ml.md>), [production](<https://devfeed.tech/tags/production.md>), [recommendations](<https://devfeed.tech/tags/recommendations.md>), [scaling](<https://devfeed.tech/tags/scaling.md>), [systems](<https://devfeed.tech/tags/systems.md>), [users](<https://devfeed.tech/tags/users.md>)

### AI overview

Notes from 2024 ML conferences covering practical lessons for building production machine-learning systems, including reward functions, simple baselines, trade-offs, time-aware models, realistic expectations, and robust evaluations.

### Source excerpt

ML systems, production & scaling, execution & collaboration, building for users, conference etiquette.

## AlignEval: Building an App for Data-Driven LLM Evaluator Development

DevFeed: [AlignEval: Building an App for Data-Driven LLM Evaluator Development](<https://devfeed.tech/articles/aligneval-building-an-app-to-make-evals-easy-fun-and-automated-46409.md>)

Original publisher: [Read original article](<https://eugeneyan.com//writing/aligneval/>)

Author: Eugene Yan

Published: 2024-10-27T00:00:00Z

Content type: article

Language: en

Sources: [Eugene Yan](<https://devfeed.tech/sources/eugene-yan.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Software Testing](<https://devfeed.tech/topics/software-testing.md>), [dataset](<https://devfeed.tech/topics/dataset.md>), [CSV](<https://devfeed.tech/topics/csv.md>), [Optimization](<https://devfeed.tech/topics/optimization.md>), [App](<https://devfeed.tech/topics/app.md>), [data](<https://devfeed.tech/topics/data.md>)

Tags: [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-evals](<https://devfeed.tech/tags/ai-evals.md>), [code](<https://devfeed.tech/tags/code.md>), [csv](<https://devfeed.tech/tags/csv.md>), [data](<https://devfeed.tech/tags/data.md>), [eval](<https://devfeed.tech/tags/eval.md>), [learning](<https://devfeed.tech/tags/learning.md>), [llm](<https://devfeed.tech/tags/llm.md>), [optimize](<https://devfeed.tech/tags/optimize.md>), [product](<https://devfeed.tech/tags/product.md>), [testing](<https://devfeed.tech/tags/testing.md>)

### AI overview

The article introduces AlignEval, an app for building and automating evaluators for LLM-powered products. It argues that effective evaluations should begin with observed data and defects, then use human criteria and metrics to assess and improve evaluator performance.

### Source excerpt

Look at and label your data, build and evaluate your LLM-evaluator, and optimize it against your labels.

## Weights & Biases LLM-Evaluator Hackathon - Hackathon Judge

DevFeed: [Weights & Biases LLM-Evaluator Hackathon - Hackathon Judge](<https://devfeed.tech/articles/weights-biases-llm-evaluator-hackathon-hackathon-judge-46379.md>)

Original publisher: [Read original article](<https://eugeneyan.com//speaking/hackathon-judge/>)

Author: Eugene Yan

Published: 2024-09-22T00:00:00Z

Content type: article

Language: en

Sources: [Eugene Yan](<https://devfeed.tech/sources/eugene-yan.md>)

Topics: [Hackathon](<https://devfeed.tech/topics/hackathon.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Prompt Engineering](<https://devfeed.tech/topics/prompt-engineering.md>), [User Interfaces](<https://devfeed.tech/topics/user-interfaces.md>), [User Experience](<https://devfeed.tech/topics/user-experience.md>)

Tags: [engineering](<https://devfeed.tech/tags/engineering.md>), [eval](<https://devfeed.tech/tags/eval.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [hackathon](<https://devfeed.tech/tags/hackathon.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-as-a-judge](<https://devfeed.tech/tags/llm-as-a-judge.md>), [ui](<https://devfeed.tech/tags/ui.md>), [ux](<https://devfeed.tech/tags/ux.md>)

### AI overview

A personal write-up about judging the Weights & Biases LLM-as-a-Judge Hackathon, where teams built projects involving knowledge graphs, LLM evaluation, evaluation prompts, and multi-turn conversations.

### Source excerpt

Being a human judge at the Weights & Biases LLM-as-a-Judge Hackathon

[Next page](<https://devfeed.tech/sources/eugene-yan.md?cursor=WyIyMDI0LTA5LTIyVDAwOjAwOjAwKzAwOjAwIiwgIjY2NWE4ZTljLTAxNzMtNDVmNi1hY2FiLTI0YTBiNTM1M2YyNyJd>)