# Beyond the model: Engineering AI infra with scientific judgement

DevFeed: [Beyond the model: Engineering AI infra with scientific judgement](<https://devfeed.tech/articles/beyond-the-model-engineering-ai-infra-with-scientific-judgement-26973.md>)

Original publisher: [Read original article](<https://medium.com/airbnb-engineering/beyond-the-model-engineering-ai-infra-with-scientific-judgement-371316d43261?source=rss----53c7c27702d5---4>)

Author: AirbnbEng

Published: 2026-09-15T17:06:18Z

Content type: article

Language: en

Sources: [The Airbnb Tech Blog - Medium](<https://devfeed.tech/sources/the-airbnb-tech-blog-medium.md>)

Topics: [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [AI Agent](<https://devfeed.tech/topics/ai-agent.md>), [Data Science](<https://devfeed.tech/topics/data-science.md>), [data](<https://devfeed.tech/topics/data.md>), [datasets](<https://devfeed.tech/topics/datasets.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [agent-harness](<https://devfeed.tech/tags/agent-harness.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agent](<https://devfeed.tech/tags/ai-agent.md>), [data](<https://devfeed.tech/tags/data.md>), [data-science](<https://devfeed.tech/tags/data-science.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [llms](<https://devfeed.tech/tags/llms.md>), [quality](<https://devfeed.tech/tags/quality.md>), [science](<https://devfeed.tech/tags/science.md>), [trust](<https://devfeed.tech/tags/trust.md>)

## AI overview

Airbnb describes an agent harness for data science that embeds scientific methodology around an AI model. The system guides agents through framing questions, selecting evidence, and recording decisions so unstructured-data investigations can be reproduced, audited, challenged, and extended across languages, geographies, and LLM-based products.

## Source excerpt

How Airbnb's agent harness transforms unstructured data exploration by encoding scientific methodology into scalable, reproducible, and audit-ready infrastructure. By: Wren Dougherty Ask a coding agent to analyze 100,000 customer support conversations and within minutes you'll have a polished taxonomy, precise prevalence numbers, and an executive-ready summary. What you can't see is the investigation that produced them: the methods it chose, the evidence it weighed, how much to trust it, or whether a second request would agree. All that reaches you is the polish. The model is undeniably intelligent, but intelligence without methodology is not science. LLMs certainly make for confident scientists, but we need them to be responsible ones. Smarter models help, but intelligence has never been the whole of science, in people or in machines. The method is as much the product as the answer. That is the idea behind the agent harness we built for data science: the methodology itself, built as infrastructure around the model. It governs how an AI agent operates, from framing a question to selecting evidence to recording decisions, so results can be reproduced, audited, and challenged, and the method shared, inspected, and built on. The challenge of unstructured data exploration In 2025, Airbnb was preparing to launch an AI customer service assistant. Before it could ship, we needed to understand exactly what kinds of situations it would face in the real world. That included rare events that could be risky for AI to interact with, and involved examining their taxonomy and prevalence to create the datasets that would help us build a more responsible product. The investigative work to do this was rigorous, but the process was deeply artisanal. Months of high-touch iteration went into each investigation, from finding the right data, reviewing samples with experts, and generating representative datasets, and the method was manually curated across notebooks, tables, docs, and indiv