# Beyond synthetic testing: Capturing and replaying real database workloads at Airbnb

DevFeed: [Beyond synthetic testing: Capturing and replaying real database workloads at Airbnb](<https://devfeed.tech/articles/beyond-synthetic-testing-capturing-and-replaying-real-database-workloads-at-airbnb-65699.md>)

Original publisher: [Read original article](<https://medium.com/airbnb-engineering/beyond-synthetic-testing-capturing-and-replaying-real-database-workloads-at-airbnb-cea7ee9b1ab2?source=rss----53c7c27702d5---4>)

Author: Zuofei Wang

Published: 2026-10-06T17:01:04Z

Content type: article

Language: en

Sources: [Airbnb](<https://devfeed.tech/sources/the-airbnb-tech-blog-medium.md>)

Topics: [Transactions](<https://devfeed.tech/topics/transactions.md>), [Test automation](<https://devfeed.tech/topics/test-automation.md>), [Distributed Systems & Parallel Computing](<https://devfeed.tech/topics/distributed-systems-parallel-computing.md>), [database-consistency](<https://devfeed.tech/topics/database-consistency.md>), [clickhouse](<https://devfeed.tech/topics/clickhouse.md>)

Tags: [airbnb](<https://devfeed.tech/tags/airbnb.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [behavior](<https://devfeed.tech/tags/behavior.md>), [capacity](<https://devfeed.tech/tags/capacity.md>), [capture](<https://devfeed.tech/tags/capture.md>), [challenges](<https://devfeed.tech/tags/challenges.md>), [client](<https://devfeed.tech/tags/client.md>), [clusters](<https://devfeed.tech/tags/clusters.md>), [complete](<https://devfeed.tech/tags/complete.md>), [critical](<https://devfeed.tech/tags/critical.md>), [custom](<https://devfeed.tech/tags/custom.md>), [data](<https://devfeed.tech/tags/data.md>), [database](<https://devfeed.tech/tags/database.md>), [databases](<https://devfeed.tech/tags/databases.md>), [debug](<https://devfeed.tech/tags/debug.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [error](<https://devfeed.tech/tags/error.md>), [fast](<https://devfeed.tech/tags/fast.md>), [fleet](<https://devfeed.tech/tags/fleet.md>), [framework](<https://devfeed.tech/tags/framework.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [sql](<https://devfeed.tech/tags/sql.md>), [synthetic](<https://devfeed.tech/tags/synthetic.md>), [technology](<https://devfeed.tech/tags/technology.md>), [testing](<https://devfeed.tech/tags/testing.md>)

## AI overview

Airbnb built a client-agnostic database traffic capture and replay system on ProxySQL to test real MySQL-compatible workloads offline. ProxySQL logs are moved to object storage; a processor separates traffic by cluster, restores transaction order, buckets queries into five-minute files, and pins captured auto-increment IDs in INSERTs. That rewrite keeps dependent reads consistent across database versions, but means replay does not test the target's native ID generation. In "Replay only" mode, traffic runs against one target at configurable speeds or QPS, with errors, latency, and resource use tracked for load testing and capacity planning. "Replay and compare" runs the same traffic against two targets and records result differences for upgrade and migration checks. Workers preserve query ordering, relative timing, and concurrency. Airbnb reports using replays to find MySQL 5.7-to-8.0 latency regressions and queries with nondeterministic ordering; scaling one cluster's write traffic by 80% raised average commit latency from about 6 ms to 34 ms. Captured data is treated as production data, and replays run only in production-equivalent environments.

## Source excerpt

How we capture real production database traffic at Airbnb and replay it offline to load-test, plan capacity, and de-risk upgrades. By: Zuofei Wang, Erluo Li Introduction At Airbnb, MySQL-compatible databases are a critical backbone of our online database infrastructure: a fleet of hundreds of clusters supporting thousands of use cases at millions of queries per second (QPS). Operating databases at scale brings hard problems, including sizing clusters for future growth, keeping behavior consistent across version upgrades and migrations, and reproducing production incidents well enough to debug them. This post describes the database traffic capture and replay system we built to help us tackle them. Challenges and motivation A complex database operation such as Airbnb's requires a number of supporting systems, and one of them is a way to capture database queries and replay them. This is needed for error recovery, governance, quality assurance testing and product improvement efforts. Previously, Airbnb's system for database query capture and replay was fragmented. Each language binding used its own logging framework to emit query traces to Kafka, which were then processed per use case and replayed by a custom replayer. Legacy Query Capture and Replay System This system was borrowed from our observability pipeline. It worked as a fast initial solution, but over time, we found that it fell short of what we needed. Capturing queries client-side and per language carried a heavy maintenance burden whose ownership was ambiguous, and it scaled poorly across our service-oriented architecture (SOA). Most importantly, it lacked enough information to convey a complete picture of each transaction. As a result, it couldn't accurately replay transactions, which made it impossible to verify that the same queries return the same results on the new database. To close these gaps, we set out to build a system for query capture and replay, with three goals: Load testing with real productio