# Why Spotify Is Not Using Bayesian A/B Testing

DevFeed: [Why Spotify Is Not Using Bayesian A/B Testing](<https://devfeed.tech/articles/why-spotify-is-not-using-bayesian-a-b-testing-156.md>)

Original publisher: [Read original article](<https://engineering.atspotify.com/2026/9/why-spotify-is-not-using-bayesian-a-b-testing/>)

Author: Spotify Engineering

Published: 2026-09-08T13:18:44Z

Content type: article

Language: en

Sources: [Spotify](<https://devfeed.tech/sources/spotify-engineering-blog.md>), [Spotify Engineering](<https://devfeed.tech/sources/spotify-engineering.md>)

Topics: [A/B Testing](<https://devfeed.tech/topics/a-b-testing.md>), [product-experimentation](<https://devfeed.tech/topics/product-experimentation.md>), [sequential-testing](<https://devfeed.tech/topics/sequential-testing.md>), [Developer Tools](<https://devfeed.tech/topics/developer-tools.md>), [Data science and engineering](<https://devfeed.tech/topics/data-science-and-engineering.md>), [Spotify](<https://devfeed.tech/topics/spotify.md>)

Tags: [a-b-testing](<https://devfeed.tech/tags/a-b-testing.md>), [bayesian](<https://devfeed.tech/tags/bayesian.md>), [bayesian-inference](<https://devfeed.tech/tags/bayesian-inference.md>), [data](<https://devfeed.tech/tags/data.md>), [data-science](<https://devfeed.tech/tags/data-science.md>), [spotify](<https://devfeed.tech/tags/spotify.md>), [testing](<https://devfeed.tech/tags/testing.md>)

## AI overview

Spotify says it has not added Bayesian inference because supporting two modes would add planning, monitoring, interpretation, and configuration costs without enough benefit for its experimentation goals. The post argues Bayesian A/B testing is a family of setups whose guarantees depend on the prior, likelihood, and stopping rule. Flat-prior posterior thresholds can reproduce frequentist peeking behavior; Bayes-factor stopping can control false positives under peeking, and a well-calibrated empirical Bayes prior can shrink winner's-curse estimates and control false discovery rates across metrics. Those benefits require representative historical data and careful calibration; pooling unlike programs or metrics, winner-biased archives, or drift can undermine results. The authors describe many Bayesian and frequentist procedures as equivalent or closely related, while noting decision rules can differ for some cost functions. They recommend choosing required guarantees based on program goals; Spotify judges prior maintenance and the costs of a second framework outweigh its benefits today.

## Source excerpt

Clearing the confusion about what Bayesian A/B testing is. The post Why Spotify Is Not Using Bayesian A/B Testing appeared first on Spotify Engineering.