# WISE: Skyscanner's Bayesian AB experimentation library and decision engine

DevFeed: [WISE: Skyscanner's Bayesian AB experimentation library and decision engine](<https://devfeed.tech/articles/wise-skyscanner-s-bayesian-ab-experimentation-library-and-decision-engine-32394.md>)

Original publisher: [Read original article](<https://medium.com/@SkyscannerEng/wise-skyscanners-bayesian-ab-experimentation-library-and-decision-engine-6841d1643482?source=rss-401f3b3c958f------2>)

Author: Skyscanner Engineering

Published: 2024-04-25T06:22:08Z

Content type: article

Language: en

Sources: [Stories by Skyscanner Engineering on Medium](<https://devfeed.tech/sources/stories-by-skyscanner-engineering-on-medium.md>)

Topics: [experiments](<https://devfeed.tech/topics/experiments.md>), [Library](<https://devfeed.tech/topics/library.md>), [Python](<https://devfeed.tech/topics/python.md>), [Statistics](<https://devfeed.tech/topics/statistics.md>)

Tags: [ab-testing](<https://devfeed.tech/tags/ab-testing.md>), [analytics](<https://devfeed.tech/tags/analytics.md>), [bayesian-statistics](<https://devfeed.tech/tags/bayesian-statistics.md>), [data-science](<https://devfeed.tech/tags/data-science.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [experiment](<https://devfeed.tech/tags/experiment.md>), [experimentation](<https://devfeed.tech/tags/experimentation.md>), [practices](<https://devfeed.tech/tags/practices.md>), [statistics](<https://devfeed.tech/tags/statistics.md>), [test](<https://devfeed.tech/tags/test.md>)

## AI overview

This article describes WISE, Skyscanner's centralized Bayesian A/B experimentation Python library and decision engine. It explains how Skyscanner uses randomized controlled experiments and why WISE was built to reduce manual analysis, standardize methodologies, and improve consistency in experimentation practices.

## Source excerpt

By Dhanush Kishore, with Jose Parreño, May Alexander, Robert Shepherd, and the Skyscanner experimentation squad At Skyscanner, product decisions are driven by experimentation. As a global leader in travel, with 110 million users every month, we rely on a data-driven approach to growing and improving our product. Product managers, engineers, and data scientists run hundreds of experiments on our in-house AB experimentation platform, Dr Jekyll, to help make decisions about what features to ship for travelers and partners. When a team develops a new feature, the next step is to run a randomized controlled experiment. Every eligible user is randomly assigned either to the test variant B, where they interact with the new feature, or to the control variant A. After running the experiment for a certain duration, we test key metrics in variant B against variant A to determine whether the feature will be beneficial if rolled out to all users. This ensures that we follow an objective, scientifically rigorous approach to making product decisions. Although we already had a strong experimentation culture and an excellent experimentation platform, we identified the following needs as we scaled our experimentation program: Reduce manual effort in analyzing experiment results: While our experimentation platform, Dr Jekyll, reported test results for various metrics in an experiment, our data analytics team had to spend a considerable amount of time interpreting the results and making a decision on what variant to ship, which was becoming a bottleneck to scaling our experimentation program Need for standardization: Different teams at Skyscanner used different approaches to AB experimentation, and inconsistencies in methodologies meant it was difficult to compare results and share learnings across teams Consistency in adherence to experimentation best practices: While many teams maintained high standards of statistical rigor, others required additional support in designing and interpr