# ab-testing Published articles for ab-testing. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## A Date-Stamped Data Warehouse Setup DevFeed: [A Date-Stamped Data Warehouse Setup]() Original publisher: [Read original article]() Author: Sahar Massachi Published: 2025-10-24T21:03:30Z Content type: article Language: en Sources: [DataExpert.io Newsletter]() Topics: [data-engineering](), [data](), [Apache Iceberg]() Tags: [ab-testing](), [data](), [data-engineering](), [storage](), [warehouse]() ### AI overview The article presents date-stamping data as a simple organizing principle for building resilient data warehouses and pipelines. It explains how this approach can manage changes over time, support experimentation and metrics, and work with systems including Hive metastore, Iceberg, Delta, and Hudi. ### Source excerpt Storage is cheap, your time is not! ## Speed with Rigor: testing smarter with group sequential design DevFeed: [Speed with Rigor: testing smarter with group sequential design]() Original publisher: [Read original article]() Author: Gabriela Lewenfus Published: 2025-08-05T17:25:29Z Content type: article Language: en Sources: [OLX]() Topics: [Testing](), [A/B Testing](), [experiments](), [data](), [implementation]() Tags: [a-b-testing](), [ab-testing](), [data](), [efficiency](), [experiment](), [experimentation](), [false-positive](), [implementation](), [pitfalls](), [sequential-testing](), [statistics](), [testing]() ### AI overview The article explains how group sequential testing can replace fixed-horizon A/B tests with monitoring at predefined intervals while controlling error rates. It discusses how this approach addresses peeking, may allow ineffective tests to stop early, and involves practical trade-offs and implementation challenges. ### Source excerpt generated by chatgptIntroduction In the world of A/B testing and experimentation, it's tempting to check results frequently and stop tests earlier if they seem significant (a practice known as peeking). However, this practice dramatically inflates the false positive rate, leading to misleading conclusions. Imagine you're running an A/B test, and after just 10 days, the results look like a clear win. You might be tempted to end the experiment early and move forward with the release of the new feature. After all, why wait another two weeks as originally planned? But here's the catch: the more often you check the results, the higher the chance you'll spot a "win" that isn't real. Frequent peeking increases the risk of false positives -- in other words, drawing the wrong conclusion just by chance. To prevent peeking while ensuring statistical power, a common practice is to set a minimum sample size that must be reached before ending the experiment. However, this fixed horizon approach can extend the experiment's duration and delay decision-making. Group sequential testing (GST) provides a statistical framework to monitor results at predefined intervals while maintaining control over error rates. At OLX, we migrated from fixed horizon experiments to GST to achieve faster insights, lower costs, and greater efficiency, enabling our teams to make smarter, data-driven decisions with speed and confidence. In this post, I'll explain how GST works, how it can accelerate experiments while still preventing the pitfalls of peeking, and how we can use it even to stop ineffective tests early. Finally, I'll discuss real-world trade-offs and implementation challenges. Peeking is a trap Before we dive into GST, let's first understand what "peeking" is and why it's so concerning. Peeking occurs when the experimenter repeatedly checks an experiment's results and finishes it once a statistically significant effect appears. This inflates the false positive rate, leading to unreliable conclu ## WISE: Skyscanner's Bayesian AB experimentation library and decision engine DevFeed: [WISE: Skyscanner's Bayesian AB experimentation library and decision engine]() Original publisher: [Read original article]() Author: Skyscanner Engineering Published: 2024-04-25T06:22:08Z Content type: article Language: en Sources: [Stories by Skyscanner Engineering on Medium]() Topics: [experiments](), [Library](), [Python](), [Statistics]() Tags: [ab-testing](), [analytics](), [bayesian-statistics](), [data-science](), [engineering](), [experiment](), [experimentation](), [practices](), [statistics](), [test]() ### AI overview This article describes WISE, Skyscanner's centralized Bayesian A/B experimentation Python library and decision engine. It explains how Skyscanner uses randomized controlled experiments and why WISE was built to reduce manual analysis, standardize methodologies, and improve consistency in experimentation practices. ### Source excerpt By Dhanush Kishore, with Jose Parreño, May Alexander, Robert Shepherd, and the Skyscanner experimentation squad At Skyscanner, product decisions are driven by experimentation. As a global leader in travel, with 110 million users every month, we rely on a data-driven approach to growing and improving our product. Product managers, engineers, and data scientists run hundreds of experiments on our in-house AB experimentation platform, Dr Jekyll, to help make decisions about what features to ship for travelers and partners. When a team develops a new feature, the next step is to run a randomized controlled experiment. Every eligible user is randomly assigned either to the test variant B, where they interact with the new feature, or to the control variant A. After running the experiment for a certain duration, we test key metrics in variant B against variant A to determine whether the feature will be beneficial if rolled out to all users. This ensures that we follow an objective, scientifically rigorous approach to making product decisions. Although we already had a strong experimentation culture and an excellent experimentation platform, we identified the following needs as we scaled our experimentation program: Reduce manual effort in analyzing experiment results: While our experimentation platform, Dr Jekyll, reported test results for various metrics in an experiment, our data analytics team had to spend a considerable amount of time interpreting the results and making a decision on what variant to ship, which was becoming a bottleneck to scaling our experimentation program Need for standardization: Different teams at Skyscanner used different approaches to AB experimentation, and inconsistencies in methodologies meant it was difficult to compare results and share learnings across teams Consistency in adherence to experimentation best practices: While many teams maintained high standards of statistical rigor, others required additional support in designing and interpr