# Speed with Rigor: testing smarter with group sequential design

DevFeed: [Speed with Rigor: testing smarter with group sequential design](<https://devfeed.tech/articles/speed-with-rigor-testing-smarter-with-group-sequential-design-20391.md>)

Original publisher: [Read original article](<https://tech.olx.com/speed-with-rigor-testing-smarter-with-group-sequential-design-af962363ce90?source=rss----761b019b483f---4>)

Author: Gabriela Lewenfus

Published: 2025-08-05T17:25:29Z

Content type: article

Language: en

Sources: [OLX](<https://devfeed.tech/sources/olx.md>)

Topics: [sequential-testing](<https://devfeed.tech/topics/sequential-testing.md>), [Software Testing](<https://devfeed.tech/topics/software-testing.md>), [A/B Testing](<https://devfeed.tech/topics/a-b-testing.md>), [product-experimentation](<https://devfeed.tech/topics/product-experimentation.md>)

Tags: [a-b-testing](<https://devfeed.tech/tags/a-b-testing.md>), [ab-testing](<https://devfeed.tech/tags/ab-testing.md>), [challenges](<https://devfeed.tech/tags/challenges.md>), [costs](<https://devfeed.tech/tags/costs.md>), [data-driven](<https://devfeed.tech/tags/data-driven.md>), [decision-making](<https://devfeed.tech/tags/decision-making.md>), [design](<https://devfeed.tech/tags/design.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [experimentation](<https://devfeed.tech/tags/experimentation.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [false-positive](<https://devfeed.tech/tags/false-positive.md>), [false-positive-rate](<https://devfeed.tech/tags/false-positive-rate.md>), [false-positives](<https://devfeed.tech/tags/false-positives.md>), [implementation](<https://devfeed.tech/tags/implementation.md>), [olx](<https://devfeed.tech/tags/olx.md>), [power](<https://devfeed.tech/tags/power.md>), [sample](<https://devfeed.tech/tags/sample.md>), [sequential-testing](<https://devfeed.tech/tags/sequential-testing.md>), [speed](<https://devfeed.tech/tags/speed.md>), [statistics](<https://devfeed.tech/tags/statistics.md>), [testing](<https://devfeed.tech/tags/testing.md>)

## AI overview

Group sequential testing (GST) lets experimenters check A/B test results at predefined interim stages and stop early while controlling the overall false-positive rate. Efficacy boundaries require especially strong evidence at early checks; futility boundaries can stop tests unlikely to show an effect, but overly aggressive futility rules risk ending tests prematurely. Compared with a fixed-horizon design, GST may reduce average sample use when boundaries are crossed, while its maximum sample size is higher to account for repeated analyses. The text gives an example where a 20-stage design's maximum can be nearly 25% larger than the fixed-horizon sample size. More frequent checks can increase early stopping opportunities but also affect that maximum. Accurate traffic estimates and planned analysis timing matter: unexpected traffic increases may compromise Type I error control, while drops can reduce sensitivity. The article describes OLX's move to GST but provides no specific implementation results in this excerpt.

## Source excerpt

generated by chatgptIntroduction In the world of A/B testing and experimentation, it's tempting to check results frequently and stop tests earlier if they seem significant (a practice known as peeking). However, this practice dramatically inflates the false positive rate, leading to misleading conclusions. Imagine you're running an A/B test, and after just 10 days, the results look like a clear win. You might be tempted to end the experiment early and move forward with the release of the new feature. After all, why wait another two weeks as originally planned? But here's the catch: the more often you check the results, the higher the chance you'll spot a "win" that isn't real. Frequent peeking increases the risk of false positives -- in other words, drawing the wrong conclusion just by chance. To prevent peeking while ensuring statistical power, a common practice is to set a minimum sample size that must be reached before ending the experiment. However, this fixed horizon approach can extend the experiment's duration and delay decision-making. Group sequential testing (GST) provides a statistical framework to monitor results at predefined intervals while maintaining control over error rates. At OLX, we migrated from fixed horizon experiments to GST to achieve faster insights, lower costs, and greater efficiency, enabling our teams to make smarter, data-driven decisions with speed and confidence. In this post, I'll explain how GST works, how it can accelerate experiments while still preventing the pitfalls of peeking, and how we can use it even to stop ineffective tests early. Finally, I'll discuss real-world trade-offs and implementation challenges. Peeking is a trap Before we dive into GST, let's first understand what "peeking" is and why it's so concerning. Peeking occurs when the experimenter repeatedly checks an experiment's results and finishes it once a statistically significant effect appears. This inflates the false positive rate, leading to unreliable conclu