# Deliveroo experimentation principles

DevFeed: [Deliveroo experimentation principles](<https://devfeed.tech/articles/deliveroo-experimentation-principles-19712.md>)

Original publisher: [Read original article](<https://deliveroo.engineering/2025/02/06/experimentation-principles.html>)

Author: Ella Johnson-Watts

Published: 2025-02-06T00:00:00Z

Content type: article

Language: en

Sources: [Deliveroo](<https://devfeed.tech/sources/deliveroo.md>)

Topics: [A/B Testing](<https://devfeed.tech/topics/a-b-testing.md>), [experiments](<https://devfeed.tech/topics/experiments.md>), [Testing](<https://devfeed.tech/topics/testing.md>), [Tooling](<https://devfeed.tech/topics/tooling.md>)

Tags: [a-b-testing](<https://devfeed.tech/tags/a-b-testing.md>), [algorithm](<https://devfeed.tech/tags/algorithm.md>), [development](<https://devfeed.tech/tags/development.md>), [testing](<https://devfeed.tech/tags/testing.md>), [tooling](<https://devfeed.tech/tags/tooling.md>), [ui](<https://devfeed.tech/tags/ui.md>)

## AI overview

Deliveroo describes how its experimentation programme evolved from ad hoc practices into a mature platform and culture. The article introduces principles such as defining a clear hypothesis and success criteria before running an experiment.

## Source excerpt

Experimentation at Deliveroo has significantly boosted our innovation as a company. It has allowed us to explore new ideas, protected us from unforeseen challenges by preventing wasted time and resources on ineffective initiatives, and promoted intellectual honesty by validating our assumptions. Over time, the breadth of our experimentation programme, our tooling and our culture have undergone significant development. Each year, we run hundreds of experiments, testing a wide array of features. These experiments range from simple UI modifications to complex algorithm adjustments. We employ a variety of methods, from standard A/B testing to advanced techniques like interleaving, multi-armed bandits and switchbacks. We've moved from teams using ad-hoc methods for deploying and analysing experiments to a mature experimentation platform, which standardises and automates much of the experiment process. But successful experimentation requires more than just technical expertise and tooling. It requires an experimentation "culture" - a shared set of principles and values that embeds experiments in the broader organisational context, and ensures that experimentation is done in a way that is maximally impactful. As we have matured technically and increasingly democratised our technical capabilities, we have also evolved a set of such principles, which codifies that cultural side. These enable us to uphold high-quality experimentation whilst achieving our current goal of broadening the pool of Deliveroo employees capable of running experiments. Principle 1: Every experiment starts with a clear hypothesis and success criteria. Experiments work best when they are specific. We formulate our hypothesis based on past experiments, anecdotes, user research, and competitive analysis - leveraging existing learnings helps us design better experiments. We avoid experimenting aimlessly just to 'see what happens', risking learning nothing. Experiments are a highly valuable resource and unst