# The Causality Gap: Measuring the True Impact of Voluntary Adoption in Digital Marketplaces

DevFeed: [The Causality Gap: Measuring the True Impact of Voluntary Adoption in Digital Marketplaces](<https://devfeed.tech/articles/the-causality-gap-measuring-the-true-impact-of-voluntary-adoption-in-digital-marketplaces-30456.md>)

Original publisher: [Read original article](<https://booking.ai/the-causality-gap-measuring-the-true-impact-of-voluntary-adoption-in-digital-marketplaces-ea68b5a35120?source=rss----4d265f07defc---4>)

Author: Lin Jia

Published: 2026-05-22T08:05:43Z

Content type: article

Language: en

Sources: [Booking.com Data Science](<https://devfeed.tech/sources/booking-com-data-science.md>)

Topics: [A/B Testing](<https://devfeed.tech/topics/a-b-testing.md>), [doubleml](<https://devfeed.tech/topics/doubleml.md>), [Data Science](<https://devfeed.tech/topics/data-science.md>), [Tool](<https://devfeed.tech/topics/tool.md>)

Tags: [a-b-testing](<https://devfeed.tech/tags/a-b-testing.md>), [causal-inference](<https://devfeed.tech/tags/causal-inference.md>), [causal-machine-learning](<https://devfeed.tech/tags/causal-machine-learning.md>), [causality](<https://devfeed.tech/tags/causality.md>), [data-science](<https://devfeed.tech/tags/data-science.md>), [doubleml](<https://devfeed.tech/tags/doubleml.md>), [e-commerce](<https://devfeed.tech/tags/e-commerce.md>), [experimentation](<https://devfeed.tech/tags/experimentation.md>), [featured](<https://devfeed.tech/tags/featured.md>), [measuring](<https://devfeed.tech/tags/measuring.md>)

## AI overview

This article explains why standard A/B tests can mismeasure features that require voluntary adoption in digital marketplaces. It presents Randomized Encouragement Design combined with Double Machine Learning as a way to estimate both causal impact among adopters and overall rollout impact.

## Source excerpt

by Lin Jia, Kexin Fei The Content of this post has been presented at Pydata Amsterdam 2026 and the slides can be found here TL; DR Whenever a feature requires voluntary adoption, standard A/B testing breaks: low adoption flattens the topline, and self-selection makes adopters incomparable to non-adopters. Combining Randomized Encouragement Design (RED) with Double Machine Learning (DoubleML) recovers two answers -- the causal lift for adopters and the rollout's overall impact. When they diverge, the gap turns an ambiguous topline into a sharp product decision -- build a better product, or build a better adoption funnel. 1. The Opt-In Barrier Across Demand and Supply Across the tech industry, many platform features rely on voluntary adoption. A customer chooses whether to claim a promotional discount. A traveller opts into a loyalty program. An e-commerce seller enables a smart-pricing tool. In every case, the platform cannot force adoption -- and the feature's true impact becomes hard to measure, on both demand and supply. At Booking.com this challenge spans both sides of the marketplace -- travellers choosing to log in, partners choosing to adopt new features. Unlike a search-ranking change that applies to 100% of traffic, opt-in features create a "trilemma" for Product Data Science: Voluntary Adoption (the "Opt-In" Barrier): Users must actively enable the feature. A standard A/B test cannot separate the product's effect from the motivation that drove users to adopt it. Extreme Heterogeneity: Travellers range from once-a-year holidaymakers to travel agencies booking thousands of nights; partners range from single-apartment hosts to hotel chains. This variance is noise on both sides. Finite Sample Sizes: Opt-in features target a finite sub-segment, so we cannot simply "run the test longer" to gain power. When these stack up, a flat topline can hide a strong product behind a weak adoption funnel. Genuinely strong features get killed, and resources flow into the wrong int