# doubleml Published articles for doubleml. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## The Causality Gap: Measuring the True Impact of Voluntary Adoption in Digital Marketplaces DevFeed: [The Causality Gap: Measuring the True Impact of Voluntary Adoption in Digital Marketplaces]() Original publisher: [Read original article]() Author: Lin Jia Published: 2026-05-22T08:05:43Z Content type: article Language: en Sources: [Booking.com Data Science]() Topics: [A/B Testing](), [doubleml](), [Data Science](), [Tool]() Tags: [a-b-testing](), [causal-inference](), [causal-machine-learning](), [causality](), [data-science](), [doubleml](), [e-commerce](), [experimentation](), [featured](), [measuring]() ### AI overview This article explains why standard A/B tests can mismeasure features that require voluntary adoption in digital marketplaces. It presents Randomized Encouragement Design combined with Double Machine Learning as a way to estimate both causal impact among adopters and overall rollout impact. ### Source excerpt by Lin Jia, Kexin Fei The Content of this post has been presented at Pydata Amsterdam 2026 and the slides can be found here TL; DR Whenever a feature requires voluntary adoption, standard A/B testing breaks: low adoption flattens the topline, and self-selection makes adopters incomparable to non-adopters. Combining Randomized Encouragement Design (RED) with Double Machine Learning (DoubleML) recovers two answers -- the causal lift for adopters and the rollout's overall impact. When they diverge, the gap turns an ambiguous topline into a sharp product decision -- build a better product, or build a better adoption funnel. 1. The Opt-In Barrier Across Demand and Supply Across the tech industry, many platform features rely on voluntary adoption. A customer chooses whether to claim a promotional discount. A traveller opts into a loyalty program. An e-commerce seller enables a smart-pricing tool. In every case, the platform cannot force adoption -- and the feature's true impact becomes hard to measure, on both demand and supply. At Booking.com this challenge spans both sides of the marketplace -- travellers choosing to log in, partners choosing to adopt new features. Unlike a search-ranking change that applies to 100% of traffic, opt-in features create a "trilemma" for Product Data Science: Voluntary Adoption (the "Opt-In" Barrier): Users must actively enable the feature. A standard A/B test cannot separate the product's effect from the motivation that drove users to adopt it. Extreme Heterogeneity: Travellers range from once-a-year holidaymakers to travel agencies booking thousands of nights; partners range from single-apartment hosts to hotel chains. This variance is noise on both sides. Finite Sample Sizes: Opt-in features target a finite sub-segment, so we cannot simply "run the test longer" to gain power. When these stack up, a flat topline can hide a strong product behind a weak adoption funnel. Genuinely strong features get killed, and resources flow into the wrong int ## Trusting the Untestable: Validation and Diagnostics for the Doubly Robust Models DevFeed: [Trusting the Untestable: Validation and Diagnostics for the Doubly Robust Models]() Original publisher: [Read original article]() Author: Shima Nassiri Published: 2026-02-12T17:07:13Z Content type: article Language: en Sources: [Lyft Engineering - Medium]() Topics: [Estimator](), [data](), [Monitoring]() Tags: [aipw](), [doubleml](), [evaluation](), [experiment](), [experimentation](), [lyft](), [measurement](), [models](), [quasi-experiment](), [rideshare](), [validation]() ### AI overview The article explains how Lyft uses quasi-experimental methods to measure causal effects when randomized A/B tests are impractical. It focuses on Augmented Inverse Propensity Weighting (AIPW), a doubly robust estimator that uses outcome and propensity-score models to estimate treatment effects from non-randomized data. ### Source excerpt written by Ross Chu and Shima Nassiri The Causal Frontier: Measurement Beyond Randomization The gold standard for determining the causal impact of a policy or product change at a company like Lyft is the A/B test (randomized experiment). By randomly assigning users to a treatment or control group, A/B tests inherently eliminate bias, providing clean estimates of the Average Treatment Effect (ATE). However, many critical business questions and large-scale initiatives simply cannot be randomized. This forces scientists to move past traditional experimentation and leverage quasi-experimental methods. We rely on non-randomized measurement in several key scenarios across Lyft: Partnerships and Policies: Assessing the incremental impact of a partnership (e.g., linking two company accounts) is often a non-randomized assignment. Since these collaborations require coordinated operational work across both companies and are typically announced or promoted broadly, this makes controlled randomization impractical. Long-Term Effect (LTE): Measuring effects that unfold over a long period, like the LTE of high prices on future rides, is typically handled by observational studies. Post-Launch Evaluation: Continuous monitoring of a policy after it has been fully rolled out requires a method that doesn't involve costly holdout groups or degradation tests. Biased Data: In cases where pre-existing experimental data is found to have an imbalance, a quasi-experimental approach can potentially leverage the biased data instead of requiring a costly rerun. Introducing Doubly Robust Models: Causal Inference Without Randomness To address these non-randomized measurement needs, Lyft relies on various quasi-experiment estimators. In this blog we specifically focus on using the Augmented Inverse Propensity Weighting (AIPW) model. This model was first established at Lyft to measure the impact of a negative user experience on future topline metrics like rides and bookings; AIPW is a form of doubly