# causal-inference Published articles for causal-inference. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## When AI art has no author: Study finds generated images often can't be traced to training data DevFeed: [When AI art has no author: Study finds generated images often can't be traced to training data]() Original publisher: [Read original article]() Author: Rachel Gordon | MIT CSAIL Published: 2026-08-18T16:35:00Z Content type: news Language: en Sources: [MIT AI News]() Topics: [Artificial Intelligence](), [AI Research](), [datasets](), [Computer Science and Artificial Intelligence Laboratory (CSAIL)]() Tags: [ablation](), [ai](), [ai-and-copyright-law](), [ai-generated-images](), [ai-governance](), [artificial-intelligence](), [arts](), [arts-technology-and-society](), [attribution-decay](), [causal-inference](), [computer-science-and-artificial-intelligence-laboratory-csail](), [computer-science-and-technology](), [counterfactual-analysis](), [counterfactual-radius](), [data](), [data-attribution](), [datasets](), [david-gifford](), [diffusion-ensembles](), [electrical-engineering-and-computer-science-eecs](), [ethics](), [generative-ai-images](), [generative-diffusion-models](), [image-similarity-metrics](), [images](), [law](), [machine-learning](), [machine-unlearning](), [mit-csail](), [mit-schwarzman-college-of-computing](), [model](), [model-interpretability](), [paper](), [privacy-preserving-machine-learning](), [research](), [school-of-engineering](), [science](), [technology-and-policy](), [technology-and-society](), [training-data-attribution](), [training-data-influence](), [zheng-dai]() ### AI overview MIT CSAIL researchers describe attribution decay, a phenomenon in which the influence of individual training examples on a generative model's outputs diminishes as datasets grow. Their method removes training examples and retrains models to test whether generated samples change. ### Source excerpt A new method for surgically removing training examples from a model reveals that as datasets grow, the link between what a model learns and what it produces dissolves. ## Correlation Lied to Us: Rethinking Product Impact with Causal Inference DevFeed: [Correlation Lied to Us: Rethinking Product Impact with Causal Inference]() Original publisher: [Read original article]() Author: Enderson Santos Published: 2026-08-04T15:31:01Z Content type: article Language: en Sources: [OLX]() Topics: [data]() Tags: [causal-inference](), [data](), [data-science](), [experimentation](), [machine-learning](), [metrics](), [performance](), [product-analytics]() ### AI overview The article examines why higher-tier OLX exposure packages appeared to perform worse than cheaper packages in some cases. It explains that sellers self-select packages, making direct package-level comparisons potentially misleading, and introduces causal inference as an approach to separate correlation from causation. ### Source excerpt Introduction At OLX, professional sellers pay for higher-tier packages because they promise more exposure. More visibility, and, in theory, better results. But when we looked at the data, something unexpected happened. In some cases, ads published with premium packages appeared to perform worse than ads using cheaper packages. That raised an uncomfortable question: If higher-tier packages provide more exposure, shouldn't they consistently perform better? At first glance, there were several possible explanations. Perhaps the extra visibility weren't creating as much value as we expected. Perhaps ranking dynamics were offsetting the additional exposure. Or perhaps the package itself wasn't the real driver of performance. It was then that we started asking a different question: Were we measuring this correctly? More specifically, were the ads across different packages actually comparable in the first place? Answering that question turned out to be far more important than comparing package-level metrics. It forced us to rethink how we measure product impact in a marketplace environment and ultimately led us to a causal inference approach designed to separate correlation from causation. In this article, I'll walk through how we approached that problem, what we learned, and how comparing similar ads changed our understanding of the true value created by exposure products. Problem Definition To understand the challenge, it's important to first understand how package exposure works at OLX. Professional sellers self select into a package when publishing their ads. The difference between packages is largely defined by how many boosts an ad receives during its lifetime. For example, in the picture below we can see that Package 1 includes 1 boost on the period of 30 days, package 2 includes 2 boosts, package 3 includes 3 boosts, and package 4 includes 4 boosts all in the same period of 30 days. The business expectation is straightforward: more boosts should create more visibili ## Variance Reduction Below the Randomization Grain DevFeed: [Variance Reduction Below the Randomization Grain]() Original publisher: [Read original article]() Author: Tilman Drerup Published: 2026-07-01T16:28:36Z Content type: article Language: en Sources: [Instacart]() Topics: [experiments]() Tags: [capacity](), [causal-inference](), [economics](), [engineering](), [experiment](), [experimentation](), [experiments](), [machine-learning](), [marketplaces](), [metrics](), [reduce](), [science](), [techniques](), [variance]() ### AI overview This article explains how marketplace experiments can reduce metric variance below the level at which treatment is randomized. It describes cluster-level randomization for containing interference and shows how fine-grained outcome predictability can improve statistical power and reduce experimentation time. ### Source excerpt Sergio Camelo, Caitlin Kearns, Matias Cersosimo, and Tilman Drerup As artificial intelligence increases the velocity of engineering and science teams, experimental throughput is set to become a bottleneck for many product decisions. Many companies can now build faster than they can experiment, with queues of good ideas running the risk of not being tested because of lack of experimental capacity. This problem is particularly severe in marketplaces, where the presence of spillover and cannibalization effects between experimental units requires cluster-level randomization techniques. That randomization, in turn, has the unfortunate tendency to substantially reduce statistical power and slow down experimentation. In this post, we show that the predictability of outcomes at fine grains can be exploited to reduce the variance of aggregate metrics, even when experiments themselves are run at a coarse level. Since statistical power depends on metric variability, this yields considerable reductions in experimentation time. The Interference Problem In marketplace settings, behavior and outcomes for individual participants are inherently intertwined. In a delivery marketplace like Instacart, for example, the dispatch system solves a bipartite matching problem between shoppers and customer orders. Since assignments are global and interdependent, matching an order to one shopper means that the same order cannot be matched to another shopper. As a result, changing the handling for a single order creates ripples that affect the orders around it. If an experimenter were to assign a treatment intervention to one of these orders while leaving neighboring orders as controls, the latter would evidently be contaminated. A common response to this problem is to randomize treatments at the level of a cluster, chosen so that interference can stay within it. In food and grocery delivery, that cluster is typically a geographical region. Since every order within a region sees the same treatme ## The Causality Gap: Measuring the True Impact of Voluntary Adoption in Digital Marketplaces DevFeed: [The Causality Gap: Measuring the True Impact of Voluntary Adoption in Digital Marketplaces]() Original publisher: [Read original article]() Author: Lin Jia Published: 2026-05-22T08:05:43Z Content type: article Language: en Sources: [Booking.com Data Science]() Topics: [A/B Testing](), [doubleml](), [Data Science](), [Tool]() Tags: [a-b-testing](), [causal-inference](), [causal-machine-learning](), [causality](), [data-science](), [doubleml](), [e-commerce](), [experimentation](), [featured](), [measuring]() ### AI overview This article explains why standard A/B tests can mismeasure features that require voluntary adoption in digital marketplaces. It presents Randomized Encouragement Design combined with Double Machine Learning as a way to estimate both causal impact among adopters and overall rollout impact. ### Source excerpt by Lin Jia, Kexin Fei The Content of this post has been presented at Pydata Amsterdam 2026 and the slides can be found here TL; DR Whenever a feature requires voluntary adoption, standard A/B testing breaks: low adoption flattens the topline, and self-selection makes adopters incomparable to non-adopters. Combining Randomized Encouragement Design (RED) with Double Machine Learning (DoubleML) recovers two answers -- the causal lift for adopters and the rollout's overall impact. When they diverge, the gap turns an ambiguous topline into a sharp product decision -- build a better product, or build a better adoption funnel. 1. The Opt-In Barrier Across Demand and Supply Across the tech industry, many platform features rely on voluntary adoption. A customer chooses whether to claim a promotional discount. A traveller opts into a loyalty program. An e-commerce seller enables a smart-pricing tool. In every case, the platform cannot force adoption -- and the feature's true impact becomes hard to measure, on both demand and supply. At Booking.com this challenge spans both sides of the marketplace -- travellers choosing to log in, partners choosing to adopt new features. Unlike a search-ranking change that applies to 100% of traffic, opt-in features create a "trilemma" for Product Data Science: Voluntary Adoption (the "Opt-In" Barrier): Users must actively enable the feature. A standard A/B test cannot separate the product's effect from the motivation that drove users to adopt it. Extreme Heterogeneity: Travellers range from once-a-year holidaymakers to travel agencies booking thousands of nights; partners range from single-apartment hosts to hotel chains. This variance is noise on both sides. Finite Sample Sizes: Opt-in features target a finite sub-segment, so we cannot simply "run the test longer" to gain power. When these stack up, a flat topline can hide a strong product behind a weak adoption funnel. Genuinely strong features get killed, and resources flow into the wrong int ## Scaling Subscriptions at The New York Times with Real-Time Causal Machine Learning DevFeed: [Scaling Subscriptions at The New York Times with Real-Time Causal Machine Learning]() Original publisher: [Read original article]() Author: Rohit Supekar Published: 2025-10-03T15:19:24Z Content type: article Language: en Sources: [New York Times]() Topics: [Machine Learning & Artificial Intelligence](), [Algorithms](), [Optimization](), [real-time]() Tags: [account](), [algorithms](), [causal-inference](), [causal-machine-learning](), [data](), [data-science](), [machine-learning](), [optimization](), [users]() ### AI overview The New York Times describes replacing its Dynamic Meter machine-learning model with real-time algorithms that decide within milliseconds whether users receive access. The system uses causal machine learning and multi-objective optimization to balance subscription, registration, engagement, conversion, and business constraints across its subscription funnel. ### Source excerpt How real-time algorithms and causal ML transformed our digital subscription funnel from static paywalls to dynamic, millisecond decision-makingIllustration by Mathieu Labrecque The New York Times became a subscription-first news and lifestyle service with the launch of its paywall in 2011. Since then, our subscription strategy has evolved substantially. Initially, users could access a limited number of free articles per month before they encountered the paywall. In 2019, we began personalizing this number using a Machine Learning (ML) model -- The Dynamic Meter. In the past few years, we have replaced this model with real-time algorithms that decide, typically within milliseconds, whether to grant access. These algorithms are tailored to balance and optimize the tradeoff between several business Key Performance Indicators (KPIs), while also allowing us the flexibility to adjust for any business constraints. This article further details the motivation behind these algorithms and their design based upon principles from causal machine learning and multi-objective optimization. Our subscription funnel The New York Times has a tiered subscription funnel (Figure 1), consisting of unregistered, registered, and subscribed users. This funnel is designed to provide non-subscribers with limited access to our content, allowing them to discover our offerings. At other times, the content may be blocked by a digital "wall". We have two types of walls -- a registration wall that asks a user to register for a free account or log in, and a paywall that asks a user to subscribe. A large number of users are unregistered -- they may be shown either a registration wall or a paywall. Once a user is in the registered state, they can be shown only a paywall. Figure 1: The New York Times subscription funnelOptimizing the subscription funnel Optimizing who sees the registration wall or the paywall -- and when -- is a very relevant question for our business. While blocking access encourages users t ## Estimating Incremental Lift in Customer Value (Delta CV) using Synthetic Control DevFeed: [Estimating Incremental Lift in Customer Value (Delta CV) using Synthetic Control]() Original publisher: [Read original article]() Author: Mahshid Moha Published: 2025-02-14T20:41:49Z Content type: article Language: en Sources: [PayPal Technology]() Topics: [Data Science](), [Transactions](), [digital wallet]() Tags: [adoption](), [causal-inference](), [customer-value](), [customers](), [data-science](), [digital-wallet](), [margin](), [monetization](), [paypal](), [products](), [revenue](), [synthetic-control](), [transactions]() ### AI overview This article explains PayPal's Delta CV metric, which estimates the incremental profit margin generated during the first year after a customer adopts a product or completes an action. It distinguishes Delta CV from customer lifetime value and describes direct and halo effects on engagement, revenue, and margin. ### Source excerpt How we measure the impact of user actions and product adoptions at PayPal In today's competitive digital landscape, understanding user interactions with your products is essential for driving revenue and building lasting customer relationships. At PayPal, our Data Science teams use causal inference to evaluate the impact of key customer actions, such as adopting a new product or adding a credit card to their wallet, on engagement (measured by Transactions per Account, or TPA), revenue, and margin to help make data-driven strategic decisions. The direct profit from a product adoption or a user action on the app could be $0 if viewed in isolation. However, that does not necessarily mean that these events are not driving engagement and monetization across the PayPal ecosystem; they can change the user engagement with other PayPal products in such a way that the user starts generating more profit. To measure the overall impact of user actions or product adoption, we introduced Delta CV (delta in Customer Value), and we defined it as a customer's incremental profit margin in the first year after adoption of a new product or completing an action. For example, if the average Delta CV for adoption of Crypto is $20, we expect customers who adopt Crypto for the first time to bring an additional $20 in margin on average in the next 12 months after the adoption. We define Delta Revenue (or TPA) in the same manner except we calculate the incremental lift in revenue (or TPA) instead of profit margin. The concept of Delta CV is very different from customer life-time value (CLV) which estimates the total profit generated by a customer over the course of their relationship with PayPal. Delta CV gives us a wholistic view on how new adoptions affect the engagement and value of an existing PayPal user. Adoption of a new product or completing an action can increase the customer's value in a few ways: The product itself may be a profit generating (Direct effect). For example, paying with ## Mindful Experimentation: Evaluate Recommendation System Performance using A/B Testing at Headspace DevFeed: [Mindful Experimentation: Evaluate Recommendation System Performance using A/B Testing at Headspace]() Original publisher: [Read original article]() Author: Headspace Published: 2021-11-29T23:34:41Z Content type: article Language: en Sources: [Stories by Headspace on Medium]() Topics: [recommendation systems](), [A/B Testing](), [data](), [Machine Learning & Artificial Intelligence]() Tags: [a-b-testing](), [causal-inference](), [data-science](), [experimentation](), [headspace](), [machine-learning](), [product-experimentation](), [recommendation-system]() ### AI overview This article describes how Headspace evaluates its personalized Content Customizer recommendation system with online controlled experiments, or A/B tests. It explains why the full recommendation system should be assessed before production and outlines experiment design, including recommendation surfaces, supported platforms, and audience selection. ### Source excerpt Author: Rohan Singh Rajput. Rohan is a Senior Data Scientist at Headspace. He combines his passion for Machine Learning with Causal Inference to improve the mindfulness and meditation practices of Headspace users. "If you can't measure it, you can't improve it." -- Peter Drucker. Motivation Content Customizer is Headspace's personalized recommendation system. Content Customizer uses historical data to train its machine learning model and provide personalized recommendations to the users, helping them discover more relevant content. There are various components involved in building a recommendation system, and the ML model is only one of them. Therefore, it is essential to evaluate the effectiveness of the recommendation system as a whole before deploying it to production. Online Controlled Experiments help us to assess our system's impact with statistical evidence. Online Controlled Experiments, a.k.a A/B testing, are the gold standard for estimating causality with high probability. A data-driven decision-making culture helps estimate the measurement's uncertainty to refute the null hypothesis based on experimental data. Furthermore, a random assignment of the users into a control-treatment group allows us to safely ignore the unobserved factors and model the parameters as random variables1. Experiment Design The following components are required to design the experiment. Recommendation Surface Area: We have a total of three surface areas for this experiment: First, in the Today tab, we will use the last three slots to display ML recommendations. Figure 1: Dynamic Playlist on Today's Tab Second is the Hero module, which is the top banner area of the Meditate/Sleep/Focus/Move tabs. Lastly, we will be using the recommended sub-tabs that also have three slots each for ML-powered content. Figure 2: Hero and Recommended Module of other four tabs In total, we have 19 places available for the experiment. The platform for Recommendation: Headspace serves on multiple platform ## How to Use Quasi-experiments and Counterfactuals to Build Great Products DevFeed: [How to Use Quasi-experiments and Counterfactuals to Build Great Products]() Original publisher: [Read original article]() Author: Antoine Rebecq Published: 2020-09-28T18:29:06Z Content type: article Language: en Sources: [Shopify Engineering](), [Shopify Engineering - Shopify Engineering]() Topics: [Statistics](), [Data Science](), [Shopify]() Tags: [analysis](), [causal-inference](), [experiments](), [statistics]() ### AI overview The Shopify Engineering article explains how A/B tests, quasi-experiments, and counterfactual estimation provide different levels of evidence for causal inference. It emphasizes that stronger causal conclusions require either sound experimental design or substantial statistical analysis and robustness checks. ### Source excerpt A/B tests are not the only tool to understand causality: quasi-experiments and counterfactuals are powerful tools for causal inference if used right.