# Two ways to measure the cumulative impact of experiments

DevFeed: [Two ways to measure the cumulative impact of experiments](<https://devfeed.tech/articles/two-ways-to-measure-the-cumulative-impact-of-experiments-2316.md>)

Original publisher: [Read original article](<https://www.datadoghq.com/blog/two-ways-to-measure-cumulative-impact/>)

Author: Lukas Goetz-Weiss; Eddie Cai

Published: 2026-08-18T00:00:00Z

Content type: opinion

Language: en

Sources: [Datadog | The Monitor blog](<https://devfeed.tech/sources/datadog-the-monitor-blog.md>)

Topics: [experiments](<https://devfeed.tech/topics/experiments.md>), [Statistics](<https://devfeed.tech/topics/statistics.md>), [Data Science](<https://devfeed.tech/topics/data-science.md>)

Tags: [analysis](<https://devfeed.tech/tags/analysis.md>), [experimentation](<https://devfeed.tech/tags/experimentation.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [product](<https://devfeed.tech/tags/product.md>), [statistics](<https://devfeed.tech/tags/statistics.md>), [testing](<https://devfeed.tech/tags/testing.md>)

## AI overview

This article explains why summing the observed lift from winning experiments overstates cumulative impact because of the winner's curse. It presents two established approaches: a randomized holdout that measures the combined effect directly, and a statistical correction that aggregates existing experiment estimates. Datadog's Cumulative Impact feature applies the correction without requiring a quarter-long holdout and can analyze an entire experimentation program or a filtered team subset.

## Source excerpt

Summing individual wins overstates true impact. See two accurate methods, holdouts and Datadog's Cumulative Impact, and how to choose between them.