# The 2026 Flaky-Test Tooling Landscape: An Honest Comparison

DevFeed: [The 2026 Flaky-Test Tooling Landscape: An Honest Comparison](<https://devfeed.tech/articles/the-2026-flaky-test-tooling-landscape-an-honest-comparison-136344.md>)

Original publisher: [Read original article](<https://www.harness.io/blog/flaky-test-tooling-landscape-2026>)

Author: Shibam Dhar

Published: 2026-11-17T00:01:00Z

Content type: comparison

Language: en

Sources: [Harness](<https://devfeed.tech/sources/harness-blog.md>)

Topics: [intermittent failures](<https://devfeed.tech/topics/intermittent-failures.md>), [Test automation](<https://devfeed.tech/topics/test-automation.md>), [CI/CD](<https://devfeed.tech/topics/cicd.md>), [auto rerun failed tests](<https://devfeed.tech/topics/auto-rerun-failed-tests.md>), [CircleCI](<https://devfeed.tech/topics/circleci.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [blog](<https://devfeed.tech/tags/blog.md>), [circleci](<https://devfeed.tech/tags/circleci.md>), [comparison](<https://devfeed.tech/tags/comparison.md>), [datadog](<https://devfeed.tech/tags/datadog.md>), [flaky](<https://devfeed.tech/tags/flaky.md>), [grounded](<https://devfeed.tech/tags/grounded.md>), [harness](<https://devfeed.tech/tags/harness.md>), [independent](<https://devfeed.tech/tags/independent.md>), [research](<https://devfeed.tech/tags/research.md>), [test](<https://devfeed.tech/tags/test.md>), [tooling](<https://devfeed.tech/tags/tooling.md>), [tools](<https://devfeed.tech/tags/tools.md>)

## AI overview

The article finds that flaky-test detection remains incomplete and that tooling differs most in how it handles quarantine, recovery, and adoption. It cites a study in which existing methods missed 9.8%-16.3% of flaky failures, and a Shaker evaluation whose detection rate was not significantly different from ordinary reruns. It contrasts approaches: Harness and Trunk keep quarantined tests running so results remain visible; Buildkite offers a muted state that does this, while skipping stops execution; BuildPulse's API-driven quarantine skips tests and requires manual retesting; CircleCI reruns failures but does not provide a recovery lifecycle; Mergify can rerun new or changed tests on pull requests; Datadog tracks test states but advanced lifecycle actions require native instrumentation and minimum library versions. The article advises measuring test time separately from pipeline time, fixing determinism and ingestion before optimizing execution, and evaluating tools on representative tests with comparable budgets and production-like environments. It notes there is no independent, equivalent benchmark across all seven products.

## Source excerpt

Seven flaky-test tools compared in 2026, grounded in independent research. What Harness, Buildkite, CircleCI, Trunk, BuildPulse, Mergify and Datadog really do. | Blog