# Tests that sometimes fail - flaky test tips

DevFeed: [Tests that sometimes fail - flaky test tips](<https://devfeed.tech/articles/tests-that-sometimes-fail-flaky-test-tips-22434.md>)

Original publisher: [Read original article](<https://samsaffron.com/archive/2019/05/15/tests-that-sometimes-fail>)

Author: Sam Saffron

Published: 2019-05-28T06:54:25Z

Content type: article

Language: en

Sources: [Sam Saffron](<https://devfeed.tech/sources/sam-saffron.md>)

Topics: [Development](<https://devfeed.tech/topics/development.md>), [debug](<https://devfeed.tech/topics/debug.md>), [CI/CD](<https://devfeed.tech/topics/cicd.md>), [Continuous Deployment (CD)](<https://devfeed.tech/topics/continuous-deployment.md>)

Tags: [blog](<https://devfeed.tech/tags/blog.md>), [c-sharp](<https://devfeed.tech/tags/c-sharp.md>), [continuous-deployment](<https://devfeed.tech/tags/continuous-deployment.md>), [debug](<https://devfeed.tech/tags/debug.md>), [development](<https://devfeed.tech/tags/development.md>), [flaky](<https://devfeed.tech/tags/flaky.md>), [infection](<https://devfeed.tech/tags/infection.md>), [net](<https://devfeed.tech/tags/net.md>), [pipeline](<https://devfeed.tech/tags/pipeline.md>), [rails](<https://devfeed.tech/tags/rails.md>), [ruby](<https://devfeed.tech/tags/ruby.md>), [saffron](<https://devfeed.tech/tags/saffron.md>), [sam](<https://devfeed.tech/tags/sam.md>), [sql](<https://devfeed.tech/tags/sql.md>), [tests](<https://devfeed.tech/tags/tests.md>), [tips](<https://devfeed.tech/tags/tips.md>), [weblog](<https://devfeed.tech/tags/weblog.md>)

## AI overview

An article about flaky tests in mature projects, explaining how they can consume debugging time, disrupt continuous deployment, and sometimes reveal underlying application flaws. It describes Discourse's practices for identifying and mitigating them.

## Source excerpt

The boy who cried wolf695x564 286 KB A liar will not be believed, even when he speaks the truth. : Aesop Once you have a project that is a few years old with a large test suite an ugly pattern emerges. Some tests that used to always work, start "sometimes" working. This starts slowly, "oh that test, yeah it sometimes fails, kick the build off again". If left unmitigated it can very quickly snowball and paralyze an entire test suite. Most developers know about this problem and call these tests "non deterministic tests", "flaky tests","random tests", "erratic tests", "brittle tests", "flickering tests" or even "heisentests". Naming is hard, it seems that this toxic pattern does not have a well established unique and standard name. Over the years at Discourse we have called this many things, for the purpose of this article I will call them flaky tests, it seems to be the most commonly adopted name. Much has been written about why flaky tests are a problem. Martin Fowler back in 2011 wrote: Non-deterministic tests have two problems, firstly they are useless, secondly they are a virulent infection that can completely ruin your entire test suite. To this I would like to add that flaky tests are an incredible cost to businesses. They are very expensive to repair often requiring hours or even days to debug and they jam the continuous deployment pipeline making shipping features slower. I would like to disagree a bit with Martin. Sometimes I find flaky tests are useful at finding underlying flaws in our application. In some cases when fixing a flaky test, the fix is in the app, not in the test. In this article I would like to talk about patterns we observed at Discourse and mitigation strategies we have adopted. Patterns that have emerged at Discourse A few months back we introduced a game. We created a topic on our development Discourse instance. Each time the test suite failed due to a flaky test we would assign the topic to the developer who originally wrote the test. Onc