# How We Handle Flaky Tests in Gradle

DevFeed: [How We Handle Flaky Tests in Gradle](<https://devfeed.tech/articles/how-we-handle-flaky-tests-in-gradle-24645.md>)

Original publisher: [Read original article](<https://blog.gradle.org/how-we-handle-flaky-tests-in-gradle>)

Author: Bo Zhang

Published: 2022-11-03T02:00:00Z

Content type: article

Language: en

Sources: [The Gradle Blog](<https://devfeed.tech/sources/the-gradle-blog.md>)

Topics: [Gradle](<https://devfeed.tech/topics/gradle.md>), [Testing](<https://devfeed.tech/topics/testing.md>), [ci](<https://devfeed.tech/topics/ci.md>), [code productivity](<https://devfeed.tech/topics/code-productivity.md>), [superProductivity](<https://devfeed.tech/topics/superproductivity.md>)

Tags: [automated](<https://devfeed.tech/tags/automated.md>), [bazel](<https://devfeed.tech/tags/bazel.md>), [best-practices](<https://devfeed.tech/tags/best-practices.md>), [build](<https://devfeed.tech/tags/build.md>), [ci](<https://devfeed.tech/tags/ci.md>), [flaky](<https://devfeed.tech/tags/flaky.md>), [gradle](<https://devfeed.tech/tags/gradle.md>), [gradle-plugin](<https://devfeed.tech/tags/gradle-plugin.md>), [junit](<https://devfeed.tech/tags/junit.md>), [retry](<https://devfeed.tech/tags/retry.md>), [teamcity](<https://devfeed.tech/tags/teamcity.md>), [techniques](<https://devfeed.tech/tags/techniques.md>), [testing](<https://devfeed.tech/tags/testing.md>), [tests](<https://devfeed.tech/tags/tests.md>), [tools](<https://devfeed.tech/tags/tools.md>)

## AI overview

This article explains how the Gradle team manages flaky automated tests. It covers retrying failed tests, automatically identifying flaky failures in CI, using dashboards and analytics to troubleshoot trends, and quarantining tests that are too unreliable. It also mentions Gradle's Test Retry Gradle plugin and alternatives such as Maven Surefire Plugin, JUnit Pioneer, and Bazel test rules.

## Source excerpt

Test flakiness is one of the main challenges of automated testing. Even though Gradle's focus is increasing developer productivity, the development of Gradle itself suffers from flaky automated tests. This blog explains some best practices when developing Gradle, which have proved effective over the years we fight with flaky tests. The Story Like in many other projects, every commit of Gradle must pass tens of thousands of automated tests. Any tiny flakiness may cause developer productivity loss. When I joined Gradle 5 years ago, the CI was full of flaky test failures - people would rerun a build, again and again, hoping to be lucky enough to get a green build. Later, we started a dedicated developer productivity team to deal with all the flakiness on CI, especially test flakiness. Here's how we do it step by step. Retry Failed Tests When a test fails, how do we determine if it's flaky or not? The easiest way is obviously retrying the failed test immediately: if the second run succeeds, the failed test is flaky. As a rule of thumb, by simply rerunning the failed test one more time, the test failures due to flakiness can be reduced by 90%. Many CI systems can recognize such flaky tests and mark the build as green automatically: In this example, the first run fails because of a network error, but the rerun succeeds. TeamCity recognizes this situation and "mutes" the test failure. If the build is connected to a Gradle Enterprise instance and that build has a published Build Scan, you can see the flaky tests in the test dashboard: In Gradle, we automatically retry the failed test classes with the Test Retry Gradle plugin. Please consult the documentation for how to adopt it in your build. Other Techniques to Retry Failed Tests If you are not using Gradle, that's okay. There are many alternatives that provide similar functionalities: Maven Surefire Plugin - Rerun failing tests JUnit Pioneer - Retrying Failing Tests Bazel flaky attribute in test rules All the tools above