# Flaky Published articles for Flaky. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Making testing work better for AI driven development DevFeed: [Making testing work better for AI driven development]() Original publisher: [Read original article]() Author: Pete Miloravac Published: 2026-09-10T09:28:14Z Content type: release Language: en Sources: [Semaphore Engineering]() Topics: [Testing](), [Development](), [Artificial Intelligence](), [Test coverage](), [RSpec]() Tags: [ai](), [flaky](), [framework](), [product-news](), [test-coverage](), [testing](), [xunit]() ### AI overview Semaphore describes product work to make testing more useful for developers and AI agents. The update covers richer test reports from testing frameworks, signal-based flaky-test detection, and code-graph-based selection of tests affected by code changes. ### Source excerpt AI agents can generate and modify code quickly. But when a test fails, speed is no longer enough. The agent needs enough context to understand what happened, why it happened, and whether the code or the test itself needs to change. In our latest product update, Marko Gaćeša is joined by Semaphore engineers Nick and [...] The post Making testing work better for AI driven development appeared first on Semaphore. ## CircleCI Smarter Testing: Stop running tests that don't matter DevFeed: [CircleCI Smarter Testing: Stop running tests that don't matter]() Original publisher: [Read original article]() Author: Nathan Fish Published: 2026-08-26T19:00:00Z Content type: article Language: en Sources: [The CircleCI Blog Feed | CircleCI]() Topics: [Testing](), [CI/CD]() Tags: [accelerate](), [auto-rerun-failed-tests](), [ci-cd](), [circleci](), [circleci-news](), [developer-productivity](), [dynamic-test-splitting](), [engineering-productivity](), [flaky](), [intelligent-test-selection](), [parallel](), [pipeline](), [quality](), [reduce](), [smarter-testing](), [test-impact-analysis](), [testing](), [tests](), [time]() ### AI overview CircleCI describes Smarter Testing, a set of features designed to reduce CI/CD test execution time by skipping tests unaffected by changes, balancing parallel nodes, and retrying flaky tests. The article says early users have seen test runs up to four times faster. ### Source excerpt Testing eats up to half your pipeline time. See how CircleCI Smarter Testing skips unaffected tests, rebalances parallel nodes, and retries flaky tests. ## Handling Application States and Retries in Cypress End-to-End Tests DevFeed: [Handling Application States and Retries in Cypress End-to-End Tests]() Original publisher: [Read original article]() Author: Gleb Bahmutov Published: 2026-07-28T04:00:00Z Content type: tutorial Language: en Sources: [Gleb Bahmutov]() Topics: [Cypress](), [test](), [App]() Tags: [api](), [can](), [cypress](), [errors](), [flaky](), [plugin](), [products](), [retry](), [test](), [testing](), [tests]() ### AI overview This tutorial explains how to design Cypress end-to-end tests for loading, empty, and error states. It recommends detecting known error states immediately and using in-place retries with cypress-recurse when appropriate, instead of relying only on whole-test retries. ### Source excerpt Imagine a typical application that loads some data. The app starts in the initial state, starts loading, then one of 3 possible states ar ## Cypress Ambassador Spotlight: Boris Selivanov DevFeed: [Cypress Ambassador Spotlight: Boris Selivanov]() Original publisher: [Read original article]() Published: 2026-07-24T13:43:32Z Content type: opinion Language: en Sources: [Cypress Blog]() Topics: [Cypress](), [Test automation](), [Testing](), [selectors](), [race-condition](), [Network](), [npm]() Tags: [ambassador](), [automation](), [cypress](), [flaky](), [how-to](), [npm](), [race-condition](), [selectors](), [testing]() ### AI overview This Cypress Ambassador Spotlight presents Boris Selivanov's advice on sharing reusable testing patterns, choosing stable selectors, and building scalable test automation. It highlights cy.intercept() for controlling network traffic and reducing flaky tests, recommends retry-ability and aliases instead of arbitrary cy.wait(ms), and suggests starting with a real application test after installing Cypress. ### Source excerpt What motivates you to be a Cypress Ambassador? I see being an Ambassador as the ultimate form of asynchronous collaboration. My goal is to share things that move the needle for other engineers. By comparing how different teams solve the same testing problems, we can turn isolated project victories into shared wins. How do you currently help others that are either using Cypress or interested in using Cypress? I'm the person teammates ping when a test goes flaky or they're not sure how to appro ## Cypress Ambassador Spotlight: Javier Flores DevFeed: [Cypress Ambassador Spotlight: Javier Flores]() Original publisher: [Read original article]() Published: 2026-07-16T21:08:08Z Content type: opinion Language: en Sources: [Cypress Blog]() Topics: [Cypress](), [Testing](), [Cucumber](), [Bootcamp](), [selectors](), [debugging](), [Automation]() Tags: [ambassador](), [automation](), [community](), [cypress](), [debugging](), [flaky](), [learning](), [projects](), [selectors](), [testing]() ### AI overview This interview highlights Javier Flores's work as a Cypress Ambassador, QA Automation Lead, mentor, and bootcamp instructor. It discusses practical Cypress testing, Cucumber exercises, interactive time-travel debugging, automatic waiting, deterministic tests, stable selectors, and beginner-friendly end-to-end testing practices. ### Source excerpt What motivates you to be a Cypress Ambassador? I want to help grow a strong, practical testing community around Cypress. I show people working code and let them build confidence by doing. That's the same philosophy behind the bootcamp I run: everyone leaves with their own repo, not just notes. How do you currently help others that are either using Cypress or interested in using Cypress? I mentor teammates and collaborate closely with developers and product owners to improve testing practices ## Endform joins the Vercel Marketplace DevFeed: [Endform joins the Vercel Marketplace]() Original publisher: [Read original article]() Author: Sam Halstead Published: 2026-07-14T00:00:00Z Content type: release Language: en Sources: [Vercel News]() Topics: [Vercel](), [Playwright](), [Command-line interface](), [Traces](), [Deployment](), [dashboards]() Tags: [cli](), [config](), [deployment](), [flaky](), [traces](), [vercel]() ### AI overview Endform is now available through the Vercel Marketplace and Vercel CLI. It runs Playwright tests in parallel on isolated machines, integrates with existing tests without configuration changes, links deployment checks to traces in its dashboard, and identifies flaky tests from historical results. ### Source excerpt Endform is now available on the Vercel Marketplace. It runs your Playwright tests in parallel, so your suite finishes in the time of your slowest test. Key capabilities: Run every test on its own isolated machine Bring your existing Playwright tests with no config changes Get a pass/fail check on every production and preview deployment, linked to full traces in the Endform dashboard Spot flaky tests by tracking results across historical runs Pay only for the test runtime you use Install Endform on the Vercel Marketplace or with the Vercel CLI: vc i endform. Read more ## Flaky Test API Now GA, New Auto-Fix Skill, Skill Quality Improvements DevFeed: [Flaky Test API Now GA, New Auto-Fix Skill, Skill Quality Improvements]() Original publisher: [Read original article]() Author: Pete Miloravac Published: 2026-06-19T11:00:00Z Content type: release Language: en Sources: [Semaphore Engineering]() Topics: [API](), [AI Agent](), [LLM evaluation / benchmarking](), [Claude](), [data]() Tags: [agent](), [ai](), [api](), [audio](), [cost](), [data](), [flaky](), [improvements](), [product-news](), [quality](), [release](), [validation](), [verify](), [video]() ### AI overview Semaphore announces generally available flaky-test data through its sem-ai API, a skill that lets an AI agent investigate and fix flaky tests, and improvements to existing skills through added examples. The article also reports typical fix costs of $1 to $1.50 when benchmarking with Claude Opus 4.8 on high effort. ### Source excerpt You can now identify your flakiest tests through the API and have an AI agent fix them, at a cost of around $1 to $1.50 per fix. What Shipped Flaky Test Data in the API (Generally Available) Flaky test data is now accessible programmatically through the sem-ai API. The API surfaces tests ranked by disruption [...] The post Flaky Test API Now GA, New Auto-Fix Skill, Skill Quality Improvements appeared first on Semaphore. ## Codex Support, Faster Task Creation, and Flaky Test Visibility DevFeed: [Codex Support, Faster Task Creation, and Flaky Test Visibility]() Original publisher: [Read original article]() Author: Pete Miloravac Published: 2026-06-04T14:43:11Z Content type: article Language: en Sources: [Semaphore Engineering]() Topics: [codex](), [ai-coding](), [Command-line interface](), [ci](), [Claude Code]() Tags: [ai-agents](), [ai-coding](), [audio](), [ci](), [cli](), [codex](), [flaky](), [product-news](), [video](), [visibility]() ### AI overview Semaphore's update covers fixes to sem-ai, faster and more reliable task creation, CLI-based task management, and a Semaphore Cloud workflow-editor option for showing skipped blocks. Planned work includes broader platform management through the CLI, flaky-test data through the API, and AI-assisted onboarding for tools including Claude Code and Codex. ### Source excerpt sem-ai keeps moving. Here's a quick rundown of what shipped last week and what's coming next. What Shipped on Sem-AI Glitches have been fixed.A few issues were blocking some users to get started with sem-ai. Those are resolved. If you tried it before and ran into friction, now's a good time to try again. See [...] The post Codex Support, Faster Task Creation, and Flaky Test Visibility appeared first on Semaphore. ## hh.ru's Test Framework Experiments for Handling Flaky Tests DevFeed: [hh.ru's Test Framework Experiments for Handling Flaky Tests]() Original publisher: [Read original article]() Author: yzhanov (hh.ru) Published: 2026-05-07T07:10:11Z Content type: article Language: ru Sources: [HeadHunter RU]() Topics: [Jenkins](), [Job](), [ui]() Tags: [flaky](), [hh](), [hh-ru](), [jenkins](), [job](), [qa](), [qa-automation](), [qa-automation-engineer](), [qa-engineer](), [qa-management](), [qa-mobile](), [qa-testing](), [retry](), [tag-4adf7d504f34](), [tag-95a52fc82d81](), [tag-b7f0358ac141](), [tag-e34f6dec12c4](), [tag-e932065bc8da](), [ui]() ### AI overview An hh.ru engineer describes experiments in the test framework to reduce the impact of flaky tests on release automation. One experiment allowed repeated retries until the queue emptied; it added about three minutes to build time and was not adopted in production. ### Source excerpt Привет, Хабр! Меня зовут Юра Жанов, я занимаюсь автоматизацией тестирования в hh.ru. Про flaky-тесты написано много, борьбу с ними не прекращаем и мы. Но сегодня немного о другом -- хочу поделиться нашим опытом минимизации неприятностей, которые наносят такие тесты. Для этого мы провели ряд экспериментов со стороны тестового фреймворка. Читать далее ## A better way to end-to-end test your webapp DevFeed: [A better way to end-to-end test your webapp]() Original publisher: [Read original article]() Author: hi@swizec.com (Swizec Teller) Published: 2026-01-06T00:00:00Z Content type: opinion Language: en Sources: [Swizec Teller]() Topics: [Testing](), [Web app](), [computer-use](), [Playwright](), [OpenAI](), [TypeScript]() Tags: [agentic](), [computer-use](), [e2e](), [e2e-testing](), [flaky](), [mocking](), [open-source](), [openai](), [playwright](), [tests]() ### AI overview The article proposes an agentic approach to end-to-end testing for web applications. Tests describe user goals, while an agent uses screenshots and browser interactions to achieve and record those goals. Later runs replay the stored actions to check for regressions, reducing test-writing overhead and sensitivity to UI implementation changes. ### Source excerpt End-to-end tests are the best way to test your webapp, but they're flaky and annoying to write. I'm trying a new agentic approach that fixes the issue. ## Introducing Self-Healing CI for Nx and Nx Cloud DevFeed: [Introducing Self-Healing CI for Nx and Nx Cloud]() Original publisher: [Read original article]() Author: Juri Strumpflohner Published: 2025-06-23T00:00:00Z Content type: release Language: en Sources: [Juri Strumpflohner]() Topics: [ci](), [AI Agent](), [Cloud]() Tags: [agents](), [ai](), [ai-agents](), [bugs](), [ci](), [code](), [configuration](), [development](), [flaky](), [nx](), [nx-cloud](), [self-healing](), [teams](), [test](), [tests](), [validation](), [workflow]() ### AI overview Nx introduces Self-Healing CI for Nx Cloud, using AI agents to detect, analyze, fix, and validate CI failures while keeping developers in control through review and approval. The system complements existing flaky-task retries by addressing failures that require code, configuration, or dependency fixes. ### Source excerpt Introducing Nx Cloud Self-Healing CI: AI agents that automatically detect, analyze, and fix your CI failures so you do not have to babysit PRs. ## Introducing Roast: Structured AI workflows made easy DevFeed: [Introducing Roast: Structured AI workflows made easy]() Original publisher: [Read original article]() Author: Obie Fernandez Published: 2025-06-18T07:00:00Z Content type: article Language: en Sources: [Shopify Engineering](), [Shopify Engineering - Shopify Engineering]() Topics: [AI Bots](), [CI/CD]() Tags: [agentic](), [agentic-ai](), [agents](), [ai](), [ai-agents](), [ci](), [ci-cd](), [code-quality](), [command-line](), [developer](), [developer-experience](), [developer-productivity](), [documentation](), [flaky](), [framework](), [orchestration](), [ruby](), [test-coverage](), [tool](), [workflows]() ### AI overview Shopify introduces Roast, an open-source framework for building structured, reproducible AI workflows. It combines declarative YAML configuration and Markdown prompts with conventional code execution to make multi-step AI processes more reliable for developer productivity tasks such as test assessment and code-quality analysis. ### Source excerpt Our Augmented Engineering DX team tackles developer productivity challenges like flaky tests and low test coverage using AI agents. We discovered breaking complex tasks into discrete steps was key to reliable AI performance, leading to the creation and open-sourcing of Roast, a tool designed to structure AI workflows effectively. ## What's new in Firebase at Cloud Next 2025 DevFeed: [What's new in Firebase at Cloud Next 2025]() Original publisher: [Read original article]() Author: Kristen Richards; Andrea Wu Published: 2025-04-09T06:07:00Z Content type: news Language: en Sources: [Firebase Blog]() Topics: [Firebase](), [Artificial Intelligence](), [Cloud](), [Genkit](), [Next.js](), [Testing]() Tags: [2025](), [ai](), [ai-logic](), [android](), [angular](), [app-distribution](), [app-hosting](), [cloud-next](), [firebase](), [firebase-studio](), [flaky](), [gemini](), [genkit](), [go](), [graphql](), [ios](), [launch](), [next-js](), [postgresql](), [python](), [react](), [react-native](), [sql-connect](), [updates](), [vertex-ai]() ### AI overview Firebase announces product updates at Cloud Next 2025, including Firebase Studio, an App Testing agent, expanded Genkit language support, Vertex AI integrations, and general availability for Data Connect and App Hosting. ### Source excerpt News, tutorials, and updates from the Firebase team. ## Recent improvements to the Maestro console DevFeed: [Recent improvements to the Maestro console]() Original publisher: [Read original article]() Author: Dan Caseley Published: 2024-12-19T08:00:00Z Content type: release Language: en Sources: [mobile.dev - Medium]() Topics: [Testing](), [Playback](), [Resilience]() Tags: [flaky](), [improvements](), [junit](), [maestro](), [playback](), [resilience](), [screenshots](), [storage](), [testing]() ### AI overview Maestro describes recent improvements to its console, including clearer playback steps, inline videos, flow run history, and fixes for JUnit reporting, video recording, and resilience when uploading screenshots and videos. ### Source excerpt Howdy, testers! We're excited to share a quick update on changes we've made to the Maestro console over the last few weeks. There's a bunch you'll have noticed already, but to m... ## SMURF: Beyond the Test Pyramid DevFeed: [SMURF: Beyond the Test Pyramid]() Original publisher: [Read original article]() Author: Google Testing Bloggers (noreply@blogger.com) Published: 2024-10-15T12:22:00Z Content type: tutorial Language: en Sources: [Google Testing Blog]() Topics: [Testing](), [Mocking]() Tags: [adam-bender](), [article](), [dependency](), [flaky](), [google](), [integration](), [maintainability](), [memory](), [production](), [resources](), [speed](), [testing](), [tests](), [tott](), [unit-tests]() ### AI overview This Google Testing on the Toilet article explains why the conventional test pyramid is insufficient for growing test suites. It introduces the SMURF mnemonic--Speed, Maintainability, Utilization, Reliability, and Fidelity--as a framework for balancing testing trade-offs. ### Source excerpt This article was adapted from a Google Testing on the Toilet (TotT) episode. You can download a printer-friendly version of this TotT episode and post it in your office. By Adam Bender The test pyramid is the canonical heuristic for guiding test suite evolution. It conveys a simple message - prefer more unit tests than integration tests, and prefer more integration tests than end-to-end tests. While useful, the test pyramid lacks the details you need as your test suite grows and you face challenging trade-offs. To scale your test suite, go beyond the test pyramid. The SMURF mnemonic is an easy way to remember the tradeoffs to consider when balancing your test suite: Speed: Unit tests are faster than other test types and can be run more often--you'll catch problems sooner. Maintainability: The aggregated cost of debugging and maintaining tests (of all types) adds up quickly. A larger system under test has more code, and thus greater exposure to dependency churn and requirement drift which, in turn, creates more maintenance work. Utilization: Tests that use fewer resources (memory, disk, CPU) cost less to run. A good test suite optimizes resource utilization so that it does not grow super-linearly with the number of tests. Unit tests usually have better utilization characteristics, often because they use test doubles or only involve limited parts of a system. Reliability: Reliable tests only fail when an actual problem has been discovered. Sorting through flaky tests for problems wastes developer time and costs resources in rerunning the tests. As the size of a system and its corresponding tests grow, non-determinism (and thus, flakiness) creeps in, and your test suite is more likely to become unreliable. Fidelity: High-fidelity tests come closer to approximating real operating conditions (e.g., real databases or traffic loads) and better predict the behavior of our production systems. Integration and end-to-end tests can better reflect realistic conditions, while unit ## Data quality testing DevFeed: [Data quality testing]() Original publisher: [Read original article]() Author: Lambert Le Manh Published: 2024-09-04T16:30:00Z Content type: article Language: en Sources: [The incident.io Blog]() Topics: [Data Quality](), [data observability](), [DataOps](), [ci](), [Transactions]() Tags: [ci](), [data-observability](), [data-quality](), [flaky](), [incident](), [incident-channel](), [incident-management](), [incident-response](), [observability](), [outage](), [post-mortem](), [slack-incident](), [testing](), [transactions](), [workflow]() ### AI overview The article explains how incident.io uses dbt's native testing features for data quality testing and data observability. It describes integrating validation into production and CI workflows, including custom tests and relationship tests, and addresses flaky failures caused by ingestion and transformation pipelines running at different times. ### Source excerpt Our data observability workflow uses data quality testing to ensure data meets accuracy, consistency, and reliability standards, enabling confident, data-driven decisions. See how we built it, the common challenges we encountered, and the solutions. ## Coroutine Testing - Picking the right Dispatcher DevFeed: [Coroutine Testing - Picking the right Dispatcher]() Original publisher: [Read original article]() Author: Kaushik Gopal Published: 2024-08-25T07:00:45Z Content type: tutorial Language: en Sources: [Kaushik Gopal's Site]() Topics: [Coroutines](), [Testing](), [test](), [Code]() Tags: [coroutine-testing](), [dispatcher](), [flaky](), [scope](), [strategy](), [testing]() ### AI overview This tutorial explains how CoroutineContext, CoroutineScope, and Dispatchers affect coroutine tests. It recommends explicitly injecting a CoroutineScope and replacing it with TestScope to control dispatchers, reduce flakiness, and make tests run faster. ### Source excerpt series This is part of a series of posts on Coroutine Testing: Picking the right Dispatcher <- Never ending tests & backgroundscope Controlling time Helpful @Junit TestRule extension (coming soon) Full USF example for Android (coming soon) Most of the problems and flakiness around coroutine testing stem from running them on different Dispatchers. This is because the choice of Dispatcher can significantly impact the behavior of coroutines. This was also the most confusing1 part for me starting out -- understanding the implications of using a Scope, Context or Dispatcher. I recommend Roman's article if you want to brush up on the fundamentals. But in a nutshell: think of CoroutineContext as a collection of elements that define the coroutine. It contains a Dispatcher, Job & a CoroutineName. When you launch a coroutine, it inherits the parent's CoroutineContext (and Dispatcher), unless you specify it explicitly. A CoroutineScope on the other hand is just a way to manage and cancel (multiple) coroutines. It also defines a context and lifecycle for the coroutines launched within it (the context could be linked to yet another Dispatcher). Any coroutine when launched, runs within a CoroutineScope. Let's take an example: Notice how the current Dispatcher of the coroutine shifts from StandardTestDispatcher -> UnconfinedTestDispatcher -> Dispatcher.IO in the span of three innocuous lines based on the coroutine builder (runTest) or scope used (TestScope, turbineScope from the 3rd party library, App scope). In my initial post I pointed out this flaky test: flaky test code on github The fix for this is as simple as explicitly injecting a TestScope and making sure the same scope is used throughout. fixed test code on github Explicitly injecting the CoroutineScope and substituting it with the TestScope works really well and is my preferred strategy. This approach allows for more control over the Dispatcher used in tests . For reasons you'll see later, these tests also run instantly (72 ## Coroutine Testing DevFeed: [Coroutine Testing]() Original publisher: [Read original article]() Author: Kaushik Gopal Published: 2024-08-25T07:00:45Z Content type: tutorial Language: en Sources: [Kaushik Gopal's Site]() Topics: [Coroutines](), [Testing](), [Kotlin](), [Concurrency](), [LineageOS]() Tags: [android](), [concurrency](), [coroutine](), [coroutine-testing](), [coroutines](), [dispatcher](), [flaky](), [junit](), [kotlin](), [migration-guide](), [testing](), [time](), [tutorials]() ### AI overview A tutorial series on testing Kotlin coroutines in Android applications. It discusses changes to Kotlin testing APIs, unreliable virtual-time advancement, flaky tests, dispatchers, background scopes, and controlling time. ### Source excerpt When the #androiddevs transitioned from Rx to coroutines the topic of testing didn't get as much attention in this new world of concurrency. It didn't help that there was a seismic change in Kotlin's testing apis with 1.6.0. A whole bunch of online resources and tutorials are now defunct courtesy this change.1 My journey in the matter started because I simply couldn't understand why test apis like advanceTimeBy wouldn't work reliably for me. The name made sense... but my time wasn't being advanced in any meaningful way. Then there's the issue of flaky tests. Here's an example: flaky test code on github Run each test individually and it will pass; run them together as one test suite and test2 alone will fail. test1 passes but it takes a full 3s to run the test. If I have 300 of these in my app, are my tests going to take 15 minutes to run? I needed to understand many core concepts in order to confidently explain all the above phenomena. I'd like to share my learnings from going down the rabbit hole, in this series of posts: series This is part of a series of posts on Coroutine Testing: <- Picking the right Dispatcher Never ending tests & backgroundscope Controlling time Helpful @Junit TestRule extension (coming soon) Full USF example for Android (coming soon) If you're looking for the most current and useful resources on coroutine testing today: Untangling Coroutine Testing - Marton Braun jetbrains official docs developer.android.com docs 1.6.0 Coroutines test migration guide ↩︎ ## Node.js 21.7.1 (Current) DevFeed: [Node.js 21.7.1 (Current)]() Original publisher: [Read original article]() Published: 2024-03-08T21:54:44Z Content type: release Language: en Sources: [Node.js Blog]() Topics: [Node.js]() Tags: [benchmark](), [flaky](), [macos](), [node-js](), [release](), [update](), [windows]() ### AI overview Node.js 21.7.1 is a current release that reverts a test-runner change so hooks run even when a test has no subtests. It also includes build, dependency, documentation, networking, stream, benchmark, and test updates. ### Source excerpt Node.js® is a free, open-source, cross-platform JavaScript runtime environment that lets developers create servers, web apps, command line tools and scripts. ## Retry flaky tests with dotnet test and PowerShell DevFeed: [Retry flaky tests with dotnet test and PowerShell]() Original publisher: [Read original article]() Author: {"twitter"=\>"hlaueriksson"} Published: 2023-03-26T19:00:00Z Content type: tutorial Language: en Sources: [Henrik Lau Eriksson]() Topics: [PowerShell](), [.NET](), [Command-line interface](), [Smoke Tests](), [Playwright](), [Selenium]() Tags: [c-sharp](), [cli](), [configuration](), [dotnet](), [flaky](), [jenkins](), [mstest](), [nunit](), [playwright](), [powershell](), [retry](), [selenium](), [test](), [testing](), [unit-tests](), [xunit]() ### AI overview This tutorial presents a PowerShell script that uses dotnet test and TRX result files to track failed tests, retry them, report the final result, and allow a configured percentage of failures. It demonstrates the script with NUnit, MSTest, and xUnit. ### Source excerpt Introducing test.ps1, a script for running flaky tests - Keeps track of failing tests and retries them - Notifies the test framework of the current retry iteration - Makes it possible to accept a certain percentage of failing tests - Outputs the test result in a coherent way ## How We Handle Flaky Tests in Gradle DevFeed: [How We Handle Flaky Tests in Gradle]() Original publisher: [Read original article]() Author: Bo Zhang Published: 2022-11-03T02:00:00Z Content type: article Language: en Sources: [The Gradle Blog]() Topics: [Gradle](), [Testing](), [ci](), [code productivity](), [superProductivity]() Tags: [automated](), [bazel](), [best-practices](), [build](), [ci](), [flaky](), [gradle](), [gradle-plugin](), [junit](), [retry](), [teamcity](), [techniques](), [testing](), [tests](), [tools]() ### AI overview This article explains how the Gradle team manages flaky automated tests. It covers retrying failed tests, automatically identifying flaky failures in CI, using dashboards and analytics to troubleshoot trends, and quarantining tests that are too unreliable. It also mentions Gradle's Test Retry Gradle plugin and alternatives such as Maven Surefire Plugin, JUnit Pioneer, and Bazel test rules. ### Source excerpt Test flakiness is one of the main challenges of automated testing. Even though Gradle's focus is increasing developer productivity, the development of Gradle itself suffers from flaky automated tests. This blog explains some best practices when developing Gradle, which have proved effective over the years we fight with flaky tests. The Story Like in many other projects, every commit of Gradle must pass tens of thousands of automated tests. Any tiny flakiness may cause developer productivity loss. When I joined Gradle 5 years ago, the CI was full of flaky test failures - people would rerun a build, again and again, hoping to be lucky enough to get a green build. Later, we started a dedicated developer productivity team to deal with all the flakiness on CI, especially test flakiness. Here's how we do it step by step. Retry Failed Tests When a test fails, how do we determine if it's flaky or not? The easiest way is obviously retrying the failed test immediately: if the second run succeeds, the failed test is flaky. As a rule of thumb, by simply rerunning the failed test one more time, the test failures due to flakiness can be reduced by 90%. Many CI systems can recognize such flaky tests and mark the build as green automatically: In this example, the first run fails because of a network error, but the rerun succeeds. TeamCity recognizes this situation and "mutes" the test failure. If the build is connected to a Gradle Enterprise instance and that build has a published Build Scan, you can see the flaky tests in the test dashboard: In Gradle, we automatically retry the failed test classes with the Test Retry Gradle plugin. Please consult the documentation for how to adopt it in your build. Other Techniques to Retry Failed Tests If you are not using Gradle, that's okay. There are many alternatives that provide similar functionalities: Maven Surefire Plugin - Rerun failing tests JUnit Pioneer - Retrying Failing Tests Bazel flaky attribute in test rules All the tools above ## Tests Under the Magnifying Lens DevFeed: [Tests Under the Magnifying Lens]() Original publisher: [Read original article]() Published: 2021-02-02T00:00:00Z Content type: article Language: en Sources: [SoundCloud Backstage Blog]() Topics: [Test coverage](), [engineering-culture](), [Tooling](), [Android](), [data](), [CRUD](), [data-processing](), [Refactoring](), [Backstage](), [Cypress](), [GitHub](), [Google]() Tags: [android](), [automation](), [backend](), [backstage](), [data](), [devxp](), [engineering](), [flaky](), [frontend](), [infrastructure](), [refactoring](), [test-coverage](), [testing](), [tooling]() ### AI overview SoundCloud describes an effort to improve testing infrastructure by aggregating test data across repositories, teams, and features. The proposed workflow treats build artifacts and test reports as data for analyzing test-suite health, retries, execution times, flakiness, pipeline costs, and developer waiting times, with the goal of guiding test fixes and refactoring. ### Source excerpt Testing is at the heart of engineering practices at SoundCloud. We strive to build well-balanced test pyramids within our code repositories... ## Node.js 14.15.0 (LTS) DevFeed: [Node.js 14.15.0 (LTS)]() Original publisher: [Read original article]() Published: 2020-10-27T16:07:29Z Content type: release Language: en Sources: [Node.js Blog]() Topics: [Node.js]() Tags: [changelog](), [documentation](), [flaky](), [linux](), [macos](), [node-js](), [release](), [support](), [windows]() ### AI overview Node.js 14.15.0 transitions the 14.x line to Active LTS, codenamed Fermium. The release includes changelog and documentation fixes and reverts a flaky WebCrypto test marking. ### Source excerpt Node.js® is a free, open-source, cross-platform JavaScript runtime environment that lets developers create servers, web apps, command line tools and scripts. ## Supabase Alpha April 2020 DevFeed: [Supabase Alpha April 2020]() Original publisher: [Read original article]() Author: Paul Copplestone Published: 2020-05-01T07:00:00Z Content type: release Language: en Sources: [Supabase Blog]() Topics: [Supabase](), [Database](), [API](), [Documentation](), [Firebase](), [Digital Ocean]() Tags: [apis](), [database](), [digital-ocean](), [documentation](), [firebase](), [flaky](), [github](), [postgres](), [realtime]() ### AI overview Supabase's April 2020 alpha update introduces fast Postgres setup, auto-generated APIs, dashboard database queries, and query analysis. The article also previews realtime listeners, schema-driven REST APIs, connectors, and automatic API and Postgres schema documentation, while noting that the product is not production ready. ### Source excerpt Two months of building [Next page]()