# Test Budget: Time Constrained CI Feedback

DevFeed: [Test Budget: Time Constrained CI Feedback](<https://devfeed.tech/articles/test-budget-time-constrained-ci-feedback-1640.md>)

Original publisher: [Read original article](<https://shopify.engineering/test-budget-time-constrained-ci-feedback>)

Author: Apostolis Stergiou

Published: 2022-03-07T15:00:00Z

Content type: article

Language: en

Sources: [Shopify Engineering](<https://devfeed.tech/sources/shopify-engineering.md>), [Shopify Engineering - Shopify Engineering](<https://devfeed.tech/sources/shopify-engineering-shopify-engineering.md>)

Topics: [Testing](<https://devfeed.tech/topics/testing.md>), [CI/CD](<https://devfeed.tech/topics/cicd.md>), [Shopify](<https://devfeed.tech/topics/shopify.md>), [Development](<https://devfeed.tech/topics/development.md>)

Tags: [continuous-integration](<https://devfeed.tech/tags/continuous-integration.md>), [shopify](<https://devfeed.tech/tags/shopify.md>), [speed](<https://devfeed.tech/tags/speed.md>), [testing](<https://devfeed.tech/tags/testing.md>), [tests](<https://devfeed.tech/tags/tests.md>), [time](<https://devfeed.tech/tags/time.md>)

## AI overview

Shopify investigates using a fixed test-time budget to speed up continuous integration feedback in a large monolith. The approach prioritizes tests to increase the chance of detecting failures early, accepting some additional risk; the analysis found failures after running 70% of the test-selection suite.

## Source excerpt

At Shopify we run more than 170,000 tests in our core monolith. Naturally, we're constantly exploring ways to make this faster, and the Test Infrastructure team analyzed the feasibility of introducing a test budget: a fixed amount of time for tests to run. The goal is to speed up the continuous integration (CI) test running phase by accepting more risk. To achieve that goal we used prioritization to reorder the test execution plan in order to increase the probability of a fast failure.