# CI Doesn't Need On-Demand: Moving Our Build Pipelines to Spot Instances

DevFeed: [CI Doesn't Need On-Demand: Moving Our Build Pipelines to Spot Instances](<https://devfeed.tech/articles/ci-doesn-t-need-on-demand-moving-our-build-pipelines-to-spot-instances-24037.md>)

Original publisher: [Read original article](<https://engineering.razorpay.com/ci-doesnt-need-on-demand-moving-our-build-pipelines-to-spot-instances-6fff1cd92ba8?source=rss----6407ad2e59af---4>)

Author: Yuvraj Singh Singhel

Published: 2026-08-05T15:07:35Z

Content type: article

Language: en

Sources: [Razorpay Engineering - Medium](<https://devfeed.tech/sources/razorpay-engineering-medium.md>)

Topics: [CI/CD](<https://devfeed.tech/topics/cicd.md>), [GitHub Actions](<https://devfeed.tech/topics/github-actions.md>), [Kubernetes](<https://devfeed.tech/topics/kubernetes.md>), [Amazon Web Services](<https://devfeed.tech/topics/aws.md>)

Tags: [aws](<https://devfeed.tech/tags/aws.md>), [ci](<https://devfeed.tech/tags/ci.md>), [ci-cd](<https://devfeed.tech/tags/ci-cd.md>), [github-actions](<https://devfeed.tech/tags/github-actions.md>), [kubernetes](<https://devfeed.tech/tags/kubernetes.md>), [pipelines](<https://devfeed.tech/tags/pipelines.md>), [self-healing](<https://devfeed.tech/tags/self-healing.md>)

## AI overview

Razorpay describes a self-healing infrastructure layer for GitHub Actions on Kubernetes that runs most CI workloads on AWS Spot Instances. The system detects spot-node termination, retries jobs, and cleans up orphaned pods; the article states that 80% of CI workloads run on spot instances with a 99.2% job success rate.

## Source excerpt

Contributors: Guptaanuj CI/CD pipelines have always had a money-vs-stability problem. Run on-demand AWS instances and your build infrastructure is rock solid, expensive, and predictable. Run on spot instances and your costs drop 70-90%, but AWS can pull the rug with 2 minutes of warning. For most teams, this is a false choice. Either pay full price for reliability, or save money and accept that builds will fail in ways nobody can debug. At Razorpay, we stopped accepting that trade-off. We built a self-healing infrastructure layer for GitHub Actions on Kubernetes that runs 80% of our CI workloads on spot instances while maintaining a 99.2% job success rate. When AWS terminates a spot node mid-build, our system detects it, retries the job, cleans up the orphaned pods, and the developer never knows. This is the story of what we built, why polling wasn't an option, and the war stories that taught us how to do retries without burning everything down. The Problem With Spot Instances On CI Spot instances are AWS capacity that nobody else wants right now. The pricing is brutal compared to on-demand: a c5.2xlarge that costs around $0.34/hour on-demand drops to roughly $0.08/hour on spot. For workloads like CI/CD, where jobs are short-lived and parallelizable, the math is obvious. The catch is in the contract. AWS reserves the right to take spot capacity back at any moment, with a 2-minute warning. That works for some workloads. For others, it's catastrophic. GitHub Actions runners on Kubernetes is the hard case. A typical CI job goes like this: GitHub assigns the job to a runner. The runner is a pod on a Kubernetes cluster running on an AWS Spot instance. The job downloads dependencies, runs tests, builds artifacts. The runner reports back to GitHub. Now insert a spot termination at minute 4 of a 7-minute build. What happens? The runner pod dies mid-job. GitHub never gets a completion signal; the job hangs until timeout, then marks as "failed". A new runner pod gets schedule