# Razorpay Engineering - Medium Razorpay's Engineering blog, decoding how we build India's Financial Infrastructure backbone - Medium This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Five Years of Kafka at Razorpay's UPI Switch DevFeed: [Five Years of Kafka at Razorpay's UPI Switch]() Original publisher: [Read original article]() Author: Kshitij Nawandar Published: 2026-09-07T09:09:58Z Content type: article Language: en Sources: [Razorpay Engineering - Medium]() Topics: [Kafka](), [Architecture & Design](), [event driven](), [Amazon Simple Queue Service (SQS)](), [distributed-systems](), [Microservices](), [Amazon Web Services]() Tags: [aws](), [aws-sns](), [distributed-systems](), [event-driven](), [kafka](), [microservices](), [razorpay](), [redshift](), [sns](), [sqs]() ### AI overview Razorpay describes five years of evolution in its UPI Switch, from a monolith using AWS SQS to Kafka-based infrastructure. The article covers architectural decisions, operational challenges, and optimization work affecting payment-processing performance, reliability, and scale. ### Source excerpt Preface The UPI Switch at Razorpay has evolved significantly in the five years since we started building it. The Switch is the platform that enables real-time payment processing with NPCI. When the team began, it was little more than an idea. Today it powers more than 70% of Razorpay's total UPI volume. Because UPI is inherently asynchronous, a messaging system sits at the heart of the Switch and has a direct impact on performance, reliability, and scale. What began as a straightforward queue became the core of the system, shaping how every new feature was designed and delivered. This post covers that evolution: the decisions that enabled growth, the ones that slowed us down, the operational issues that forced us to rethink our assumptions, and the optimizations that ultimately stabilized our Kafka-based infrastructure. This is the story of what we got right, what we got wrong, and how we eventually built something stable enough to grow on. The First Version: Monolith and SQS When we began building the UPI Switch, we weren't thinking about massive scale, distributed systems, or elegant event routing. So we built Switch v1 as a monolith. No microservices, no distributed orchestration: just one solid block of code doing everything. That was the right call. We needed to move fast, experiment, and learn, and we followed the Keep It Simple, Stupid (KISS) principle deliberately. For messaging, we picked AWS SQS: reliable, managed, and low on cognitive load. We didn't need ordering guarantees at the time, so a standard queue worked fine. We started with just two queues, and this setup held its ground. It handled a peak of 400 TPS during the IPL. The limitations showed up as the ecosystem grew. A single event, like a successful payment, needed to fan out into multiple workflows: Update NPCI with an API call Send callbacks to merchants about payment status Push structured data into our warehouse (AWS Redshift) To handle this, we started bolting on AWS SNS plus SQS for fan-ou ## How Razorpay Cut Its Metrics Bill by 62% Without Losing a Dashboard DevFeed: [How Razorpay Cut Its Metrics Bill by 62% Without Losing a Dashboard]() Original publisher: [Read original article]() Author: Dhairya Mehta Published: 2026-08-24T10:25:21Z Content type: article Language: en Sources: [Razorpay Engineering - Medium]() Topics: [Monitoring](), [migration](), [incident](), [Self-hosted](), [telemetry]() Tags: [cost](), [high-availability](), [incident](), [metrics](), [migration](), [monitoring](), [self-hosted]() ### AI overview Razorpay reduced daily metrics ingestion from about 450 billion to about 170 billion samples, cutting its metrics bill by 62% without deleting useful dashboards or breaking alerts. The article describes an audit conducted during a planned migration from a self-hosted VictoriaMetrics cluster to a managed monitoring platform, covering scrape intervals, duplicate high-availability scrapes, and high-cardinality metrics. ### Source excerpt Contributor: Saijal Shrivastava We cut our metrics ingestion by 62% without deleting a single useful dashboard or breaking an alert. Daily ingestion dropped from about 450 billion samples to about 170 billion. The surprising part was not one large optimization. It was three simple discoveries hiding in plain sight: Scrape intervals can multiply ingestion. High-availability scraping can silently send duplicate metrics when deduplication happens only at the storage layer. High-cardinality metrics are easy to create, expensive to keep, and hard to remove without ownership. We found these while preparing to move from a self-hosted VictoriaMetrics cluster to a managed monitoring platform. The migration started as a reliability project. The audit turned it into a cost and signal-quality project. We were not just moving monitoring data. We were moving years of accumulated assumptions about what was worth scraping. The Incident that Forced the Audit Our self-hosted VictoriaMetrics cluster had served us well for years. Then one storage node hit EBS volume throttling. Traffic shifted to the remaining storage nodes, the extra load pushed another node past its limits, and the storage layer collapsed. We saw the same pattern whenever a storage node became unavailable: traffic redistributed, the remaining nodes overloaded, and the cluster moved back toward failure. The collection, ingestion, and query components were still running, so the system looked alive from the outside. But the storage layer was down. Metrics stopped being persisted. Dashboards went blank. Alerts stopped evaluating. Recovery took hours. Teams fell back to logs and manual checks to understand production health. Figure 1: Cascading failure in the pipeline That made the reliability problem obvious. We needed a more stable monitoring architecture. We evaluated several options: self-hosted designs with disaster recovery built in, and fully managed alternatives. But when we priced the managed options, one number ## CI Doesn't Need On-Demand: Moving Our Build Pipelines to Spot Instances DevFeed: [CI Doesn't Need On-Demand: Moving Our Build Pipelines to Spot Instances]() Original publisher: [Read original article]() Author: Yuvraj Singh Singhel Published: 2026-08-05T15:07:35Z Content type: article Language: en Sources: [Razorpay Engineering - Medium]() Topics: [CI/CD](), [GitHub Actions](), [Kubernetes](), [Amazon Web Services]() Tags: [aws](), [ci](), [ci-cd](), [github-actions](), [kubernetes](), [pipelines](), [self-healing]() ### AI overview Razorpay describes a self-healing infrastructure layer for GitHub Actions on Kubernetes that runs most CI workloads on AWS Spot Instances. The system detects spot-node termination, retries jobs, and cleans up orphaned pods; the article states that 80% of CI workloads run on spot instances with a 99.2% job success rate. ### Source excerpt Contributors: Guptaanuj CI/CD pipelines have always had a money-vs-stability problem. Run on-demand AWS instances and your build infrastructure is rock solid, expensive, and predictable. Run on spot instances and your costs drop 70-90%, but AWS can pull the rug with 2 minutes of warning. For most teams, this is a false choice. Either pay full price for reliability, or save money and accept that builds will fail in ways nobody can debug. At Razorpay, we stopped accepting that trade-off. We built a self-healing infrastructure layer for GitHub Actions on Kubernetes that runs 80% of our CI workloads on spot instances while maintaining a 99.2% job success rate. When AWS terminates a spot node mid-build, our system detects it, retries the job, cleans up the orphaned pods, and the developer never knows. This is the story of what we built, why polling wasn't an option, and the war stories that taught us how to do retries without burning everything down. The Problem With Spot Instances On CI Spot instances are AWS capacity that nobody else wants right now. The pricing is brutal compared to on-demand: a c5.2xlarge that costs around $0.34/hour on-demand drops to roughly $0.08/hour on spot. For workloads like CI/CD, where jobs are short-lived and parallelizable, the math is obvious. The catch is in the contract. AWS reserves the right to take spot capacity back at any moment, with a 2-minute warning. That works for some workloads. For others, it's catastrophic. GitHub Actions runners on Kubernetes is the hard case. A typical CI job goes like this: GitHub assigns the job to a runner. The runner is a pod on a Kubernetes cluster running on an AWS Spot instance. The job downloads dependencies, runs tests, builds artifacts. The runner reports back to GitHub. Now insert a spot termination at minute 4 of a 7-minute build. What happens? The runner pod dies mid-job. GitHub never gets a completion signal; the job hangs until timeout, then marks as "failed". A new runner pod gets schedule ## How We Refresh Razorpay's Data Warehouse 10x Faster with Graphs and Indexes DevFeed: [How We Refresh Razorpay's Data Warehouse 10x Faster with Graphs and Indexes]() Original publisher: [Read original article]() Author: Amit Prabhu Published: 2026-07-14T14:06:16Z Content type: article Language: en Sources: [Razorpay Engineering - Medium]() Topics: [data](), [airflow](), [Apache Spark](), [Amazon S3](), [Apache Iceberg](), [microservices architecture](), [parquet](), [Architecture & Design]() Tags: [airflow](), [analytics](), [apache-iceberg](), [architecture](), [batch](), [data](), [graphs](), [infrastructure](), [parquet](), [pipelines](), [razorpay](), [spark](), [trino](), [warehouse]() ### AI overview Razorpay describes its data warehouse refresh pipeline, which builds wide denormalized Facts by joining data from multiple microservices. The article covers the original Airflow- and Spark-based full-refresh process, the underlying lake formats and query layer, and the scaling challenges that led the team to reconsider refresh strategy, data layout, and high-cardinality dimensions. ### Source excerpt Contributors: Utkarsh Koppikar Rohan Background Razorpay provides the payment infrastructure for millions of merchants globally. Behind every payment, settlement, and refund is a microservices architecture where each service owns its own database. While this keeps services independent and scalable, it creates a challenge for stakeholders who need to see across those boundaries. The Data Platform team manages the infrastructure that bridges this gap. Transactional data flows into the lake via CDC pipelines, ingested onto S3 in Delta Lake, Apache Iceberg, or plain Parquet formats. On top of the lake, we build domain-specific warehouse tables -- wide, pre-joined tables that co-locate all the data a consumer needs, queryable via Trino. These power two use cases: Analytics (internal dashboards on Tableau and Superset) and Reporting (merchants and regulated entities who download structured data exports; Razorpay generates nearly a million such reports per month). The warehouse tables that power both use cases are called Facts. A Fact is a flat denormalised table on S3, produced by joining 10 to 30 microservice tables and materialising the result once. A settlement Fact, for example, merges payments, refunds, adjustments, and card details into a single wide row so that a dashboard or report reads from a single table instead of joining across services in real time. It is closer to a domain-specific materialised view than a classical data warehouse fact table. We maintain over 50 such Facts, and approximately 40% of all merchant reports are served directly from them. As data volumes and the number of entities per fact grew, the batch generation pipeline began to show its limits, prompting us to rethink the refresh strategy, the data layout, and how to handle high-cardinality dimensions. The rest of this post covers that journey. The Full Refresh Pipeline: Our Baseline and the Pain The original full-refresh pipeline was straightforward. Schedule: Airflow schedules Spark jobs o ## How Razorpay runs network-isolated Hermes AI agents for employees DevFeed: [How Razorpay runs network-isolated Hermes AI agents for employees]() Original publisher: [Read original article]() Author: ashwath kumar Published: 2026-07-12T14:22:53Z Content type: article Language: en Sources: [Razorpay Engineering - Medium]() Topics: [AI Agent](), [Kubernetes](), [Open Source](), [Cloud](), [data](), [Embeddings](), [Graphs]() Tags: [ai](), [ai-agent](), [architecture](), [autonomous](), [cloud](), [credentials](), [embeddings](), [github](), [google](), [graph](), [kubernetes](), [network](), [open-source](), [slack]() ### AI overview Razorpay describes its platform for running Hermes, an open-source AI agent, for employees. Each agent runs in an isolated Kubernetes namespace with encrypted storage, a separate cloud identity, and its own network policy. The article explains how the platform supports persistent autonomous sessions while limiting access to other employees' data and the public internet. ### Source excerpt How we run a personal AI agent for everyone at Razorpay: always on, multi-model, and safe by construction. Contributors: Siddharth Tripathi Today, more than 220 Razorpay employees each have their own always-on AI agent. On a typical day, about 84 of them are actively working. Every one runs in its own isolated Kubernetes namespace, with its own encrypted storage, its own cloud identity, and its own network policy. It learns new skills as its owner works, and it keeps running long after they close their laptop: one employee's agent has already logged more than 15,000 sessions, 90% of them while its owner was asleep or away. Provisioning a new one takes under two minutes. Running the agents was never the hard part. Running them safely was: 220 of them, each with shell access and live credentials, without any single agent becoming a path into another employee's data or out to the open internet. The answer came down to one design choice, and everything in this post is a consequence of it: Isolation is a property of the infrastructure, not the application. Hermes itself is an open-source agent by Nous Research; what we built is the platform that runs it safely & isolated, for the whole company. Proof It's Real: One Instance, Eight Weeks In Before any of the architecture, here's the proof that people actually use this. The following is one real instance from our cluster, over its first eight weeks (numbers pulled live, the person anonymised). In eight weeks, this one user's Hermes ran 15,039 sessions. Only 391 of those were the person sitting down to chat with it; the other 13,570 were autonomous runs the agent kicked off on its own schedule while its owner was asleep or in meetings. That ratio is the whole idea in one statistic: the assistant does most of its work when you're not there. What is it doing in those runs? They'd wired up 21 always-on scheduled jobs that turn Hermes into a personal intelligence service: Ingest. Every hour it pulls from 23 Slack channels (plus ## How Razorpay Built a Customer Data Platform for Queryable Segments at Scale DevFeed: [How Razorpay Built a Customer Data Platform for Queryable Segments at Scale]() Original publisher: [Read original article]() Author: Varun Meka Published: 2026-06-26T08:06:33Z Content type: article Language: en Sources: [Razorpay Engineering - Medium]() Topics: [data](), [Transactions](), [App](), [systems](), [Apache Spark]() Tags: [app](), [banking](), [card](), [customer](), [data](), [devices](), [engineering](), [payments](), [platform](), [razorpay](), [saas](), [scale](), [shopping](), [transactions]() ### AI overview Razorpay describes building an in-house Customer Data Platform to unify fragmented customer and transaction data into queryable audience segments. The supplied excerpt says the platform serves segments across more than 500 million user profiles in under 30 milliseconds while keeping personally identifiable information isolated in source systems. ### Source excerpt Turning Scattered Data Into Queryable Segments at Scale: How Razorpay Built Its Customer Data Platform A consent-native CDP that serves audience segments across 500M+ user profiles in under 30ms, with PII isolated to the source systems. The Problem We Were Solving A customer opens her favourite online shopping app, adds a few items to her cart, and pays INR 1,200 via UPI, powered invisibly by Razorpay. A week later she returns and pays using a saved Visa card from her laptop. Later that month, she places a larger INR 8,500 order through net banking from work. Three transactions. Three different payment instruments. Three different devices. To the merchant's engineering team, and to Razorpay's data systems, these could look like three completely different people, unless you've done the hard work of figuring out they're all the same customer. Now suppose this is a D2C fashion brand approaching their Diwali sale. The merchant's growth team has a clear plan: "Identify customers who have transacted at least once in the last 30 days, have spent more than INR 5,000 cumulatively this quarter, and haven't enrolled in our loyalty programme. Send them an early-access nudge with a personalized discount 48 hours before the public sale opens." A year ago, answering that question at Razorpay meant filing a cross-team data request, waiting for an analyst to write a custom Spark job, and getting an answer in 2-3 days. By the time the merchant had the segment, the Diwali sale was already live. The early-access window had closed. The campaign got sent to a broader, less-targeted audience, wasting spend on customers who would have bought anyway and leaving cold customers untouched. Now multiply that pain by millions of merchants. Razorpay powers payments and growth for over 12 million merchants. From D2C fashion brands and SaaS startups to subscription platforms, ed-tech companies, and the 2 million+ local merchants accepting QR payments every day. Together, they process billions of transaction ## From 750 Hours to 2 Hours: AI-Powered Security Triage at Razorpay DevFeed: [From 750 Hours to 2 Hours: AI-Powered Security Triage at Razorpay]() Original publisher: [Read original article]() Author: Prathamesh Joshi Published: 2026-06-09T14:56:35Z Content type: article Language: en Sources: [Razorpay Engineering - Medium]() Topics: [Security](), [Artificial Intelligence](), [Vulnerabilities](), [Architecture & Design](), [Code](), [API](), [GitHub](), [Sanitization](), [API keys]() Tags: [ai](), [api](), [architecture](), [code](), [credentials](), [github](), [hardcoded-credentials](), [sast](), [security](), [vulnerabilities]() ### AI overview Razorpay describes an AI-powered security triage system built to reduce the manual workload caused by large volumes of SAST, dependency, and secret-detection alerts. Its first live layer retrieves issue context, source code, data flows, and sanitization logic to distinguish genuine vulnerabilities from false positives, with reported accuracy of 75-80%. ### Source excerpt Co-authors: Mahlaqahaque Mh, Keertiv, Hari Prasad Pujari How we taught AI to read code like a senior security engineer Every day, Razorpay engineers ship thousands of lines of code. Every line births new security findings. SAST scanners flag suspicious patterns. Dependency checkers find vulnerable libraries. Secret detection tools catch hardcoded credentials. The alerts pile up. Hundreds become thousands. The backlog becomes noise. We hit a breaking point. Developers faced security ticket counts climbing into the thousands, with most of them turning out to be false positives. The classic "alert that cried wolf" scenario played out daily. When everything is marked critical, nothing is. Developers stopped trusting security findings altogether. Security engineers weren't having a better time. Validating issues manually while fielding constant ad-hoc requests from frustrated developers. Trying to stop a waterfall with a teaspoon. The human bottleneck became the limiting factor in our security posture. The core problem was simple. Traditional static analysis tools excel at finding patterns that might be vulnerabilities. They lack context. They can't distinguish between a properly sanitized SQL query and a vulnerable one. Between a test API key and a production secret. Between a dangerous data flow and one protected by business logic. For every 10 alerts, 7-8 were false positives. Manual triage became the bottleneck. Security couldn't scale with engineering velocity. That's when we built what we call the Autonomous Security Special Ops system. An AI-powered engine that handles the heavy lifting so humans can focus on what actually matters. The Three-layer Intelligence System Rather than throwing more human hours at the problem, we built an AI architecture operating in three layers. L1: Context-Aware AI Triage (Live). Our intelligent first responder. Powered by 29 specialized sub-skills , it reads code context like a senior security engineer. When a SAST finding lands, L1 ## Razorpay Oncall Agent: From 30-Minute Investigations to 90-Second AI Analysis DevFeed: [Razorpay Oncall Agent: From 30-Minute Investigations to 90-Second AI Analysis]() Original publisher: [Read original article]() Author: Anuj Gupta Published: 2026-04-29T06:56:11Z Content type: article Language: en Sources: [Razorpay Engineering - Medium]() Topics: [Artificial Intelligence](), [Langgraph](), [incident](), [SRE](), [observability](), [Large Language Model](), [Retrieval-Augmented Generation]() Tags: [agent](), [ai](), [incident](), [langgraph](), [logs](), [metrics](), [observability](), [retrieval-augmented-generation](), [sre]() ### AI overview Razorpay describes building a multi-agent AI system to automate production incident investigations. The Oncall Agent uses LangGraph, an LLM, alerting tools, and two retrieval-augmented generation systems containing architecture, dependency, and diagnostic runbook context. ### Source excerpt Our on-call engineers were spending 30 minutes investigating every production alert. Here's what happened when we automated it. At 3 AM, alerts don't care about your sleep schedule. When our payment infrastructure threw an error last month, our on-call engineer spent 32 minutes jumping between six different monitoring systems before understanding what was broken. One tool for metrics. Another for logs. Third tool for pod health. And multiple more for infrastructure, deployment history and database health. By the time they identified the root cause (a bad deployment), payment failures had already impacted customers for nearly 40 minutes. This wasn't their fault. They followed our runbook perfectly. The problem was that no single system could tell them "here's what's wrong and why." They had to manually connect dots across disconnected observability tools. That's when we asked ourselves: what if AI could do this investigation for us? The Metric Nobody Optimizes For The SRE world talks endlessly about Mean Time to Detect (how fast you catch problems) and Mean Time to Resolve (how fast you fix them). But there's a critical phase hiding between them: Mean Time to Investigate. MTTI is the gap from "we know it's broken" to "we know what to fix." At Razorpay, this phase was consuming 20-40 minutes per incident. With 15-20 incidents weekly, that's 6-8 hours of engineering time spent doing repetitive investigative work. Worse, the quality was inconsistent. Senior engineers knew exactly which systems to check for payment alerts. Junior engineers sometimes checked irrelevant dashboards or missed critical correlations. The investigation depended entirely on who was on-call that night. What We Built (And Why It Works) Razorpay Oncall Agent is a multi-agent AI system that automates incident investigation. The architecture is built on LangGraph, a framework for creating stateful workflows with conditional logic, and uses LLM as the reasoning engine. Here's how the components work t ## Razorpay's Linked Payments Architecture for Combining Payment Methods DevFeed: [Razorpay's Linked Payments Architecture for Combining Payment Methods]() Original publisher: [Read original article]() Author: Vatsal Mehta Published: 2026-04-08T09:56:17Z Content type: article Language: en Sources: [Razorpay Engineering - Medium]() Topics: [Architecture & Design](), [Microservices](), [Transactions](), [Database]() Tags: [architectures](), [authorization](), [fintech](), [microservices](), [monolithic-architecture](), [payment](), [payment-gateway](), [payment-processing](), [payments](), [razorpay](), [recovery](), [technical](), [transactions]() ### AI overview Razorpay describes Linked Payments, an architecture that lets customers combine a gift card with card, UPI, or another payment method for one order. The article explains the resulting challenges around authorization sequencing, failure recovery, and split settlement across payment-method microservices. ### Source excerpt You have a $50 gift card. Your cart is $75. The checkout says "pick one payment method." This is a solvable problem. If you've ever tried to use a gift card for an online purchase, you know this frustration intimately. The gift card covers most of it, but not quite all. The checkout forces you to choose: use the gift card and abandon some items, or ignore the gift card and pay the full amount another way. Either choice feels wrong. This isn't a technical limitation of payment processing. It's an architectural one. Most payment systems treat each method as an isolated, complete transaction. You pay with a card OR UPI OR a gift card. The concept of composing multiple methods to fulfill a single order simply doesn't exist in traditional payment gateway architectures. At Razorpay, this limitation was costing merchants real money. Gift card redemption rates suffered because customers abandoned partial-value cards. Average order values stayed lower because customers couldn't combine store credit with additional payment. The business case for solving this was clear. That's why we built Linked Payments, a system that treats payment methods as composable building blocks. Customers can now use a gift card for $50, then cover the remaining $25 via card, UPI, or any other method. The system handles authorization sequencing, failure recovery, and settlement splitting automatically. The Complexity Hidden in "Just Combine Them" The challenge sounds simple until you consider what payment systems actually do. Traditional payment flows are beautifully simple. Customer initiates payment. System authorizes the full amount from one method. If authorization succeeds, capture the funds. Settle to the merchant. Either the payment worked or it didn't. One authorization, one capture, one settlement. Linked payments shatter this simplicity. Now you have multiple authorizations for a single order. Sequential dependencies where the second payment only happens if the first succeeds. Partial fail