# Adyen Tech Insights from the team building the world's payments infrastructure. - Medium This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Automating Code Modernization with OpenRewrite - By Stefano Dalla Palma, Adyen DevFeed: [Automating Code Modernization with OpenRewrite - By Stefano Dalla Palma, Adyen]() Original publisher: [Read original article]() Author: Adyen Published: 2026-08-22T06:45:48Z Content type: article Language: en Sources: [Adyen Tech]() Topics: [Refactoring](), [Jackson, Java Dates, Exception, LocalDate](), [migration](), [Tooling](), [Development](), [Software Engineering]() Tags: [code-review](), [internal-developer-platform](), [jackson](), [migration](), [modernization](), [refactoring](), [software](), [software-engineering](), [tech](), [tooling]() ### AI overview An Adyen engineering team explores using OpenRewrite to automate code modernization and reduce the manual effort of framework, API, and static-analysis updates. The article describes a completed migration of an internal Jackson 2 wrapper to Jackson 3 across hundreds of modules and discusses integrating the tool into the build and CI workflow. ### Source excerpt Automating Code Modernization with OpenRewriteBy Stefano Dalla Palma - Development Tooling Engineer, Adyen In a large-scale engineering organization such as Adyen, where hundreds of developers produce hundreds of merge requests per day across thousands of modules, continuously evolving code to meet the latest standards is essential for long-term platform health and performance. However, executing routine updates manually, such as adopting new framework patterns, keeping up with API enhancements, or fine-tuning static analysis rules can introduce friction that takes developers away from building core features. When I joined Adyen's internal developer platform team, I wanted to understand whether any of that friction could be automated away. Around that time, I came across an episode of Software Engineering Radio (link) where the creator of OpenRewrite made an argument that stuck with me: "We are asking framework authors to start taking responsibility for providing recipes when they make breaking changes." I loved that framing. The hassle of keeping up shouldn't fall entirely on the consumers of a library, but on the people who changed the contract in the first place. While we couldn't change how external maintainers worked, we could adopt that mindset internally. I saw an opportunity: if a tool could absorb the friction of a breaking change, we could handle the cleanup on behalf of our developers. But my enthusiasm wasn't going to convince anyone. The engineers I pitched to wanted to see it work before they'd trust it, and I don't blame them. The only way to earn trust was to do the work: pick a real use case, drive it to completion, and let the results speak. One team had a ticket to migrate an internal Jackson 2 wrapper to Jackson 3, a tricky refactoring with deep caller dependencies across hundreds of modules, exactly the kind of high-effort, lower-urgency work that teams understandably prioritize behind customer-facing delivery. I asked if I could take a shot at ## A Deep Dive into Table partitioning Part 5: Partitioning an environment not individual tables DevFeed: [A Deep Dive into Table partitioning Part 5: Partitioning an environment not individual tables]() Original publisher: [Read original article]() Author: Adyen Published: 2026-07-13T09:18:45Z Content type: article Language: en Sources: [Adyen Tech]() Topics: [Databases](), [PostgreSQL]() Tags: [adyen](), [database](), [deep-dive](), [partitioning](), [performance](), [postgresql](), [table-partitioning]() ### AI overview This fifth article in a table-partitioning series explains why partitioning must account for an entire relational database environment rather than isolated tables. It discusses lessons from PostgreSQL partitioning, including table locks, partition maintenance, indexes, foreign keys, and the careful use of default partitions. ### Source excerpt By Derk van Veen, Database Engineer, Adyen If we had known better, this would have been the first blog post about partitioning to publish. The reality is, the first article was published two years and two weeks ago, as I write these opening sentences. So this article will not only give insight into the most important decisions you will have to make when considering your partitioning planning, but as a bonus, it will give some insights into how much it takes to write an article like this. One of my values is sharing knowledge and giving back to the community. I share what I learn and take people along on my journey and investigations into the problems we've faced. For this reason, the previous articles have documented these learnings, and this one is about a lesson we learned a little too late. The Journey so far Up to this point: We've shared the what, why and when to partition in the first blog post. In the second article, we covered how to maintain partitioned tables, as they provide some additional challenges when it comes to adding/removing partitions and managing indexes and foreign keys. Only in the third article did we disclose why we decided to work on our own partitioning framework: the existing partitioning solutions didn't provide us with the tools to minimize locks on tables. Our databases are always up and pressured, and the locks around partitioning were simply impossible to deal with. The key lesson from this article was that the default partition is the root of (almost) all evil regarding table partitioning in PostgreSQL. In the last article, we shared a cool solution to a problem we faced, where the default partition actually became the hero of the story. Yes, default partitions are usually the root of partitioning evil, but they also provide a welcome exception to the rule. Just be very careful with its usage. The Journey Continues... What ingredients can be missing after reading these four articles? You already know all about partitioning tables by ## Building a GlobalStream: Replication Challenges and Optimizations DevFeed: [Building a GlobalStream: Replication Challenges and Optimizations]() Original publisher: [Read original article]() Author: Adyen Published: 2026-07-13T09:18:08Z Content type: article Language: en Sources: [Adyen Tech]() Topics: [Replication](), [Kafka](), [Streaming](), [Latency](), [Optimization](), [systems]() Tags: [data-replication](), [kafka](), [latency](), [mirrormaker](), [open-source](), [optimization](), [payments](), [replication](), [streaming](), [tech]() ### AI overview Adyen describes the challenges of replicating Kafka data across continents with MirrorMaker 2. The article covers data-location tradeoffs, Kafka Connect behavior, parameter tuning, and optimizations intended to sustain high-throughput replication while controlling latency. ### Source excerpt By Gaurav Singh & Luciano Sabença, Streaming Platform Team Global data streaming requires careful tradeoffs between performance, compliance, and reliability. At Adyen, replication is essential: data must remain close to producers to reduce latency and meet regulations, while also being centralized for analytics. In this blog, we share our experience optimizing MirrorMaker 2 for large-scale, cross-continent Kafka replication, the parameter tuning we applied, and the hidden behaviors in Kafka Connect that ultimately unlocked the required throughput. Why Replicate? There are many things you need to take into account when designing a data streaming platform for a company such as Adyen. One of the most important aspects of it is data location and replication. Due to performance, reliability, and compliance, keeping data close to producers is usually a good idea. After all, you don't want to wait in line at the cashier to confirm the payment for that fat burger you just bought on that nice beach in Australia while your payment was being sent to Europe. However, it's also a common practice to have all data in a centralized cluster(s) in centralized locations for analytical purposes. These two need a bridge, which is where mirroring comes in. An example replication flow diagramMirror Maker When it comes to copying data between Kafka clusters, Mirror Maker is the default open source tool. MirrorMaker 2 is the current version, and it's built on top of Kafka Connect, a platform designed to make it easier to integrate Kafka with other tools such as databases and distributed file systems, and -- why not? -- another Kafka cluster. Setting up Mirror Maker is fairly straightforward, but tuning it to the performance Adyen requires isn't. Let's go over the journey into the depths of Kafka Connect and parameter tuning to find a solution! Playing the Volume Game Kafka is a highly flexible tool and serves as the backbone for many different architectural patterns. It can be used for batch ## Inside Cilium CNI: solving mysterious Kubernetes pod setup timeouts DevFeed: [Inside Cilium CNI: solving mysterious Kubernetes pod setup timeouts]() Original publisher: [Read original article]() Author: Adyen Published: 2026-07-13T09:17:40Z Content type: article Language: en Sources: [Adyen Tech]() Topics: [Cilium](), [Kubernetes](), [Kubernetes networking](), [networking](), [eBPF](), [Latency](), [hdfs](), [Apache Spark](), [big-data](), [cpu]() Tags: [big-data](), [cilium](), [cni](), [cpu](), [data](), [devops](), [ebpf](), [hdfs](), [kubernetes](), [kubernetes-networking](), [latency](), [spark](), [speed](), [tracing]() ### AI overview An Adyen engineer investigates Cilium CNI timeouts that prevented new Kubernetes pods from starting. The article attributes the issue to sequential connection-tracking-table garbage collection, which became a bottleneck under workloads involving HDFS, Trino, and short-lived Spark jobs. ### Source excerpt By Jorrick Sleijster - Senior Data Platform Engineer, Adyen I was fully aware a year ago that a single configuration line could break the Kubernetes networking stack. But if they told me that leftovers from Kubernetes pods which terminated hours prior could block new ones from starting, I would have thought they were joking. In high-performance networking, 35 seconds is a lifetime. This was the latency required to iterate through our connection tracking table of 7 million entries at a maximum speed of 200,000 entries per second. At our 16-million-entry peak, this sequential lookup could take up to 80 seconds, leading to Cilium CNI timeouts preventing new pods from starting on affected nodes. We uncovered this linear-time behavior at Adyen by tracing syscalls, inspecting codebases, and analyzing eBPF internals. This investigation revealed how our varied workloads turned the connection tracking table's garbage collection algorithm into a critical bottleneck. Our setup: why we're different At Adyen, we run Cilium CNI across all our 100+ Kubernetes clusters. When we switched from Calico to Cilium, we knew we'd face challenges adapting it to our production workloads. Our production big data Kubernetes clusters have a unique usage pattern compared to the other Kubernetes environments within Adyen: Data extraction from HDFS. Our infrastructure relies on more than 500 datanodes. Trino represents one of our most demanding HDFS workloads, processing analytical queries against data stored on HDFS. Due to the distributed nature of HDFS, each file you download requires a new connection to any of these 500 nodes. Therefore, during peak hours, a single pod can produce approximately 50,000 connections every minute. Pod churn. Many pods we spawn on the Kubernetes cluster run batch jobs, such as Spark jobs. They stay around for anywhere from a second to a couple of hours. Wide variety of workloads. Some workloads are very CPU-intensive, like Spark pods executing complex joins and tra ## A Deep Dive into Table partitioning Part 5: Partitioning an environment not individual tables DevFeed: [A Deep Dive into Table partitioning Part 5: Partitioning an environment not individual tables]() Original publisher: [Read original article]() Author: Adyen Published: 2025-09-17T10:29:53Z Content type: article Language: en Sources: [Adyen Tech]() Topics: [Databases](), [PostgreSQL](), [Framework]() Tags: [adyen](), [database](), [locks](), [partitioning](), [peformance](), [postgresql](), [table-partitioning]() ### AI overview The fifth article in a table-partitioning series explains why partitioning must account for relationships among tables in a PostgreSQL database, rather than treating tables as standalone. It also recaps earlier lessons about partition maintenance, minimizing locks, and the careful use of default partitions. ### Source excerpt By Derk van Veen, Database Engineer, Adyen If we had known better, this would have been the first blog post about partitioning to publish. The reality is, the first article was published two years and two weeks ago, as I write these opening sentences. So this article will not only give insight into the most important decisions you will have to make when considering your partitioning planning, but as a bonus, it will give some insights into how much it takes to write an article like this. One of my values is sharing knowledge and giving back to the community. I share what I learn and take people along on my journey and investigations into the problems we've faced. For this reason, the previous articles have documented these learnings, and this one is about a lesson we learned a little too late. The Journey so far Up to this point: We've shared the what, why and when to partition in the first blog post. In the second article, we covered how to maintain partitioned tables, as they provide some additional challenges when it comes to adding/removing partitions and managing indexes and foreign keys. Only in the third article did we disclose why we decided to work on our own partitioning framework: the existing partitioning solutions didn't provide us with the tools to minimize locks on tables. Our databases are always up and pressured, and the locks around partitioning were simply impossible to deal with. The key lesson from this article was that the default partition is the root of (almost) all evil regarding table partitioning in PostgreSQL. In the last article, we shared a cool solution to a problem we faced, where the default partition actually became the hero of the story. Yes, default partitions are usually the root of partitioning evil, but they also provide a welcome exception to the rule. Just be very careful with its usage. The Journey Continues... What ingredients can be missing after reading these four articles? You already know all about partitioning tables by ## Adyen API Diff Tool DevFeed: [Adyen API Diff Tool]() Original publisher: [Read original article]() Author: Adyen Published: 2025-08-19T08:33:01Z Content type: article Language: en Sources: [Adyen Tech]() Topics: [API](), [Tool](), [User Experience](), [Developer experience](), [HTTP]() Tags: [adyen](), [api](), [changelog](), [design](), [developer-experience](), [developers](), [features](), [http](), [integration](), [open-api](), [tool](), [versioning]() ### AI overview Adyen introduces an API Diff Tool that helps developers compare API versions, inspect changes by version, endpoint, and HTTP method, identify breaking changes, and navigate to related API reference details. ### Source excerpt Track new features, recent updates and breaking changes APIs evolve quickly with new features, design changes, and deprecations, making it challenging for consumers to keep up. This impact can be significant: developers might miss valuable improvements, misunderstand the behavior of new features, delay the adoption of critical compliance updates. This is why we have put passion and effort in the Adyen API Diff Tool, a new tool designed to help developers and technical users easily track and compare changes between API versions. Whether you're maintaining an existing integration or building something new, staying on top of API changes is critical. Previously, this information was scattered across various sources like product announcements, documentation, release notes, and GitHub, resulting in a suboptimal user experience. This changes today with our new powerful API Diff Tool. Features Our goal is simple: to make it easy to understand what's changed in the Adyen APIs between any two versions. The Diff Tool includes built-in filters to: Compare API versions View changes by version, endpoint and HTTP method Group changes by endpoint Spot quickly what's new, changed, or removed Focus on Breaking Changes only Labels, colors, and accordions provide a clean and effective way to visualize the changes. It is embedded directly in the API Explorer. From any specific change in the log, you can easily navigate to the relevant API reference or endpoint details. Benefits for Developers and other Users The API Diff Tool is designed to remove uncertainty when working with Adyen APIs. Whether you're integrating for the first time or upgrading to a new version, it is a valuable tool to: Understand what changed: compare any two API versions and see exactly what's been added, removed, or modified. Spot breaking changes early: identify changes that could impact your integration, so you can plan ahead. Evolve faster and safer: stay up-to-date with the latest API releases to integrate the ## A Developer's Guide to HMAC Validation for Adyen Webhooks DevFeed: [A Developer's Guide to HMAC Validation for Adyen Webhooks]() Original publisher: [Read original article]() Author: Adyen Published: 2025-06-16T09:38:43Z Content type: tutorial Language: en Sources: [Adyen Tech]() Topics: [Security](), [integrity](), [payload](), [implementation](), [JSON](), [HTTP](), [Testing](), [Open Source]() Tags: [adyen](), [api](), [authentication](), [best-practices](), [developer](), [encryption](), [guide](), [hmac](), [http](), [implementation](), [integration](), [json](), [open-source](), [payload](), [payments](), [security](), [testing](), [validation](), [webhooks]() ### AI overview This guide explains how Adyen uses HMAC signatures to protect the authenticity and integrity of webhook payloads. It distinguishes payment webhooks, which include the signature in the JSON payload, from other webhooks, which provide it in an HTTP header, and discusses custom validation and Adyen's open-source libraries. ### Source excerpt By Beppe Catanese, Developer Relations, AdyenImage by authorIntroduction When it comes to payments, security isn't optional -- it's essential. If you're integrating with Adyen, ensuring the incoming webhooks' authenticity and integrity is very important. That's where Hash-based Message Authentication Code (HMAC) plays a critical role in securing your Adyen integration. Image by author Implementing, testing, and troubleshooting HMAC validation can be challenging. This guide explains how HMAC validation works, highlights the challenges, and provides tools and best practices for secure and reliable implementation. HMAC at Adyen All Adyen webhooks use HMAC to ensure the integrity and authenticity of the payloads delivered to your integrations. The HMAC key should be enabled when setting up a new webhook (either in the Customer Area Webhook page or using the Management API). Adyen will use the HMAC key to sign the payload by creating an HMAC signature. You must validate the HMAC signature, delivered with the webhook, using the same HMAC key. Adyen webhooks fall into two main categories, each with its approach to HMAC implementation. Let's explore the two scenarios. 1. Payments Webhooks For payments-related webhooks, the calculation of the signature involves using a subset of fields, and it's embedded directly within the JSON payload under the `additionalData` object: { "live":"false", "notificationItems":[ { "NotificationRequestItem":{ "additionalData":{ "hmacSignature":"+JWKfq4ynALK+FFzGgHnp1jSMQJMBJeb87dlph24sXw=" }, ... } } ] }2. Other Webhooks (Adyen for Platforms, Management) For non-payment webhooks, the signature is calculated using the entire JSON payload. Instead of being included in the payload itself, the signature is provided in the HTTP Header `hmacSignature`: content-length: 1614 content-type: application/json hmacsignature: SMQZFOq3oIdugmf97u9TB+5256jjXgUX3MRjK+RlGNQ=Implementing HMAC Validation Developers have two options for implementing HMAC validation: ## Adyen's AI strategy and applied research engineering DevFeed: [Adyen's AI strategy and applied research engineering]() Original publisher: [Read original article]() Author: Adyen Published: 2025-06-05T07:26:51Z Content type: article Language: en Sources: [Adyen Tech]() Topics: [AI Strategy](), [Artificial Intelligence](), [Finance](), [Model Context Protocol]() Tags: [adyen](), [ai](), [ai-strategy](), [data-engineering](), [engineering](), [finance](), [payments](), [research]() ### AI overview The article describes Adyen's approach to applying AI in fintech, including AI-first productivity tooling and AI products such as Adyen Uplift. It frames applied research engineering as a way to connect AI developments with finance and large-scale production use. ### Source excerpt Andreu Mora - SVP / Global Head of Engineering Data, Adyen In this day and age, any company -- and especially tech companies -- needs to find a way to adapt and embrace AI. There's zero chance that the world and society won't change because of AI. On one side, we see how AI labs and the scientific community are spearheading the creation of frontier models and on the other side, how companies clearly see the value of AI but are still pinning down how they can apply these new technologies day to day in a meaningful way, for example in critical / large-scale production flows. Initiatives are being created and shipped that propel the technology forward in the right direction (e.g. MCP, A2A), but require network effects, outcome guarantees and other challenges to be solved in order to become a full-fledged reality. The world of fintech, and finance in general will change because of AI, and Adyen is exceptionally positioned -- given our tech-first mindset, scale and high quality data -- to play a primary role in this evolution. We have a great opportunity at being the fintech company that bridges the worlds of AI and finance. ✏ If you are new to Adyen, Adyen is a global financial technology platform, founded in Amsterdam in 2006, that powers end-to-end payments, data insights, and financial products for many of the world's leading businesses -- including Uber, eBay, and LinkedIn. We are a team of 4500 people who operate in 28 offices worldwide, serve customers in over 115 nationalities, and processed more than USD 1.4 trillion in payment volume in 2024 alone, with a steady growth of ~25% YOY. To this end, we are executing on our AI strategy, which roughly is comprised of three pillars: Adapting our ways of working and productivity tooling to be AI-first. We are doing this using AI products that accelerate our delivery and augment the depth of our work. As examples, we are using conference companions, code companions and AI that helps us search and interact with our knowledge ## Data Agent Benchmark for Multi-step Reasoning (DABstep) DevFeed: [Data Agent Benchmark for Multi-step Reasoning (DABstep)]() Original publisher: [Read original article]() Author: Adyen Published: 2025-02-05T09:19:27Z Content type: article Language: en Sources: [Adyen Tech]() Topics: [Benchmark](), [Data analysis](), [LLMs](), [AI Bots]() Tags: [adyen](), [agents](), [ai-models](), [benchmark](), [data](), [data-agents](), [data-analysis](), [data-science](), [genaimodels](), [hugging-face]() ### AI overview Adyen and Hugging Face introduce DABstep, a benchmark with more than 450 data analysis tasks for evaluating language models and AI agents on multi-step reasoning. The article reports that the strongest reasoning-based agents achieved 16% accuracy, indicating that complex real-world data analysis remains difficult for current models. ### Source excerpt Alex Egg (Adyen), Martin Iglesias (Adyen), Friso Kingma (Adyen), Andreu Mora (Adyen), Leandro Von Werra (HuggingFace), Thomas Wolf (HuggingFace) https://huggingface.co/spaces/adyen/DABstep 👉🏽 You can access DABStep with this link here: https://huggingface.co/spaces/adyen/DABstep Language models are becoming increasingly capable and can solve tasks autonomously as agents. There are many exciting use cases, especially at the intersection of reasoning, code, and data. However, proper evaluation benchmarks on real-world problems are lacking and hinder progress in the field. To tackle this challenge, Adyen and Hugging Face built the Data Agent Benchmark for Multi-step Reasoning (DABstep) together. DABstep consists of over 450 data analysis tasks designed to evaluate the capabilities of state-of-the-art LLMs and AI agents. Our findings reveal that DABstep presents a significant challenge for current AI models, with the most capable Reasoning-based agents achieving only 16% accuracy, highlighting significant progress to be made in the field. DABStep requires AI models to: dive in details of data and be rigorous (no hallucinations) reason over free form text and databases connect with real life use-cases (not just math or code) In this blog post, we'll cover the design and construction of the benchmark, explore evaluation results, and discuss the significant gap between current models and the ability to solve complex data analysis tasks effectively.. Motivation Data analysis is both an art and a science that requires technical skill, domain knowledge and creativity, and thus, it's rarely straightforward. Even seasoned data analysts face challenges like: Simple but time-consuming tasks: The sheer volume of even simple tasks often turns straightforward analysis into hours of repetitive work. Complex context and high cognitive load: Some tasks require analysis to juggle intricate domain-specific knowledge, making them both time-intensive and mentally draining. For example, (1) ## Database corruption in PostgreSQL: our journey to improving our upgrade process DevFeed: [Database corruption in PostgreSQL: our journey to improving our upgrade process]() Original publisher: [Read original article]() Author: Adyen Published: 2025-01-24T18:42:23Z Content type: article Language: en Sources: [Adyen Tech]() Topics: [PostgreSQL](), [Database](), [upgrade](), [Disk image](), [Transactions]() Tags: [adyen](), [backup](), [clusters](), [database-corruption](), [postgresql](), [scale](), [storage](), [toast-storage](), [transactions](), [upgrade]() ### AI overview Adyen engineers describe how corrupted PostgreSQL TOAST data surfaced after a database upgrade from version 9.6 to 13. The corruption affected millions of rows, could not be restored from a pre-upgrade backup, and led to a nearly year-long investigation into data access, detection, and upgrade-process improvements. ### Source excerpt by Cagri Biroglu & Derk van Veen, Database Engineers, Adyen PostgreSQL is a core component of Adyen's payment platform, enabling high-throughput, low-latency transaction processing on a global scale. Its stability, scalability, and extensibility make it a critical part of our infrastructure, supporting the reliability required for financial operations. A few years ago, we upgraded our PostgreSQL fleet from version 9.6 to 13. Upgrades are always long and tedious, but we prepared well for them, and the upgrade on even the hardest clusters with an extremely high number of TPS(transactions per second) and a size of multiple hundreds of TB(terabytes) went without major difficulties. But a few years later, we suddenly saw occasional errors related to transactions not being visible or invalid x_min or x_max. We tried to analyze the issue but didn't have much time, as transaction wraparound is always just around the corner in this environment. We had some quiet weeks, where no error was seen, and we thought that whatever it was, it was now behind us. We were wrong. TOAST storage The errors became a frequent nuisance, and we had no other option but to figure out what was happening. As it turned out, we had an issue with corrupted TOAST data entered in a table in the two weeks before we did the upgrade from 9.6 to 13 on this table's cluster. After the first analysis, we realized: This corruption affected millions of rows. We couldn't restore this table from a backup before the upgrade. If possible, we want to regain access to our data. We only knew about this corruption because of a clean-up job. It was unclear how many other tables were affected by the same issue. TOAST storage is not something we have understood and mastered yet. Transaction logic on TOAST is even more complicated than transaction logic on main tables. This is the start of a journey that will last almost a year. Background on TOAST storage Let's start with some background on TOAST data, as the corruptions a