# data-replication Published articles for data-replication. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## KCP: How to Migrate to Confluent Cloud in Days, Not Weeks DevFeed: [KCP: How to Migrate to Confluent Cloud in Days, Not Weeks]() Original publisher: [Read original article]() Author: Ahmed Saef Zamzam Published: 2026-09-14T07:00:00Z Content type: tutorial Language: en Sources: [Confluent: Data in motion]() Topics: [Kafka](), [migration](), [Confluent Cloud](), [Replication](), [Infrastructure as code](), [Provisioning](), [Access Control](), [Structured-data]() Tags: [access-control](), [confluent](), [confluent-cloud](), [data-replication](), [infrastructure-as-code-iac](), [kafka](), [migration](), [open-source](), [provisioning](), [replication](), [schema]() ### AI overview The article explains how Confluent's open source KCP tool automates migration from Amazon MSK to Confluent Cloud. KCP supports discovery, infrastructure provisioning, ACL and schema mapping, and migration, while Cluster Linking provides offset-preserving data replication. Support for self-managed Kafka migrations is described as coming soon. ### Source excerpt Use Kafka Copy Paste (KCP) to automate Kafka migration with infrastructure generation, ACL and schema mapping, and offset-preserving data replication. ## Building a GlobalStream: Replication Challenges and Optimizations DevFeed: [Building a GlobalStream: Replication Challenges and Optimizations]() Original publisher: [Read original article]() Author: Adyen Published: 2026-07-13T09:18:08Z Content type: article Language: en Sources: [Adyen Tech]() Topics: [Replication](), [Kafka](), [Streaming](), [Latency](), [Optimization](), [systems]() Tags: [data-replication](), [kafka](), [latency](), [mirrormaker](), [open-source](), [optimization](), [payments](), [replication](), [streaming](), [tech]() ### AI overview Adyen describes the challenges of replicating Kafka data across continents with MirrorMaker 2. The article covers data-location tradeoffs, Kafka Connect behavior, parameter tuning, and optimizations intended to sustain high-throughput replication while controlling latency. ### Source excerpt By Gaurav Singh & Luciano Sabença, Streaming Platform Team Global data streaming requires careful tradeoffs between performance, compliance, and reliability. At Adyen, replication is essential: data must remain close to producers to reduce latency and meet regulations, while also being centralized for analytics. In this blog, we share our experience optimizing MirrorMaker 2 for large-scale, cross-continent Kafka replication, the parameter tuning we applied, and the hidden behaviors in Kafka Connect that ultimately unlocked the required throughput. Why Replicate? There are many things you need to take into account when designing a data streaming platform for a company such as Adyen. One of the most important aspects of it is data location and replication. Due to performance, reliability, and compliance, keeping data close to producers is usually a good idea. After all, you don't want to wait in line at the cashier to confirm the payment for that fat burger you just bought on that nice beach in Australia while your payment was being sent to Europe. However, it's also a common practice to have all data in a centralized cluster(s) in centralized locations for analytical purposes. These two need a bridge, which is where mirroring comes in. An example replication flow diagramMirror Maker When it comes to copying data between Kafka clusters, Mirror Maker is the default open source tool. MirrorMaker 2 is the current version, and it's built on top of Kafka Connect, a platform designed to make it easier to integrate Kafka with other tools such as databases and distributed file systems, and -- why not? -- another Kafka cluster. Setting up Mirror Maker is fairly straightforward, but tuning it to the performance Adyen requires isn't. Let's go over the journey into the depths of Kafka Connect and parameter tuning to find a solution! Playing the Volume Game Kafka is a highly flexible tool and serves as the backbone for many different architectural patterns. It can be used for batch ## Essential Non Functional Requirements DevFeed: [Essential Non Functional Requirements]() Original publisher: [Read original article]() Author: Saurabh Dashora Published: 2026-07-07T08:33:34Z Content type: tutorial Language: en Sources: [System Design Codex]() Topics: [Requirements](), [Architecture & Design](), [Scalability](), [Availability](), [Latency](), [reliability](), [Replication](), [Users]() Tags: [availability](), [data-replication](), [failover](), [haproxy](), [kubernetes](), [latency](), [nginx](), [requirements]() ### AI overview This tutorial explains non-functional requirements as quality attributes that determine how software systems behave under real-world conditions. It covers availability, latency, scalability, and durability, with implementation approaches including load balancers, health checks, failover, CDNs, data replication, horizontal scaling, Kubernetes, and concurrent API design. ### Source excerpt Don't miss them ## When CDC Becomes a Crutch: A Philosophy on Data Replication DevFeed: [When CDC Becomes a Crutch: A Philosophy on Data Replication]() Original publisher: [Read original article]() Published: 2026-04-06T00:00:00Z Content type: opinion Language: en Sources: [Gaurav Sarma's Blog]() Topics: [Replication](), [consistency](), [Databases](), [Microservice](), [Kafka]() Tags: [consistency](), [data-replication](), [database](), [distributed](), [replication]() ### AI overview The article argues that Change Data Capture can become a crutch when teams replicate database tables across services without a clear data-ownership or consistency strategy. Although CDC tools make replication easy, maintaining and reconciling divergent replicas remains costly, especially after schema changes, pipeline lag, or ingestion bugs. ### Source excerpt Your orders service needs the user's email to send a confirmation. The payments service needs the user's tier to apply a discount... ## Data replication across backend services with Kafka and Protobuf DevFeed: [Data replication across backend services with Kafka and Protobuf]() Original publisher: [Read original article]() Author: Emmanuel Joubaud Published: 2023-04-14T10:01:57Z Content type: tutorial Language: en Sources: [JobTeaser]() Topics: [Back end](), [Kafka](), [Replication](), [data-architecture](), [API](), [Structured-data]() Tags: [api](), [architecture](), [backend](), [data](), [data-replication](), [distributed-systems](), [kafka](), [microservices](), [replication](), [serialization](), [serialization-format](), [software-architecture](), [software-engineering]() ### AI overview This tutorial explains Jobteaser's Silos architecture for replicating data across backend services. It describes using Kafka topics and Google Protobuf messages so consumer services can maintain local database copies while avoiding synchronous service-to-service API calls. ### Source excerpt The Jobteaser application contains a lot of different relatively independent modules to help universities provide career guidance to students: a job board, a career event management system, a career advice appointment management system... When we decided to migrate our application's backend from a monolith to a service-oriented architecture, we strived to keep each module as isolated as possible from the others in the event of an incident. If the career appointment system was down, students should still be able to browse and apply to job ads. That isolation is achieved through what we've called our Silos architecture. The gist is we avoid synchronous API calls between backend services, and prefer asynchronous data integration between services. There are a lot of different ways to implement asynchronous communication between services, and few companies share the details of theirs so we had to figure out a lot of stuff on our own. Now that we've refined our system, we thought we'd share the details of our approach, based on simple data replication using Kafka and Protobuf. How it worksWhen a write operation happens, data is changed first on the Source of Truth service, who then publishes it to a Kafka topic that consumers can subscribe to, in order to replicate their own local copy in their databaseSource of Truth and consumers Every table in our data model has an owner service, also called the Source of Truth (SoT) for that data, or the producer. For instance, the job board service may be the SoT for the job ad data. The owner service has 3 main responsibilities: it receives and validates all write requests for the data it owns (Create, Update, Delete) it stores the data to its own database, i.e. the Source of Truth, the authoritative state of the data if it's data that other services might want to access, it publishes its latest state as a message into a Kafka topic Consumer services that are interested in accessing a given table can then subscribe to the Kafka topic ## On Monoliths and Microservices DevFeed: [On Monoliths and Microservices]() Original publisher: [Read original article]() Published: 2015-10-01T00:00:00Z Content type: article Language: en Sources: [Juri Strumpflohner]() Topics: [Microservice](), [Microservices](), [Architecture & Design](), [REST API](), [Replication](), [data-governance](), [Availability]() Tags: [architecture](), [availability](), [communication](), [data-governance](), [data-replication](), [microservices](), [replication](), [rest](), [rest-api]() ### AI overview An overview of monolithic and microservices architecture that recommends vertically decomposed, self-contained services with independent teams and no shared state or database. It describes REST-based communication, data governance, replication through Atom feeds, and the trade-off between temporary inconsistency and service availability. ### Source excerpt Lorem ipsum dolor sit amet