# transaction-log

A database log that records transactions and their modifications, supporting recovery after system failures.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Three Records Your Support Team Needs After a Background Job Fails

DevFeed: [Three Records Your Support Team Needs After a Background Job Fails](<https://devfeed.tech/articles/three-records-your-support-team-needs-after-a-background-job-fails-143910.md>)

Original publisher: [Read original article](<https://hackernoon.com/three-records-your-support-team-needs-after-a-background-job-fails?source=rss>)

Author: Khakan Ali

Published: 2026-11-19T14:37:05Z

Content type: tutorial

Language: en

Sources: [HackerNoon](<https://devfeed.tech/sources/hackernoon.md>)

Topics: [Transactions](<https://devfeed.tech/topics/transactions.md>), [Test-driven development](<https://devfeed.tech/topics/tdd.md>), [Test automation](<https://devfeed.tech/topics/test-automation.md>), [transaction-log](<https://devfeed.tech/topics/transaction-log.md>), [Job](<https://devfeed.tech/topics/job.md>)

Tags: [acceptance](<https://devfeed.tech/tags/acceptance.md>), [background](<https://devfeed.tech/tags/background.md>), [background-jobs](<https://devfeed.tech/tags/background-jobs.md>), [build](<https://devfeed.tech/tags/build.md>), [catalog](<https://devfeed.tech/tags/catalog.md>), [change](<https://devfeed.tech/tags/change.md>), [code](<https://devfeed.tech/tags/code.md>), [commit](<https://devfeed.tech/tags/commit.md>), [complete](<https://devfeed.tech/tags/complete.md>), [crash](<https://devfeed.tech/tags/crash.md>), [crash-recovery](<https://devfeed.tech/tags/crash-recovery.md>), [crashes](<https://devfeed.tech/tags/crashes.md>), [customer](<https://devfeed.tech/tags/customer.md>), [data](<https://devfeed.tech/tags/data.md>), [database](<https://devfeed.tech/tags/database.md>), [database-transactions](<https://devfeed.tech/tags/database-transactions.md>), [distributed-systems](<https://devfeed.tech/tags/distributed-systems.md>), [fault-tolerance](<https://devfeed.tech/tags/fault-tolerance.md>), [idempotent-background-jobs](<https://devfeed.tech/tags/idempotent-background-jobs.md>), [idempotent-processing](<https://devfeed.tech/tags/idempotent-processing.md>), [learn](<https://devfeed.tech/tags/learn.md>), [python](<https://devfeed.tech/tags/python.md>), [python-sqlite-importer](<https://devfeed.tech/tags/python-sqlite-importer.md>), [retries](<https://devfeed.tech/tags/retries.md>), [sqlite](<https://devfeed.tech/tags/sqlite.md>), [tests](<https://devfeed.tech/tags/tests.md>), [transactions](<https://devfeed.tech/tags/transactions.md>)

### AI overview

The local Python/SQLite importer makes a failed job inspectable by saving three records: the accepted import's exact bytes and hash, each row committed with its catalog change in one transaction, and an outcome derived from completed-row counts. Reusing the same import ID and bytes safely skips completed rows in this single-database design; changed bytes under that ID are rejected. Process-exit tests show that a retry completes work after a pre-commit failure, and after a post-commit failure it preserves the committed row without rewriting it, even when acknowledgment was absent. Competing local workers produced exactly two catalog writes for two rows, with SQLite serializing writes and a 15-second lock wait. The example ran locally with Python 3.13.3 and SQLite 3.49.2. It does not establish safety for remote side effects, distributed systems, power loss, or production deployment; an uncertain remote action requires provider idempotency behavior and reconciliation.

### Source excerpt

Learn how to build a crash-recoverable Python and SQLite importer using transactions, idempotent retries, and tests for interrupted background jobs.

## Optimistic Deadlock Handling with Redo Log-Based Speculative Versioning under High Contention

DevFeed: [Optimistic Deadlock Handling with Redo Log-Based Speculative Versioning under High Contention](<https://devfeed.tech/articles/optimistic-deadlock-handling-with-redo-log-based-speculative-versioning-under-high-contention-132676.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2607.19697>)

Author: Hyejin Yoo, Seongjae Moon, Jonghyeok Park, Viktor Leis, Sang-Won Lee

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Databases](<https://devfeed.tech/sources/arxiv-databases.md>)

Topics: [Deadlock](<https://devfeed.tech/topics/deadlock.md>), [Distributed Systems & Parallel Computing](<https://devfeed.tech/topics/distributed-systems-parallel-computing.md>), [versioning](<https://devfeed.tech/topics/versioning.md>), [transaction-log](<https://devfeed.tech/topics/transaction-log.md>)

Tags: [concurrency-control](<https://devfeed.tech/tags/concurrency-control.md>), [cs-db](<https://devfeed.tech/tags/cs-db.md>), [deadlock](<https://devfeed.tech/tags/deadlock.md>), [dependency-graph](<https://devfeed.tech/tags/dependency-graph.md>), [detection](<https://devfeed.tech/tags/detection.md>), [log](<https://devfeed.tech/tags/log.md>), [mysql](<https://devfeed.tech/tags/mysql.md>), [oltp](<https://devfeed.tech/tags/oltp.md>), [ordering](<https://devfeed.tech/tags/ordering.md>), [postgresql](<https://devfeed.tech/tags/postgresql.md>), [robust](<https://devfeed.tech/tags/robust.md>), [threads](<https://devfeed.tech/tags/threads.md>), [throughput](<https://devfeed.tech/tags/throughput.md>), [tpc-c](<https://devfeed.tech/tags/tpc-c.md>), [transaction](<https://devfeed.tech/tags/transaction.md>), [transactions](<https://devfeed.tech/tags/transactions.md>), [updates](<https://devfeed.tech/tags/updates.md>), [using](<https://devfeed.tech/tags/using.md>), [version](<https://devfeed.tech/tags/version.md>), [versioning](<https://devfeed.tech/tags/versioning.md>), [waiting](<https://devfeed.tech/tags/waiting.md>)

### AI overview

RCC aims to improve concurrency under high write contention by creating speculative record versions from redo log entries, allowing conflicting transactions to proceed without waiting for predecessors to commit. It tracks dependencies and orders conflicts at access time to ensure serializability, installs updates lazily at commit, and handles aborts by discarding speculative versions. Its optimistic deadlock handling grants locks before asynchronously checking the dependency graph for cycles. RCC-C uses column-level redo logs to avoid record-level false conflicts. The authors report implementation on MySQL, PostgreSQL, and DBx1000; in TPC-C at 128 threads on a 128-core server, RCC achieves up to 3.3x the throughput of state-of-the-art schemes, with further improvement from RCC-C. On high-contention YCSB, RCC remains robust through 128 threads while other schemes degrade beyond 32.

### Source excerpt

arXiv:2607.19697v2 Announce Type: replace Abstract: Modern OLTP engines rely on multi-versioning to eliminate read-write conflicts, yet their concurrency is severely limited by write-write conflicts. With immediate, in-place updates, only one transaction updates a record at a time, and the others wait for it to commit or abort. Conflicting transactions are thus serialized. We propose RCC, a Redo log-based Concurrency Control scheme for high write contention. A transaction updates a record out of place by creating a speculative write version from its redo log entry. With multiple speculative versions per record, conflicting transactions are pipelined without waiting for predecessors to commit. RCC guarantees serializability using lock-based access-time conflict ordering and dependency graph tracking. Updates are installed lazily at commit, so a transaction aborts by discarding its speculative versions. Based on this, RCC introduces two novel techniques. Optimistic deadlock handling decouples deadlock detection from lock acquisition: it grants locks without checking for deadlocks and detects cycles asynchronously in the dependency graph. Columnar RCC (RCC-C) removes record-level false conflicts with column-level redo logs. We implemented RCC on MySQL, PostgreSQL, and DBx1000. In TPC-C at 128 threads on a 128-core server, RCC improves throughput by up to 3.3x over the state-of-the-art schemes, and RCC-C improves it further by avoiding false deadlocks and aborts. On high-contention YCSB, RCC remains robust up to 128 threads, whereas other schemes degrade beyond 32.

## Why disaster recovery takes longer than your RTO (9 hidden causes)

DevFeed: [Why disaster recovery takes longer than your RTO (9 hidden causes)](<https://devfeed.tech/articles/why-disaster-recovery-takes-longer-than-your-rto-9-hidden-causes-129325.md>)

Original publisher: [Read original article](<https://www.red-gate.com/simple-talk/databases/rto-why-recovery-takes-longer/>)

Author: Edward Pollack

Published: 2026-11-11T12:00:00Z

Content type: article

Language: en

Sources: [Simple Talk](<https://devfeed.tech/sources/simple-talk.md>)

Topics: [Disaster Recovery](<https://devfeed.tech/topics/disaster-recovery.md>), [downtime](<https://devfeed.tech/topics/downtime.md>), [transaction-log](<https://devfeed.tech/topics/transaction-log.md>), [deprecated](<https://devfeed.tech/topics/deprecated.md>)

Tags: [2](<https://devfeed.tech/tags/2.md>), [2026](<https://devfeed.tech/tags/2026.md>), [3](<https://devfeed.tech/tags/3.md>), [about](<https://devfeed.tech/tags/about.md>), [actions](<https://devfeed.tech/tags/actions.md>), [cause](<https://devfeed.tech/tags/cause.md>), [database](<https://devfeed.tech/tags/database.md>), [database-administration](<https://devfeed.tech/tags/database-administration.md>), [databases](<https://devfeed.tech/tags/databases.md>), [disaster](<https://devfeed.tech/tags/disaster.md>), [disaster-recovery](<https://devfeed.tech/tags/disaster-recovery.md>), [downtime](<https://devfeed.tech/tags/downtime.md>), [featured](<https://devfeed.tech/tags/featured.md>), [first](<https://devfeed.tech/tags/first.md>), [hidden](<https://devfeed.tech/tags/hidden.md>), [learn](<https://devfeed.tech/tags/learn.md>), [post](<https://devfeed.tech/tags/post.md>), [recovery](<https://devfeed.tech/tags/recovery.md>), [rto](<https://devfeed.tech/tags/rto.md>), [simple](<https://devfeed.tech/tags/simple.md>), [takes](<https://devfeed.tech/tags/takes.md>), [talk](<https://devfeed.tech/tags/talk.md>), [time](<https://devfeed.tech/tags/time.md>), [you-ll](<https://devfeed.tech/tags/you-ll.md>)

### AI overview

RTO estimates often fall short because they count restore time but omit the full path back to working service. The article defines recovery time as detection (including monitoring sample intervals), response, restore or failover, application reconnection, and validation. It recommends documenting response roles and automation; testing backup restores and timing them at realistic scale; checking storage and network capacity; planning for failover and scripted application changes; tracking encryption keys, certificates, subscriptions, licenses, and cloud service retirements; and validating recovered data and applications before users return. It warns that a system can appear available while using the wrong restore point, creating data loss and difficult recovery choices. RTO is user-facing application availability, while RPO concerns acceptable data loss.

### Source excerpt

Recovery Time Objective (RTO) is the maximum downtime you'll accept. Learn why real recovery runs longer, and the nine gaps that cause it.... The post Why disaster recovery takes longer than your RTO (9 hidden causes) appeared first on Simple Talk.

## PROTECT-DB: Protecting Data using Replicated State Machines: Efficient Corruption Detection & Recovery

DevFeed: [PROTECT-DB: Protecting Data using Replicated State Machines: Efficient Corruption Detection & Recovery](<https://devfeed.tech/articles/protect-db-protecting-data-using-replicated-state-machines-efficient-corruption-detection-recovery-125659.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2605.11953>)

Author: Anant Utgikar, S. Sudarshan

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>), [arXiv: Databases](<https://devfeed.tech/sources/arxiv-databases.md>)

Topics: [Data Management](<https://devfeed.tech/topics/data-management.md>), [Transactions](<https://devfeed.tech/topics/transactions.md>), [transaction-log](<https://devfeed.tech/topics/transaction-log.md>), [threat detection](<https://devfeed.tech/topics/threat-detection.md>), [Disaster Recovery](<https://devfeed.tech/topics/disaster-recovery.md>)

Tags: [blockchain](<https://devfeed.tech/tags/blockchain.md>), [concurrent](<https://devfeed.tech/tags/concurrent.md>), [corruption](<https://devfeed.tech/tags/corruption.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cs-db](<https://devfeed.tech/tags/cs-db.md>), [data](<https://devfeed.tech/tags/data.md>), [database](<https://devfeed.tech/tags/database.md>), [databases](<https://devfeed.tech/tags/databases.md>), [detection](<https://devfeed.tech/tags/detection.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [execution](<https://devfeed.tech/tags/execution.md>), [extension](<https://devfeed.tech/tags/extension.md>), [log](<https://devfeed.tech/tags/log.md>), [postgresql](<https://devfeed.tech/tags/postgresql.md>), [recovery](<https://devfeed.tech/tags/recovery.md>), [repair](<https://devfeed.tech/tags/repair.md>), [replica](<https://devfeed.tech/tags/replica.md>), [state-machines](<https://devfeed.tech/tags/state-machines.md>), [study](<https://devfeed.tech/tags/study.md>), [system](<https://devfeed.tech/tags/system.md>), [transactions](<https://devfeed.tech/tags/transactions.md>)

### AI overview

PROTECT-DB uses Byzantine-fault-tolerant replicated state machines built on a deterministic PostgreSQL extension to protect database state. Replicas deterministically execute transactions recorded in a shared log or blockchain. The system aims to detect corruption quickly and repair it while transaction execution continues. The abstract says a performance study demonstrates efficiency and practicality but provides no measurements or detailed results.

### Source excerpt

arXiv:2605.11953v2 Announce Type: replace Abstract: Data is critical for the operation of any organization and needs to be protected, especially against attacks that compromise the state of the database. In this paper, we explore an approach based on Byzantine-fault tolerant replicated state machines, built on top of a deterministic extension of PostgreSQL. Each replica deterministically executes transactions recorded in a shared log/blockchain. Our focus is on creating a practical system that is designed for efficient and quick detection of corruption, as well as quick repair concurrent with execution of transactions. We also present a performance study showing the efficiency and practicality of our approach. We believe our work lays the foundations for the practical use of the BFT replicated state machine approach in the context of databases.

## Change Data Capture in SQLite, Without the Triggers

DevFeed: [Change Data Capture in SQLite, Without the Triggers](<https://devfeed.tech/articles/change-data-capture-in-sqlite-without-the-triggers-121133.md>)

Original publisher: [Read original article](<https://turso.tech/blog/change-data-capture-in-sqlite-without-the-triggers>)

Author: Jeff Olson

Published: 2026-11-05T00:00:00Z

Content type: article

Language: en

Sources: [Turso](<https://devfeed.tech/sources/turso-blog.md>)

Topics: [Turso](<https://devfeed.tech/topics/turso.md>), [SQLite](<https://devfeed.tech/topics/sqlite.md>), [Event-Streaming](<https://devfeed.tech/topics/event-streaming.md>), [event-driven data processing](<https://devfeed.tech/topics/event-driven-data-processing.md>), [transaction-log](<https://devfeed.tech/topics/transaction-log.md>), [Publish-subscribe pattern](<https://devfeed.tech/topics/pubsub.md>), [Debezium Connector](<https://devfeed.tech/topics/debezium-connector.md>)

Tags: [2](<https://devfeed.tech/tags/2.md>), [built-in](<https://devfeed.tech/tags/built-in.md>), [capture](<https://devfeed.tech/tags/capture.md>), [cdc](<https://devfeed.tech/tags/cdc.md>), [change](<https://devfeed.tech/tags/change.md>), [change-data-capture](<https://devfeed.tech/tags/change-data-capture.md>), [consumer](<https://devfeed.tech/tags/consumer.md>), [data](<https://devfeed.tech/tags/data.md>), [database](<https://devfeed.tech/tags/database.md>), [delete](<https://devfeed.tech/tags/delete.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [here](<https://devfeed.tech/tags/here.md>), [how-to](<https://devfeed.tech/tags/how-to.md>), [insert](<https://devfeed.tech/tags/insert.md>), [records](<https://devfeed.tech/tags/records.md>), [software-architecture](<https://devfeed.tech/tags/software-architecture.md>), [sqlite](<https://devfeed.tech/tags/sqlite.md>), [triggers](<https://devfeed.tech/tags/triggers.md>), [turso](<https://devfeed.tech/tags/turso.md>), [update](<https://devfeed.tech/tags/update.md>), [works](<https://devfeed.tech/tags/works.md>), [write](<https://devfeed.tech/tags/write.md>)

### AI overview

Turso Database adds built-in SQLite CDC that records committed inserts, updates, and deletes in `turso_cdc` without per-table triggers. Consumers can poll the ordered stream, group records by transaction, and resume from a saved watermark; they should make processing idempotent because crashes can replay already-applied changes. Modes retain primary keys, before or after images, or full details, and schema changes are recorded. The feature is actively developed: pin versions, and snapshot column lists before migrations because decoding old records with today's columns can misalign them. CDC captures row facts, not domain meaning or user identity; those require application-side context. The excerpt also cautions that retention cleanup on synced databases may remove changes Sync has not pushed.

### Source excerpt

Built-in CDC in Turso Database records every insert, update, and delete for you. Here is how it works, where it is rough, and how to write a consumer that survives restarts.

## AWS contributes a large-transaction binary log optimization to MySQL

DevFeed: [AWS contributes a large-transaction binary log optimization to MySQL](<https://devfeed.tech/articles/aws-contributes-a-large-transaction-binary-log-optimization-to-mysql-112630.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/database/aws-contributes-a-large-transaction-binary-log-optimization-to-mysql/>)

Author: Xueting Wu

Published: 2026-11-02T16:57:08Z

Content type: news

Language: en

Sources: [AWS Database Blog](<https://devfeed.tech/sources/aws-database-blog.md>)

Topics: [MySQL](<https://devfeed.tech/topics/mysql.md>), [transaction-log](<https://devfeed.tech/topics/transaction-log.md>), [database-optimization](<https://devfeed.tech/topics/database-optimization.md>), [Amazon Aurora](<https://devfeed.tech/topics/amazon-aurora.md>), [Amazon Web Services](<https://devfeed.tech/topics/aws.md>), [Amazon EC2](<https://devfeed.tech/topics/amazon-ec2.md>)

Tags: [advanced-300](<https://devfeed.tech/tags/advanced-300.md>), [amazon-aurora](<https://devfeed.tech/tags/amazon-aurora.md>), [announcements](<https://devfeed.tech/tags/announcements.md>), [aws](<https://devfeed.tech/tags/aws.md>), [bolt](<https://devfeed.tech/tags/bolt.md>), [crash-recovery](<https://devfeed.tech/tags/crash-recovery.md>), [large](<https://devfeed.tech/tags/large.md>), [latency](<https://devfeed.tech/tags/latency.md>), [mysql](<https://devfeed.tech/tags/mysql.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [rds-for-mysql](<https://devfeed.tech/tags/rds-for-mysql.md>), [throughput](<https://devfeed.tech/tags/throughput.md>), [transaction](<https://devfeed.tech/tags/transaction.md>)

### AI overview

BOLT removes the large-transaction binlog copy that can block all MySQL commits: once a transaction exceeds a configurable threshold (128 MB by default), MySQL promotes its spill file into the binlog sequence. A header event lets crash recovery seek to the transaction's end instead of scanning its contents. In the described controlled benchmarks, a 50 GiB transaction's commit took about 2.5 seconds with BOLT versus about 7 minutes without it; binlog recovery took about 0.1 seconds versus about 4 minutes. Throughput also rose in tested workloads containing large transactions. These results are benchmark-specific, and remaining commit time includes durable spill-file writes and InnoDB commit. BOLT is enabled by default in Aurora MySQL; a future MySQL innovation release is planned. Large transactions can still reduce replica parallelism and cause lag, so the post recommends batching them where the application allows.

### Source excerpt

When MySQL commits a large transaction, writing it to the binary log can freeze every other commit on the server for minutes. This post introduces Binary Log Optimization for Large Transactions (BOLT), an AWS contribution to community MySQL that makes commit latency independent of transaction size, speeds crash recovery, and improves throughput.

## Turns out database backups are NOT trivial

DevFeed: [Turns out database backups are NOT trivial](<https://devfeed.tech/articles/turns-out-database-backups-are-not-trivial-113330.md>)

Original publisher: [Read original article](<https://stack.convex.dev/database-backups-are-not-trivial>)

Author: Stack

Published: 2026-10-29T15:30:13Z

Content type: article

Language: en

Sources: [Stack](<https://devfeed.tech/sources/stack.md>)

Topics: [backups](<https://devfeed.tech/topics/backups.md>), [Database](<https://devfeed.tech/topics/database.md>), [database-consistency](<https://devfeed.tech/topics/database-consistency.md>), [transaction-log](<https://devfeed.tech/topics/transaction-log.md>), [Transactions](<https://devfeed.tech/topics/transactions.md>), [DuckDB](<https://devfeed.tech/topics/duckdb.md>), [clickhouse](<https://devfeed.tech/topics/clickhouse.md>)

Tags: [abstraction](<https://devfeed.tech/tags/abstraction.md>), [agent](<https://devfeed.tech/tags/agent.md>), [algorithm](<https://devfeed.tech/tags/algorithm.md>), [analytics](<https://devfeed.tech/tags/analytics.md>), [api](<https://devfeed.tech/tags/api.md>), [backing-up](<https://devfeed.tech/tags/backing-up.md>), [backup](<https://devfeed.tech/tags/backup.md>), [backups](<https://devfeed.tech/tags/backups.md>), [convex](<https://devfeed.tech/tags/convex.md>), [data](<https://devfeed.tech/tags/data.md>), [database](<https://devfeed.tech/tags/database.md>), [production](<https://devfeed.tech/tags/production.md>), [tables](<https://devfeed.tech/tags/tables.md>)

### AI overview

A row-by-row backup of a changing database can combine records from different moments and describe a state that never existed. Convex's data sync avoids this by alternating between reading table rows and catching up on the document log, keeping its snapshot within the retention window; it reports `upToDate` when the copy is consistent. Pages may contain repeated rows and arrive out of timestamp order, so callers must apply truncates and values and save pages safely. The endpoint requires a Pro plan and charges for database reads and bandwidth; its cursor can resume syncing within the deployment's retention window. Convex also offers dashboard and CLI backups, scheduled backups on Pro, and faster, cheaper physical backups for dedicated deployments, though those cannot be downloaded or inspected. For analytics at scale, the article says Convex still recommends its Fivetran connector.

### Source excerpt

I thought backing up a database was simple, you just loop over the tables and copy the rows out. Nipunn sat me down at a whiteboard and showed me why that can give you a backup of data that never existed, and how Convex gets around it.

## Data backup and recovery: backup vs. replication and when to use both

DevFeed: [Data backup and recovery: backup vs. replication and when to use both](<https://devfeed.tech/articles/data-backup-and-recovery-backup-vs-replication-and-when-to-use-both-106254.md>)

Original publisher: [Read original article](<https://www.veeam.com/blog/data-backup-and-recovery.html>)

Author: Dylan Sams

Published: 2026-10-28T14:08:53Z

Content type: article

Language: en

Sources: [Veeam Software Official Blog](<https://devfeed.tech/sources/veeam-software-official-blog.md>)

Topics: [backups](<https://devfeed.tech/topics/backups.md>), [Featured-Backing Up](<https://devfeed.tech/topics/featured-backing-up.md>), [Disaster Recovery](<https://devfeed.tech/topics/disaster-recovery.md>), [copy](<https://devfeed.tech/topics/copy.md>), [transaction-log](<https://devfeed.tech/topics/transaction-log.md>), [Data integrity in software engineering](<https://devfeed.tech/topics/data-integrity-in-software-engineering.md>)

Tags: [applications](<https://devfeed.tech/tags/applications.md>), [backup](<https://devfeed.tech/tags/backup.md>), [backup-and-recovery](<https://devfeed.tech/tags/backup-and-recovery.md>), [backup-and-replication](<https://devfeed.tech/tags/backup-and-replication.md>), [backup-and-replication-software](<https://devfeed.tech/tags/backup-and-replication-software.md>), [backup-and-restore-software](<https://devfeed.tech/tags/backup-and-restore-software.md>), [backups](<https://devfeed.tech/tags/backups.md>), [bandwidth](<https://devfeed.tech/tags/bandwidth.md>), [bare-metal](<https://devfeed.tech/tags/bare-metal.md>), [business](<https://devfeed.tech/tags/business.md>), [data-backup](<https://devfeed.tech/tags/data-backup.md>), [data-backup-and-recovery](<https://devfeed.tech/tags/data-backup-and-recovery.md>), [data-backup-and-recovery-solutions](<https://devfeed.tech/tags/data-backup-and-recovery-solutions.md>), [database-backup-and-recovery](<https://devfeed.tech/tags/database-backup-and-recovery.md>), [recovery](<https://devfeed.tech/tags/recovery.md>), [replication](<https://devfeed.tech/tags/replication.md>), [veeam](<https://devfeed.tech/tags/veeam.md>)

### AI overview

Backup preserves historical restore points; replication keeps a standby that can resume service faster but also carries forward deletion, corruption, and encryption. The article recommends choosing protection by each workload's recovery point objective (RPO) and recovery time objective (RTO), with top-tier workloads using both. A completed backup job does not prove its contents are restorable or clean: confirm coverage, then test restore points in an isolated network with application checks, malware scanning, clean-point identification, and immutability. Enterprise recovery also requires workload coverage, database-aware processing and transaction logs, and coordinated runbooks. Veeam Data Platform is presented as managing backup and replication under one policy layer; edition capabilities differ, and the excerpt advises checking current release support before relying on specific features.

### Source excerpt

Your jobs completed last night. Retention is enforced, repositories are provisioned, and the dashboard has been green all week. None of that answers the two questions you'll get in the first 10 minutes of an incident: How fast can we be back? And back to what? The post Data backup and recovery: backup vs. replication and when to use both appeared first on Veeam Software Official Blog.

## Database Replication: Types, Benefits & How It Works

DevFeed: [Database Replication: Types, Benefits & How It Works](<https://devfeed.tech/articles/database-replication-types-benefits-how-it-works-105579.md>)

Original publisher: [Read original article](<https://talent500.com/blog/database-replication/>)

Author: Ekakshar

Published: 2026-10-28T10:51:10Z

Content type: article

Language: en

Sources: [Talent500](<https://devfeed.tech/sources/backend-archives-talent500-blog.md>)

Topics: [copy](<https://devfeed.tech/topics/copy.md>), [Database](<https://devfeed.tech/topics/database.md>), [read-replica](<https://devfeed.tech/topics/read-replica.md>), [Disaster Recovery](<https://devfeed.tech/topics/disaster-recovery.md>), [database-consistency](<https://devfeed.tech/topics/database-consistency.md>), [database-scalability](<https://devfeed.tech/topics/database-scalability.md>), [transaction-log](<https://devfeed.tech/topics/transaction-log.md>), [Distributed Systems & Parallel Computing](<https://devfeed.tech/topics/distributed-systems-parallel-computing.md>), [Debezium Connector](<https://devfeed.tech/topics/debezium-connector.md>), [Apache Cassandra](<https://devfeed.tech/topics/cassandra.md>)

Tags: [availability](<https://devfeed.tech/tags/availability.md>), [backend](<https://devfeed.tech/tags/backend.md>), [challenges](<https://devfeed.tech/tags/challenges.md>), [data](<https://devfeed.tech/tags/data.md>), [database](<https://devfeed.tech/tags/database.md>), [database-replication](<https://devfeed.tech/tags/database-replication.md>), [disaster](<https://devfeed.tech/tags/disaster.md>), [disaster-recovery](<https://devfeed.tech/tags/disaster-recovery.md>), [failover](<https://devfeed.tech/tags/failover.md>), [first](<https://devfeed.tech/tags/first.md>), [helps](<https://devfeed.tech/tags/helps.md>), [multiple](<https://devfeed.tech/tags/multiple.md>), [practices](<https://devfeed.tech/tags/practices.md>), [recovery](<https://devfeed.tech/tags/recovery.md>), [reliability](<https://devfeed.tech/tags/reliability.md>), [replication](<https://devfeed.tech/tags/replication.md>), [scalability](<https://devfeed.tech/tags/scalability.md>), [servers](<https://devfeed.tech/tags/servers.md>), [synchronization](<https://devfeed.tech/tags/synchronization.md>), [types](<https://devfeed.tech/tags/types.md>), [works](<https://devfeed.tech/tags/works.md>)

### AI overview

Database replication copies and propagates data across database instances to support availability, disaster recovery, read scaling and regional access. The article distinguishes primary-replica setups, which concentrate writes on one primary, from multi-primary and peer-to-peer designs that distribute writes but require conflict handling; cascading replication can reduce direct primary connections while adding downstream lag. Synchronous replication waits for configured replica acknowledgements and can reduce loss of acknowledged transactions at the cost of latency or blocked writes. Asynchronous replication lowers write delays but permits stale reads and possible loss of recent changes during failover; semi-synchronous guarantees depend on what the database acknowledges. Implementation typically initializes replicas from a snapshot or backup, captures changes through logs or CDC, transfers and applies them, then monitors lag, errors and log retention. Replicas do not replace backups, since deletions or corruption may propagate. Reliable deployments also need routing and promotion procedures, tested recovery, consistency choices, access controls and cost planning. PostgreSQL, MySQL

### Source excerpt

Database Replication helps maintain data availability and reliability by copying data across multiple database servers. Explore replication types, synchronization methods, failover strategies, benefits, and challenges, along with best practices for improving scalability, fault tolerance, and disaster recovery. The post Database Replication: Types, Benefits & How It Works appeared first on Talent500 blog.

## Expanding Customer Notifications for ClickHouse Managed Postgres

DevFeed: [Expanding Customer Notifications for ClickHouse Managed Postgres](<https://devfeed.tech/articles/expanding-customer-notifications-for-clickhouse-managed-postgres-101161.md>)

Original publisher: [Read original article](<https://clickhouse.com/blog/expanding-customer-notifications-managed-postgres>)

Author: Amog Iska

Published: 2026-10-26T15:22:20Z

Content type: release

Language: en

Sources: [ClickHouse Blog](<https://devfeed.tech/sources/clickhouse-blog.md>)

Topics: [notifications](<https://devfeed.tech/topics/notifications.md>), [downtime](<https://devfeed.tech/topics/downtime.md>), [transaction-log](<https://devfeed.tech/topics/transaction-log.md>), [Heroku Postgres](<https://devfeed.tech/topics/heroku-postgres.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [22](<https://devfeed.tech/tags/22.md>), [about](<https://devfeed.tech/tags/about.md>), [act](<https://devfeed.tech/tags/act.md>), [alert](<https://devfeed.tech/tags/alert.md>), [alerts](<https://devfeed.tech/tags/alerts.md>), [also](<https://devfeed.tech/tags/also.md>), [analytics](<https://devfeed.tech/tags/analytics.md>), [application](<https://devfeed.tech/tags/application.md>), [applications](<https://devfeed.tech/tags/applications.md>), [clickhouse](<https://devfeed.tech/tags/clickhouse.md>), [customer](<https://devfeed.tech/tags/customer.md>), [limit](<https://devfeed.tech/tags/limit.md>), [long-running](<https://devfeed.tech/tags/long-running.md>), [managed](<https://devfeed.tech/tags/managed.md>), [notifications](<https://devfeed.tech/tags/notifications.md>), [postgres](<https://devfeed.tech/tags/postgres.md>), [replication](<https://devfeed.tech/tags/replication.md>), [running](<https://devfeed.tech/tags/running.md>), [transactions](<https://devfeed.tech/tags/transactions.md>)

### AI overview

ClickHouse Managed Postgres is adding five notifications to help identify storage, availability, transaction, and connection issues. An inactive-slot alert will flag slots with at least 10 GiB of unconfirmed WAL observed over two hours, excluding slots identified as its managed CDC service; for ClickPipes, the text says WalShadow uses physical WAL replication and avoids logical slots. Planned-switchover alerts identify the affected service after a standby takes over, such as during scaling; unplanned-failover alerts report recovery after a primary becomes unavailable. Both interrupt existing connections, but applications can reconnect using the same connection string. A long-running-transaction alert covers active or idle transactions open for at least two hours, which may hold locks or impede vacuum. A connection alert triggers at 95% of the limit; suggested checks include application sessions and pool settings. The text says delivery options for notifications currently available include console, email, and Slack.

### Source excerpt

ClickHouse Managed Postgres is adding five new customer notifications: inactive replication slots, planned switchovers, unplanned failovers, long-running transactions, and connections approaching their limit.

## The paper that split the database in four: inside Azure SQL Database Hyperscale, aka Socrates

DevFeed: [The paper that split the database in four: inside Azure SQL Database Hyperscale, aka Socrates](<https://devfeed.tech/articles/the-paper-that-split-the-database-in-four-inside-azure-sql-database-hyperscale-aka-socrates-101493.md>)

Original publisher: [Read original article](<https://devblogs.microsoft.com/azure-sql/the-paper-that-split-the-database-in-four-inside-azure-sql-database-hyperscale-aka-socrates/>)

Author: Anna Hoffman

Published: 2026-10-25T11:23:29Z

Content type: article

Language: en

Sources: [Azure SQL Dev Corner](<https://devfeed.tech/sources/azure-sql-dev-corner.md>)

Topics: [Azure SQL](<https://devfeed.tech/topics/azure-sql.md>), [transaction-log](<https://devfeed.tech/topics/transaction-log.md>), [sql-server](<https://devfeed.tech/topics/sql-server.md>), [read-replica](<https://devfeed.tech/topics/read-replica.md>), [database-optimization](<https://devfeed.tech/topics/database-optimization.md>), [sql-database](<https://devfeed.tech/topics/sql-database.md>), [consistency](<https://devfeed.tech/topics/consistency.md>), [Azure](<https://devfeed.tech/topics/azure.md>)

Tags: [also](<https://devfeed.tech/tags/also.md>), [azure](<https://devfeed.tech/tags/azure.md>), [azure-sql](<https://devfeed.tech/tags/azure-sql.md>), [azure-sql-database](<https://devfeed.tech/tags/azure-sql-database.md>), [brand](<https://devfeed.tech/tags/brand.md>), [but](<https://devfeed.tech/tags/but.md>), [database](<https://devfeed.tech/tags/database.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [first](<https://devfeed.tech/tags/first.md>), [hood](<https://devfeed.tech/tags/hood.md>), [hyperscale](<https://devfeed.tech/tags/hyperscale.md>), [inside](<https://devfeed.tech/tags/inside.md>), [microsoft](<https://devfeed.tech/tags/microsoft.md>), [microsoft-research](<https://devfeed.tech/tags/microsoft-research.md>), [microsoft-sql](<https://devfeed.tech/tags/microsoft-sql.md>), [paper](<https://devfeed.tech/tags/paper.md>), [partners](<https://devfeed.tech/tags/partners.md>), [split](<https://devfeed.tech/tags/split.md>), [sql](<https://devfeed.tech/tags/sql.md>), [sql-database](<https://devfeed.tech/tags/sql-database.md>)

### AI overview

Azure SQL Database Hyperscale, codenamed Socrates, avoids the full-replica architecture's data-sized limits by splitting query processing, transaction logging, data pages, and durable storage across compute nodes, XLOG, page servers, and XStore. Compute nodes cache pages in memory and RBPEX; on a cache miss, GetPage@LSN asks a page server for a page current to a specified log position. XLOG distributes committed log records, while XStore snapshots support backups. This design removes full data copying when adding compute or replicas and makes backup and restore less dependent on database size. The text reports scale operations changing from O(data) to O(1), about half the storage overhead, and higher update-heavy log throughput than the full-replica design. It also cites skewed workloads where small caches served a larger share of reads. Hyperscale retains T-SQL compatibility, with tradeoffs involving remote page fetches and reliance on caching; the excerpt gives no detailed operational comparison of those effects.

### Source excerpt

Under the Hood: The Research Behind Microsoft SQL, part 1 The Microsoft SQL team is always innovating. They do this in day-to-day efforts, but there are also brand new efforts that are constantly being explored, investigated, tested, and researched. Our engineering team partners closely with MSR (Microsoft Research) on some of these efforts. The team [...] The post The paper that split the database in four: inside Azure SQL Database Hyperscale, aka Socrates appeared first on Azure SQL Dev Corner.

## Let's build a database!

DevFeed: [Let's build a database!](<https://devfeed.tech/articles/let-s-build-a-database-113332.md>)

Original publisher: [Read original article](<https://stack.convex.dev/lets-build-a-database>)

Author: Stack

Published: 2026-10-09T15:00:37Z

Content type: tutorial

Language: en

Sources: [Stack](<https://devfeed.tech/sources/stack.md>)

Topics: [Database](<https://devfeed.tech/topics/database.md>), [SQLite](<https://devfeed.tech/topics/sqlite.md>), [transaction-log](<https://devfeed.tech/topics/transaction-log.md>), [Transactions](<https://devfeed.tech/topics/transactions.md>), [acid](<https://devfeed.tech/topics/acid.md>), [database-consistency](<https://devfeed.tech/topics/database-consistency.md>), [Heroku Postgres](<https://devfeed.tech/topics/heroku-postgres.md>)

Tags: [build](<https://devfeed.tech/tags/build.md>), [convex](<https://devfeed.tech/tags/convex.md>), [database](<https://devfeed.tech/tags/database.md>), [file](<https://devfeed.tech/tags/file.md>), [fundamentals](<https://devfeed.tech/tags/fundamentals.md>), [json](<https://devfeed.tech/tags/json.md>), [postgres](<https://devfeed.tech/tags/postgres.md>), [sqlite](<https://devfeed.tech/tags/sqlite.md>), [wrapper](<https://devfeed.tech/tags/wrapper.md>)

### AI overview

Convex's SQLite or Postgres layer stores bytes; the article attributes its transactional and reactive behavior to Convex's own backend. It builds a model from an append-only transaction log, a committer that validates read sets and retries conflicts, timestamps and versioned indexes for snapshot reads, and subscriptions that rerun queries after relevant commits. Mutations must be deterministic and side effect free so retries are safe; actions handle external effects, while a scheduler can enqueue them atomically with a successful mutation. Inspection of a local SQLite deployment shows document versions stored as timestamped rows linked to prior versions, with indexes also persisted in tables. The article presents this as an explanatory model and notes that other Convex features are outside its scope.

### Source excerpt

Is Convex just a Postgres wrapper? To answer that we build a database from scratch with the same properties as Convex, starting from a single JSON file, then crack open the SQLite file Convex uses to see what it's actually doing.

## Lakebase Postgres branch-based restores for fast recovery at scale

DevFeed: [Lakebase Postgres branch-based restores for fast recovery at scale](<https://devfeed.tech/articles/lakebase-postgres-branch-based-restores-for-fast-recovery-at-scale-63389.md>)

Original publisher: [Read original article](<https://www.databricks.com/blog/lakebase-postgres-branch-based-restores-fast-recovery-scale>)

Author: Cassie Murray; Carlota Soto

Published: 2026-10-01T17:53:52Z

Content type: article

Language: en

Sources: [Databricks](<https://devfeed.tech/sources/databricks.md>)

Topics: [self-hosted databases vs managed services](<https://devfeed.tech/topics/self-hosted-databases-vs-managed-services.md>), [transaction-log](<https://devfeed.tech/topics/transaction-log.md>), [Disaster Recovery](<https://devfeed.tech/topics/disaster-recovery.md>)

Tags: [database](<https://devfeed.tech/tags/database.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [lakebase](<https://devfeed.tech/tags/lakebase.md>), [postgres](<https://devfeed.tech/tags/postgres.md>), [recovery](<https://devfeed.tech/tags/recovery.md>), [scale](<https://devfeed.tech/tags/scale.md>)

### AI overview

The article contrasts traditional Postgres point-in-time recovery, which can require provisioning compute, restoring snapshots, replaying WAL, and waiting for data to hydrate, with Lakebase Postgres branch-based restores. It describes Lakebase's decoupled compute and storage and immutable history, which allow a restore to create a branch at a selected timestamp without copying data. The article says this metadata operation can make past database states queryable quickly, including at large database sizes, and could support product features such as versioning or undo.

### Source excerpt

In managed OLTP, restores have always been painfully slow and they get slower at...

## Dasha 1.8: index recommendations, I/O analysis, schema checks and log insights

DevFeed: [Dasha 1.8: index recommendations, I/O analysis, schema checks and log insights](<https://devfeed.tech/articles/dasha-1-8-index-recommendations-i-o-analysis-schema-checks-and-log-insights-61633.md>)

Original publisher: [Read original article](<https://www.postgresql.org/about/news/dasha-18-index-recommendations-io-analysis-schema-checks-and-log-insights-3387/>)

Published: 2026-09-29T00:00:00Z

Content type: release

Language: en

Sources: [PostgreSQL news](<https://devfeed.tech/sources/postgresql-news.md>)

Topics: [table-partitioning](<https://devfeed.tech/topics/table-partitioning.md>), [read-replica](<https://devfeed.tech/topics/read-replica.md>), [Transactions](<https://devfeed.tech/topics/transactions.md>), [schema design](<https://devfeed.tech/topics/schema-design.md>), [transaction-log](<https://devfeed.tech/topics/transaction-log.md>), [rate-limiting](<https://devfeed.tech/topics/rate-limiting.md>)

Tags: [context](<https://devfeed.tech/tags/context.md>), [cve](<https://devfeed.tech/tags/cve.md>), [database](<https://devfeed.tech/tags/database.md>), [indexes](<https://devfeed.tech/tags/indexes.md>), [performance](<https://devfeed.tech/tags/performance.md>), [postgresql](<https://devfeed.tech/tags/postgresql.md>)

### AI overview

Dasha 1.8 adds PostgreSQL index recommendations, I/O analysis, schema checks and log insights. It analyzes statistics across cluster hosts and replicas, suggests B-tree indexes with supporting query and execution details, and adds schema findings with remediation SQL. Log search supports additional providers, and query plan comparisons can identify statements whose plans worsened across time windows.

### Source excerpt

Dasha is an open source performance dashboard for PostgreSQL fleets. It connects to your clusters with a read-only role, shows what the databases are doing right now, and explains what to do about it. Nothing is installed on the database hosts: no agent, and no extension beyond the ones you already have. One instance serves many clusters and all of their replicas. Most PostgreSQL statistics are per-instance and are not replicated, so Dasha reads every host of a cluster and reasons over the combined picture. An index that looks unused on the primary may be serving the entire read workload on a standby. New since the last announcement Index recommendations. Dasha reads pg_stat_statements on every host of a cluster and proposes the B-tree indexes that are missing. Each candidate comes with its column order, a ready CREATE INDEX CONCURRENTLY, the statements it covers and their share of execution time. Write counters and partitioned tables (ATTACH PARTITION) are taken into account. A candidate can be backed by auto_explain plans from the log: sequential scans of its table, what they cost and how many rows their filters discarded. A misestimated row count raises a warning to run ANALYZE first. I/O page built on pg_stat_io (PostgreSQL 16+). Server I/O is broken down by backend type, object and context (normal, vacuum, bulkread, bulkwrite), with WAL on PostgreSQL 18. There is a live mode and snapshot history, and cards for shared_buffers efficiency, vacuum cost and bulk operations. Schema checks. 17 checks for structural defects that stay invisible until something breaks, including sequences close to exhaustion, tables without a primary or unique key, forgotten UNLOGGED relations, a public schema anyone can create objects in (CVE-2018-1058), UUIDs stored as varchar, nullable foreign keys, NOT VALID constraints that were never validated and reserved keywords used as names. Each finding has a severity and remediation SQL. Log insights. Log search now works with OpenSearch, El

## The Best ETL Tools for Real-Time Data Integration in 2026

DevFeed: [The Best ETL Tools for Real-Time Data Integration in 2026](<https://devfeed.tech/articles/the-best-etl-tools-for-real-time-data-integration-in-2026-58720.md>)

Original publisher: [Read original article](<https://hackernoon.com/the-best-etl-tools-for-real-time-data-integration-in-2026?source=rss>)

Author: Pipelineguy

Published: 2026-09-23T07:42:22Z

Content type: comparison

Language: en

Sources: [HackerNoon](<https://devfeed.tech/sources/hackernoon.md>)

Topics: [etl](<https://devfeed.tech/topics/etl.md>), [real-time](<https://devfeed.tech/topics/real-time.md>), [data-architecture](<https://devfeed.tech/topics/data-architecture.md>), [Event-Streaming](<https://devfeed.tech/topics/event-streaming.md>), [Streaming](<https://devfeed.tech/topics/streaming.md>), [Database](<https://devfeed.tech/topics/database.md>), [transaction-log](<https://devfeed.tech/topics/transaction-log.md>), [Transactions](<https://devfeed.tech/topics/transactions.md>), [AWS Glue](<https://devfeed.tech/topics/aws-glue.md>), [Cloud Dataflow](<https://devfeed.tech/topics/cloud-dataflow.md>), [databricks](<https://devfeed.tech/topics/databricks.md>), [PostgreSQL](<https://devfeed.tech/topics/postgresql.md>)

Tags: [apache-kafka](<https://devfeed.tech/tags/apache-kafka.md>), [aws-glue](<https://devfeed.tech/tags/aws-glue.md>), [batch-processing](<https://devfeed.tech/tags/batch-processing.md>), [change-data-capture](<https://devfeed.tech/tags/change-data-capture.md>), [cloud-dataflow](<https://devfeed.tech/tags/cloud-dataflow.md>), [compare](<https://devfeed.tech/tags/compare.md>), [data](<https://devfeed.tech/tags/data.md>), [data-engineering](<https://devfeed.tech/tags/data-engineering.md>), [data-integration](<https://devfeed.tech/tags/data-integration.md>), [data-pipelines](<https://devfeed.tech/tags/data-pipelines.md>), [data-streaming](<https://devfeed.tech/tags/data-streaming.md>), [database](<https://devfeed.tech/tags/database.md>), [databricks](<https://devfeed.tech/tags/databricks.md>), [etl](<https://devfeed.tech/tags/etl.md>), [event-streaming](<https://devfeed.tech/tags/event-streaming.md>), [flink](<https://devfeed.tech/tags/flink.md>), [good-company](<https://devfeed.tech/tags/good-company.md>), [kafka](<https://devfeed.tech/tags/kafka.md>), [postgresql](<https://devfeed.tech/tags/postgresql.md>), [real-time](<https://devfeed.tech/tags/real-time.md>)

### AI overview

A comparison of ETL tools for real-time data integration, explaining how freshness requirements and architectures such as batch processing, micro-batching, change data capture, and event streaming affect tool selection.

### Source excerpt

Real-time doesn't mean the same thing to every ETL tool. Compare Estuary, Confluent, Fivetran, and more to find the right fit for your data.

## From Batch Snapshots to Near-Real-Time Data

DevFeed: [From Batch Snapshots to Near-Real-Time Data](<https://devfeed.tech/articles/from-batch-snapshots-to-near-real-time-data-20029.md>)

Original publisher: [Read original article](<https://technology.doximity.com/articles/from-batch-snapshots-to-near-real-time-data>)

Author: Doximity

Published: 2026-08-04T16:29:00Z

Content type: article

Language: en

Sources: [Doximity](<https://devfeed.tech/sources/doximity.md>)

Topics: [streaming data analytics](<https://devfeed.tech/topics/streaming-data-analytics.md>), [Event-Streaming](<https://devfeed.tech/topics/event-streaming.md>), [Apache Kafka](<https://devfeed.tech/topics/apache-kafka.md>), [Snowflake streaming data architecture](<https://devfeed.tech/topics/snowflake-streaming-data-architecture.md>), [Data pipelines](<https://devfeed.tech/topics/data-pipelines.md>), [schema-evolution](<https://devfeed.tech/topics/schema-evolution.md>), [transaction-log](<https://devfeed.tech/topics/transaction-log.md>)

Tags: [analytics](<https://devfeed.tech/tags/analytics.md>), [application](<https://devfeed.tech/tags/application.md>), [apply](<https://devfeed.tech/tags/apply.md>), [article](<https://devfeed.tech/tags/article.md>), [base](<https://devfeed.tech/tags/base.md>), [batch](<https://devfeed.tech/tags/batch.md>), [bootstrap](<https://devfeed.tech/tags/bootstrap.md>), [but](<https://devfeed.tech/tags/but.md>), [can](<https://devfeed.tech/tags/can.md>), [capture](<https://devfeed.tech/tags/capture.md>), [cascade](<https://devfeed.tech/tags/cascade.md>), [cdc](<https://devfeed.tech/tags/cdc.md>), [change](<https://devfeed.tech/tags/change.md>), [change-data-capture](<https://devfeed.tech/tags/change-data-capture.md>), [changes](<https://devfeed.tech/tags/changes.md>), [data](<https://devfeed.tech/tags/data.md>), [real-time](<https://devfeed.tech/tags/real-time.md>), [real-time-data](<https://devfeed.tech/tags/real-time-data.md>), [snapshots](<https://devfeed.tech/tags/snapshots.md>), [time](<https://devfeed.tech/tags/time.md>)

### AI overview

Doximity added CDC to make application database changes available in minutes while retaining its batch pipeline for transactionally consistent snapshots and recovery. In a 12-day measurement window, 95% of sampled events from active tables reached the queryable intermediate layer within eight minutes of publication to Kafka. The excerpt identifies four design decisions: use trusted batch snapshots to bootstrap CDC, preserve metadata while routing changes across source databases, expose new changes through a base-plus-delta view before warehouse merging, and synthesize cascade deletes that MySQL does not emit as individual binary-log events. The excerpt begins describing point-in-time snapshots using Amazon Aurora, but does not provide the full implementation details or tradeoffs for the four decisions.

### Source excerpt

Change Data Capture (CDC) is often presented as a straightforward pipeline: read a database transaction log, publish each change, and apply those changes to another system. That description is accurate, but it leaves out many of the decisions that determine whether the resulting data can be trusted. At Doximity, we already had a batch pipeline that periodically copied snapshots of application databases into our data warehouse. Those snapshots were reliable, but their freshness was measured in hours. We introduced CDC to make changes available in minutes so downstream transformations and operational analytics would not have to wait for the next batch snapshot. We continued using the batch pipeline for the consistency and recovery guarantees it already provided. Over a 12-day measurement window, 95% of events from a stratified sample of active tables reached the queryable intermediate layer within eight minutes of being published to Kafka. The most interesting parts of the project were not the connections from a source database to Kafka or from Kafka to Snowflake, but four questions we had to answer around them: How could we reuse our existing, transactionally consistent batch snapshots as an on-demand starting point for CDC, without reprocessing every existing row? How could we onboard new tables and absorb schema changes from many source databases across our products without growing operational overhead for each one? How could new changes become queryable without waiting for the warehouse to merge them into place? How could we handle cascading child deletes that MySQL performs but never emits as individual binary-log events? Our answers are the four design decisions in this article: a trusted batch snapshot, metadata-preserving routing, a base-plus-delta view, and synthetic cascade deletes. Together, they turned a stream of row changes into a system we could bootstrap, scale, validate, and recover. The sections that follow explain the tradeoffs and guardrails so rea

## It's Time to Break Up with Archive Tables

DevFeed: [It's Time to Break Up with Archive Tables](<https://devfeed.tech/articles/it-s-time-to-break-up-with-archive-tables-50495.md>)

Original publisher: [Read original article](<https://sqlperformance.com/2026/07/partitioning/its-time-to-break-up-with-archive-tables>)

Author: Aaron Bertrand

Published: 2026-07-27T09:00:52Z

Content type: article

Language: en

Sources: [SQLPerformance.com](<https://devfeed.tech/sources/sqlperformance-com.md>)

Topics: [table-partitioning](<https://devfeed.tech/topics/table-partitioning.md>), [sql-server](<https://devfeed.tech/topics/sql-server.md>), [Database](<https://devfeed.tech/topics/database.md>), [transaction-log](<https://devfeed.tech/topics/transaction-log.md>), [backups](<https://devfeed.tech/topics/backups.md>)

Tags: [backups](<https://devfeed.tech/tags/backups.md>), [database-design](<https://devfeed.tech/tags/database-design.md>), [partitioning](<https://devfeed.tech/tags/partitioning.md>), [sql](<https://devfeed.tech/tags/sql.md>), [sql-performance](<https://devfeed.tech/tags/sql-performance.md>), [sql-server](<https://devfeed.tech/tags/sql-server.md>), [table-partitioning](<https://devfeed.tech/tags/table-partitioning.md>)

### AI overview

Aaron Bertrand explains why moving aging SQL Server data row by row into archive tables can create unnecessary writes and operational overhead. He presents table partitioning as an alternative for managing data retention and workload impact.

### Source excerpt

Aaron Bertrand talks about archive tables in SQL Server and how to improve the process using table partitioning. The post It's Time to Break Up with Archive Tables appeared first on SQLPerformance.com.

## Database Storage Optimization for the High-Performance DBA

DevFeed: [Database Storage Optimization for the High-Performance DBA](<https://devfeed.tech/articles/database-storage-optimization-for-the-high-performance-dba-50490.md>)

Original publisher: [Read original article](<https://sqlperformance.com/2026/05/sql-performance/database-storage-optimization-for-the-high-performance-dba>)

Author: Kevin Kline

Published: 2026-05-28T18:06:34Z

Content type: article

Language: en

Sources: [SQLPerformance.com](<https://devfeed.tech/sources/sqlperformance-com.md>)

Topics: [Database](<https://devfeed.tech/topics/database.md>), [sql-server](<https://devfeed.tech/topics/sql-server.md>), [Monitoring](<https://devfeed.tech/topics/monitoring.md>), [Benchmark](<https://devfeed.tech/topics/benchmark.md>), [incident](<https://devfeed.tech/topics/incident.md>), [Latency](<https://devfeed.tech/topics/latency.md>), [observability](<https://devfeed.tech/topics/observability.md>), [reliability-engineering](<https://devfeed.tech/topics/reliability-engineering.md>), [transaction-log](<https://devfeed.tech/topics/transaction-log.md>), [sessions](<https://devfeed.tech/topics/sessions.md>)

Tags: [2025](<https://devfeed.tech/tags/2025.md>), [alert-fatigue](<https://devfeed.tech/tags/alert-fatigue.md>), [background](<https://devfeed.tech/tags/background.md>), [burnout](<https://devfeed.tech/tags/burnout.md>), [database](<https://devfeed.tech/tags/database.md>), [high-performance](<https://devfeed.tech/tags/high-performance.md>), [incident](<https://devfeed.tech/tags/incident.md>), [latency](<https://devfeed.tech/tags/latency.md>), [monitoring](<https://devfeed.tech/tags/monitoring.md>), [observability](<https://devfeed.tech/tags/observability.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [performance](<https://devfeed.tech/tags/performance.md>), [reliability](<https://devfeed.tech/tags/reliability.md>), [sql-performance](<https://devfeed.tech/tags/sql-performance.md>), [sql-server](<https://devfeed.tech/tags/sql-server.md>), [storage](<https://devfeed.tech/tags/storage.md>), [systems](<https://devfeed.tech/tags/systems.md>), [transaction-log](<https://devfeed.tech/tags/transaction-log.md>)

### AI overview

This article explains how SQL Server storage problems contribute to recurring incidents, including I/O bottlenecks, TempDB contention, transaction log growth, inconsistent latency, and blocked sessions. It frames storage optimization as a practical way to reduce incident volume and improve database reliability.

### Source excerpt

After a while, if you lead database teams, the pattern becomes familiar. Your DBAs spend too much time firefighting. Complexity keeps rising. The time available for real improvement keeps shrinking. The pressure shows up in the data. The 2025 State of Database Report puts numbers behind that reality. DBAs spend an average of 27 hours a week [...] The post Database Storage Optimization for the High-Performance DBA appeared first on SQLPerformance.com.

## SWIFT Ledger: a new chapter for global finance

DevFeed: [SWIFT Ledger: a new chapter for global finance](<https://devfeed.tech/articles/swift-ledger-a-new-chapter-for-global-finance-56922.md>)

Original publisher: [Read original article](<https://www.thoughtworks.com/insights/articles/SWIFT-Ledger-new-chapter-for-global-finance>)

Author: Alla Gancz, Sid Zalaki, Nathan Hilt

Published: 2025-10-17T00:00:00Z

Content type: article

Language: en

Sources: [Thoughtworks Insights](<https://devfeed.tech/sources/thoughtworks-insights.md>)

Topics: [Blockchain](<https://devfeed.tech/topics/blockchain.md>), [transaction-log](<https://devfeed.tech/topics/transaction-log.md>), [interoperability](<https://devfeed.tech/topics/interoperability.md>), [Transactions](<https://devfeed.tech/topics/transactions.md>), [real-time](<https://devfeed.tech/topics/real-time.md>)

Tags: [article](<https://devfeed.tech/tags/article.md>), [blockchain](<https://devfeed.tech/tags/blockchain.md>), [cross-border](<https://devfeed.tech/tags/cross-border.md>), [digital-transformation](<https://devfeed.tech/tags/digital-transformation.md>), [financial-institutions](<https://devfeed.tech/tags/financial-institutions.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [interoperability](<https://devfeed.tech/tags/interoperability.md>), [ledger](<https://devfeed.tech/tags/ledger.md>), [real-time](<https://devfeed.tech/tags/real-time.md>), [technology-strategy](<https://devfeed.tech/tags/technology-strategy.md>)

### AI overview

The article examines SWIFT's proposed blockchain-based shared ledger, developed with ConsenSys and backed by global banks, to support real-time, 24/7 cross-border settlement. It describes a shift from transmitting payment instructions to coordinating tokenized value through a shared transaction log, smart contracts, and interoperability across digital money networks.

### Source excerpt

SWIFT Ledger: a new chapter for global finance

## Optimizing Databases on Kubernetes: Kubernetes Backup and Recovery with CNPG and ZFS Snapshots

DevFeed: [Optimizing Databases on Kubernetes: Kubernetes Backup and Recovery with CNPG and ZFS Snapshots](<https://devfeed.tech/articles/optimizing-databases-on-kubernetes-kubernetes-backup-and-recovery-with-cnpg-and-zfs-snapshots-22280.md>)

Original publisher: [Read original article](<https://www.ardanlabs.com/blog/2025/01/optimizing-databases-on-kubernetes-ep5-kubernetes-backup-and-recovery-with-cnpg-and-zfs-snapshots.html>)

Published: 2024-12-30T00:00:00Z

Content type: article

Language: en

Sources: [William Kennedy](<https://devfeed.tech/sources/william-kennedy.md>)

Topics: [Featured-Backing Up](<https://devfeed.tech/topics/featured-backing-up.md>), [CloudNativePG](<https://devfeed.tech/topics/cloudnativepg.md>), [ZFS](<https://devfeed.tech/topics/zfs-file-system-features.md>), [Disaster Recovery](<https://devfeed.tech/topics/disaster-recovery.md>), [transaction-log](<https://devfeed.tech/topics/transaction-log.md>)

Tags: [automated](<https://devfeed.tech/tags/automated.md>), [backup](<https://devfeed.tech/tags/backup.md>), [backup-and-recovery](<https://devfeed.tech/tags/backup-and-recovery.md>), [backups](<https://devfeed.tech/tags/backups.md>), [cloud-block-storage-benchmarking](<https://devfeed.tech/tags/cloud-block-storage-benchmarking.md>), [data](<https://devfeed.tech/tags/data.md>), [databases](<https://devfeed.tech/tags/databases.md>), [durability](<https://devfeed.tech/tags/durability.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [fast](<https://devfeed.tech/tags/fast.md>), [kubernetes](<https://devfeed.tech/tags/kubernetes.md>), [kubernetes-backup](<https://devfeed.tech/tags/kubernetes-backup.md>), [kubernetes-database-performance](<https://devfeed.tech/tags/kubernetes-database-performance.md>), [kubernetes-database-throughput](<https://devfeed.tech/tags/kubernetes-database-throughput.md>), [kubernetes-local-storage-pros-and-cons](<https://devfeed.tech/tags/kubernetes-local-storage-pros-and-cons.md>), [kubernetes-production-grade-storage](<https://devfeed.tech/tags/kubernetes-production-grade-storage.md>), [kubernetes-storage-benchmarking](<https://devfeed.tech/tags/kubernetes-storage-benchmarking.md>), [kubernetes-storage-efficiency](<https://devfeed.tech/tags/kubernetes-storage-efficiency.md>), [kubernetes-storage-trade-offs](<https://devfeed.tech/tags/kubernetes-storage-trade-offs.md>), [kubernetes-zfs-advantages](<https://devfeed.tech/tags/kubernetes-zfs-advantages.md>), [logs](<https://devfeed.tech/tags/logs.md>), [optimizing-postgresql-on-kubernetes](<https://devfeed.tech/tags/optimizing-postgresql-on-kubernetes.md>), [performance](<https://devfeed.tech/tags/performance.md>), [pgbench-kubernetes-testing](<https://devfeed.tech/tags/pgbench-kubernetes-testing.md>), [point](<https://devfeed.tech/tags/point.md>), [postgresql](<https://devfeed.tech/tags/postgresql.md>), [production](<https://devfeed.tech/tags/production.md>), [rancher-local-path-performance](<https://devfeed.tech/tags/rancher-local-path-performance.md>), [snapshots](<https://devfeed.tech/tags/snapshots.md>), [storage](<https://devfeed.tech/tags/storage.md>), [techniques](<https://devfeed.tech/tags/techniques.md>), [zfs](<https://devfeed.tech/tags/zfs.md>), [zfs-compression-benefits](<https://devfeed.tech/tags/zfs-compression-benefits.md>), [zfs-lz4-compression](<https://devfeed.tech/tags/zfs-lz4-compression.md>), [zfs-vs-cloud-storage](<https://devfeed.tech/tags/zfs-vs-cloud-storage.md>)

### AI overview

The excerpt describes a PostgreSQL backup and recovery approach using CNPG and ZFS: automate backups and ship WAL to S3-compatible object storage, with barman cloud cited for managing replication logs, to support point-in-time recovery. ZFS copy-on-write snapshots are presented as a way to clone databases quickly with low storage overhead, aided by compression. The episode also flags balancing performance and storage needs, and notes possible configuration and durability challenges; it does not provide the actual YAML configuration details in the supplied text.

### Source excerpt

Introduction: In the final episode of the Optimizing Databases on Kubernetes series, Jérôme Petazzoni dives into advanced backup and recovery techniques for PostgreSQL, showcasing how CNPG (Cloud Native PostgreSQL) and ZFS snapshots ensure durability and fast recovery in production environments. This episode focuses on safeguarding data with comprehensive backup strategies, point-in-time recovery, and rapid database cloning, while leveraging ZFS's efficiency for optimized storage performance. Comprehensive Backup Strategies: Exploring backups with CNPG, including write-ahead logs (WAL) and volume snapshots.