# wal Published articles for wal. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Designing a reliable unplanned failover DevFeed: [Designing a reliable unplanned failover]() Original publisher: [Read original article]() Author: Shlomi Noach Published: 2026-11-05T00:00:00Z Content type: article Language: en Sources: [PlanetScale]() Topics: [read-replica](), [Availability](), [distributed algorithm](), [control-plane]() Tags: [admin](), [detection](), [durability](), [engineering](), [failover](), [freeze](), [neki](), [postgres](), [primary](), [wal]() ### AI overview Neki avoids promoting a replica based on Admin's failed health check alone. It compares replica reports of physical replication: a reachable replica still streaming from the primary blocks failover, while a reachable report of broken replication with no contradiction supports recovery; if Admin cannot reach any replica, it waits. Once it decides the primary is dead, Neki disconnects reachable replicas by changing `primary_conninfo`, preventing reconnection to a returning primary and freezing their WAL positions. Admin prefers a promotable replica that has replayed through the highest received WAL position, waits briefly for catch-up, then selects by `receive_lsn` and `replay_lsn`. The new primary is promoted and the others repointed; a replica ahead of it may need rewind or rebuild. With synchronous `ANY 1`, commits require an acknowledging replica, but acknowledged WAL can still be lost if the primary and the replica holding it fail together. Cross-availability-zone acknowledgements reduce that risk at the cost of commit latency. Async durability offers no guarantee that an unreachable primary has stopped accepting writes. Unrecoverable cases include having no reachable candidate. ### Source excerpt How Neki detects a failed Postgres primary, and chooses a new one from the WAL that survived. ## Building an Ingest Safety Monitor for Your IIoT Pipeline in SQL DevFeed: [Building an Ingest Safety Monitor for Your IIoT Pipeline in SQL]() Original publisher: [Read original article]() Author: John Shi Published: 2026-10-16T12:55:08Z Content type: tutorial Language: en Sources: [Timescale]() Topics: [SQL]() Tags: [headroom](), [iiot](), [monitor](), [pipeline](), [postgresql](), [replica](), [replica-lag](), [safety](), [sql](), [volume](), [wal]() ### AI overview The article describes an SQL-based safety monitor for an IIoT ingestion pipeline. It recommends sampling WAL volume, replica lag, and dead tuples every minute and calculating a headroom percentage that can trigger alerts. ### Source excerpt Ingest fails suddenly, not gradually. Sample WAL volume, replica lag, and dead tuples every minute, and get one headroom percentage you can alert on. ## Resolving PostgreSQL replication lag with heartbeat tables in change data capture scenarios DevFeed: [Resolving PostgreSQL replication lag with heartbeat tables in change data capture scenarios]() Original publisher: [Read original article]() Author: Donghua Luo Published: 2026-09-30T16:47:04Z Content type: tutorial Language: en Sources: [AWS Database Blog]() Topics: [PostgreSQL](), [AWS Database Migration Service](), [Debezium Connector](), [data migrations]() Tags: [amazon-aurora](), [amazon-cloudwatch](), [aws-database-migration-service](), [best-practices](), [change-data-capture](), [debezium](), [intermediate-200](), [postgresql](), [postgresql-compatible](), [rds-for-postgresql](), [replication](), [wal]() ### AI overview This tutorial explains how PostgreSQL replication slots can stall during change data capture migrations when tracked tables are quiet even though the database is generating writes elsewhere. It describes using heartbeat tables with AWS Database Migration Service or Debezium to advance replication and prevent WAL accumulation, and mentions CloudWatch and SQL diagnostics for monitoring slot health. ### Source excerpt Replication lag during a change data capture (CDC) migration can be counterintuitive: the source is busy, yet replication falls behind and WAL piles up. This post shows how to resolve PostgreSQL replication lag with heartbeat tables using AWS DMS or Debezium, and how to monitor replication slot health with Amazon CloudWatch and SQL diagnostics. ## Cursor Uses S3 WAL to Scale Git Storage to More than 300 Pushes per Second DevFeed: [Cursor Uses S3 WAL to Scale Git Storage to More than 300 Pushes per Second]() Original publisher: [Read original article]() Author: Leela Kumili Published: 2026-09-30T13:47:00Z Content type: news Language: en Sources: [InfoQ]() Topics: [Amazon S3](), [cell-based-architecture](), [Data integrity in software engineering](), [consistency]() Tags: [agents](), [ai-architecture](), [ai-assisted-coding](), [architecture-design](), [aws](), [consistency](), [continuous-integration](), [cursor-continuity-git-storage](), [database-replication](), [development](), [distributed-systems](), [github](), [hashing](), [ide](), [latency](), [news](), [nvme](), [protocol](), [s3](), [scaling](), [source-of-truth](), [wal]() ### AI overview Cursor's Continuity Git storage architecture uses an S3-backed write-ahead log as the durable source of truth, while local NVMe repositories serve as warm caches. Cursor reports linear read scaling with up to 100 replicas and more than 300 pushes per second in synthetic tests. The article also notes that these performance figures have not been independently verified. ### Source excerpt Cursor has introduced Continuity, a Git storage architecture that uses an S3 backed write ahead log as the source of truth. The design turns local NVMe repositories into warm caches and separates replica coordination from consistency. Cursor reports linear read scaling with up to 100 replicas and more than 300 pushes per second with S3 Express One Zone in synthetic tests. By Leela Kumili ## Does CDC Slow Down a Production Database? The Real Cost of Log-Based CDC DevFeed: [Does CDC Slow Down a Production Database? The Real Cost of Log-Based CDC]() Original publisher: [Read original article]() Author: Emily Lucek Published: 2026-09-21T12:50:17Z Content type: article Language: en Sources: [Estuary]() Topics: [Production Engineering](), [self-hosted databases vs managed services](), [read-replica](), [database-consistency]() Tags: [behavior](), [cases](), [cdc](), [data](), [database](), [database-performance](), [has](), [postgresql](), [production](), [replication](), [vacuum](), [wal]() ### AI overview The article explains how PostgreSQL log-based change data capture (CDC) affects production database performance. It covers WAL generation, decoding resource use, configuration costs such as REPLICA IDENTITY FULL, and how stalled replication slots and old vacuum horizons can lead to retained WAL or table bloat. The impact varies by workload and available capacity. ### Source excerpt See where PostgreSQL CDC adds overhead, from WAL and logical decoding to replication slots, vacuum pressure, and production query performance. ## How PlanetScale Postgres Protects Logical Replication Slots During Cutovers DevFeed: [How PlanetScale Postgres Protects Logical Replication Slots During Cutovers]() Original publisher: [Read original article]() Author: Simeon Griggs Published: 2026-09-21T00:00:00Z Content type: tutorial Language: en Sources: [PlanetScale]() Topics: [Replication](), [Availability](), [Database](), [Cloud]() Tags: [engineering](), [high-availability](), [postgres](), [reliability](), [replication](), [wal]() ### AI overview This article explains why Postgres cutovers must account for logical replication slots, not only whether a replica has replayed the write-ahead log. It describes how missing slot state can interrupt downstream consumers such as search indexes, analytics tools, and queues, and why PlanetScale blocks unsafe promotions. ### Source excerpt Postgres understands high-availability architecture, but its defaults assume you don't have replication slots. This can get you into trouble in a cutover event. The combination of what Postgres can do and what PlanetScale lets you do is what keeps connected applications safe. ## PostgreSQL CDC Backfills at Scale: Running a Multi-Day Backfill While Production Keeps Writing DevFeed: [PostgreSQL CDC Backfills at Scale: Running a Multi-Day Backfill While Production Keeps Writing]() Original publisher: [Read original article]() Author: Emily Lucek; Dani Pálma Published: 2026-09-10T09:45:31Z Content type: tutorial Language: en Sources: [Estuary]() Topics: [PostgreSQL](), [Production Engineering]() Tags: [algorithm](), [change-data-capture](), [load](), [postgresql](), [recovery](), [scale](), [wal]() ### AI overview The guide explains how large PostgreSQL CDC backfills interact with ongoing production writes. It covers WAL retention and replication-slot risks, how chunk size affects source load, acknowledgment intervals and restart costs, and options for reducing or recovering historical reads. It also describes Estuary's watermark-and-fence approach for reconciling backfill rows with live changes. ### Source excerpt Learn how PostgreSQL CDC backfills affect WAL retention, source load, chunk sizing, and recovery while production continues writing. ## PostgreSQL Replication Slots: How to Monitor, Recover, and Prevent WAL Bloat DevFeed: [PostgreSQL Replication Slots: How to Monitor, Recover, and Prevent WAL Bloat]() Original publisher: [Read original article]() Author: Dani Pálma; Emily Lucek Published: 2026-08-19T07:27:39Z Content type: tutorial Language: en Sources: [Estuary]() Topics: [PostgreSQL](), [Resilience]() Tags: [bloat](), [cdc](), [postgresql](), [prevent](), [recover](), [replication](), [wal]() ### AI overview This guide explains how PostgreSQL replication slots support CDC continuity and how stalled consumers, long-running transactions, WAL limits, and server changes can cause retained WAL growth or slot loss. It shows how to monitor slot positions and storage, size WAL retention, recover and reconcile gaps, and prepare for failures. ### Source excerpt Replication slots protect CDC continuity, but stalled consumers can retain excessive WAL or make slots unusable. ## How to Write to SSDs - Co-Designing DBMS and Flash Storage DevFeed: [How to Write to SSDs - Co-Designing DBMS and Flash Storage]() Original publisher: [Read original article]() Published: 2026-06-22T00:00:00Z Content type: article Language: en Sources: [Gaurav Sarma's]() Topics: [Database](), [systems](), [Optimization](), [Parallelism]() Tags: [b-tree](), [checkpoint](), [cycles](), [database](), [databases](), [latency](), [mysql](), [optimize](), [optimizing](), [postgresql](), [ssd](), [storage](), [systems](), [wal](), [write-amplification]() ### AI overview The article explains how database management systems and SSDs jointly amplify writes. It reports that a 4 KiB logical page write can become about 18.85 KiB of flash writes on a Samsung PM9A3, and presents DBMS-SSD co-design, including avoiding in-place updates, as a way to address the combined amplification. ### Source excerpt . [How to Write to SSDs](optimising-ssd-writes-for-dbms-cover... ## When upserts don't update but still write: Debugging Postgres performance at scale DevFeed: [When upserts don't update but still write: Debugging Postgres performance at scale]() Original publisher: [Read original article]() Author: Anthonin Bonnefoy Published: 2026-03-23T00:00:00Z Content type: article Language: en Sources: [Datadog]() Topics: [Boilerplate](), [performance-engineering](), [Heroku Postgres]() Tags: [batching](), [btree](), [cluster](), [clusters](), [code](), [concurrency](), [concurrency-control](), [data](), [database](), [datadog-agent](), [debugger](), [engineering](), [postgres](), [postgres-performance](), [postgresql](), [wal]() ### AI overview A high-volume host metadata upsert unexpectedly increased PostgreSQL disk writes and WAL syncs because `ON CONFLICT DO UPDATE` locks a conflicting row even when its WHERE condition prevents an update. Datadog engineers used `pg_walinspect` to identify the WAL activity and replaced the upsert with a data-modifying CTE using `ON CONFLICT DO NOTHING` and an outer UPDATE. This avoided unnecessary WAL records, with a race-condition trade-off accepted for their host-tracking use case. ### Source excerpt When a high-volume upsert doubled disk writes, Datadog engineers traced the issue to Postgres WAL behavior and rewrote the query to eliminate hidden costs. ## How Databases Checkpoint to Disk Without Stopping the World DevFeed: [How Databases Checkpoint to Disk Without Stopping the World]() Original publisher: [Read original article]() Published: 2026-03-15T00:00:00Z Content type: tutorial Language: en Sources: [Gaurav Sarma's]() Topics: [Database](), [PostgreSQL]() Tags: [checkpoint](), [crash-recovery](), [databases](), [latency](), [postgresql](), [snapshot](), [wal]() ### AI overview A tutorial on database checkpointing explains why pausing writes to flush all dirty pages causes latency spikes and throughput stalls. It introduces fuzzy or online checkpointing, including PostgreSQL's WAL-based approach, which allows writes to continue while dirty pages are flushed and uses WAL replay during crash recovery. ### Source excerpt Your database has gigabytes of dirty pages in memory. At some point they need to hit disk... ## Contributing to Debezium: Fixing Logical Replication at Scale DevFeed: [Contributing to Debezium: Fixing Logical Replication at Scale]() Original publisher: [Read original article]() Author: Conor Gallagher Published: 2025-12-18T23:00:00Z Content type: article Language: en Sources: [Zalando]() Topics: [Replication](), [Event-Streaming](), [PostgreSQL](), [Streaming](), [Database](), [Kubernetes]() Tags: [backend](), [databases](), [debezium](), [event-streaming](), [infrastructure](), [kubernetes](), [kubernetes-clusters](), [open-source](), [postgresql](), [replication](), [scale](), [streaming](), [wal](), [zalando]() ### AI overview Zalando describes how its engineers contributed two Debezium features to address PostgreSQL logical replication problems in a Kubernetes-based event-streaming platform. The article focuses on runaway WAL growth in low-activity databases and a compatibility issue involving pgjdbc keepalive flushing during a Debezium upgrade. ### Source excerpt How we contributed two features to Debezium to solve WAL growth and enable safer logical replication for our event streaming infrastructure ## Patching the PostgreSQL JDBC Driver DevFeed: [Patching the PostgreSQL JDBC Driver]() Original publisher: [Read original article]() Author: Declan Murphy Published: 2023-11-08T23:00:00Z Content type: article Language: en Sources: [Zalando]() Topics: [PostgreSQL](), [Replication](), [Event-Streaming](), [Database]() Tags: [backend](), [event-streaming](), [infrastructure](), [integration](), [open-source](), [patching](), [postgresql](), [replication](), [wal](), [zalando]() ### AI overview A Zalando contribution to the PostgreSQL JDBC driver fixes a logical-replication issue that could cause runaway WAL growth. The article explains how the problem affected Postgres-sourced event streams and how the upstream driver fix benefits Debezium and other JDBC clients. ### Source excerpt Contributing to the PostgreSQL JDBC Driver to address the issue of runaway WAL growth in Logical Replication ## Postgres backups: Logical vs. Physical an overview DevFeed: [Postgres backups: Logical vs. Physical an overview]() Original publisher: [Read original article]() Author: Map Published: 2017-09-03T20:55:56Z Content type: tutorial Language: en Sources: [Craig Kerstiens]() Topics: [backups](), [Database](), [data]() Tags: [backing-up](), [backup](), [backups](), [checksums](), [dump](), [environments](), [logical](), [overview](), [pg-dump](), [portable](), [postgres](), [postgresql](), [production](), [sql](), [wal]() ### AI overview This article explains PostgreSQL's two backup types: logical backups, which are portable and can target selected tables but add database load, and physical backups, which consist of the database's bytes on disk. It also discusses checksums, corruption detection, and the role of the write-ahead log. ### Source excerpt It's not a very disputed topic that you should backup your database, and further test your backups. What is a little less discussed, at least for Postgres, is the types of backups that exist. Within Postgres there are two forms of backups and understanding them is a useful foundation for anyone working with Postgres. The two backup types are Physical: which consist of the actual bytes on disk, Logical: which is a more portable format. Let's dig into each a bit more so you can better assess which makes sense for you. Logical backups Logical backups are the most well known type within Postgres. This is what you get when you run pg_dump against a database. There are a number of different formats you can get from logical backups and Postgres does a good job of making it easy to compress and configure this backup how you see fit. When a logical backup is run against a database it is not throttled, this introduces a noticable load on your database. As it's reading the data from disk and generating (in layman terms) a bunch of SQL INSERT statements, it has to actually see the data. It's of note that older Postgres databases (read: prior to 9.3) there were no checksums against your database. Checksums are just one tool for you to help check against data corruption. Because a logical dump has to actually read and generate the data to insert it will discover any corruption that exists for you. This portable format is also very useful to pull down copies from production to different environments. I.e. if you need a copy of production data down on your local laptop pg_dump is the way to do it. Logical backups are also database specific, but then allow you to dump only certain tables. All in all logical backups bring some good features, but come at two cost: Load on your system The backup contains data as of the time when it ran Physical backups Physical backups are another option when it comes to backing up your database. As we mentioned earlier it is the physical bytes on disk