# database performance Published articles for database performance. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Troubleshooting row lock contention in Amazon Aurora PostgreSQL: Part 1 - Understanding row lock contention in PostgreSQL DevFeed: [Troubleshooting row lock contention in Amazon Aurora PostgreSQL: Part 1 - Understanding row lock contention in PostgreSQL]() Original publisher: [Read original article]() Author: Sameer Kumar Published: 2026-09-14T16:02:08Z Content type: tutorial Language: en Sources: [AWS Database Blog]() Topics: [Amazon Aurora](), [PostgreSQL](), [Concurrency](), [Monitoring](), [Transactions](), [Amazon CloudWatch](), [Amazon RDS](), [Architecture & Design](), [Extension]() Tags: [advanced-300](), [amazon-aurora](), [amazon-cloudwatch](), [amazon-rds](), [architecture](), [availability](), [concurrency](), [database](), [database-performance](), [extension](), [locks](), [monitoring](), [postgresql](), [production](), [rds-for-postgresql](), [technical-how-to](), [transactions](), [troubleshooting]() ### AI overview This first part of a two-part series explains row lock contention in PostgreSQL and Amazon Aurora PostgreSQL. It covers how concurrent transactions competing for the same rows can reduce throughput and cause timeouts despite healthy CPU and I/O, then introduces PostgreSQL locking internals and monitoring techniques using system views, functions, the pgrowlocks extension, and log_lock_waits. The article notes that the same behavior and investigation approach apply to Amazon RDS for PostgreSQL. ### Source excerpt Row lock contention can collapse database throughput during a flash sale even when CPU and I/O look healthy. In Part 1 of this series, learn how PostgreSQL row locking works and how to monitor lock contention in Amazon Aurora PostgreSQL and Amazon RDS for PostgreSQL using system views, the pgrowlocks extension, and the log_lock_waits parameter. ## JDBC batching can still issue one round trip per row unless driver statement rewriting is enabled DevFeed: [JDBC batching can still issue one round trip per row unless driver statement rewriting is enabled]() Original publisher: [Read original article]() Author: hello@ankit-rana.com Published: 2026-09-03T00:00:00Z Content type: tutorial Language: en Sources: [Ankit Rana | Mechanical Sympathy]() Topics: [Databases](), [Latency](), [client](), [Code]() Tags: [batching](), [bulk-load](), [client](), [cost](), [database](), [database-performance](), [durability](), [jdbc](), [latency](), [mysql](), [postgresql](), [server](), [sql]() ### AI overview For small database writes, network round trips and transaction overhead can dominate the actual insert work. The article explains that JDBC batching may accept rows into a client-side batch while still sending separate statements unless driver-side statement rewriting is enabled, and that autocommit adds a transaction and durability barrier per row. ### Source excerpt For small writes the per-statement overhead dominates the actual work, so throughput is set by round trips rather than by the database. JDBC batching is the usual fix and it silently does nothing on the wire unless the driver is told to rewrite the statements, so addBatch can look correct while still issuing one round trip per row. Autocommit compounds it by turning every row into its own transaction and its own durability barrier, which is the difference between one fsync and a hundred thousand. ## Foreign keys are not overhead, and the thing you measured was a missing index DevFeed: [Foreign keys are not overhead, and the thing you measured was a missing index]() Original publisher: [Read original article]() Author: hello@ankit-rana.com Published: 2026-08-30T00:00:00Z Content type: opinion Language: en Sources: [Ankit Rana | Mechanical Sympathy]() Topics: [Database](), [integrity]() Tags: [database](), [database-performance](), [foreign-keys](), [indexing](), [integrity](), [overhead](), [postgres](), [postgresql](), [referential-integrity](), [schema-design]() ### AI overview The article explains that the measured cost of a slow parent delete is usually caused by a missing index on the child table's foreign-key column, not by foreign-key checking itself. It argues that dropping the constraint merely moves referential-integrity enforcement into application code and can allow orphaned records. ### Source excerpt A foreign key check is an index lookup against a primary key that is almost certainly already in cache, which is close to the cheapest thing a database does. The expensive case people measure is a parent delete with no index on the child's referencing column, which forces a full scan of the child table per deleted row, and that is a missing index rather than a cost of the constraint. Dropping the key does not remove the work, it moves the integrity guarantee into application code where nothing reports when it stops holding. ## Benchmarking vector indexes DevFeed: [Benchmarking vector indexes]() Original publisher: [Read original article]() Author: Evgeniy Patlan Published: 2026-08-27T13:35:32Z Content type: article Language: en Sources: [Blog - Percona]() Topics: [benchmarking](), [Ground truth / benchmark quality](), [Databases](), [ann](), [datasets](), [Sorting]() Tags: [ai-vector](), [ann](), [benchmark](), [benchmarking](), [benchmarks](), [blog](), [blog-post](), [data](), [database-performance](), [database-trends](), [databases](), [datasets](), [embeddings](), [index](), [measurements](), [model](), [open-source](), [percona](), [points](), [report](), [run](), [search](), [sorting](), [vector-search](), [vectorsearch]() ### AI overview This article explains how Percona's vector-bench measures vector indexes by building database engines from pinned versions, running them in identical containers with the same data and hardware, and comparing consistent measurements. It describes embeddings, exact search, approximate nearest neighbour search, and ground truth as the brute-force reference needed to evaluate index accuracy. ### Source excerpt Nearly every database has vector search now, and every one of them has a blog post with a big number in it. Almost none of those numbers can be checked, because the thing that makes them meaningful is usually missing. We built a vector-bench to stop guessing. You name the engines you want, build them ... Continued The post Benchmarking vector indexes appeared first on Percona. ## Connection pool sizing is a queueing theory problem, not a tuning knob DevFeed: [Connection pool sizing is a queueing theory problem, not a tuning knob]() Original publisher: [Read original article]() Author: hello@ankit-rana.com Published: 2026-08-22T00:00:00Z Content type: article Language: en Sources: [Ankit Rana | Mechanical Sympathy]() Topics: [connection pool](), [queueing theory](), [Latency](), [Database](), [Concurrency]() Tags: [capacity-planning](), [concurrency](), [connection-pool](), [cpu](), [database](), [database-performance](), [hikaricp](), [inference](), [latency](), [network](), [postgresql](), [query](), [queueing-theory]() ### AI overview This article explains how to size a connection pool using Little's law: concurrency equals throughput multiplied by connection holding time. It argues that increasing the pool beyond the database's parallel execution capacity can move the queue, increase latency, and leave throughput flat. It also identifies excessive holding time from network round trips, N+1 queries, long transactions, and model inference calls as common causes of pool exhaustion. ### Source excerpt Pool size follows from Little's law: concurrency equals throughput multiplied by holding time, so 500 requests per second each holding a connection for 20ms needs ten connections rather than a round number someone picked. Past the point where the database can execute requests in parallel, adding connections moves the queue rather than removing it, and latency grows while throughput stays flat. The number that actually reaches the database is the pool size multiplied by the instance count, which is usually the number nobody has calculated. ## Covering Indexes and Index-Only Scans in PostgreSQL DevFeed: [Covering Indexes and Index-Only Scans in PostgreSQL]() Original publisher: [Read original article]() Author: hello@ankit-rana.com Published: 2026-08-18T00:00:00Z Content type: tutorial Language: en Sources: [Ankit Rana | Mechanical Sympathy]() Topics: [PostgreSQL](), [Query (disambiguation)](), [data]() Tags: [b-tree](), [cache](), [covering](), [covering-index](), [database-performance](), [explain](), [heap](), [indexes](), [indexing](), [innodb](), [pages](), [postgresql](), [query](), [storage](), [vacuum]() ### AI overview This tutorial explains why a normal index scan may still be slow: after finding matching entries, the database follows a pointer into the table for each row. Covering indexes store the selected columns in the index and can avoid those heap reads. In PostgreSQL, index-only scans also depend on the visibility map marking pages all-visible, while SELECT * prevents the technique from being fully effective. ### Source excerpt A normal index scan finds matching rows and then follows a pointer into the table for every one of them, which is a random read per row. A covering index stores the columns the query selects, so the engine answers entirely from the index and skips those reads. In PostgreSQL this only works when the visibility map marks the pages all-visible, so an unvacuumed table will report Heap Fetches in EXPLAIN and give back most of the gain. SELECT star defeats the technique completely. ## Why your index is not being used, and why the planner is usually right DevFeed: [Why your index is not being used, and why the planner is usually right]() Original publisher: [Read original article]() Author: hello@ankit-rana.com Published: 2026-08-16T00:00:00Z Content type: tutorial Language: en Sources: [Ankit Rana | Mechanical Sympathy]() Topics: [Query (disambiguation)](), [MySQL](), [bug]() Tags: [bug](), [cardinality](), [database-performance](), [explain](), [function](), [indexes](), [indexing](), [mysql](), [postgresql](), [predicate](), [query](), [query-planner]() ### AI overview This tutorial explains why database indexes may not be used even when they exist. It focuses on inaccurate cardinality estimates caused by stale statistics, predicates that prevent index matching, implicit casts, and cases where sequential scans are the cheaper choice. ### Source excerpt An unused index is almost never a planner bug. It is usually a predicate the planner cannot match to the index, such as a function or an implicit cast applied to the column, or a cardinality estimate that is wrong because statistics are stale. When the estimate is right and the planner still refuses, it is often correct: past a few percent of the table, random access through an index costs more than reading the table sequentially. The diagnostic that matters is the gap between estimated and actual rows in EXPLAIN ANALYZE. ## How ORM N+1 Queries Create Hidden Request Latency DevFeed: [How ORM N+1 Queries Create Hidden Request Latency]() Original publisher: [Read original article]() Author: hello@ankit-rana.com Published: 2026-08-02T00:00:00Z Content type: tutorial Language: en Sources: [Ankit Rana | Mechanical Sympathy]() Topics: [Object-relational mapping](), [Latency](), [Monitoring](), [lazy loading](), [Database](), [connection pool]() Tags: [connection-pool](), [database](), [database-performance](), [hibernate](), [latency](), [lazy-loading](), [monitoring](), [n-plus-one](), [observability](), [orm](), [query](), [request]() ### AI overview This article explains how ORM lazy loading can turn a loop over orders into an N+1 query pattern. Individual queries may remain fast and avoid slow-query alerts, while the accumulated network round trips increase request latency and hold connection-pool resources. It recommends monitoring queries per request and asserting query counts in tests. ### Source excerpt N+1 queries hide from monitoring because no individual query is slow. Four hundred queries at 0.4ms each never appear in the slow query log, never move p99 query latency, and still cost the request 300ms because the expense is the round trip rather than the execution. The metric that catches it is queries per request, and the only durable fix is asserting on that count in tests, since the code that causes it looks like a field access rather than I/O. ## Pagination at Scale: Why OFFSET and SKIP Will Eventually Break Your API DevFeed: [Pagination at Scale: Why OFFSET and SKIP Will Eventually Break Your API]() Original publisher: [Read original article]() Author: hello@ankit-rana.com Published: 2026-03-17T00:00:00Z Content type: tutorial Language: en Sources: [Ankit Rana | Mechanical Sympathy]() Topics: [API](), [Database](), [MongoDB]() Tags: [api](), [api-design](), [b-tree](), [cursor](), [database-performance](), [latency](), [mongodb](), [pagination](), [scalability](), [scale](), [sql]() ### AI overview This article explains why OFFSET and SKIP pagination become slower and more memory-intensive as page depth increases. It presents cursor-based, or keyset, pagination as an alternative that uses an indexed range seek and reads a fixed number of rows, keeping fetch cost largely constant with list depth. The trade-off is that cursor pagination does not provide reliable absolute page numbers. ### Source excerpt OFFSET does not seek. The engine fetches offset plus limit rows, sorts them, and discards the first N, so cost grows linearly with page depth. Cursor pagination passes the last seen key instead of a page number, letting the B+ tree seek in logarithmic time and read a fixed number of rows, which keeps latency flat at any depth. The price is losing absolute page numbers. ## How UUIDv4 Primary Keys Affect Database Insert Performance DevFeed: [How UUIDv4 Primary Keys Affect Database Insert Performance]() Original publisher: [Read original article]() Author: hello@ankit-rana.com Published: 2026-03-16T00:00:00Z Content type: article Language: en Sources: [Ankit Rana | Mechanical Sympathy]() Topics: [Database](), [MySQL](), [ordering]() Tags: [b-tree](), [database](), [database-performance](), [fragmentation](), [innodb](), [latency](), [mysql](), [performance](), [system-design](), [uuid]() ### AI overview This article explains how random UUIDv4 primary keys can reduce insert performance in clustered indexes. Random inserts land in the middle of B+ tree pages, causing page splits, fragmentation, additional writes, and buffer-pool churn. It recommends UUIDv7 or ULID, or an auto-increment clustered key with UUIDs in a unique secondary index. ### Source excerpt The problem with UUIDv4 primary keys is not uniqueness, it is that random keys fight the storage engine's physical layout. In a clustered index the primary key is the table, so random inserts land mid-page, trigger page splits, leave pages half-full, and can inflate a 50 GB dataset to roughly 100 GB while wasting buffer pool on empty space. Use UUIDv7 or ULID, or keep an auto-increment clustered key with the UUID in a unique secondary index. ## How io\_uring improves database performance DevFeed: [How io\_uring improves database performance]() Original publisher: [Read original article]() Published: 2025-12-11T00:00:00Z Content type: tutorial Language: en Sources: [Gaurav Sarma's Blog]() Topics: [io\_uring](), [Databases](), [Linux Kernel](), [NVMe]() Tags: [asynchronous](), [database-performance](), [io](), [io-uring](), [nvme](), [performance]() ### AI overview This article explains how io_uring improves database performance through shared submission and completion queues, batching, fewer system calls, zero-copy operation, and features such as SQPoll and registered buffers. It also discusses DMA and the relevance of fast NVMe storage. ### Source excerpt . [How io_uring Improves Database Performance](how-iouring-improves-database-performance-cover... ## Locality, and Temporal-Spatial Hypothesis DevFeed: [Locality, and Temporal-Spatial Hypothesis]() Original publisher: [Read original article]() Author: Marc Brooker Published: 2025-10-05T00:00:00Z Content type: article Language: en Sources: [Marc Brooker's Blog](), [Marc Brooker's Blog]() Topics: [Databases](), [Caching](), [systems](), [IO](), [Streaming](), [Time Series](), [observability]() Tags: [cache](), [database](), [database-performance](), [io](), [linux](), [metrics](), [observability](), [performance](), [postgres](), [streaming](), [systems](), [time-series]() ### AI overview The article examines locality in database and computer-system performance, focusing on how temporal and spatial locality influence caching, read-ahead, and data layout. It develops the temporal-spatial locality hypothesis and compares its relevance for streaming, time-series, metrics, observability, and hash-based database systems. ### Source excerpt Locality, and Temporal-Spatial Hypothesis Good fences make good neighbors? Last week at PGConf NYC, I had the pleasure of hearing Andres Freund talking about the great work he's been doing to bring async IO to Postgres 18. One particular result caught my eye: a large difference in performance between forward and reverse scans, seemingly driven by read ahead1. The short version is that IO layers (like Linux's) optimize performance by proactively pre-fetching data ahead of the current read point in a file, so it's already cached when needed. Notably, most of these systems don't do this backwards. This leads to a big difference in performance between forward scans (where the pages are already in the cache when they're needed) and backward scans (where the database needs to block on IO to fetch the next page). This lead me to thinking more about a particular hypothesis behind many database designs: a temporal-spatial locality hypothesis2. Before we get there, let's talk about locality more generally, because it might be the single most important idea in database performance (and computer systems performance generally). Temporal locality is the idea that data accessed recently is likely to be accessed again soon. This idea is what's behind CPU caches, database buffer pools, and most caches you'll come across in computer systems. Spatial locality is the idea that when we access data, we're likely to access nearby data soon. Almost all database systems take advantage of these forms of locality, and would lost significant performance without taking advantage of them. Stacks of books could be written about these ideas. Stacks of books have been written about these ideas. We could talk about cache-oblivious algorithms, or non-polluting read and write instructions, or have an argument about linked lists. Instead, I want to zoom in to a particular idea in databases: temporal-spatial hypothesis. The hypothesis I mean has a definition something like this: Temporal-spatial localit ## SEON Achieves 80% Faster Database Performance for Anti-Fraud Platform with ClickHouse on AWS DevFeed: [SEON Achieves 80% Faster Database Performance for Anti-Fraud Platform with ClickHouse on AWS]() Original publisher: [Read original article]() Author: AWS Published: 2025-07-23T09:16:59Z Content type: article Language: en Sources: [ClickHouse Blog]() Topics: [clickhouse](), [Amazon Web Services (AWS)](), [Database](), [data analytics](), [real-time](), [Scalability](), [Security, Privacy and Abuse Prevention](), [API](), [Machine learning]() Tags: [analytics](), [api](), [aws](), [clickhouse](), [data](), [data-analytics](), [database](), [database-performance](), [fraud](), [machine-learning](), [performance](), [real-time](), [scalability](), [storage]() ### AI overview SEON worked with ClickHouse and AWS to improve the performance and scalability of its anti-fraud and anti-money-laundering platform. The solution reduced processing time by more than 80 percent while supporting near-real-time transaction analysis and large, customized rule sets. ### Source excerpt Learn how anti-fraud platform SEON worked with ClickHouse and used AWS to improve near real-time data analytics by 80% and reduce data storage costs by 30% ## Property Finder Optimizes Database Performance, Reduces Costs by 50% with ClickHouse on AWS DevFeed: [Property Finder Optimizes Database Performance, Reduces Costs by 50% with ClickHouse on AWS]() Original publisher: [Read original article]() Author: AWS Published: 2025-07-21T00:00:00Z Content type: article Language: en Sources: [ClickHouse Blog]() Topics: [clickhouse](), [Amazon Web Services (AWS)](), [Database](), [data-engineering](), [Website]() Tags: [amazon-web-services](), [aws](), [clickhouse](), [data-engineering](), [database](), [database-performance](), [performance]() ### AI overview Property Finder migrated to ClickHouse running on Amazon Web Services to improve database and analytics performance across regions. The article reports a 50% reduction in operating costs, a fivefold improvement in website performance, and a 96% improvement in database query speed. ### Source excerpt "ClickHouse and AWS are the most trusted collaborators we have. We've benefited massively from our joint product roadmap" Khaled Thabit - Director of Data Engineering, Property Finder ## Heroku Postgres 17 with the New Upgrade Process: Faster Performance, Easier Upgrade DevFeed: [Heroku Postgres 17 with the New Upgrade Process: Faster Performance, Easier Upgrade]() Original publisher: [Read original article]() Author: Jonathan Brown Published: 2025-05-29T15:32:49Z Content type: release Language: en Sources: [Heroku]() Topics: [Heroku](), [PostgreSQL](), [upgrade](), [Database](), [JSON](), [Monitoring](), [Latency]() Tags: [availability](), [database](), [database-performance](), [heroku](), [heroku-postgres](), [monitoring](), [news](), [performance](), [performance-optimization](), [platform-updates](), [postgresql](), [sql-json](), [upgrade]() ### AI overview Heroku announces the general availability of Heroku Postgres 17 and makes a new version-upgrade method the default. The release includes query and write performance improvements, enhanced JSON support, lower vacuum memory usage, and expanded observability features. ### Source excerpt We're excited to announce the general availability of Heroku Postgres version 17, packed with new features and enhancements to your database performance. And that's not all - we're also introducing a game-changing feature that streamlines your upgrade experience. This new method of version upgrade is now the default, so you can try it to upgrade [...] The post Heroku Postgres 17 with the New Upgrade Process: Faster Performance, Easier Upgrade appeared first on Heroku. ## Introducing Warehouses: Compute-compute separation in ClickHouse Cloud DevFeed: [Introducing Warehouses: Compute-compute separation in ClickHouse Cloud]() Original publisher: [Read original article]() Author: Dmitry Pavlov Published: 2025-01-27T00:00:00Z Content type: article Language: en Sources: [ClickHouse Blog]() Topics: [clickhouse](), [Cloud](), [Database]() Tags: [blog](), [clickhouse](), [cloud](), [database](), [database-performance](), [operations](), [performance](), [resource]() ### AI overview This blog introduces compute-compute separation in ClickHouse Cloud. The approach assigns isolated compute resources to different users, workloads, or database operations so that resource-intensive queries are less likely to affect the performance and stability of other workloads. It is presented as a way to improve workload isolation, resource management, and cost efficiency, including by separating reads from writes and protecting critical operations such as INSERTs and UPDATEs. ### Source excerpt This blog describes how compute-compute is helping ClickHouse Cloud customers achieve better tenant isolation and optimize their overall resource consumption and costs. ## Choosing the right Postgres indexes DevFeed: [Choosing the right Postgres indexes]() Original publisher: [Read original article]() Author: Milly Leadley Published: 2024-10-25T16:45:00Z Content type: tutorial Language: en Sources: [The incident.io Blog]() Topics: [Database](), [Sorting]() Tags: [blog](), [database](), [database-performance](), [how-to](), [incident](), [incident-channel](), [incident-management](), [incident-response](), [learn](), [outage](), [performance](), [post-mortem](), [postgres](), [slack-incident](), [sorting](), [strategy]() ### AI overview A practical guide to choosing Postgres indexes to improve query performance. It explains sequential and index scans, the tradeoffs of disk space and slower writes, suitable indexing scenarios, b-tree indexes, unique indexes, and column ordering in multi-column indexes. ### Source excerpt Indexes can dramatically boost your database performance, but knowing when to use them isn't always obvious. This blog covers what indexes are, when to use them, how to choose the right type, and tips for spotting missing ones. Whether you're optimizing queries, enforcing uniqueness, or improving sorting, you'll learn how to fine-tune your indexing strategy without overcomplicating it. ## Battling database performance DevFeed: [Battling database performance]() Original publisher: [Read original article]() Author: Rory Bain Published: 2023-04-20T00:00:00Z Content type: article Language: en Sources: [The incident.io Blog]() Topics: [Database](), [observability](), [incident](), [Traces](), [Grafana](), [Go Language](), [API]() Tags: [app](), [database](), [database-performance](), [errors](), [go](), [grafana](), [http](), [incident](), [incident-channel](), [incident-management](), [incident-response](), [logs](), [observability](), [on-call](), [outage](), [performance](), [post-mortem](), [slack-incident](), [traces]() ### AI overview The article describes intermittent database-related application timeouts that occurred over two weeks. It explains how incident investigation, Grafana dashboards, traces, and connection-pool instrumentation revealed requests waiting nearly 20 seconds for database connections, and discusses performance and observability changes made to address the issue. ### Source excerpt Earlier this year, we experienced intermittent timeouts in our application -- here's how we tried to address the underlying issue over the next two weeks. ## Measuring Realtime Database performance with Firebase Performance Monitoring DevFeed: [Measuring Realtime Database performance with Firebase Performance Monitoring]() Original publisher: [Read original article]() Author: Sam Stern Published: 2021-03-12T00:00:00Z Content type: tutorial Language: en Sources: [Firebase Blog]() Topics: [Firebase](), [Realtime Database](), [Monitoring](), [real user monitoring](), [Latency](), [SDK](), [WebSocket](), [React]() Tags: [backend](), [database-performance](), [firebase](), [javascript](), [latency](), [monitoring](), [performance](), [performance-monitoring](), [react](), [real-user-monitoring](), [realtime](), [realtime-database](), [tutorials](), [web](), [websocket]() ### AI overview This tutorial explains how to measure Realtime Database performance with Firebase Performance Monitoring. It uses custom traces to measure database operations over the long-running WebSocket connection and examines how latency varies by user geography. ### Source excerpt News, tutorials, and updates from the Firebase team. ## Introducing HaloDB, a fast, embedded key-value storage engine written in Java DevFeed: [Introducing HaloDB, a fast, embedded key-value storage engine written in Java]() Original publisher: [Read original article]() Author: amberwilsonla-blog Published: 2018-09-19T23:48:42Z Content type: release Language: en Sources: [Yahoo]() Topics: [Databases](), [Java](), [benchmarking](), [Open Source](), [Software]() Tags: [architecture](), [benchmarking](), [data](), [database-performance](), [embedded](), [go](), [java](), [open-source](), [performance](), [yahoo-engineering]() ### AI overview Yahoo developers introduces HaloDB, an embedded key-value storage engine written in Java. Built for an IO-bound workload with high read and write throughput, HaloDB uses append-only log files and an in-memory index. The article reports a 50% improvement in write capacity while maintaining submillisecond read latency at the 99th percentile, and announces the project's availability under the Apache license. ### Source excerpt yahoodevelopers: By Arjun Mannaly, Senior Software Engineer At Oath, multiple ad platforms use a high throughput, low latency distributed key-value database that runs in data centers all over the world. The database stores billions of records and handles millions of read and write requests per second at millisecond latencies. The data we have in this database must be persistent, and the working set is larger than what we can fit in memory. Therefore, a key component of the database performance is a fast storage engine. Our current solution had served us well, but it was primarily designed for a read-heavy workload and its write throughput started to be a bottleneck as write traffic increased. There were other additional concerns as well; it took hours to repair a corrupted DB, or iterate over and delete records. The storage engine also didn't expose enough operational metrics. The primary concern though was the write performance, which based on our projections, would have been a major obstacle for scaling the database. With these concerns in mind, we began searching for an alternative solution. We searched for a key-value storage engine capable of dealing with IO-bound workloads, with submillisecond read latencies under high read and write throughput. After concluding our research and benchmarking alternatives, we didn't find a solution that worked for our workload, thus we were inspired to build HaloDB. Now, we're glad to announce that it's also open source and available to use under the terms of the Apache license. HaloDB has given our production boxes a 50% improvement in write capacity while consistently maintaining a submillisecond read latency at the 99th percentile. Architecture HaloDB primarily consists of append-only log files on disk and an index of keys in memory. All writes are sequential writes which go to an append-only log file and the file is rolled-over once it reaches a configurable size. Older versions of records are removed to make space by a bac ## Profiling your Realtime Database Performance DevFeed: [Profiling your Realtime Database Performance]() Original publisher: [Read original article]() Author: Tyler Rockwood Published: 2017-03-14T00:00:00Z Content type: tutorial Language: en Sources: [Firebase Blog]() Topics: [Firebase](), [Command-line interface](), [Database]() Tags: [app-quality](), [cli](), [database-performance](), [firebase](), [firebase-cli](), [launch](), [profiling](), [realtime-database](), [speed]() ### AI overview Firebase introduces a profiler built into the Firebase CLI for monitoring Realtime Database performance. The database:profile command reports path-level speed, bandwidth, and unindexed-query data, with an optional raw output mode for more detailed operation information. ### Source excerpt News, tutorials, and updates from the Firebase team. ## IdentityCache: Improving Performance one Cached Model at a Time DevFeed: [IdentityCache: Improving Performance one Cached Model at a Time]() Original publisher: [Read original article]() Author: Camilo Lopez Published: 2013-04-04T13:24:00Z Content type: article Language: en Sources: [Shopify Engineering](), [Shopify Engineering - Shopify Engineering]() Topics: [Caching](), [Shopify](), [Database](), [Library](), [web applications]() Tags: [activerecord](), [caching](), [database](), [database-performance](), [library](), [performance](), [shopify]() ### AI overview Shopify open-sources IdentityCache, an internal library that caches ActiveRecord models in Memcached. The article explains how moving database reads to the cache improved performance and resilience as traffic and write-heavy sales events increased. ### Source excerpt A month ago Shopify was at BigRubyConf where we mentioned an internal library we use for caching ActiveRecord models called IdentityCache. We're pleased to say that the library has been extracted out of the Shopify code base and has been open sourced! At Shopify, our core application has been database performance bound for much of our platform's history. That means that the most straightforward way of making Shopify more performant and resilient is to move work out of the database layer. ## Understanding Postgres Performance DevFeed: [Understanding Postgres Performance]() Original publisher: [Read original article]() Author: Map Published: 2012-10-01T20:55:56Z Content type: tutorial Language: en Sources: [Craig Kerstiens]() Topics: [PostgreSQL](), [Caching](), [Database](), [Heroku Postgres](), [Amazon EC2]() Tags: [cache](), [database](), [database-performance](), [ec2](), [heroku](), [heroku-postgres](), [index](), [indexes](), [postgres](), [postgres-performance](), [postgresql]() ### AI overview A practical guide to assessing PostgreSQL performance for application developers. It explains how to check cache hit rates, when to increase available database cache, and how index usage and table size can indicate opportunities for improvement. ### Source excerpt Update theres a more recent post that expands further on where to start optimizing specific queries, and of course if you want to dig into optimizing your infrastructure High Performance PostgreSQL is still a great read For many application developers their database is a black box. Data goes in, comes back out and in between there developers hope its a pretty short time span. Without becoming a DBA there's a few pieces of data that most application developers can easily grok which will help them understand if their database is performing adequately. This post will provide some quick tips that allow you to determine whether your database performance is slowing down your app, and if so what you can do about it. Understanding your Cache and its Hit Rate The typical rule for most applications is that only a fraction of its data is regularly accessed. As with many other things data can tend to follow the 80/20 rule with 20% of your data accounting for 80% of the reads and often times its higher than this. Postgres itself actually tracks access patterns of your data and will on its own keep frequently accessed data in cache. Generally you want your database to have a cache hit rate of about 99%. You can find your cache hit rate with: SELECT sum(heap_blks_read) as heap_read, sum(heap_blks_hit) as heap_hit, sum(heap_blks_hit) / (sum(heap_blks_hit) + sum(heap_blks_read)) as ratio FROM pg_statio_user_tables; We can see in this dataclip that the cache rate for Heroku Postgres is 99.99%. If you find yourself with a ratio significantly lower than 99% then you likely want to consider increasing the cache available to your database, you can do this on Heroku Postgres by performing a fast database changeover or on something like EC2 by performing a dump/restore to a larger instance size. Understanding Index Usage The other primary piece for improving performance is indexes. Several frameworks will add indexes on your primary keys, though if you're searching on other fields or joini