# Are you sure you want to use MMAP in your database management system?

DevFeed: [Are you sure you want to use MMAP in your database management system?](<https://devfeed.tech/articles/are-you-sure-you-want-to-use-mmap-in-your-database-management-system-25074.md>)

Original publisher: [Read original article](<https://databasearchitects.blogspot.com/2022/01/are-you-sure-you-want-to-use-mmap-in.html>)

Author: Viktor Leis (noreply@blogger.com)

Published: 2022-01-16T14:07:00Z

Content type: article

Language: en

Sources: [Database Architects](<https://devfeed.tech/sources/database-architects.md>)

Topics: [Databases](<https://devfeed.tech/topics/databases.md>), [Caching](<https://devfeed.tech/topics/caching.md>), [Transactions](<https://devfeed.tech/topics/transactions.md>), [IO](<https://devfeed.tech/topics/io.md>), [systems](<https://devfeed.tech/topics/systems.md>), [Kernel](<https://devfeed.tech/topics/kernel.md>), [NVMe](<https://devfeed.tech/topics/nvme.md>)

Tags: [asynchronous](<https://devfeed.tech/tags/asynchronous.md>), [cache](<https://devfeed.tech/tags/cache.md>), [caching](<https://devfeed.tech/tags/caching.md>), [database](<https://devfeed.tech/tags/database.md>), [kernel](<https://devfeed.tech/tags/kernel.md>), [nvme](<https://devfeed.tech/tags/nvme.md>), [pcie](<https://devfeed.tech/tags/pcie.md>), [performance](<https://devfeed.tech/tags/performance.md>), [systems](<https://devfeed.tech/tags/systems.md>), [transactions](<https://devfeed.tech/tags/transactions.md>)

## AI overview

The article discusses why database management systems often avoid mmap despite operating-system page caching. It explains that mmap limits control over write-back, transactions, crash recovery, asynchronous I/O, and error handling, and argues that Linux page-cache behavior may not keep pace with modern NVMe storage bandwidth.

## Source excerpt

Many database management systems carefully manage disk I/O operations and explicitly cache pages in main memory. Operating systems implement a page cache to speed up recurring disk accesses as well, and even allow transparent access to disk files through the mmap system call. Why do most database systems then even implement I/O handling and a caching component if the OS provides these features through mmap? Andrew Pavlo, Andrew Crotty, and myself tried to answer this question in a CIDR 2022 paper. This is quite a contentious question as the Hacker News discussion of the paper shows. The paper argues that using mmap in database systems is almost always a bad idea. To implement transactions and crash recovery with mmap, the DBMS has to write any change out-of-place because there is no way to prevent write back of a particular page. This makes it impossible to implement classical ARIES-style transactions. Furthermore, data access through mmap can take a handful of nanoseconds (if the data is in the CPU cache) or milliseconds (if it's on disk). If a page is not cached, it will be read through a synchronous page fault and there is no interface for asynchronous I/O. I/O errors, on the other hand, are communicated through signals rather than a local error code. These problems are caused by mmap's interface, which is too high-level and does not give the database system enough control. In addition to discussing these interface problems, the paper also shows that Linux' page cache and mmap implementation cannot achieve the bandwidth of modern storage devices. One PCIe 4.0 NVMe SSD can read over 6 GB/s and upcoming PCIe 5.0 SSDs will almost double this number. To achieve this performance, one needs to schedule hundreds or even thousands (if one has multiple SSDs) of concurrent I/O requests. Doing this in a synchronous fashion by starting hundreds of threads will not work well. Other kernel-level performance issues are single-threaded page eviction and TLB shootdowns. Overall,