# How much does the read/write buffer size matter for socket throughput?

DevFeed: [How much does the read/write buffer size matter for socket throughput?](<https://devfeed.tech/articles/how-much-does-the-read-write-buffer-size-matter-for-socket-throughput-20758.md>)

Original publisher: [Read original article](<https://www.evanjones.ca/read-write-buffer-size.html>)

Published: 2023-07-16T16:04:32Z

Content type: article

Language: en

Sources: [Evan Jones](<https://devfeed.tech/sources/evan-jones.md>)

Topics: [IO](<https://devfeed.tech/topics/io.md>), [systems](<https://devfeed.tech/topics/systems.md>), [Linux](<https://devfeed.tech/topics/linux.md>), [Benchmark](<https://devfeed.tech/topics/benchmark.md>), [networking](<https://devfeed.tech/topics/networking.md>), [Rust](<https://devfeed.tech/topics/rust.md>), [Ubuntu](<https://devfeed.tech/topics/ubuntu.md>), [Google Cloud Platform (GCP)](<https://devfeed.tech/topics/google-cloud.md>)

Tags: [amd](<https://devfeed.tech/tags/amd.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [blocking](<https://devfeed.tech/tags/blocking.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [kernel](<https://devfeed.tech/tags/kernel.md>), [linux](<https://devfeed.tech/tags/linux.md>), [networking](<https://devfeed.tech/tags/networking.md>), [performance](<https://devfeed.tech/tags/performance.md>), [processors](<https://devfeed.tech/tags/processors.md>), [rust](<https://devfeed.tech/tags/rust.md>), [tcp](<https://devfeed.tech/tags/tcp.md>), [test](<https://devfeed.tech/tags/test.md>), [ubuntu](<https://devfeed.tech/tags/ubuntu.md>), [unix](<https://devfeed.tech/tags/unix.md>)

## AI overview

This article reports experiments on how blocking I/O buffer sizes affect read() and write() throughput. It finds that approximately 32 KiB is a useful starting point, while large writes may benefit from buffers around 256 KiB to 1 MiB. Results vary by hardware and software.

## Source excerpt

The read() and write() system calls take a variable-length byte array as an argument. As a simplified model, the time for the system call should be some constant "per-call" time, plus time directly proportional to the number of bytes in the array. That is, the time for each call should be time = (per_call_minimum_time) + (array_len) x (per_byte_time). With this model, using a larger buffer should increase throughput, asymptotically approaching 1/per_byte_time. I was curious: do real system calls behave this way? What are the ideal buffer sizes for read() and write() if we want to maximize throughput? I decided to do some experiments with blocking I/O. These are not rigorous, and I suspect the results will vary significantly if the hardware and software are different than one the system I tested. The really short answer is that a buffer of 32 KiB is a good starting point on today's systems, and I would want to measure the performance to go beyond that. However, for large writes, performance can increase. On Linux, the simple model holds for small buffers (≤ 4 KiB), but once the program approaches the maximum throughput, the throughput becomes highly variable and in many cases decreases as the buffers get larger. For blocking I/O, approximately 32 KiB is large enough to hit the maximum throughput for read(), but write() throughput improves with buffers up to around 256 KiB - 1 MiB. The reason for the asymmetry is that the Linux kernel will only write less than the entire buffer (a "short write") if there is an error (e.g. a signal causing EINTR). Thus, larger write buffers means the operating system needs to switch to the process less often. On the other head, "short reads", where a read() returns less than the maximum length, become increasingly common as the buffer size increases, which diminishes the benefit. There is a SO_RCVLOWAT socket option to change this that I did not test. The experiments were run on two 16 CPU Google Cloud T2D instances, which use AMD EPYC