# Why pipes sometimes get "stuck": buffering

DevFeed: [Why pipes sometimes get "stuck": buffering](<https://devfeed.tech/articles/why-pipes-sometimes-get-stuck-buffering-21115.md>)

Original publisher: [Read original article](<https://jvns.ca/blog/2024/11/29/why-pipes-get-stuck-buffering/>)

Author: Julia Evans

Published: 2024-11-29T08:23:31Z

Content type: article

Language: en

Sources: [Julia Evans](<https://devfeed.tech/sources/julia-evans.md>)

Topics: [Terminal](<https://devfeed.tech/topics/terminal.md>), [Command-line interface](<https://devfeed.tech/topics/cli.md>), [Streams](<https://devfeed.tech/topics/streams.md>)

Tags: [deep-dive](<https://devfeed.tech/tags/deep-dive.md>), [glibc](<https://devfeed.tech/tags/glibc.md>), [implementation](<https://devfeed.tech/tags/implementation.md>), [libc](<https://devfeed.tech/tags/libc.md>), [performance](<https://devfeed.tech/tags/performance.md>), [terminal](<https://devfeed.tech/tags/terminal.md>)

## AI overview

This article explains why output from commands in Unix pipelines can appear to get stuck. Programs such as grep often use block buffering when writing to a pipe, so matches may remain in memory until the buffer fills or the program exits. When output goes directly to a terminal, line buffering usually makes each line appear immediately. The behavior depends on the program's implementation and, in the case discussed, libc and glibc.

## Source excerpt

Here's a niche terminal problem that has bothered me for years but that I never really understood until a few weeks ago. Let's say you're running this command to watch for some specific output in a log file: tail -f /some/log/file | grep thing1 | grep thing2 If log lines are being added to the file relatively slowly, the result I'd see is... nothing! It doesn't matter if there were matches in the log file or not, there just wouldn't be any output. I internalized this as "uh, I guess pipes just get stuck sometimes and don't show me the output, that's weird", and I'd handle it by just running grep thing1 /some/log/file | grep thing2 instead, which would work. So as I've been doing a terminal deep dive over the last few months I was really excited to finally learn exactly why this happens. why this happens: buffering The reason why "pipes get stuck" sometimes is that it's VERY common for programs to buffer their output before writing it to a pipe or file. So the pipe is working fine, the problem is that the program never even wrote the data to the pipe! This is for performance reasons: writing all output immediately as soon as you can uses more system calls, so it's more efficient to save up data until you have 8KB or so of data to write (or until the program exits) and THEN write it to the pipe. In this example: tail -f /some/log/file | grep thing1 | grep thing2 the problem is that grep thing1 is saving up all of its matches until it has 8KB of data to write, which might literally never happen. programs don't buffer when writing to a terminal Part of why I found this so disorienting is that tail -f file | grep thing will work totally fine, but then when you add the second grep, it stops working!! The reason for this is that the way grep handles buffering depends on whether it's writing to a terminal or not. Here's how grep (and many other programs) decides to buffer its output: Check if stdout is a terminal or not using the isatty function If it's a terminal, use line b