Re: Clogged pipe?

Pete:

> On FreeBSD 13.1-RELEASE-p7 I have a small test file, named testfile, 
> which contains three short lines of text, "one," "two," and "three" 
> without the quotes.
> 
> This command waits indefinitely without producing any output:
> 
>    tail -f testfile | cat -n | sed -u 's/^/X/'

Stdio buffering.  See setbuf(3).

By default, output using stdio functions is buffered.  If it goes
to a tty, it is line-buffered, i.e., output is saved up until a
whole line terminated by '\n' is complete, then that line is written.
Output to files or pipes is block-buffered.  Output is saved up
until 8 kB or 16 kB or such, and only then is it written.  The
buffer is also flushed on program exit.

> But, these five commands all work as expected, immediately outputting 
> versions of the three lines of text:
> 
>    tail -f testfile | cat -n

tail(1) specifically avoids stdio buffering.
cat(1) here writes to a tty, so output is only line-buffered.

>    cat -n testfile | sed -u 's/^/X/'

cat(1) terminates, so it writes all its output.
The -u option to sed(1) disables buffering, although it would default
to line-buffering here anyway if output goes to a tty.

>    tail -f testfile | sed -u 's/^/X/'

tail(1) specifically avoids stdio buffering.
The -u option to sed(1) disables buffering, although it would default
to line-buffering here anyway if output goes to a tty.

>    tail testfile | cat -n | sed -u 's/^/X/'

tail(1) terminates, so it writes all its output.
cat(1) terminates, so it writes all its output.
The -u option to sed(1) disables buffering, although it would default
to line-buffering here anyway if output goes to a tty.

>    tail -f testfile | cat | sed -u 's/^/X/'

tail(1) specifically avoids stdio buffering.
Looks like plain cat(1) also avoids buffering and only introduces
buffering when you specify line-oriented filter functions such as
-n.
The -u option to sed(1) disables buffering, although it would default
to line-buffering here anyway if output goes to a tty.

> Interestingly (or maybe not), the issue doesn't occur on my Debian box; 
> all six commands produce immediate output there in both dash and bash.

Differences in default buffering behavior.

Note that you can explicitly disable cat(1)'s output buffering with
the -u option.

There is also stdbuf(1).

-- 
Christian "naddy" Weisgerber                          naddy@mips.inka.de