Pipes, cat buffer size

[Tim Kientzle]

>>>... the writer could write 1-byte buffers and
>>>the reader will be forced to read each byte individually.
>>
>>No; take a look at /sys/kern/sys_pipe.c .  Depending on how much data
>>is in the pipe, it switches between async in-kernel buffering (<8192
>>bytes), and direct page wiring between sender and receiver (basically
>>zero-copy).
> 
> Ok, maybe it's just not behaving as I thought it should. See this test program:

However, when I add "sleep(1)" to your test program,
I see the following output:

$ dd bs=1 if=/dev/zero | ./reader
read 2282 bytes
read 65536 bytes
read 65536 bytes
read 65536 bytes
read 65536 bytes
read 65536 bytes
read 65536 bytes
read 65536 bytes
read 65536 bytes
read 65536 bytes
read 65536 bytes

In your original example, because your sample reader does
nothing, it sees each write separately.  If the reader
was actually doing some work then the pipe would buffer
up data while your reader was busy.

This looks like exactly the right behavior:  The reader
will only block if there is no data in the pipe at all;
the writer will only block if it gets "too far ahead" of
the reader.  Except in those cases, each program gets
to do I/O as fast as it can.

If your program needs larger blocks, it should keep reading
until it gets enough data.  (The -B option of GNU tar is
an example of this sort of behavior.)

Tim