[RFC] stdbuf: Force stdio's streams initial buffering mode (patch)

[Jeremie Le Hen]

On Sat, Feb 19, 2011 at 07:50:43PM +0100, Jeremie Le Hen wrote:
> I've been annoyed multiple time when running a command such like
>     iostat -x 1 | grep -v ad10 | cat -n
> 
> The problem stems from two factors:
>   - grep's stdio sees that its stdout is not a terminal, so stdout is
>     full buffered and not line-buffered;
>   - iostat produces output too slowly so the aforementioned buffer takes
>     numerous seconds to be filled and flushed to the last command.
> 
> This problems is not specific to FreeBSD, it is actually a consequence
> of POSIX specification.  I've checked this on Solaris and Linux.
> 
> I've attached a small patch for stdio, so if the environment variable
> STDIO_IOLBF is set, the output streams will be line-oriented by default.
>     iostat -x 1 | env STDIO_IOLBF=1 grep -v ad10 | cat -n

I improved the whole picture.  Now there is a shared library
libstdbuf.so which can be loaded with LD_PRELOAD and configured through
a environment variables.  It is able to initial control buffering for
stdin, stdout and stderr (no buffering, line buffering, block
buffering).  There is also an utility named stdbuf(1) which can be used
to run a command with the appropriate environment variables.  Those are
named after a similar tool in recent versions of GNU coreutils; of
course, I also borrowed the interface for POLA.

Here is how to use it (example taken from the manpage):

    vmstat 1 | stdbuf -o L awk '$2 > 1 || $3 > 1' | cat -n

I think that using a preloaded shared library is better performance-wise
because libc doesn't bother looking up configuration variables in the
environment upon each execve(2), especially for something which is to be
rarely used.

Manpages for both stdbuf(1) and libstdbuf(3) are provided.  There is
surely room for improvement, so feel free to propose corrections to
them.

The patch can be applied as-is I think, although I've developped it
against an old -CURRENT source tree (about 6 months ago).

I'm looking for some feedback and review and hopefully this feature will
be committed for FreeBSD 9.0-RELEASE.

Regards,
-- 
Jeremie Le Hen

Humans are born free and equal.  But some are more equal than others.
					    Coluche
-------------- next part --------------
A non-text attachment was scrubbed...
Name: stdbuf.diff
Type: text/x-diff
Size: 16602 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20110804/892762cc/stdbuf.bin