[RFC] stdbuf: Force stdio's streams initial buffering mode (patch)

[Jeremie Le Hen]

Hi,

On Thu, Aug 04, 2011 at 07:19:28PM +0200, Jeremie Le Hen wrote:
> On Sat, Feb 19, 2011 at 07:50:43PM +0100, Jeremie Le Hen wrote:
> > I've been annoyed multiple time when running a command such like
> >     iostat -x 1 | grep -v ad10 | cat -n
> > 
> > The problem stems from two factors:
> >   - grep's stdio sees that its stdout is not a terminal, so stdout is
> >     full buffered and not line-buffered;
> >   - iostat produces output too slowly so the aforementioned buffer takes
> >     numerous seconds to be filled and flushed to the last command.
> > 
> > This problems is not specific to FreeBSD, it is actually a consequence
> > of POSIX specification.  I've checked this on Solaris and Linux.
> > 
> > I've attached a small patch for stdio, so if the environment variable
> > STDIO_IOLBF is set, the output streams will be line-oriented by default.
> >     iostat -x 1 | env STDIO_IOLBF=1 grep -v ad10 | cat -n
> 
> I improved the whole picture.  Now there is a shared library
> libstdbuf.so which can be loaded with LD_PRELOAD and configured through
> a environment variables.  It is able to initial control buffering for
> stdin, stdout and stderr (no buffering, line buffering, block
> buffering).  There is also an utility named stdbuf(1) which can be used
> to run a command with the appropriate environment variables.  Those are
> named after a similar tool in recent versions of GNU coreutils; of
> course, I also borrowed the interface for POLA.
> 
> Here is how to use it (example taken from the manpage):
> 
>     vmstat 1 | stdbuf -o L awk '$2 > 1 || $3 > 1' | cat -n
> 
> I think that using a preloaded shared library is better performance-wise
> because libc doesn't bother looking up configuration variables in the
> environment upon each execve(2), especially for something which is to be
> rarely used.
> 
> Manpages for both stdbuf(1) and libstdbuf(3) are provided.  There is
> surely room for improvement, so feel free to propose corrections to
> them.
> 
> The patch can be applied as-is I think, although I've developped it
> against an old -CURRENT source tree (about 6 months ago).
> 
> I'm looking for some feedback and review and hopefully this feature will
> be committed for FreeBSD 9.0-RELEASE.

It's now too late for 9.0-RELEASE :).  But I'm still willing to see this
feature committed eventually!

While talking about this implementation with a friend, he told me that
having a separate shared library sounds a little bit hackish and two or
three getenv(3) in the libc startup path shouldn't eat too much CPU
cycles.

Any opinion on this?  Anyone willing to commit this?

Thanks.  Regards,
-- 
Jeremie Le Hen

Men are born free and equal.  Later on, they're on their own.
				Jean Yanne