Asynchronous pipe I/O
rihad at mail.ru
Wed Nov 5 04:23:09 PST 2008
Imagine this shell pipeline:
sh prog1 | sh prog2
As given above, prog1 blocks if prog2 hasn't yet read previously written
data (actually, newline separated commands) or is busy. What I want is
for prog1 to never block:
sh prog1 | buffer | sh prog2
I first thought that the aptly named misc/buffer port would do exactly
what I wanted: buffering prog1 output for prog2 to read it at its
earliest convenience. That way prog1 would never block (unless it hit
buffer's memory limits). Alas, misc/buffer was originally designed for
tape backups, and despite its author's stating its applicability to
"This is a program designed initially to speed up writing tapes on
remote tape drives, but may be used as a general pipe buffering utility."
buffer never starts writing unless the limit given by -s is crossed,
which is 10 kbytes by default and cannot be less than 496 bytes, which
is too much for me. Ideally I want it to start writing immediately,
whenever new data hits its pools. Unfortunately, the -p 0 option doesn't
Size in bytes of each block. The default blocksize is
10k to match the normal output of the tar(1) program.
Only start a write when the given percentage of the internal
queue is full. A percentage around 75 often proves best.
Defaults to zero.
Wouldn't such an intermediary tool be a great way to boost performance
for certain types of solutions?
Thanks for any tips (Sorry if this was an inappropriate place to ask)
More information about the freebsd-questions