Pipes, cat buffer size
Dan Nelson
dnelson at allantgroup.com
Sat Oct 18 21:35:08 UTC 2008
In the last episode (Oct 18), Ivan Voras said:
> I'm working on a program that's intended to be used as a "filter", as
> in "something | myprogram > file". I'm trying it with cat and I'm
> seeing my read()s return small blocks, 64 kB in size. I suppose this
> is because cat writes in 64 kB blocks. So:
>
> a) Is there a way to programatically, per-process, set the pipe buffer
> size? The program in question is a compressor and it's particularly
> inefficient when given small blocks and I'm wondering if the system can
> buffer enough data for it.
Why not keep reading until you reach your desired compression block
size? Bzip2's default blocksize is 900k, for example.
> b) Is there any objection to the following patch to cat:
It might be simpler to just use "dd if=myfile obs=1m" instead of
patching cat.
--
Dan Nelson
dnelson at allantgroup.com
More information about the freebsd-hackers
mailing list