sort(1) memory usage

[Dag-Erling Smørgrav]

Dag-Erling Smørgrav <des at des.no> writes:
> Erik Trulsson <ertr1013 at student.uu.se> writes:
> > Yep, it seems that GNU sort allocates a quite large buffer by default when
> > the size of the input is unknown (such as when it reads input from stdin.)
> > A quick check in the source code indicates that it tries to size this buffer
> > according to how much memory the system has (and according to any limits set
> > on how much memory the process is allowed to use.)
> Uh, OK.  This scaling doesn't seem to work correctly.  It seems to
> allocate 27 MB on 32-bit machines and 54 MB on 64-bit machines,
> regardless of memory size.

Looking at the code, it seems to go to extreme lengths to get it
absolutely wrong.  For instance, if hw.physmem / 8 > hw.usermem, it will
pick the former, which means it's pretty much guaranteed to either fail
or hose your system (or both).

In the immortal words of Blazing Star: YOU FAIL IT

Count this as a vote for ditching GNU sort in favor of a BSD-licensed
implementation (from {Net,Open}BSD for instance).

DES
-- 
Dag-Erling Smørgrav - des at des.no