sort(1) memory usage

[David Schultz]

On Sun, Feb 03, 2008, Dag-Erling Smørgrav wrote:
> Dag-Erling Smørgrav <des at des.no> writes:
> > Erik Trulsson <ertr1013 at student.uu.se> writes:
> > > Yep, it seems that GNU sort allocates a quite large buffer by default when
> > > the size of the input is unknown (such as when it reads input from stdin.)
> > > A quick check in the source code indicates that it tries to size this buffer
> > > according to how much memory the system has (and according to any limits set
> > > on how much memory the process is allowed to use.)
> > Uh, OK.  This scaling doesn't seem to work correctly.  It seems to
> > allocate 27 MB on 32-bit machines and 54 MB on 64-bit machines,
> > regardless of memory size.
> 
> Looking at the code, it seems to go to extreme lengths to get it
> absolutely wrong.  For instance, if hw.physmem / 8 > hw.usermem, it will
> pick the former, which means it's pretty much guaranteed to either fail
> or hose your system (or both).
> 
> In the immortal words of Blazing Star: YOU FAIL IT
> 
> Count this as a vote for ditching GNU sort in favor of a BSD-licensed
> implementation (from {Net,Open}BSD for instance).

We had been using a BSD-licensed sort(1), but ache@ changed it
back to GNU sort several years ago. Anyone know why? If I had to
guess I'd say i18n, but that's not very hard to deal with these
days given strcoll(3).

That said, I'm unaware of any technical differences between the two.