rand_harvestq high cpu usage when /dev/urandom is used

[Conrad Meyer]

Hi Ali,

On Mon, Aug 20, 2018 at 4:27 AM, Ali Abdallah <aliovx at gmail.com> wrote:
> Hello,
>
> I was just sorting randomly some jpg image files using:
>
> ls *.jpg  | sort -R --random-source=/dev/urandom
>
> The above command never exited. Later I noticed that
> one of my CPU is always running 100%. top -S reveals that it is
> rand_harvestq kernel service.
>
> Is this is a bug? This occurs on 12-ALPHA1 and 11.2

There is probably at least one bug in sort(1).  sort has special
behavior if the --random-source matches its default (/dev/random), but
otherwise doesn't understand device files or pipes very well.  Since
urandom isn't exactly the same path as /dev/random, sort fails pretty
hard.

sort attempts to seed its internal RNG by MD5ing the provided random
source path.  For its default path, /dev/random, it grabs at most
MAX_DEFAULT_RANDOM_SEED_DATA_SIZE (or 1024) bytes.  This is hugely
excessive and MD5ing it is totally unnecessary, but still mostly
harmless.

For non-default files, it just passes the path to MD5File, which will
read() until EOF.  Since /dev/urandom will never return EOF, sort
--random-source=/dev/urandom will get stuck in MD5File forever.  This
is totally stupid.

I'm not sure why rand_harvestq would take 100% of a CPU core even as a
result of excessive consumption of /dev/urandom, but it is certainly
possible that sort(1) is consuming 100% of a CPU core reading from
urandom and MD5ing the result.

All the best,
Conrad