Re: Stressing malloc(9)

On Mon, Apr 22, 2024 at 2:07 PM Karl Denninger <karl@denninger.net> wrote:
>
> On 4/22/2024 12:46, Alan Somers wrote:
>
> When I said "33kiB" I meant "33 pages", or 132 kB.  And the solution
> turns out to be very easy.  Since I'm using ZFS on top of geli, with
> the default recsize of 128kB, I'll just set
> vfs.zfs.vdev.aggregation_limit to 128 kB.  That way geli will never
> need to allocate more than 128kB contiguously.  ZFS doesn't even need
> those big allocations to be contiguous; it's just aggregating smaller
> operations to reduce disk IOPs.  But aggregating up to 1MB (the
> default) is overkill; any rotating HDD should easily be able to max
> out its consecutive write IOPs with 128kB operation size.  I'll add a
> read-only sysctl for g_eli_alloc_sz too.  Thanks Mark.
>
> -Alan
>
> Setting this on one of my production machines that uses zfs behind geli drops the load average quite materially with zero impact on throughput that I can see (thus far.)  I will run this for a while but it certainly doesn't appear to have any negatives associated with it and does appear to improve efficiency quite a bit.

Great news!  Also, FTR I should add that this advice only applies to
people who use HDDs.  For SSDs zfs uses a different aggregation limit,
and the default value is already low enough.
-Alan