Re: Speed improvements in ZFS

From: Mateusz Guzik <mjguzik_at_gmail.com>
Date: Sun, 20 Aug 2023 20:02:31 UTC 
On 8/20/23, Alexander Leidinger <Alexander@leidinger.net> wrote:
> Am 2023-08-20 19:10, schrieb Mateusz Guzik:
>> On 8/18/23, Alexander Leidinger <Alexander@leidinger.net> wrote:
>
>>> I have a 51MB text file, compressed to about 1MB. Are you interested
>>> to
>>> get it?
>>>
>>
>> Your problem is not the vnode limit, but nullfs.
>>
>> https://people.freebsd.org/~mjg/netchild-periodic-find.svg
>
> 122 nullfs mounts on this system. And every jail I setup has several
> null mounts. One basesystem mounted into every jail, and then shared
> ports (packages/distfiles/ccache) across all of them.
>
>> First, some of the contention is notorious VI_LOCK in order to do
>> anything.
>>
>> But more importantly the mind-boggling off-cpu time comes from
>> exclusive locking which should not be there to begin with -- as in
>> that xlock in stat should be a slock.
>>
>> Maybe I'm going to look into it later.
>
> That would be fantastic.
>

I did a quick test, things are shared locked as expected.

However, I found the following:
        if ((xmp->nullm_flags & NULLM_CACHE) != 0) {
                mp->mnt_kern_flag |= lowerrootvp->v_mount->mnt_kern_flag &
                    (MNTK_SHARED_WRITES | MNTK_LOOKUP_SHARED |
                    MNTK_EXTENDED_SHARED);
        }

are you using the "nocache" option? it has a side effect of xlocking

-- 
Mateusz Guzik <mjguzik gmail.com>