[Bug 275594] High CPU usage by arc_prune; analysis and fix

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=275594

--- Comment #33 from Seigo Tanimura <seigo.tanimura@gmail.com> ---
(In reply to Thomas Mueller from comment #32)
> After upgrading from 12-STABLE to 13-STABLE (due to ports changes
> triggered by 12 being EOLed), I'm now observing high CPU usage of
> kernel{arc_prune} on 13-STABLE too.

Do you see any other threads using the CPU as much as kernel{arc_prune}? eg

- vnlru
- Any threads that access files somehow while running poudriere-bulk(8) (eg
cc1)

If so, what you have seen is the same as mine.  Kernel{arc_prune} and the
threads above contend for the vnode list lock.  Each of them spins in the
kernel until it acquires the lock, which can be found by top(1) if you have any
idle CPUs.  You may have to reduce the builders to let top(1) work.

I was not aware at the time of the last massive poudriere-bulk(8) on
13.2-RELEASE, but it is now likely that the same issue occured on it as well.

The comparision of my poudriere-bulk(8) results, both on the same host except
for the OS versions:

                         | 13.2-RELEASE | 14.0-RELEASE
-------------------------+--------------+-------------
Build Date               |  13 Apr 2023 |  19 Jan 2024
ZFS Fix                  |           No |          Yes
# of Packages            |         1147 |         2128
# of Successful Packages |         1136 |         2127
Elapsed Time             |     18:44:33 |     06:54:28
Packages / Hour          |           61 |          309


> Questions:
> Would migrating to ZFS on root mitigate the issues?

I would say no; that would give even move pressure to ARC.

> Is 13-STABLE in focus for this PR?

Not for now, but it should be.  In addition, FreeBSD-EN-23:18.openzfs should
include 13-STABLE as well.

I have one baremetal 13.2-RELEASE host with ZFS, but it does not suffer from
the issue as of now.  This host serves the volumes to the bhyve(8) VMs mainly,
so it does not use vnodes heavily.

-- 
You are receiving this mail because:
You are the assignee for the bug.