sysctl is too slow

From: Alan Somers <asomers_at_freebsd.org>
Date: Mon, 16 Aug 2021 18:50:46 UTC
ztop feels very sluggish on a server with 5000 ZFS datasets.  Dtrace shows
that almost all of its time is spent in sys_sysctl.  ktrace shows that both
ztop and sysctl(8) call sys_sysctl a total of five times for each sysctl
they care about:

1) To get the next oid
2) To get the sysctl's name
3) To get the oidfmt
4) To get the size of the value
5) To get the value itself.

Each of these steps takes about equal time, and together all five take
about 100us.  If the time per call is mostly syscall overhead, then the
process could be sped up by 80% by combining all of these things into a
single syscall: return the next oid, its name, its format, the size of its
value, and optimistically the value itself, assuming the user passed a
sufficiently large buffer.

Am I missing something?  Is there any other reason why sysctl is so slow?
Or should I forget about it, and try to export ZFS's dataset stats through
devstat instead?
-Alan