zfs listing and CPU

[Paul Kraus]

> On Aug 11, 2017, at 2:28 AM, Eugene M. Zheganin <emz at norma.perm.ru> wrote:
> 
> Why does the zfs listing eat so much of the CPU ?

> 47114 root           1  20    0 40432K  3840K db->db  4   0:05 26.84% zfs
> 47099 root           1  20    0 40432K  3840K zio->i 17   0:05 26.83% zfs
> 47106 root           1  20    0 40432K  3840K db->db 21   0:05 26.81% zfs
> 47150 root           1  20    0 40432K  3428K db->db 13   0:03 26.31% zfs
> 47141 root           1  20    0 40432K  3428K zio->i 28   0:03 26.31% zfs
> 47135 root           1  20    0 40432K  3312K g_wait  9   0:03 25.51% zfs

> This is from winter 2017 11-STABLE (r310734), one of the 'zfs'es is cloning, and all the others are 'zfs list -t all'. I have like 25 gigs of free RAM, do I have any chance of speeding this up using may be some caching or some sysctl tuning ? We are using a simple ZFS web API that may issue concurrent or sequential listing requests, so as you can see they sometimes do stack.

How many snapshots do you have ? I have only seen this behavior with LOTS (not hundreds, but thousands) of snapshots.

What does your `iostat -x 1` look like ? I expect that you are probably saturating your drives with random I/O.