pw operations slow under zfs load

[javocado]

Sorry, the boot disk (SSD) is plain old ufs.

On Wed, Jul 29, 2015 at 2:11 PM, javocado <javocado at gmail.com> wrote:

> Hi,
>
> We have a pretty busy ZFS pool running on an 8.3 AMD system. We are
> noticing that when the pool is busy pw-related operations seem to  take a
> long time to complete:
>
> # time pw unlock 1000
> 0.007u 0.036s 0:39.72 0.0%      45+1953k 0+113io 0pf+0w
>
> # time pw lock 1000
> 0.032u 0.022s 1:09.63 0.0%      24+1132k 0+114io 0pf+0w
>
> Wile the command is running, we note that the process is locked in the D
> state:
>
> root  85051  0.0  0.0  5832   960   0  D+    1:53PM   0:00.02
> /usr/sbin/pwd_mkdb -u 1000 /etc/master.passwd
>
> We also note that there is next to 0 disk activity on the boot volume:
>
> # gstat -f ad
>
> dT: 1.005s  w: 1.000s  filter: ad
>  L(q)  ops/s    r/s   kBps   ms/r    w/s   kBps   ms/w   %busy Name
>     0      0      0      0    0.0      0      0    0.0    0.0| ad6
>     0      0      0      0    0.0      0      0    0.0    0.0| ad8
>     0      0      0      0    0.0      0      0    0.0    0.0| ad10
>     0      0      0      0    0.0      0      0    0.0    0.0| ad12
>
> And plenty of free mem:
>
> Mem: 400M Active, 3391M Inact, 128G Wired, 1935M Cache, 14G Buf, 6055M Free
>
> So, what's going on here? How does a busy pool with it's own set of drives
> (which operate off an HBA) affect the speed of operations involving the
> boot volume (an SSD connected to the mobo)?
>
>