"High" cpu usage when using ZFS cache device

[Ivan Voras]

On 16 November 2010 13:15, Christer Solskogen
<christer.solskogen at gmail.com> wrote:
> On Tue, Nov 16, 2010 at 12:47 PM, Ivan Voras <ivoras at freebsd.org> wrote:
>
>> You can easily test it - use the stick as a simple disk device with UFS and
>> see how much CPU does it take simply to talk to the device.
>
> See, that is why I think it is a ZFS issue. Because I did that.
> I created a UFS filesystem on the same usb stick. Mounted it and did a
> "dd if=/dev/zero of=/mnt/file".
> The systemload goes +0.6 instead if +10.3.
>
> See:
> CPU:  0.0% user,  0.0% nice,  0.6% system,  0.0% interrupt, 99.3% idle
> Mem: 832M Active, 960M Inact, 7017M Wired, 2600K Cache, 1237M Buf, 3063M Free
> Swap: 8192M Total, 8192M Free
>
>  PID USERNAME    THR PRI NICE   SIZE    RES STATE   C   TIME   WCPU COMMAND
> 38261 root          1  46    0  5776K  1112K wdrain  7   0:07  4.98% dd
>
> But when using it as cache device for zfs:
>
> CPU:  0.0% user,  0.0% nice, 11.9% system,  0.0% interrupt, 88.1% idle
> Mem: 832M Active, 193M Inact, 5782M Wired, 2592K Cache, 1237M Buf, 5066M Free
> Swap: 8192M Total, 8192M Free
>
> The funny thing is that when I add the device (and some cache is added
> to it) the load is normal. But the load goes up when nothing is
> written to it (or beeing read from it)

You mean you have system load on an otherwise idle system?

Try this:

1) start "top" with parameters -H -S, see if anything is using the CPU time

2) start "gstat", see if anything is using IO, and if it's
particularly slow or busying the device too much