"High" cpu usage when using ZFS cache device

[Alexander Leidinger]

Quoting Christer Solskogen <christer.solskogen at gmail.com> (from Tue,  
16 Nov 2010 13:15:32 +0100):

> On Tue, Nov 16, 2010 at 12:47 PM, Ivan Voras <ivoras at freebsd.org> wrote:
>
>> You can easily test it - use the stick as a simple disk device with UFS and
>> see how much CPU does it take simply to talk to the device.
>
> See, that is why I think it is a ZFS issue. Because I did that.
> I created a UFS filesystem on the same usb stick. Mounted it and did a
> "dd if=/dev/zero of=/mnt/file".
> The systemload goes +0.6 instead if +10.3.
>
> See:
> CPU:  0.0% user,  0.0% nice,  0.6% system,  0.0% interrupt, 99.3% idle
> Mem: 832M Active, 960M Inact, 7017M Wired, 2600K Cache, 1237M Buf, 3063M Free
> Swap: 8192M Total, 8192M Free
>
>   PID USERNAME    THR PRI NICE   SIZE    RES STATE   C   TIME   WCPU COMMAND
> 38261 root          1  46    0  5776K  1112K wdrain  7   0:07  4.98% dd
>
> But when using it as cache device for zfs:
>
> CPU:  0.0% user,  0.0% nice, 11.9% system,  0.0% interrupt, 88.1% idle
> Mem: 832M Active, 193M Inact, 5782M Wired, 2592K Cache, 1237M Buf, 5066M Free
> Swap: 8192M Total, 8192M Free
>
> The funny thing is that when I add the device (and some cache is added
> to it) the load is normal. But the load goes up when nothing is
> written to it (or beeing read from it)

How do you measure that nothing is read or written to it?

Please check with
   gstat -f '^<DEVICE>$'
if there are really no reads/writes to the device (please replace  
<DEVICE> with the name of your USB device, e.g. da0).

If you see writes, I would say
  - this is the reason for the load
  - your cache is on the way to be filled with
    useful data

If gstat shows zero activity, I suggest to run 'top -S' and look at  
the process(es) which consume about 10% CPU (do not take care about  
the idle process). Based upon this we can maybe suggest further things  
to investigate.

Bye,
Alexander.

-- 
Is a tattoo real, like a curb or a battleship?  Or are we suffering in
Safeway?

http://www.Leidinger.net    Alexander @ Leidinger.net: PGP ID = B0063FE7
http://www.FreeBSD.org       netchild @ FreeBSD.org  : PGP ID = 72077137