ZFS performance degradation over time

[Garrett Moore]

Sure:

[root at leviathan ~]# sysctl kstat.zfs.misc.arcstats
kstat.zfs.misc.arcstats.hits: 32092629
kstat.zfs.misc.arcstats.misses: 1064835
kstat.zfs.misc.arcstats.demand_data_hits: 30542262
kstat.zfs.misc.arcstats.demand_data_misses: 848959
kstat.zfs.misc.arcstats.demand_metadata_hits: 1550367
kstat.zfs.misc.arcstats.demand_metadata_misses: 215876
kstat.zfs.misc.arcstats.prefetch_data_hits: 0
kstat.zfs.misc.arcstats.prefetch_data_misses: 0
kstat.zfs.misc.arcstats.prefetch_metadata_hits: 0
kstat.zfs.misc.arcstats.prefetch_metadata_misses: 0
kstat.zfs.misc.arcstats.mru_hits: 18329884
kstat.zfs.misc.arcstats.mru_ghost_hits: 114483
kstat.zfs.misc.arcstats.mfu_hits: 13762745
kstat.zfs.misc.arcstats.mfu_ghost_hits: 172573
kstat.zfs.misc.arcstats.deleted: 1735926
kstat.zfs.misc.arcstats.recycle_miss: 2076926
kstat.zfs.misc.arcstats.mutex_miss: 545
kstat.zfs.misc.arcstats.evict_skip: 532474
kstat.zfs.misc.arcstats.hash_elements: 6784
kstat.zfs.misc.arcstats.hash_elements_max: 14351
kstat.zfs.misc.arcstats.hash_collisions: 149862
kstat.zfs.misc.arcstats.hash_chains: 338
kstat.zfs.misc.arcstats.hash_chain_max: 4
kstat.zfs.misc.arcstats.p: 25819136
kstat.zfs.misc.arcstats.c: 107609280
kstat.zfs.misc.arcstats.c_min: 107609280
kstat.zfs.misc.arcstats.c_max: 860874240
kstat.zfs.misc.arcstats.size: 40148272
kstat.zfs.misc.arcstats.hdr_size: 1411072
kstat.zfs.misc.arcstats.l2_hits: 0
kstat.zfs.misc.arcstats.l2_misses: 0
kstat.zfs.misc.arcstats.l2_feeds: 0
kstat.zfs.misc.arcstats.l2_rw_clash: 0
kstat.zfs.misc.arcstats.l2_writes_sent: 0
kstat.zfs.misc.arcstats.l2_writes_done: 0
kstat.zfs.misc.arcstats.l2_writes_error: 0
kstat.zfs.misc.arcstats.l2_writes_hdr_miss: 0
kstat.zfs.misc.arcstats.l2_evict_lock_retry: 0
kstat.zfs.misc.arcstats.l2_evict_reading: 0
kstat.zfs.misc.arcstats.l2_free_on_write: 0
kstat.zfs.misc.arcstats.l2_abort_lowmem: 0
kstat.zfs.misc.arcstats.l2_cksum_bad: 0
kstat.zfs.misc.arcstats.l2_io_error: 0
kstat.zfs.misc.arcstats.l2_size: 0
kstat.zfs.misc.arcstats.l2_hdr_size: 0
kstat.zfs.misc.arcstats.memory_throttle_count: 10349

Additionally, from top:
Mem: 2869M Active, 440M Inact, 479M Wired, 91M Cache, 418M Buf, 63M Free

Getting <20MB/s combined read/write at this point (ie copying files from one
directory to another, both source and destination on the zpool).


On Sat, Jan 9, 2010 at 7:56 PM, Jeremy Chadwick <freebsd at jdc.parodius.com>wrote:

> On Sat, Jan 09, 2010 at 07:40:53PM -0500, Garrett Moore wrote:
> > If no-one has any questions, I'll try Artem's suggestion of wasting a
> bunch
> > of memory in Perl/Python and forcing some memory to be swapped out. (I
> don't
> > want to do it yet in case someone wants a specific number before I do
> that).
>
> It would help if you could provide the entire output from:
>
> sysctl kstat.zfs.misc.arcstats
>
> --
> | Jeremy Chadwick                                   jdc at parodius.com |
> | Parodius Networking                       http://www.parodius.com/ |
> | UNIX Systems Administrator                  Mountain View, CA, USA |
> | Making life hard for others since 1977.              PGP: 4BD6C0CB |
>
> _______________________________________________
> freebsd-stable at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscribe at freebsd.org"
>