ZFS performance degradation over time

[Garrett Moore]

After being up for a few days and having good performance, I turned on
rTorrent and downloaded two 700MB files. After these torrents completed, my
performance has tanked again. My combined read/write speed to the array
won't exceed about 25MB/s.

It seems that after having downloaded a few torrents, resources are grabbed
and then not released. From top:
Mem: 2326M Active, 962M Inact, 484M Wired, 82M Cache, 418M Buf, 87M Free

Nothing in userland is using a significant amount of memory. eg rTorrent is
using 41MB according to top. Killing rTorrent does not alleviate the
performance problems.

arcstats.size is hovering around 30-40MB.

[root at leviathan ~]# sysctl kstat.zfs.misc.arcstats.size
kstat.zfs.misc.arcstats.size: 28953448
[root at leviathan ~]# sysctl vm.stats.vm.v_inactive_count
vm.stats.vm.v_inactive_count: 237831
[root at leviathan ~]# sysctl vm.stats.vm.v_active_count
vm.stats.vm.v_active_count: 595762
[root at leviathan ~]# sysctl vm.stats.vm.v_cache_count
vm.stats.vm.v_cache_count: 21472


If no-one has any questions, I'll try Artem's suggestion of wasting a bunch
of memory in Perl/Python and forcing some memory to be swapped out. (I don't
want to do it yet in case someone wants a specific number before I do that).