ZFS performance degradation over time

[Alexander Leidinger]

On Fri, 8 Jan 2010 15:05:06 -0800 Jeremy Chadwick
<freebsd at jdc.parodius.com> wrote:

> All that said -- I know what the OP is referring to, as I've seen it
> myself (on RELENG_7, and possibly early releases of 8.0).  The only
> way to relieve the pain, AFAIK, is to reboot.

I assume the problem is that the memory handling in ZFS is not the best
regarding memory fragmentation (at least in FreeBSD, I do not know of
the memory handling in Solaris has a similar behavior).

Regarding the RELENG_7 systems you have which use ZFS, are they SMP
systems? If yes, do you see problems when a lot of parallel accesses
(let's say several find in parallel) are made? All the commits to ZFS
for 7-stable are triggered by a problem I have in this regard on a
7-stable system, but so far nothing helped.

> I do see some MFC's done about 13 hours ago to RELENG_7 and RELENG_8
> that talk about the ARC and "paging pressure", which to me means
> decreased performance when it occurs... or maybe it helps with the
> kmem exhaustion problem?  The brief description in the commit is
> simply not enough to suffice; it's almost like we need a "FreeBSD ZFS
> Newsletter" that documents what all the changes are that get
> committed, what they fix, and what's being worked on/tested in HEAD
> (for potential MFC).

The change you talk about tells ZFS to cleanup the ARC when the max
size is passed, instead when there is real need to get some free
memory. The dresult is that the arc_max setting is more strictly
followed, and that there is no expensive operation to be done to free
the ARC when the system needs free memory.

Bye,
Alexander.