FreeBSD 10.1 Memory Exhaustion

[Matt Churchyard]

Yes, I'm one of those and I suspect it's very common.
I generally just use official FreeBSD releases and let the devs decide what patches are/aren't applied.
However, I limit max ARC on all my ZFS systems to leave a few GB or so (my latest system has limit of 28G with 32GB total)

Last time I didn't limit ARC was a new system about a year ago and it paniced due to memory after a few days
I don't bother letting it near all my RAM anymore.

For me (and probably many other ZFS users that want their machines to stay up more than a few days) it's much easier to do this, than to run manually patched kernels that *might* fix it, but *might* also cause other problems. Also allows me control over how much ZFS has, and how much I leave for my other applications.

Would be nice if it 'just worked', but I'll be very reluctant to take the limits off.

-Matt

-----Original Message-----
From: owner-freebsd-fs at freebsd.org [mailto:owner-freebsd-fs at freebsd.org] On Behalf Of Sean Chittenden
Sent: 14 July 2015 16:10
To: Adrian Gschwend
Cc: FreeBSD Filesystems
Subject: Re: FreeBSD 10.1 Memory Exhaustion

I think the reason this is not seen more often is because people frequently throw limits on the arc in /boot/loader.conf:

vfs.zfs.arc_min="18G"
vfs.zfs.arc_max="149G"

ZFS ARC *should* not require those settings, but does currently for mixed workloads (i.e. databases) in order to be "stable".  By setting fixed sizes on the ARC, UMA and ARC are much more cooperative in that they have their own memory regions to manage so this behavior is not seen as often.

To be clear, however, it should not be necessary to set parameters like these in /boot/loader.conf in order to obtain consistent operational behavior.  I'd be curious to know if someone running 10.2 BETA without patches is able to trigger this behavior or not.  There was work done that reported helped with this between 10.1 and now.  To what extent it helped, however, I don't have any advice yet.

-sc



On Tue, Jul 14, 2015 at 3:34 AM, Adrian Gschwend <ml-ktk at netlabs.org> wrote:

> On 14.07.15 11:26, Matthew Seaman wrote:
>
>
> > On 07/13/15 12:58, Karl Denninger wrote:
> >> Put this on your box and see if the problem goes away.... :-)
>
> [...]
>
> > I know that you, Karl, and a number of others have been advocating 
> > to get this patch set committed.  Having now personally run into the 
> > sort of problems that this addresses I can say that I would very 
> > much like to see this go in.  Conditional of course on this actually 
> > solving the problems I and others have been experiencing without 
> > introducing significant regressions elsewhere. It's only had a day's 
> > testing from me so far, but it's looking good.  If it survives a 
> > week without the system locking up, I'll be convinced.
>
> I was the one which posted the message last year which triggered Karl 
> to analyze it as he saw similar issues:
>
> https://lists.freebsd.org/pipermail/freebsd-fs/2014-March/019043.html
>
> https://lists.freebsd.org/pipermail/freebsd-fs/2014-March/019057.html
>
> Since then I run on Karls patch and never had any issue anymore. Not 
> that my boxes were basically unusable without the patch.
>
> So I'm basically hoping since then that the patch will be committed soon.
>
> >    * The memory exhaustion effect or equivalent memory pressures can be
> >      triggered at will
> >    * The test doesn't require unfeasibly large resources to run
> >    * The behaviour provides a good model for real-world deployments
> >
> > Maybe these tests would be too large-scale to run every day in 
> > Jenkins, but having them available as part of, say, the release 
> > process, seems like a no-brainer to me.
>
> I wouldn't consider my setup as "unfeasibly large resources", in fact 
> I triggered it with a bunch of jails running on a machine and 
> providing various Internet-services for a small Open Source community. 
> I was always surprised that not more people ran into this issue as I 
> had it since 8.x.
>
> regards
>
> Adrian
>
>
>
> _______________________________________________
> freebsd-fs at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-fs
> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"
>



--
Sean Chittenden
_______________________________________________
freebsd-fs at freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"