ZFS and deadlock with {nullfs,NFS}

[Joao Barros]

On 6/20/07, John Nielsen <lists at jnielsen.net> wrote:
> On Wednesday 20 June 2007 03:56:13 pm Dan Mack wrote:
> > On Wed, 20 Jun 2007, John Nielsen wrote:
> > > On Wednesday 20 June 2007 12:30:23 pm Dan Mack wrote:
> > >> On Wed, 20 Jun 2007, Kris Kennaway wrote:
> > >>
> > >> <snip>
> > >>
> > >>> 404 at the moment, but look for processes involving zil* in the
> > >>> backtrace.  I had to disable zil (vfs.zfs.zil_disable=1 tunable) to
> > >>> prevent low-memory deadlocks on my machines.  Since then it's been
> > >>> fine.
> > >>>
> > >>> You may also wish to use my patches (see the archives) to improve
> > >>> performance and low-memory behaviour.
> > >>>
> > >>> Kris
> > >>
> > >> Does someone have these recommended sysctls embodied in an example
> > >> /boot/loader.conf yet?   Here is mine, does it look reasonable to keep
> > >> ZFS from running into the kmem_ memory panics?  I have no idea if I
> > >> found all of your recommendations so it would be nice if they were
> > >> summarized in one place.
> > >>
> > >>   |  # /boot/loader.conf  i386 / 1GB memory / SMP
> > >>   |  kern.maxvnodes="50000"
> > >
> > > Note that kern.maxvnodes has to be set in /etc/sysctl.conf to have any
> > > effect.
> > >
> > > JN
> >
> > FWIW ... I'm still experiencing panics; this time with when running cvsup
> > to update /usr/ports:
> >
> > borg# grep -i Panic /var/crash/info*
> > /var/crash/info.0:  Panic String: kmem_malloc(16384): kmem_map too small:
> > 268419072 total allocated /var/crash/info.1:  Panic String:
> > kmem_malloc(131072): kmem_map too small: 266354688 total allocated
> > /var/crash/info.2:  Panic String: kmem_malloc(4096): kmem_map too small:
> > 268435456 total allocated /var/crash/info.3:  Panic String:
> > kmem_malloc(28672): kmem_map too small: 268435456 total allocated
> >
> >
> > Anyone ideas on other tunables that might keep the system running?  I'm
> > not too concerned about performance.   Otherwise I'll have to go spend
> > some money on an AMD64 mainboard, memory, and cpu :-(
>
> The only tunable I had that you didn't was this one:
>
> vfs.zfs.arc_max="83886080"
>
> FWIW, my i386 system has 1.5GB RAM. More RAM might help, and/or you probably
> still need to do some balancing (read: fiddling) with the tunables you're
> already adjusting.
>
> JN
> _______________________________________________
> freebsd-current at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscribe at freebsd.org"
>

With a 4 drive raidz, i386 and 1GB ram I only changed one setting:

kern.maxvnodes="50000"
Without it a simple recursive chmod or chown would crash the kernel.
A 'make world' would fail too.

FYI my vfs.zfs look like this:

# sysctl vfs.zfs.
vfs.zfs.arc_min: 16777216
vfs.zfs.arc_max: 167772160
vfs.zfs.mdcomp_disable: 0
vfs.zfs.prefetch_disable: 0
vfs.zfs.zio.taskq_threads: 0
vfs.zfs.recover: 0
vfs.zfs.vdev.cache.size: 10485760
vfs.zfs.vdev.cache.max: 16384
vfs.zfs.cache_flush_disable: 0
vfs.zfs.zil_disable: 0
vfs.zfs.debug: 0

-- 
Joao Barros