kernel trap with zfs root after shutdown

[Henri Hennebert]

Hello,

I enjoy reading "A conversation with Jeff Bonwick and Bill Moore" in ACM 
Queue. PJD paper "Porting the ZFS file system to the FreeBSD operation 
system" was really interesting... As a result, I switch all my 
filesystems to zfs -- Yes I know: WARNING: ZFS is considered to be an 
experimental feature in FreeBSD -- and it's quite stable for me.

Nevertheless I encouter a glitch *after* shutdown:

Kernel: 7.0-BETA1 - cvsup Oct 23 with patch
http://people.freebsd.org/~pjd/patches/vm_kern.c.2.patch.
Note: without this patch I get a kmem_map too small after some io load.


I setup my root fs under zfs. the boot fs is under gmirror+ufs.

loader.conf:

#--- Tuning for ZFS - http://wiki.freebsd.org/ZFSTuningGuide
vm.kmem_size="600M"                     # Size of kernel memory
vm.kmem_size_max="600M"                 # Max kmem size
vfs.zfs.debug="0"                       # ZFS - debug messages
vfs.zfs.prefetch_disable="1"            # ZFS - disable prefetch
vfs.zfs.arc_max="128M"                  # ZFS - maximum ARC size : 128MB
#--- Mount root filesystem from ZFS
zfs_load="YES"                          # Load ZFS module
vfs.root.mountfrom="zfs:pool0"          # Root filesystem is pool0

zpool status:

   pool: pool0
  state: ONLINE
  scrub: none requested
config:

         NAME        STATE     READ WRITE CKSUM
         pool0       ONLINE       0     0     0
           mirror    ONLINE       0     0     0
             da0s2   ONLINE       0     0     0
             da1s2   ONLINE       0     0     0

errors: No known data errors

   pool: pool1
  state: ONLINE
  scrub: none requested
config:

         NAME        STATE     READ WRITE CKSUM
         pool1       ONLINE       0     0     0
           da0s3     ONLINE       0     0     0
           da1s3     ONLINE       0     0     0

errors: No known data errors

   pool: pool2
  state: ONLINE
  scrub: none requested
config:

         NAME        STATE     READ WRITE CKSUM
         pool2       ONLINE       0     0     0
           raidz1    ONLINE       0     0     0
             ad4s3   ONLINE       0     0     0
             ad6s3   ONLINE       0     0     0

errors: No known data errors

After shutdown I get a kernel trap:

[root at morzine MORZINE]# kgdb kernel.debug /var/crash/vmcore.20
[GDB will not be able to debug user-mode threads: 
/usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"]
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain 
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd".

Unread portion of the kernel message buffer:
<118>Oct 28 09:58:48 morzine syslogd: exiting on signal 15
Waiting (max 60 seconds) for system process `vnlru' to stop...done
Waiting (max 60 seconds) for system process `bufdaemon' to stop...done
Waiting (max 60 seconds) for system process `syncer' to stop...
Syncing disks, vnodes remaining...0 0 0 0 0 done
All buffers synced.
kernel trap 12 with interrupts disabled


Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address   = 0x188
fault code              = supervisor read, page not present
instruction pointer     = 0x20:0xc0593425
stack pointer           = 0x28:0xf668fa68
frame pointer           = 0x28:0xf668fa80
code segment            = base 0x0, limit 0xfffff, type 0x1b
                         = DPL 0, pres 1, def32 1, gran 1
processor eflags        = resume, IOPL = 0
current process         = 1 (init)
trap number             = 12
panic: page fault
cpuid = 0
KDB: stack backtrace:
db_trace_self_wrapper(c080a4a9,f668f944,c059f96f,c0825e2c,0,...) at 
db_trace_self_wrapper+0x26
kdb_backtrace(c0825e2c,0,c07eebd8,f668f950,0,...) at kdb_backtrace+0x29
panic(c07eebd8,c08270e8,c5527f68,1,1,...) at panic+0x10f
trap_fatal(c0826fea,c,c588f000,f668f9c0,c5527d48,...) at trap_fatal+0x333
trap(f668fa28) at trap+0x11f
calltrap() at calltrap+0x6
--- trap 0xc, eip = 0xc0593425, esp = 0xf668fa68, ebp = 0xf668fa80 ---
_mtx_lock_sleep(c69d1a18,c5528440,0,0,0,...) at _mtx_lock_sleep+0x85
vrele(c69d1990,c5b80d0c,c5528440,c080eb5b,91f,...) at vrele+0x5b
vflush(c5b80d0c,1,2,c5528440,c5b5a844,...) at vflush+0x69b
zfs_umount(c5b80d0c,80000,c5528440,c5528440,c080e762,...) at zfs_umount+0xd5
dounmount(c5b80d0c,80000,c5528440,ead65aec,0,...) at dounmount+0x4ef
vfs_unmountall(c08083ac,0,c0808410,10e,f668fc50,...) at vfs_unmountall+0x33
boot(c5528440,8,0,fffffffe,c5528440,...) at boot+0x444
reboot(c5528440,f668fcfc,4,8048260,9e7c9054,...) at reboot+0x67
syscall(f668fd38) at syscall+0x335
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (55, FreeBSD ELF32, reboot), eip = 0x8050be3, esp = 
0xbfbfe90c, ebp = 0xbfbfe9d8 ---
Uptime: 3d16h44m52s
Physical memory: 2030 MB
Dumping 468 MB: 453 437 421 405 389 373 357 341 325 309 293 277 261 245 
229 213 197 181 165 149 133 117 101 85 69 53 37 21 5

#0  doadump () at pcpu.h:195
195             __asm __volatile("movl %%fs:0,%0" : "=r" (td));
(kgdb)

Anyway, thanks for this great port

Henri