7.2 dies in zfs

Randy Bush randy at psg.com
Fri Nov 20 02:48:07 UTC 2009


i think the issue is how to tune for zfs

i386 with 4G of RAM

    FreeBSD psg.com 7.2-STABLE FreeBSD 7.2-STABLE #2: Wed Nov 18 03:04:55 GMT 2009     root at psg.com:/usr/obj/usr/src/sys/PSG  i386

RELENG_7 cvsupped Nov 18 02:42 GMT

    panic: kmem_malloc(65536): kmem_map too small: 535019520 total allocated
    cpuid = 0
    Uptime: 13h15m1s
    Physical memory: 3958 MB
    Dumping 637 MB: 622 606 590 574 558 542 526 510 494 478 462 446 430 414 398 382 366 350 334 318 302 286 270 254 238 222 206 190 174 158 142 126 110 94 78 62 46 30 14
    Dump complete
    Automatic reboot in 15 seconds - press a key on the console to abort

and it did not auto reboot

    # cat /boot/loader.conf.local
    ipfw_load=YES
    umass_load=YES
    zfs_load=YES
    vm.kmem_size=536870912
    vm.kmem_size_max=1073741824
    vfs.zfs.prefetch_disable=1

it has zfs 

    # zpool status
      pool: tank
     state: ONLINE
     scrub: none requested
    config:

	    NAME        STATE     READ WRITE CKSUM
	    tank        ONLINE       0     0     0
	      mirror    ONLINE       0     0     0
		twed1   ONLINE       0     0     0
		twed2   ONLINE       0     0     0

    errors: No known data errors

but boots and has root on ufs

    # df -H
    Filesystem        Size    Used   Avail Capacity  Mounted on
    /dev/twed0s1a     260M    199M     40M    83%    /
    devfs             1.0k    1.0k      0B   100%    /dev
    /dev/twed0s1h      65M    2.3M     57M     4%    /root
    procfs            4.1k    4.1k      0B   100%    /proc
    tank              147G     17M    147G     0%    /tank
    tank/usr          167G     20G    147G    12%    /usr
    tank/usr/home     216G     68G    147G    32%    /usr/home
    tank/var          149G    2.3G    147G     2%    /var
    tank/var/spool    148G    531M    147G     0%    /var/spool
    /dev/md0          130M     12k    119M     0%    /tmp

# kgdb kernel.debug /usr/home/crash/vmcore.20
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd"...

Unread portion of the kernel message buffer:
panic: kmem_malloc(65536): kmem_map too small: 535019520 total allocated
cpuid = 0
Uptime: 13h15m1s
Physical memory: 3958 MB
Dumping 637 MB: 622 606 590 574 558 542 526 510 494 478 462 446 430 414 398 382 366 350 334 318 302 286 270 254 238 222 206 190 174 158 142 126 110 94 78 62 46 30 14

Reading symbols from /boot/kernel/zfs.ko...Reading symbols from /boot/kernel/zfs.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/zfs.ko
Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from /boot/kernel/opensolaris.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/opensolaris.ko
Reading symbols from /boot/kernel/ipfw.ko...Reading symbols from /boot/kernel/ipfw.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ipfw.ko
Reading symbols from /boot/kernel/umass.ko...Reading symbols from /boot/kernel/umass.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/umass.ko
Reading symbols from /boot/kernel/cam.ko...Reading symbols from /boot/kernel/cam.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/cam.ko
Reading symbols from /boot/kernel/usb.ko...Reading symbols from /boot/kernel/usb.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/usb.ko
#0  doadump () at pcpu.h:196
196             __asm __volatile("movl %%fs:0,%0" : "=r" (td));
(kgdb) back
#0  doadump () at pcpu.h:196
#1  0xc052b0b6 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418
#2  0xc052b39e in panic (fmt=Variable "fmt" is not available.
) at /usr/src/sys/kern/kern_shutdown.c:574
#3  0xc06dfb54 in kmem_malloc (map=0xc107108c, size=65536, flags=2) at /usr/src/sys/vm/vm_kern.c:305
#4  0xc06d6317 in page_alloc (zone=0x0, bytes=65536, pflag=0xf67ee4a7 "\002", wait=2) at /usr/src/sys/vm/uma_core.c:952
#5  0xc06d8e20 in uma_large_malloc (size=65536, wait=2) at /usr/src/sys/vm/uma_core.c:2706
#6  0xc05189e8 in malloc (size=65536, mtp=0xc0989060, flags=2) at /usr/src/sys/kern/kern_malloc.c:393
#7  0xc0897a61 in zfs_kmem_alloc (size=65536, kmflags=2) at /usr/src/sys/modules/zfs/../../cddl/compat/opensolaris/kern/opensolaris_kmem.c:74
#8  0xc090bf4a in zio_buf_alloc (size=65536) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:207
#9  0xc08f3472 in vdev_cache_read (zio=0xd39d0708) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_cache.c:188
#10 0xc090c145 in zio_vdev_io_start (zio=0xd39d0708) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1816
#11 0xc090c7f0 in zio_execute (zio=0xd39d0708) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:998
#12 0xc08f6bda in vdev_mirror_io_start (zio=0xd77e7708) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c:303
#13 0xc090c7f0 in zio_execute (zio=0xd77e7708) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:998
#14 0xc08f6bda in vdev_mirror_io_start (zio=0xdbd22960) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c:303
#15 0xc090c7f0 in zio_execute (zio=0xdbd22960) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:998
#16 0xc08ad39d in arc_read_nolock (pio=0xd1844960, spa=0xc7746000, bp=0xcc548640, done=0xc08b0600 <dbuf_read_done>, private=0xd9c89e38, priority=0, zio_flags=1, arc_flags=0xf67ee854, zb=0xf67ee834) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:2762
#17 0xc08ad878 in arc_read (pio=0xd1844960, spa=0xc7746000, bp=0xcc548640, pbuf=0xc9b29134, done=0xc08b0600 <dbuf_read_done>, private=0xd9c89e38, priority=0, zio_flags=1, arc_flags=0xf67ee854, zb=0xf67ee834) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:2507
#18 0xc08b0ada in dbuf_read (db=0xd9c89e38, zio=0xd1844960, flags=14) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:521
#19 0xc08b1142 in dbuf_findbp (dn=0xcba92000, level=Variable "level" is not available.
) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:1381
#20 0xc08b1269 in dbuf_hold_impl (dn=0xcba92000, level=0 '\0', blkid=0, fail_sparse=0, tag=0x0, dbp=0xf67ee8f0) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:1617
#21 0xc08b2529 in dbuf_hold (dn=0xcba92000, blkid=0, tag=0x0) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:1689
#22 0xc08b48cc in dmu_buf_hold (os=0xc774b3d0, object=167123, offset=0, tag=0x0, dbp=0xf67ee95c) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c:101
#23 0xc0900044 in zap_lockdir (os=0xc774b3d0, obj=167123, tx=0x0, lti=RW_READER, fatreader=1, adding=0, zapp=0xf67eeba0) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zap_micro.c:388
#24 0xc09009fd in zap_cursor_retrieve (zc=0xf67eeb9c, za=0xf67eea84) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zap_micro.c:1004
#25 0xc0925a2b in zfs_freebsd_readdir (ap=0xf67eec00) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:2156
#26 0xc07382f2 in VOP_READDIR_APV (vop=0xc098b560, a=0xf67eec00) at vnode_if.c:1407
#27 0xc05becae in kern_getdirentries (td=0xd5a01000, fd=19, buf=0x29061000 <Address 0x29061000 out of bounds>, count=4096, basep=0xf67eec74) at vnode_if.h:747
#28 0xc05beec1 in getdirentries (td=0xd5a01000, uap=0xf67eecfc) at /usr/src/sys/kern/vfs_syscalls.c:3776
#29 0xc072cfc5 in syscall (frame=0xf67eed38) at /usr/src/sys/i386/i386/trap.c:1101
#30 0xc0711380 in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:262
#31 0x00000033 in ?? ()
Previous frame inner to this frame (corrupt stack?)


randy


More information about the freebsd-fs mailing list