unionfs panic in 7.1

Daichi GOTO daichi at freebsd.org
Sat Feb 21 01:52:32 PST 2009


Steve Wills wrote:
> Hi,
> 
> I've found an reproducable panic in unionfs on 7.1-R.
> 
> /usr/src/sys/fs/unionfs/union_subr.c is:
>      $FreeBSD: src/sys/fs/unionfs/union_subr.c,v 1.92.2.7.2.2 2008/12/15 
> 03:58:55 daichi Exp $
> 
> kgdb output is below.
> 
> kgdb /boot/kernel/kernel.symbols /var/crash/vmcore.0
> GNU gdb 6.1.1 [FreeBSD]
> Copyright 2004 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and you 
> are
> welcome to change it and/or distribute copies of it under certain 
> conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB.  Type "show warranty" for details.
> This GDB was configured as "amd64-marcel-freebsd"...
> 
> Unread portion of the kernel message buffer:
> 
> stack pointer            = 0x10:0xffffffffb0e1a8b0
> frame pointer            = 0x10:0xffffffffb0e1a8d0
> code segment        = base 0x0, limit 0xfffff, type 0x1b
>             = DPL 0, pres 1, long 1, def32 0, gran 1
> processor eflags    = interrupt enabled, resume, IOPL = 0
> current process        = 72036 (make)
> panic: from debugger
> cpuid = 0
> Uptime: 49m32s
> Physical memory: 2034 MB
> Dumping 441 MB: 426 410 394 378 362 346 330 314 298 282 266 250 234 218 
> 202 186 170 154 138 122 106 90 74 58 42 26 10
> 
> Reading symbols from /boot/kernel/if_tap.ko...Reading symbols from 
> /boot/kernel/if_tap.ko.symbols...done.
> done.
> Loaded symbols for /boot/kernel/if_tap.ko
> Reading symbols from /boot/kernel/snd_hda.ko...Reading symbols from 
> /boot/kernel/snd_hda.ko.symbols...done.
> done.
> Loaded symbols for /boot/kernel/snd_hda.ko
> Reading symbols from /boot/kernel/sound.ko...Reading symbols from 
> /boot/kernel/sound.ko.symbols...done.
> done.
> Loaded symbols for /boot/kernel/sound.ko
> Reading symbols from /boot/kernel/atapicam.ko...Reading symbols from 
> /boot/kernel/atapicam.ko.symbols...done.
> done.
> Loaded symbols for /boot/kernel/atapicam.ko
> Reading symbols from /boot/kernel/smb.ko...Reading symbols from 
> /boot/kernel/smb.ko.symbols...done.
> done.
> Loaded symbols for /boot/kernel/smb.ko
> Reading symbols from /boot/kernel/smbus.ko...Reading symbols from 
> /boot/kernel/smbus.ko.symbols...done.
> done.
> Loaded symbols for /boot/kernel/smbus.ko
> Reading symbols from /boot/kernel/linprocfs.ko...Reading symbols from 
> /boot/kernel/linprocfs.ko.symbols...done.
> done.
> Loaded symbols for /boot/kernel/linprocfs.ko
> Reading symbols from /boot/kernel/linux.ko...Reading symbols from 
> /boot/kernel/linux.ko.symbols...done.
> done.
> Loaded symbols for /boot/kernel/linux.ko
> Reading symbols from /boot/kernel/zfs.ko...Reading symbols from 
> /boot/kernel/zfs.ko.symbols...done.
> done.
> Loaded symbols for /boot/kernel/zfs.ko
> Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from 
> /boot/kernel/opensolaris.ko.symbols...done.
> done.
> Loaded symbols for /boot/kernel/opensolaris.ko
> Reading symbols from /boot/kernel/if_bridge.ko...Reading symbols from 
> /boot/kernel/if_bridge.ko.symbols...done.
> done.
> Loaded symbols for /boot/kernel/if_bridge.ko
> Reading symbols from /boot/kernel/bridgestp.ko...Reading symbols from 
> /boot/kernel/bridgestp.ko.symbols...done.
> done.
> Loaded symbols for /boot/kernel/bridgestp.ko
> Reading symbols from /usr/local/modules/fuse.ko...done.
> Loaded symbols for /usr/local/modules/fuse.ko
> Reading symbols from /boot/kernel/radeon.ko...Reading symbols from 
> /boot/kernel/radeon.ko.symbols...done.
> done.
> Loaded symbols for /boot/kernel/radeon.ko
> Reading symbols from /boot/kernel/drm.ko...Reading symbols from 
> /boot/kernel/drm.ko.symbols...done.
> done.
> Loaded symbols for /boot/kernel/drm.ko
> Reading symbols from /boot/kernel/unionfs.ko...Reading symbols from 
> /boot/kernel/unionfs.ko.symbols...done.
> done.
> Loaded symbols for /boot/kernel/unionfs.ko
> Reading symbols from /boot/kernel/nullfs.ko...Reading symbols from 
> /boot/kernel/nullfs.ko.symbols...done.
> done.
> Loaded symbols for /boot/kernel/nullfs.ko
> #0  doadump () at pcpu.h:195
> 195    pcpu.h: No such file or directory.
>     in pcpu.h
> (kgdb) bt
> #0  doadump () at pcpu.h:195
> #1  0xffffffff804c70a8 in boot (howto=260) at 
> /usr/src/sys/kern/kern_shutdown.c:418
> #2  0xffffffff804c750c in panic (fmt=Variable "fmt" is not available.
> ) at /usr/src/sys/kern/kern_shutdown.c:574
> #3  0xffffffff801c1707 in db_panic (addr=Variable "addr" is not available.
> ) at /usr/src/sys/ddb/db_command.c:446
> #4  0xffffffff801c1d6f in db_command (last_cmdp=0xffffffff80aa6448, 
> cmd_table=0x0, dopager=1) at /usr/src/sys/ddb/db_command.c:413
> #5  0xffffffff801c1f80 in db_command_loop () at 
> /usr/src/sys/ddb/db_command.c:466
> #6  0xffffffff801c3b69 in db_trap (type=Variable "type" is not available.
> ) at /usr/src/sys/ddb/db_main.c:228
> #7  0xffffffff804f3955 in kdb_trap (type=12, code=0, 
> tf=0xffffffffb0e1a800) at /usr/src/sys/kern/subr_kdb.c:524
> #8  0xffffffff807a3180 in trap_fatal (frame=0xffffffffb0e1a800, 
> eva=Variable "eva" is not available.
> ) at /usr/src/sys/amd64/amd64/trap.c:759
> #9  0xffffffff807a3554 in trap_pfault (frame=0xffffffffb0e1a800, 
> usermode=0) at /usr/src/sys/amd64/amd64/trap.c:680
> #10 0xffffffff807a3eda in trap (frame=0xffffffffb0e1a800) at 
> /usr/src/sys/amd64/amd64/trap.c:449
> #11 0xffffffff80788dde in calltrap () at 
> /usr/src/sys/amd64/amd64/exception.S:209
> #12 0xffffffff804b9f0e in _mtx_lock_sleep (m=0xffffff006c384638, 
> tid=18446742974280188784, opts=Variable "opts" is not available.
> ) at /usr/src/sys/kern/kern_mutex.c:339
> #13 0xffffffff80555767 in vn_start_write (vp=0xffffff006c4983f0, 
> mpp=Variable "mpp" is not available.
> ) at /usr/src/sys/kern/vfs_vnops.c:902
> #14 0xffffffff805574a1 in vn_close (vp=0xffffff006c4983f0, flags=1, 
> file_cred=0xffffff0015947500, td=0xffffff0004e74370) at 
> /usr/src/sys/kern/vfs_vnops.c:287
> #15 0xffffffff80557589 in vn_closefile (fp=0xffffff006c020e00, 
> td=0xffffff0004e74370) at /usr/src/sys/kern/vfs_vnops.c:867
> #16 0xffffffff8049331f in fdrop (fp=0xffffff006c020e00, 
> td=0xffffff0004e74370) at file.h:299
> #17 0xffffffff80494579 in closef (fp=0xffffff006c020e00, 
> td=0xffffff0004e74370) at /usr/src/sys/kern/kern_descrip.c:2033
> #18 0xffffffff80494d5d in kern_close (td=0xffffff0004e74370, fd=Variable 
> "fd" is not available.
> ) at /usr/src/sys/kern/kern_descrip.c:1125
> #19 0xffffffff807a37d6 in syscall (frame=0xffffffffb0e1ac80) at 
> /usr/src/sys/amd64/amd64/trap.c:907
> #20 0xffffffff80788feb in Xfast_syscall () at 
> /usr/src/sys/amd64/amd64/exception.S:330
> #21 0x000000000044030c in ?? ()
> Previous frame inner to this frame (corrupt stack?)
> (kgdb) frame 12
> #12 0xffffffff804b9f0e in _mtx_lock_sleep (m=0xffffff006c384638, 
> tid=18446742974280188784, opts=Variable "opts" is not available.
> ) at /usr/src/sys/kern/kern_mutex.c:339
> 339                owner = (struct thread *)(v & ~MTX_FLAGMASK);
> (kgdb) list
> 334             * If the owner is running on another CPU, spin until the
> 335             * owner stops running or the state of the lock changes.
> 336             */
> 337            v = m->mtx_lock;
> 338            if (v != MTX_UNOWNED) {
> 339                owner = (struct thread *)(v & ~MTX_FLAGMASK);
> 340    #ifdef ADAPTIVE_GIANT
> 341                if (TD_IS_RUNNING(owner)) {
> 342    #else
> 343                if (m != &Giant && TD_IS_RUNNING(owner)) {
> (kgdb) p v
> $1 = 0
> (kgdb) p m
> $2 = (struct mtx *) 0xffffff006c384638
> (kgdb) p *m
> $3 = {lock_object = {lo_name = 0x0, lo_type = 0x0, lo_flags = 
> 1281410792, lo_witness_data = {lod_list = {stqe_next = 0xffffff004c60c708},
>       lod_witness = 0xffffff004c60c708}}, mtx_lock = 0, mtx_recurse = 0}
> (kgdb) frame 13
> #13 0xffffffff80555767 in vn_start_write (vp=0xffffff006c4983f0, 
> mpp=Variable "mpp" is not available.
> ) at /usr/src/sys/kern/vfs_vnops.c:902
> 902        MNT_ILOCK(mp);
> (kgdb) list
> 897                return (0);
> 898            }
> 899        }
> 900        if ((mp = *mpp) == NULL)
> 901            return (0);
> 902        MNT_ILOCK(mp);
> 903        if (vp == NULL)
> 904            MNT_REF(mp);
> 905        /*
> 906         * Check on status of suspension.
> (kgdb) p mp
> $4 = (struct mount *) 0xffffff006c3845e8
> (kgdb) p *mp
> $5 = {mnt_lock = {lk_object = {lo_name = 0x1 <Address 0x1 out of 
> bounds>, lo_type = 0xffffffffb1029552 "unionfs", lo_flags = 2969738112, 
> lo_witness_data = {
>         lod_list = {stqe_next = 0xffffff003c980000}, lod_witness = 
> 0xffffff003c980000}}, lk_interlock = 0xffffff0015d46378, lk_flags = 0, 
> lk_sharecount = 0,
>     lk_waitcount = 775638512, lk_exclusivecount = -256, lk_prio = -1, 
> lk_timo = 0, lk_lockholder = 0x0, lk_newlock = 0x0}, mnt_mtx = 
> {lock_object = {
>       lo_name = 0x0, lo_type = 0x0, lo_flags = 1281410792, 
> lo_witness_data = {lod_list = {stqe_next = 0xffffff004c60c708}, 
> lod_witness = 0xffffff004c60c708}},
>     mtx_lock = 0, mtx_recurse = 0}, mnt_gen = 0, mnt_list = {tqe_next = 
> 0x0, tqe_prev = 0x0}, mnt_op = 0xffffffffb1029552, mnt_vfc = 
> 0xffffffffb1029552,
>   mnt_vnodecovered = 0x4390000, mnt_syncer = 0x0, mnt_ref = -2136101936, 
> mnt_nvnodelist = {tqh_first = 0x80, tqh_last = 0x50000000000000},
>   mnt_nvnodelistsize = 51, mnt_writeopcount = 0, mnt_kern_flag = -1, 
> mnt_flag = 4294967295, mnt_noasync = 0, mnt_opt = 0xffffffff8086e910,
>   mnt_optnew = 0xffffffff8086e910, mnt_maxsymlinklen = 16973824, 
> mnt_stat = {f_version = 0, f_type = 0, f_flags = 4, f_bsize = 0,
>     f_iosize = 18446742976017063016, f_blocks = 4294967297, f_bfree = 0, 
> f_bavail = 0, f_files = 0, f_ffree = 0, f_syncwrites = 0,
>     f_asyncwrites = 18446742976013551312, f_syncreads = 0, f_asyncreads 
> = 18446742976013551424, f_spare = {0, 0, 0, 18446742976013551456, 0, 0, 
> 0, 0,
>       18446744071572912256, 16384}, f_namemax = 784962992, f_owner = 
> 4294967040, f_fsid = {val = {0, 0}},
>     f_charspare = 
> "\000\000\000\000\000\000\000\000ËE8l\000ˇˇˇËE8l\000ˇˇˇ", '\0' <repeats 
> 24 times>, 
> "\001\000\000\000\000\000\000\000\023\016\206\200ˇˇˇˇ\2001ß\200ˇˇˇˇXy⁄l\000ˇˇˇ", 
> f_fstypename = "xsv\004\000ˇˇˇ–K8l\000ˇˇˇ",
>     f_mntfromname = "\030D8l\000ˇˇˇ", '\0' <repeats 16 times>, 
> "\200o\034\201ˇˇˇˇ∂L\003\003", '\0' <repeats 12 times>, 
> "∏\227\tJ\000ˇˇˇ∏¨`L\000ˇˇˇ\000\000\000\000\000\000\000\000\b\000\000\000\000\000\000\000 
> \031L0\000\000\000",
>     f_mntonname = "\a", '\0' <repeats 15 times>, 
> "\023\016\206\200ˇˇˇˇ\023\016\206\200ˇˇˇˇ\000\0009\004", '\0' <repeats 
> 12 times>, "‡\230≠\200ˇˇˇˇ@", '\0' <repeats 13 times>, 
> "P\0003\000\000\000\000\000\000\000ˇˇˇˇˇˇˇˇ"}, mnt_cred = 0x0, mnt_data 
> = 0xffffffff8086e910, mnt_time = -2138642160, mnt_iosize_max = 16973824,
>   mnt_export = 0x0, mnt_label = 0x4, mnt_hashseed = 0, mnt_markercnt = 
> 0, mnt_holdcnt = 1815627896, mnt_holdcntwaiters = -256, 
> mnt_secondary_writes = 10,
>   mnt_secondary_accwrites = 0, mnt_gjprovider = 0x0, mnt_explock = 
> {lk_object = {lo_name = 0x0, lo_type = 0x0, lo_flags = 0, 
> lo_witness_data = {lod_list = {
>           stqe_next = 0x0}, lod_witness = 0x0}}, lk_interlock = 
> 0xffffff006c3848c8, lk_flags = 2589843616, lk_sharecount = -1, 
> lk_waitcount = -1707370640,
>     lk_exclusivecount = -1, lk_prio = -1, lk_timo = -1707370720, 
> lk_lockholder = 0x9, lk_newlock = 0x0}}
> (kgdb)
> 
> I reproduce this by unionfs mounting a ports dir used by the ports 
> tinderbox. If anyone would like more info, please let me know, I have 
> the core around, and can reproduce, help debug, resend in case my mailer 
> has mangled this, etc.

I have some research around that issue.

First I should say, I cannot judge that unionfs leads
that problem or not. JIMO, I guess that is not depends
on unionfs itself. Very similar panis is reported without
unionfs. At least, I cannot figure out what is the main
cause of this problem. sorry.

> Thanks,
> Steve
> 
> _______________________________________________
> freebsd-stable at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscribe at freebsd.org"

-- 
   Daichi GOTO, http://people.freebsd.org/~daichi


More information about the freebsd-stable mailing list