panic: softdep_setup_inomapdep: found inode already exists in 6.2

Fri May 11 11:09:54 UTC 2007

Kostik Belousov wrote:
> On Thu, May 03, 2007 at 01:23:52AM +0300, Cristian KLEIN wrote:
>> On Mar, Mai 1, 2007 11:42 pm, Eric Anderson wrote:
>>> On 04/28/07 18:19, Cristian KLEIN wrote:
>>>
>>>> Hi everybody,
>>>>
>>>>
>>>> I am running a FreeBSD 6.2-p3, on which I am experiencing exactly the
>>>> same simtoms as one item of the TODO list of 6.0:
>>>> http://www.freebsd.org/releases/6.0R/todo.html
>>>>
>>>>
>>>> panic: softdep_setup_inomapdep: found inode 	Needs testing 	Tor Egge
>>>> Found by stress tests at
>>>> http://www.holm.cc/stress/log/cons138.html
>>>>
>>>>
>>>> Does anybody know whether this bug should have been solved in 6.2?
>>>> Should
>>>> I file a PR?
>>>>
>>>
>>> Sorry if I missed it, but were you able to provide a backtrace?  If you
>>> can, you should compile your kernel with debugging, so at least you could
>>> make a little more out of the crash.  See the handbook if you need help on
>>> that.
>> Hi,
>>
>> I haven't mentioned any technical details yet, as I wasn't sure whether
>> this issue is known or not.
>>
>> First, let me tell you what I did. I wanted to better protect data on
>> /jail/mail/home by doing daily snapshots, saved like
>> /jail/mail/home/.snap/2007-04-03-03-22-02. I mounted one of these
>> snapshots (not the latest) in /mnt/home and rsync'd it to a server. I
>> might have rsync'd while taking a new snapshot.
>>
>> Server started crashing randomly, either while rsync-ing, or in the
>> morning, during heavy load. Now I removed the snapshots and the system is
>> stable.
>>
>> Other random information that might be useful: I am using gmirror and
>> jail. Disabling SMP has not effect, WITNESS didn't say anything. The
>> filesystem has userquotas. The filesystem stores maildir and has about
>> 1.6Minodes.
>>
>> Config is GENERIC + SMP + QUOTA - unused hardware devices.
>>
>> root# uname -a
>> FreeBSD bavaria.xxxxxx 6.2-RELEASE-p3 FreeBSD 6.2-RELEASE-p3 #5: Fri Apr
>> 27 20:01:20 EEST 2007
>> cristi at bavaria.xxxxxx:/usr/obj/usr/src/sys/BAVARIA-SMP  i386
>>
>> root# kgdb kernel.debug /var/crash/vmcore.3
>> [GDB will not be able to debug user-mode threads:
>> /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"]
>> GNU gdb 6.1.1 [FreeBSD]
>> Copyright 2004 Free Software Foundation, Inc.
>> GDB is free software, covered by the GNU General Public License, and you are
>> welcome to change it and/or distribute copies of it under certain
>> conditions.
>> Type "show copying" to see the conditions.
>> There is absolutely no warranty for GDB.  Type "show warranty" for details.
>> This GDB was configured as "i386-marcel-freebsd".
>>
>> Unread portion of the kernel message buffer:
>> panic: softdep_setup_inomapdep: found inode already exists
>> cpuid = 0
>> KDB: enter: panic
>> Uptime: 5h42m12s
>> Dumping 1023 MB (2 chunks)
>>   chunk 0: 1MB (159 pages) ... ok
>>   chunk 1: 1023MB (261837 pages) 1007 991 975 959 943 927 911 895 879 863
>> 847 831 815 799 783 767 751 735 719 703 687 671 655 639 623 607 591 575
>> 559 543 527 511 495 479 463 447 431 415 399 383 367 351 335 319 303 287
>> 271 255 239 223 207 191 175 159 143 127 111 95 79 63 47 31 15
>>
>> #0  doadump () at pcpu.h:165
>> 165             __asm __volatile("movl %%fs:0,%0" : "=r" (td));
>> (kgdb) bt
>> #0  doadump () at pcpu.h:165
>> #1  0xc0545e18 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:409
>> #2  0xc0546149 in panic (fmt=0xc075660c "softdep_setup_inomapdep: found
>> inode already exists")
>>     at /usr/src/sys/kern/kern_shutdown.c:565
>> #3  0xc068c64a in softdep_setup_inomapdep (bp=0xd8c2ba68, ip=0x0, newinum=0)
>>     at /usr/src/sys/ufs/ffs/ffs_softdep.c:1527
>> #4  0xc067d4dd in ffs_nodealloccg (ip=0xc68e818c, cg=0, ipref=Unhandled
>> dwarf expression opcode 0x93
>> ) at /usr/src/sys/ufs/ffs/ffs_alloc.c:1762
>> #5  0xc067bb83 in ffs_hashalloc (ip=0xc68e818c, cg=0, pref=Unhandled dwarf
>> expression opcode 0x93
>> ) at /usr/src/sys/ufs/ffs/ffs_alloc.c:1248
>> #6  0xc067b232 in ffs_valloc (pvp=0xc6883440, mode=33024, cred=0xc50d3a80,
>> vpp=0xe733d4e8)
>>     at /usr/src/sys/ufs/ffs/ffs_alloc.c:932
>> #7  0xc06a6bd7 in ufs_makeinode (mode=33024, dvp=0xc6883440,
>> vpp=0xe733d8f0, cnp=0xe733d904)
>>     at /usr/src/sys/ufs/ufs/ufs_vnops.c:2220
>> #8  0xc06a348f in ufs_create (ap=0x0) at
>> /usr/src/sys/ufs/ufs/ufs_vnops.c:189
>> #9  0xc071e879 in VOP_CREATE_APV (vop=0x0, a=0xe733d7fc) at vnode_if.c:204
>> #10 0xc0684018 in ffs_snapshot (mp=0xc4e28000,
>>     snapfile=0xc699c200 "/jail/mail/home/.snap/2007-04-28-03-34-31") at
>> vnode_if.h:111
>> #11 0xc06964d7 in ffs_mount (mp=0xc4e28000, td=0xc50e1300) at
>> /usr/src/sys/ufs/ffs/ffs_vfsops.c:312
>> #12 0xc059eb6e in vfs_domount (td=0xc50e1300, fstype=0xc4efac90 "ufs",
>> fspath=0xc4efa670 "/jail/mail/home",
>>     fsflags=16842752, fsdata=0xc4efaca0) at
>> /usr/src/sys/kern/vfs_mount.c:928
>> #13 0xc059e1da in vfs_donmount (td=0x0, fsflags=16842752, fsoptions=0x0)
>> at /usr/src/sys/kern/vfs_mount.c:676
>> #14 0xc05a0f84 in kernel_mount (ma=0xc4efac50, flags=0) at pcpu.h:162
>> #15 0xc06966fb in ffs_cmount (ma=0xc4efac50, data=0x0, flags=0,
>> td=0xc50e1300)
>>     at /usr/src/sys/ufs/ffs/ffs_vfsops.c:392
>> #16 0xc059e3f0 in mount (td=0xc50e1300, uap=0xe733dd04) at
>> /usr/src/sys/kern/vfs_mount.c:742
>> #17 0xc070b5bb in syscall (frame=
>>       {tf_fs = 59, tf_es = 59, tf_ds = 59, tf_edi = -1077944004, tf_esi =
>> -1077941276, tf_ebp = -1077943864, tf_isp = -416031388, tf_ebx =
>> -1077943824, tf_edx = -1, tf_ecx = -1077940507, tf_eax = 21,
>> tf_trapno = 12, tf_err = 2, tf_eip = 671885143, tf_cs = 51,
>> tf_eflags = 582, tf_esp = -1077944036, tf_ss = 59})
>>     at /usr/src/sys/i386/i386/trap.c:983
>> #18 0xc06f60df in Xint0x80_syscall () at
>> /usr/src/sys/i386/i386/exception.s:200
>> #19 0x00000033 in ?? ()
>> Previous frame inner to this frame (corrupt stack?)
>>
>> Now that I have KDB on serial console I might be able to crash (or even
>> better, test patches) at night on that system.
>>
>> Please don't hesitate to request further information. I will try to
>> elaborate a recipe for crashing the system.
> Do you have ddb backtrace for this instance of panic ? Or, for any other
> such panic, please, show both ddb backtrace and kgdb output of "bt full".

Sorry for my late answer.

No, I don't have the ddb backtrace. I will work on crashing my laptop,
so I'll be able to provide the ddb backtrace too.