core dump in cvsup caused by _once()?

Tue Dec 1 20:42:10 UTC 2009

On Tue, Dec 01, 2009 at 12:59:25PM -0600, Sean C. Farley wrote:
> On Tue, 1 Dec 2009, John Baldwin wrote:
> 
> >On Saturday 28 November 2009 5:15:01 am Gary Jennejohn wrote:
> >>Since I installed a new world and kernel on November 26 I'm seeing
> >>core dumps with cvsup, even though I reinstalled cvsup yesterday.
> >>
> >>Here the output from a gdb session without any debugging symbols:
> >>
> >>Core was generated by `cvsup'.
> >>Program terminated with signal 4, Illegal instruction.
> >>Reading symbols from /lib/libz.so.5...(no debugging symbols found)...done.
> >>Loaded symbols for /lib/libz.so.5
> >>Reading symbols from /lib/libm.so.5...(no debugging symbols found)...done.
> >>Loaded symbols for /lib/libm.so.5
> >>Reading symbols from /lib/libc.so.7...(no debugging symbols found)...done.
> >>Loaded symbols for /lib/libc.so.7
> >>Reading symbols from /libexec/ld-elf.so.1...(no debugging symbols
> >found)...done.
> >>Loaded symbols for /libexec/ld-elf.so.1
> >>#0  0x00000008009edcf7 in gmtime_r () from /lib/libc.so.7
> >>(gdb) bt
> >>#0  0x00000008009edcf7 in gmtime_r () from /lib/libc.so.7
> >>#1  0x00000008009ed79e in gmtime_r () from /lib/libc.so.7
> >>#2  0x00000008009ee420 in gmtime_r () from /lib/libc.so.7
> >>#3  0x00000008009ee638 in gmtime_r () from /lib/libc.so.7
> >>#4  0x00000008009f1988 in _once () from /lib/libc.so.7
> >>#5  0x00000008009ed41f in timeoff () from /lib/libc.so.7
> >>#6  0x00000008009eeca7 in gmtime () from /lib/libc.so.7
> >>#7  0x00000000004a643a in calloc ()
> >>#8  0x000000000043aec7 in ?? ()
> >>#9  0x0000000000448eaa in ?? ()
> >>#10 0x0000000000409ece in ?? ()
> >>#11 0x00000000004191a4 in ?? ()
> >>#12 0x0000000000417cbe in ?? ()
> >>#13 0x000000000041529f in ?? ()
> >>#14 0x0000000000414d7a in ?? ()
> >>#15 0x000000000049f980 in calloc ()
> >>#16 0x000000000048fa3d in fnmatch ()
> >>#17 0x00007fffffffd3e8 in ?? ()
> >>#18 0x00007fffffffe950 in ?? ()
> >>#19 0x00007fffffffea40 in ?? ()
> >>#20 0x00007fffffffea28 in ?? ()
> >>#21 0x0000000000000000 in ?? ()
> >>#22 0x0000000000000000 in ?? ()
> >>#23 0x00001fa00000037f in ?? ()
> >>#24 0x0000000000000000 in ?? ()
> >>#25 0x00000000006476c0 in ?? ()
> >>#26 0x00000000006476c0 in ?? ()
> >>#27 0x0000000000494d89 in fnmatch ()
> >>Previous frame inner to this frame (corrupt stack?)
> >>
> >>Seems to me that _once() was a very recent addition.  Can't say for
> >>certain whether this is the culprit, but it looks suspicious to me.
> >
> >Can you do 'x/i $rip'?  Also, if you could rebuild libc with debug symbols
> >that could be helpful (just cd /usr/src/lib/libc; make clean; make
> >DEBUG_FLAGS=-g install).
> 
> Here is what I get from cvsupd:
> GNU gdb 6.1.1 [FreeBSD]
> Copyright 2004 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and you are
> welcome to change it and/or distribute copies of it under certain 
> conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB.  Type "show warranty" for details.
> This GDB was configured as "amd64-marcel-freebsd"...
> Core was generated by `cvsupd'.
> Program terminated with signal 4, Illegal instruction.
> Reading symbols from /lib/libz.so.5...done.
> Loaded symbols for /lib/libz.so.5
> Reading symbols from /lib/libm.so.5...done.
> Loaded symbols for /lib/libm.so.5
> Reading symbols from 
> /usr/FreeBSD/branches/stable/8/src/lib/libc/libc.so.7...done.
> Loaded symbols for /usr/FreeBSD/branches/stable/8/src/lib/libc/libc.so.7
> Reading symbols from /libexec/ld-elf.so.1...done.
> Loaded symbols for /libexec/ld-elf.so.1
> #0  0x00000008005c0d20 in _rtld_error () from /libexec/ld-elf.so.1
> (gdb) where
> #0  0x00000008005c0d20 in _rtld_error () from /libexec/ld-elf.so.1
> #1  0x00000008005c156b in dladdr () from /libexec/ld-elf.so.1
> #2  0x00000008005c1643 in dladdr () from /libexec/ld-elf.so.1
> #3  0x00000008005be7bd in ?? () from /libexec/ld-elf.so.1
> #4  0x0000000000816ed8 in ?? ()
> #5  0x0000000000000000 in ?? ()
> #6  0x0000000000000006 in ?? ()
> #7  0x0000000000000043 in ?? ()
> #8  0x000000000072aba8 in ?? ()
> #9  0x0000000800a368e1 in _nsyycheck () from 
> /usr/FreeBSD/branches/stable/8/src/lib/libc/libc.so.7
> #10 0x000000000072abbb in ?? ()
> #11 0x0000000000008000 in ?? ()
> #12 0x000000000072abbe in ?? ()
> #13 0x0000000000000216 in ?? ()
> #14 0x0000000000000000 in ?? ()
> #15 0x00000008005ed600 in ?? ()
> #16 0x0000000000000161 in ?? ()
> #17 0x0000000800a09049 in tzload (name=0x800a368e1 "posixrules", 
> sp=0x7353b8, doextend=0) at 
> /usr/FreeBSD/branches/stable/8/src/lib/libc/stdtime/localtime.c:422
> #18 0x0000000800a08a1e in tzparse (name=0x72b1cd "CDT,M3.2.0,M11.1.0", 
> sp=0x7353b8, lastditch=Variable "lastditch" is not available.
> ) at /usr/FreeBSD/branches/stable/8/src/lib/libc/stdtime/localtime.c:1003
> #19 0x0000000800a096f6 in tzload (name=Variable "name" is not available.) 
> at /usr/FreeBSD/branches/stable/8/src/lib/libc/stdtime/localtime.c:580
> #20 0x0000000800a09a86 in tzsetwall_basic (rdlocked=1) at 
> /usr/FreeBSD/branches/stable/8/src/lib/libc/stdtime/localtime.c:1229
> #21 0x0000000800a09deb in mktime (tmp=0x739ff8) at 
> /usr/FreeBSD/branches/stable/8/src/lib/libc/stdtime/localtime.c:2119
> #22 0x00000000004ae085 in Date__ToTime (M3_D5xROs_d=0x5eed80) at 
> DateBsd.m3:77
> #23 0x00000000004709dc in TimeStamp__Init () at TimeStamp.m3:46
> #24 0x0000000000470aa2 in TimeStamp__New (M3_CD9pHn__result=0x73a1c8) at 
> TimeStamp.m3:60
> #25 0x000000000046fc1e in Random__RandomSeed () at Random.m3:67
> #26 0x000000000046fab2 in Random__Init (M3_B04YLH_t=0x825b38, 
> M3_AicXUJ_fixed=0 '\0') at Random.m3:42
> #27 0x000000000044b9d5 in SortedRCSDeltaTbl__Init (M3_EKdMGR_tbl=0x825af8) 
> at SortedTable.mg:106
> #28 0x0000000000450d99 in RCSFile__Init (M3_BcmbT8_rf=0x825990, 
> M3_Bjvku1_desc=0x825a40) at RCSFile.m3:483
> #29 0x00000000004510c2 in RCSFile__OpenReadonly (M3_Bd56fi_p=0x825838) at 
> RCSFile.m3:574
> #30 0x000000000046305f in Attic__RCSFileOpenReadonly 
> (M3_DMtSqf_path=0x73b3f8) at Attic.m3:120
> #31 0x00000000004166bc in RCSComp__CheckoutSend (M3_BQOzaz_self=0x65a610, 
> M3_CzVV2w_sfr=0x65e300, M3_Bd56fi_name=0x825778, M3_Bd56fi_tag=0x651a00, 
> M3_Bd56fi_date=0x651a00,
>     M3_AicXUJ_deleteIfDead=0 '\0', M3_AicXUJ_isFixup=0 '\0') at 
>     RCSComp.m3:1715
> #32 0x000000000040d08a in RCSComp__CompCollection (M3_BQOzaz_self=0x65a610, 
> M3_CzVV2w_sfr=0x65e300) at RCSComp.m3:238
> #33 0x000000000040c4d8 in RCSComp__CompBatch (M3_BQOzaz_self=0x65a610) at 
> RCSComp.m3:155
> #34 0x000000000040bc90 in RCSComp__Apply (M3_BQOzaz_self=0x65a610) at 
> RCSComp.m3:78
> #35 0x00000000004a7240 in ThreadPosix__DetermineContext 
> (M3_AJWxb1_oldSP=0x35) at ThreadPosix.m3:1127
> #36 0x0000000000689058 in ?? ()
> #37 0x00007fffffffe0a0 in ?? ()
> #38 0x000000000049c68c in RTMisc__Align (M3_AJWxb1_a=Cannot access memory 
> at address 0x64c) at RTMisc.m3:31
> Previous frame inner to this frame (corrupt stack?)
> (gdb) x/i $rip
> 0x8005c0d20 <_rtld_error+3296>: mov    %rdi,0xffffffffffffffa0(%rbp)
> (gdb) info threads 
> * 1 process 100176  0x00000008005c0d20 in _rtld_error () from 
> /libexec/ld-elf.so.1
> 
> BTW, I noticed the m3 call ThreadPosix__DetermineContext(), yet cvsupd 
> is not linked against a thread library.  The amd64 binary is linked to 
> libz, libm and libc.  The i386 binary links against those as well as 
> libutil and libmd.

Could you, please, also recompile rtld with debugging symbols ?

SIGILL might be generated by kernel when signal frame cannot be copied
out to usermode stack. Check out the registers content and size of
stack too.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-current/attachments/20091201/16cc0837/attachment.pgp