svn commit: r310445 - in head/sys/mips: include mips

Adrian Chadd adrian.chadd at gmail.com
Fri Dec 23 20:59:40 UTC 2016 

That's a ... deeep stack. :( How big is each stackframe?


-adrian


On 22 December 2016 at 19:49, John Baldwin <jhb at freebsd.org> wrote:
> On Friday, December 23, 2016 03:27:11 AM John Baldwin wrote:
>> Author: jhb
>> Date: Fri Dec 23 03:27:11 2016
>> New Revision: 310445
>> URL: https://svnweb.freebsd.org/changeset/base/310445
>>
>> Log:
>>   Teach DDB how to unwind across a kernel stack overflow.
>>
>>   Kernel stack overflows in MIPS call panic() directly from an assembly
>>   handler after storing the interrupted context's registers in a
>>   trapframe.  Rather than inferring the location of ra, sp, and pc from
>>   the instruction stream, recognize the pc of a kernel stack overflow
>>   and pull the registers from the trapframe.
>>
>>   Sponsored by:       DARPA / AFRL
>
> We should possibly use this same logic for other subroutines that use
> trapframes.  Trying to run a program that core dumps over NFS in my
> qemu mips64 machine triggers the stack overflow on a kernel built with
> gcc5.3 from ports.  The DDB stack trace doesn't get very far:
>
> panic: kernel stack overflow - trapframe at 0xffffffff80699eb0
> KDB: stack backtrace:
> db_trace_self+1c (?,?,?,?) ra ffffffff801620d8 sp ffffffff80699ac0 sz 16
> ffffffff801620ac+2c (?,?,?,?) ra ffffffff80357788 sp ffffffff80699ad0 sz 800
> ffffffff80357744+44 (?,?,?,?) ra ffffffff80304ff4 sp ffffffff80699df0 sz 16
> vpanic+f4 (?,?,?,?) ra ffffffff80305d84 sp ffffffff80699e00 sz 48
> panic+30 (?,ffffffff80699eb0,ffffffffffffffec,ffffffff802c901c) ra ffffffff805388bc sp ffffffff80699e30 sz 96
> MipsTLBInvalidException+360 (?,?,?,?) ra 0 sp ffffffff80699e90 sz 0
> --- Kernel Stack Overflow ---
> ffffffff802c9018+4 (?,?,?,?) ra ffffffff805544e4 sp c00000002a5b5fa0 sz 96
> ffffffff80554400+e4 (?,?,?,?) ra ffffffff802c90f4 sp c00000002a5b6000 sz 48
> ffffffff802c9018+dc (?,?,?,?) ra ffffffff805563dc sp c00000002a5b6030 sz 96
> cpu_intr+248 (?,?,?,?) ra ffffffff80538110 sp c00000002a5b6090 sz 80
> MipsKernIntr+188 (?,?,?,?) ra 0 sp c00000002a5b60e0 sz 368
> pid 607
> KDB: enter: panic
>
> (In particular, we probably should be using the trapframe logic employed
> here for MipsKernIntr.)
>
> I've patched kgdb to add a MIPS kernel target and taught it how to handle
> trapframes, etc. (and it can cross-debug, so I used an amd64 host to examine
> the dump) and it gives a more useful stack trace.  Unfortunately, increasing
> the kstack size on MIPS as a workaround for this issue seems to be very
> hard as the code in cpu_switch() hard codes exactly two TLB entries to cover
> the kernel stack and PCB to avoid TLB faults during a context switch.  I think
> if you create a kthread with a non-default kstack size on MIPS it's probably
> a ticking time bomb in that you might get a TLB miss when accessing the pcb.
>
> savectx () at /usr/home/john/work/git/freebsd/sys/mips/mips/swtch.S:171
> 171             SAVE_U_PCB_CONTEXT(ra, PCB_REG_PC, a0)
> (kgdb) where
> #0  savectx () at /usr/home/john/work/git/freebsd/sys/mips/mips/swtch.S:171
> #1  0xffffffff80304dfc in doadump (textdump=textdump at entry=0)
>     at /usr/home/john/work/git/freebsd/sys/kern/kern_shutdown.c:297
> #2  0xffffffff8015d868 in db_dump (dummy=<optimized out>,
>     dummy2=<optimized out>, dummy3=<optimized out>, dummy4=<optimized out>)
>     at /usr/home/john/work/git/freebsd/sys/ddb/db_command.c:546
> #3  0xffffffff8015e474 in db_command (
>     last_cmdp=last_cmdp at entry=0xffffffff8069e778 <db_last_command>,
>     cmd_table=<optimized out>,
>     cmd_table at entry=0xffffffff8069e730 <db_cmd_table>, dopager=dopager at entry=1)
>     at /usr/home/john/work/git/freebsd/sys/ddb/db_command.c:453
> #4  0xffffffff8015eab8 in db_command_loop ()
>     at /usr/home/john/work/git/freebsd/sys/ddb/db_command.c:506
> #5  0xffffffff80162040 in db_trap (type=<optimized out>, code=<optimized out>)
>     at /usr/home/john/work/git/freebsd/sys/ddb/db_main.c:248
> #6  0xffffffff803583e8 in kdb_trap (type=type at entry=9, code=code at entry=0,
>     tf=tf at entry=0xffffffff80699ca0 <pcpu_space+7328>)
>     at /usr/home/john/work/git/freebsd/sys/kern/subr_kdb.c:654
> #7  0xffffffff8054ba10 in trap (trapframe=0xffffffff80699ca0 <pcpu_space+7328>)
>     at /usr/home/john/work/git/freebsd/sys/mips/mips/trap.c:828
> #8  <signal handler called>
> #9  kdb_enter (why=0xffffffff805b6138 "panic", msg=<optimized out>)
>     at /usr/home/john/work/git/freebsd/sys/kern/subr_kdb.c:444
> #10 0xffffffff8030503c in vpanic (fmt=<optimized out>,
>     ap=ap at entry=0xffffffff80699e58 <pcpu_space+7768>)
>     at /usr/home/john/work/git/freebsd/sys/kern/kern_shutdown.c:752
> ---Type <return> to continue, or q <return> to quit---
> #11 0xffffffff80305d84 in panic (fmt=<optimized out>)
>     at /usr/home/john/work/git/freebsd/sys/kern/kern_shutdown.c:690
> #12 <signal handler called>
> #13 0xffffffff802c901c in intr_event_handle (ie=0x9800000000a42400,
>     frame=frame at entry=0x0)
>     at /usr/home/john/work/git/freebsd/sys/kern/kern_intr.c:1397
> #14 0xffffffff805544e4 in gt_pci_intr (v=0x9800000000ab0800)
>     at /usr/home/john/work/git/freebsd/sys/mips/malta/gt_pci.c:236
> #15 0xffffffff802c90f4 in intr_event_handle (ie=0x9800000000a43900,
>     frame=frame at entry=0xc00000002a5b6100)
>     at /usr/home/john/work/git/freebsd/sys/kern/kern_intr.c:1436
> #16 0xffffffff805563dc in cpu_intr (tf=0xc00000002a5b6100)
>     at /usr/home/john/work/git/freebsd/sys/mips/mips/intr_machdep.c:264
> #17 <signal handler called>
> #18 le_pci_wrcsr (sc=0x9800000000ae9400, port=<optimized out>,
>     val=<optimized out>)
>     at /usr/home/john/work/git/freebsd/sys/dev/le/if_le_pci.c:187
> #19 0xffffffff801893e4 in am79900_start_locked (sc=0x9800000000ae9400)
>     at /usr/home/john/work/git/freebsd/sys/dev/le/am79900.c:594
> #20 0xffffffff8018b1c0 in lance_start (ifp=<optimized out>)
>     at /usr/home/john/work/git/freebsd/sys/dev/le/lance.c:243
> #21 0xffffffff804137e4 in if_start (ifp=ifp at entry=0x9800000000ae4800)
>     at /usr/home/john/work/git/freebsd/sys/net/if.c:3630
> #22 0xffffffff80413998 in if_transmit (ifp=0x9800000000ae4800,
>     m=<optimized out>) at /usr/home/john/work/git/freebsd/sys/net/if.c:3642
> #23 0xffffffff804174fc in ether_output_frame (
> ---Type <return> to continue, or q <return> to quit---
>     ifp=ifp at entry=0x9800000000ae4800, m=0x980000000162d100)
>     at /usr/home/john/work/git/freebsd/sys/net/if_ethersubr.c:457
> #24 0xffffffff80417b94 in ether_output (ifp=0x9800000000ae4800,
>     m=0x980000000162d100, dst=0x98000000032878f0, ro=<optimized out>)
>     at /usr/home/john/work/git/freebsd/sys/net/if_ethersubr.c:429
> #25 0xffffffff8045cf28 in ip_output (m=m at entry=0x980000000162d100,
>     opt=<optimized out>, ro=0x98000000032878d0, flags=<optimized out>,
>     imo=<optimized out>, imo at entry=0x0, inp=<optimized out>)
>     at /usr/home/john/work/git/freebsd/sys/netinet/ip_output.c:664
> #26 0xffffffff80470348 in tcp_output (tp=0x9800000001f4f000)
>     at /usr/home/john/work/git/freebsd/sys/netinet/tcp_output.c:1432
> #27 0xffffffff80480d1c in tcp_usr_send (so=0x98000000032a8a20,
>     flags=<optimized out>, m=0x9800000001621a00, nam=0x0,
>     control=<optimized out>, td=0x9800000003275a60)
>     at /usr/home/john/work/git/freebsd/sys/netinet/tcp_usrreq.c:956
> #28 0xffffffff803a6ef8 in sosend_generic (so=0x98000000032a8a20,
>     addr=<optimized out>, uio=0x0, top=0x9800000001621a00,
>     control=<optimized out>, flags=<optimized out>, td=<optimized out>)
>     at /usr/home/john/work/git/freebsd/sys/kern/uipc_socket.c:1359
> #29 0xffffffff803a7060 in sosend (so=<optimized out>, addr=addr at entry=0x0,
>     uio=uio at entry=0x0, top=top at entry=0x9800000001621a00,
>     control=control at entry=0x0, flags=flags at entry=0, td=<optimized out>)
>     at /usr/home/john/work/git/freebsd/sys/kern/uipc_socket.c:1403
> #30 0xffffffff8049220c in clnt_vc_call (cl=<optimized out>,
>     ext=0xc00000002a5b69e8, proc=<optimized out>, args=0x980000000162d200,
>     resultsp=0xc00000002a5b6b40, utimeout=...)
> ---Type <return> to continue, or q <return> to quit---
>     at /usr/home/john/work/git/freebsd/sys/rpc/clnt_vc.c:398
> #31 0xffffffff804908a4 in clnt_reconnect_call (cl=0x9800000001630400,
>     ext=0xc00000002a5b69e8, proc=<optimized out>, args=0x980000000162d200,
>     resultsp=0xc00000002a5b6b40, utimeout=...)
>     at /usr/home/john/work/git/freebsd/sys/rpc/clnt_rc.c:271
> #32 0xffffffff801dd360 in newnfs_request (nd=0xc00000002a5b6b40,
>     nmp=0x980000000163b800, clp=clp at entry=0x0, nrp=0x980000000163b928,
>     vp=0x98000000044aa000, td=td at entry=0x9800000003275a60,
>     cred=cred at entry=0x9800000001670000, prog=prog at entry=100003, vers=3,
>     retsum=retsum at entry=0x0, toplevel=toplevel at entry=1, xidp=xidp at entry=0x0,
>     sep=sep at entry=0x0)
>     at /usr/home/john/work/git/freebsd/sys/fs/nfs/nfs_commonkrpc.c:746
> #33 0xffffffff8022c738 in nfscl_request (nd=nd at entry=0xc00000002a5b6b40,
>     vp=vp at entry=0x98000000044aa000, p=p at entry=0x9800000003275a60,
>     cred=cred at entry=0x9800000001670000, stuff=stuff at entry=0x0)
>     at /usr/home/john/work/git/freebsd/sys/fs/nfsclient/nfs_clport.c:951
> #34 0xffffffff802116a8 in nfsrpc_commit (vp=0x98000000044aa000,
>     offset=offset at entry=65536, cnt=cnt at entry=65536,
>     cred=cred at entry=0x9800000001670000, p=p at entry=0x9800000003275a60,
>     nap=nap at entry=0xc00000002a5b6cc0,
>     attrflagp=attrflagp at entry=0xc00000002a5b6d88, stuff=stuff at entry=0x0)
>     at /usr/home/john/work/git/freebsd/sys/fs/nfsclient/nfs_clrpcops.c:3640
> #35 0xffffffff80220e74 in ncl_commit (vp=vp at entry=0x98000000044aa000,
>     offset=offset at entry=65536, cnt=<optimized out>,
>     cred=cred at entry=0x9800000001670000, td=td at entry=0x9800000003275a60)
>     at /usr/home/john/work/git/freebsd/sys/fs/nfsclient/nfs_clvnops.c:2553
> ---Type <return> to continue, or q <return> to quit---
> #36 0xffffffff8022130c in ncl_flush (vp=0x98000000044aa000,
>     waitfor=<optimized out>, cred=cred at entry=0x0, td=0x9800000003275a60,
>     commit=commit at entry=1,
>     called_from_renewthread=called_from_renewthread at entry=0)
>     at /usr/home/john/work/git/freebsd/sys/fs/nfsclient/nfs_clvnops.c:2774
> #37 0xffffffff8022220c in nfs_fsync (ap=<optimized out>)
>     at /usr/home/john/work/git/freebsd/sys/fs/nfsclient/nfs_clvnops.c:2619
> #38 0xffffffff80559508 in VOP_FSYNC_APV (vop=<optimized out>,
>     a=a at entry=0xc00000002a5b6fa0) at vnode_if.c:1331
> #39 0xffffffff803bcbd4 in VOP_FSYNC (td=<optimized out>,
>     waitfor=<optimized out>, vp=<optimized out>) at vnode_if.h:549
> #40 bufsync (bo=<optimized out>, waitfor=<optimized out>)
>     at /usr/home/john/work/git/freebsd/sys/kern/vfs_bio.c:4605
> #41 0xffffffff803e6088 in bufobj_invalbuf (bo=bo at entry=0x98000000044aa0d0,
>     flags=<optimized out>, slpflag=<optimized out>, slptimeo=<optimized out>)
>     at /usr/home/john/work/git/freebsd/sys/kern/vfs_subr.c:1633
> #42 0xffffffff803e63a8 in vinvalbuf (vp=vp at entry=0x98000000044aa000,
>     flags=flags at entry=1, slpflag=slpflag at entry=0, slptimeo=slptimeo at entry=0)
>     at /usr/home/john/work/git/freebsd/sys/kern/vfs_subr.c:1711
> #43 0xffffffff8022df5c in ncl_vinvalbuf (vp=vp at entry=0x98000000044aa000,
>     flags=flags at entry=1, td=td at entry=0x9800000003275a60,
>     intrflg=<optimized out>, intrflg at entry=1)
>     at /usr/home/john/work/git/freebsd/sys/fs/nfsclient/nfs_clbio.c:1387
> #44 0xffffffff8021d5ec in nfs_setattr (ap=0xc00000002a5b7298)
>     at /usr/home/john/work/git/freebsd/sys/fs/nfsclient/nfs_clvnops.c:937
> #45 0xffffffff80558564 in VOP_SETATTR_APV (vop=<optimized out>,
> ---Type <return> to continue, or q <return> to quit---
>     a=a at entry=0xc00000002a5b7298) at vnode_if.c:799
> #46 0xffffffff80230eac in VOP_SETATTR (cred=0x9800000001670000,
>     vap=0xc00000002a5b71f0, vp=0x98000000044aa000) at vnode_if.h:335
> #47 ncl_write (ap=<optimized out>)
>     at /usr/home/john/work/git/freebsd/sys/fs/nfsclient/nfs_clbio.c:1280
> #48 0xffffffff80558b70 in VOP_WRITE_APV (vop=<optimized out>,
>     a=a at entry=0xc00000002a5b74f0) at vnode_if.c:1000
> #49 0xffffffff803f9708 in VOP_WRITE (cred=<optimized out>,
>     ioflag=<optimized out>, uio=0xc00000002a5b7690, vp=<optimized out>)
>     at vnode_if.h:413
> #50 vn_io_fault_doio (args=args at entry=0xc00000002a5b76d8,
>     uio=uio at entry=0xc00000002a5b7690, td=td at entry=0x9800000003275a60)
>     at /usr/home/john/work/git/freebsd/sys/kern/vfs_vnops.c:952
> #51 0xffffffff803f9894 in vn_io_fault1 (vp=vp at entry=0x98000000044aa000,
>     uio=uio at entry=0xc00000002a5b7690, args=args at entry=0xc00000002a5b76d8,
>     td=td at entry=0x9800000003275a60)
>     at /usr/home/john/work/git/freebsd/sys/kern/vfs_vnops.c:1060
> #52 0xffffffff803fca30 in vn_rdwr (rw=rw at entry=UIO_WRITE,
>     vp=vp at entry=0x98000000044aa000, base=base at entry=0x160085000,
>     len=len at entry=16384, offset=offset at entry=131072,
>     segflg=segflg at entry=UIO_USERSPACE, ioflg=ioflg at entry=16641,
>     active_cred=active_cred at entry=0x9800000001670000,
>     file_cred=file_cred at entry=0x0, aresid=aresid at entry=0xc00000002a5b77a0,
>     td=td at entry=0x9800000003275a60)
>     at /usr/home/john/work/git/freebsd/sys/kern/vfs_vnops.c:576
> #53 0xffffffff803fcc68 in vn_rdwr_inchunks (rw=rw at entry=UIO_WRITE,
> ---Type <return> to continue, or q <return> to quit---
>     vp=0x98000000044aa000, base=0x160085000, len=16384, offset=131072,
>     offset at entry=98304, segflg=segflg at entry=UIO_USERSPACE,
>     ioflg=ioflg at entry=16641, active_cred=0x9800000001670000, file_cred=0x0,
>     aresid=aresid at entry=0x0, td=0x9800000003275a60)
>     at /usr/home/john/work/git/freebsd/sys/kern/vfs_vnops.c:641
> #54 0xffffffff802954f4 in core_write (seg=UIO_USERSPACE, offset=98304,
>     len=<optimized out>, base=<optimized out>, p=0xc00000002a5b7860)
>     at /usr/home/john/work/git/freebsd/sys/kern/imgact_elf.c:1229
> #55 core_output (tmpbuf=0x0, p=0xc00000002a5b7860, offset=98304,
>     len=<optimized out>, base=<optimized out>)
>     at /usr/home/john/work/git/freebsd/sys/kern/imgact_elf.c:1243
> #56 elf64_coredump (td=<optimized out>, vp=0x98000000044aa000,
>     limit=9223372036854775807, flags=<optimized out>)
>     at /usr/home/john/work/git/freebsd/sys/kern/imgact_elf.c:1384
> #57 0xffffffff8030805c in coredump (td=td at entry=0x9800000003275a60)
>     at /usr/home/john/work/git/freebsd/sys/kern/kern_sig.c:3477
> #58 0xffffffff8030b818 in sigexit (td=td at entry=0x9800000003275a60,
>     sig=sig at entry=7)
>     at /usr/home/john/work/git/freebsd/sys/kern/kern_sig.c:3072
> #59 0xffffffff8030c19c in postsig (sig=sig at entry=7)
>     at /usr/home/john/work/git/freebsd/sys/kern/kern_sig.c:2985
> #60 0xffffffff8036f13c in ast (framep=0xc00000002a5b7d30)
>     at /usr/home/john/work/git/freebsd/sys/kern/subr_trap.c:314
> #61 <signal handler called>
> #62 0x0000000120000ff4 in ?? ()
> Backtrace stopped: previous frame identical to this frame (corrupt stack?)
>
> --
> John Baldwin
>

More information about the svn-src-head mailing list