Kernel panics in tcp_twclose
Palle Girgensohn
girgen at FreeBSD.org
Tue Sep 22 16:49:47 UTC 2015
> 22 sep 2015 kl. 18:46 skrev Palle Girgensohn <girgen at FreeBSD.org>:
>
> Hi all,
>
>
>> 21 sep 2015 kl. 15:53 skrev Palle Girgensohn <girgen at FreeBSD.org>:
>>
>>>
>>> 21 sep 2015 kl. 10:21 skrev Julien Charbon <jch at FreeBSD.org>:
>>>
>>>
>>> Hi Konstantin, Hi Palle,
>>>
>>> On 18/09/15 18:06, Konstantin Belousov wrote:
>>>> On Fri, Sep 18, 2015 at 03:56:25PM +0200, Julien Charbon wrote:
>>>>> Hi Palle,
>>>>>
>>>>> On 18/09/15 11:12, Palle Girgensohn wrote:
>>>>>> We see daily panics on our production systems (web server, apache
>>>>>> running MPM event, openjdk8. Kernel with VIMAGE. Jails using netgraph
>>>>>> interfaces [not epair]).
>>>>>>
>>>>>> The problem started after the summer. Normal port upgrades seems to
>>>>>> be the only difference. The problem occurs with 10.2-p2 kernel as
>>>>>> well as 10.1-p4 and 10.1-p15.
>>>>>>
>>>>>> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203175
>>>>>>
>>>>>> Any ideas?
>>>>>
>>>>> Thanks for you detailed report. I am not aware of any tcp_twclose()
>>>>> related issues (without VIMAGE) since FreeBSD 10.0 (does not mean there
>>>>> are none). Few interesting facts (at least for me):
>>>>>
>>>>> - Your crash happens when unlocking a inp exclusive lock with INP_WUNLOCK()
>>>>>
>>>>> - Something is already wrong before calling turnstile_broadcast() as it
>>>>> is called with ts = NULL:
>>>> In the kernel without witness this is a 99%-sure indication of attempt to
>>>> unlock not owned lock.
>>>
>>> Thanks, this is useful. So far I did not find any path where
>>> tcp_twclose() can call INP_WUNLOCK without having the exclusive lock
>>> held, that makes this issue interesting.
>>>
>>>>> I won't go to far here as I am not expert enough in VIMAGE, but one
>>>>> question anyway:
>>>>>
>>>>> - Can you correlate this kernel panic to a particular event? Like for
>>>>> example a VIMAGE/VNET jail destruction.
>>>>>
>>>>> I will test that on my side on a 10.2 machine.
>>>
>>> I did not find any issues while testing 10.2 + VIMAGE on my side. Thus
>>> Palle what I would suggest:
>>>
>>> - First, test with stable/10 to see if by chance this issue has already
>>> been fixed in stable branch.
>>>
>>> - Second, if issue is still in stable/10, compile 10.2 kernel with
>>> these options:
>>>
>>> options DDB
>>> options DEADLKRES
>>> options INVARIANTS
>>> options INVARIANT_SUPPORT
>>> options WITNESS
>>> options WITNESS_SKIPSPIN
>>>
>>> To see where the original fault is coming from.
>>
>> Hi,
>>
>> We just had two crashes within 15 minutes using 10.2 with these two added:
>>
>> https://svnweb.freebsd.org/changeset/base/287261
>>
>> https://svnweb.freebsd.org/changeset/base/287780
>>
>> We don't always get a core dump, but the second time, we did.
>>
>> very similar stack trace, but not identical:
>>
>> (kgdb) #0 doadump (textdump=<value optimized out>) at pcpu.h:219
>> #1 0xffffffff80949a82 in kern_reboot (howto=260)
>> at /usr/src/sys/kern/kern_shutdown.c:451
>> #2 0xffffffff80949e65 in vpanic (fmt=<value optimized out>,
>> ap=<value optimized out>) at /usr/src/sys/kern/kern_shutdown.c:758
>> #3 0xffffffff80949cf3 in panic (fmt=0x0)
>> at /usr/src/sys/kern/kern_shutdown.c:687
>> #4 0xffffffff80d5d0bb in trap_fatal (frame=<value optimized out>,
>> eva=<value optimized out>) at /usr/src/sys/amd64/amd64/trap.c:851
>> #5 0xffffffff80d5d3bd in trap_pfault (frame=0xfffffe1760bc1840,
>> usermode=<value optimized out>) at /usr/src/sys/amd64/amd64/trap.c:674
>> #6 0xffffffff80d5ca5a in trap (frame=0xfffffe1760bc1840)
>> at /usr/src/sys/amd64/amd64/trap.c:440
>> #7 0xffffffff80d42dd2 in calltrap ()
>> at /usr/src/sys/amd64/amd64/exception.S:236
>> #8 0xffffffff8099861c in turnstile_broadcast (ts=0x0, queue=1)
>> at /usr/src/sys/kern/subr_turnstile.c:838
>> #9 0xffffffff80948100 in __rw_wunlock_hard (c=0xfffff811c43487a0, tid=1,
>> file=0x1 <Address 0x1 out of bounds>, line=1)
>> at /usr/src/sys/kern/kern_rwlock.c:988
>> #10 0xffffffff80b067c4 in tcp_twclose (tw=<value optimized out>,
>> reuse=<value optimized out>) at /usr/src/sys/netinet/tcp_timewait.c:540
>> #11 0xffffffff80b06e0b in tcp_tw_2msl_scan (reuse=0)
>> at /usr/src/sys/netinet/tcp_timewait.c:748
>> #12 0xffffffff80b04b0e in tcp_slowtimo ()
>> at /usr/src/sys/netinet/tcp_timer.c:198
>> #13 0xffffffff809b7a04 in pfslowtimo (arg=0x0)
>> at /usr/src/sys/kern/uipc_domain.c:508
>> #14 0xffffffff8095f91b in softclock_call_cc (c=0xffffffff81620bf0,
>> cc=0xffffffff8169dc00, direct=0) at /usr/src/sys/kern/kern_timeout.c:685
>> #15 0xffffffff8095fd44 in softclock (arg=0xffffffff8169dc00)
>> at /usr/src/sys/kern/kern_timeout.c:814
>> #16 0xffffffff8091592b in intr_event_execute_handlers (
>> p=<value optimized out>, ie=0xfffff801102e0d00)
>> at /usr/src/sys/kern/kern_intr.c:1264
>> #17 0xffffffff80915d76 in ithread_loop (arg=0xfffff801102adee0)
>> at /usr/src/sys/kern/kern_intr.c:1277
>> #18 0xffffffff8091347a in fork_exit (
>> callout=0xffffffff80915ce0 <ithread_loop>, arg=0xfffff801102adee0,
>> frame=0xfffffe1760bc1c00) at /usr/src/sys/kern/kern_fork.c:1018
>> #19 0xffffffff80d4330e in fork_trampoline ()
>> at /usr/src/sys/amd64/amd64/exception.S:611
>> #20 0x0000000000000000 in ?? ()
>>
>>
>>
>> I'll try stable/10 now. Would you suggest a "clean" stable/10, or could 287621 and 287780 help?
>>
>> I'll add the debugging suggested options right away.
>>
>> Palle
>
>
> I have a new core dump from ^/stable/10 with:
>
>
> options DDB
> options DEADLKRES
> options INVARIANTS
> options INVARIANT_SUPPORT
> options WITNESS
> options WITNESS_SKIPSPIN
>
>
> What can I do with the core dump? "corrupt stack"...
>
> (kgdb) #0 doadump (textdump=1) at pcpu.h:219
> #1 0xffffffff8094b337 in kern_reboot (howto=260)
> at /usr/src/sys/kern/kern_shutdown.c:451
> #2 0xffffffff8094b845 in vpanic (fmt=<value optimized out>,
> ap=<value optimized out>) at /usr/src/sys/kern/kern_shutdown.c:758
> #3 0xffffffff8094b6d9 in kassert_panic (fmt=<value optimized out>)
> at /usr/src/sys/kern/kern_shutdown.c:646
> #4 0xffffffff80b1ee59 in tcp_usr_detach (so=<value optimized out>)
> at /usr/src/sys/netinet/tcp_usrreq.c:202
> #5 0xffffffff809cd291 in sofree (so=0xfffff801dd302000)
> at /usr/src/sys/kern/uipc_socket.c:747
> #6 0xffffffff809cdb00 in soclose (so=<value optimized out>)
> at /usr/src/sys/kern/uipc_socket.c:849
> #7 0xffffffff808fe659 in _fdrop (fp=0xfffff802a593db40, td=0x0) at file.h:343
> #8 0xffffffff80901092 in closef (fp=0xfffff802a593db40,
> td=0xfffff80eebc894a0) at /usr/src/sys/kern/kern_descrip.c:2338
> #9 0xffffffff808feb5d in closefp (fdp=0xfffff80b20cce000,
> fd=<value optimized out>, fp=0xfffff802a593db40, td=0xfffff80eebc894a0,
> holdleaders=<value optimized out>)
> at /usr/src/sys/kern/kern_descrip.c:1194
> #10 0xffffffff80d7bc3a in amd64_syscall (td=0xfffff80eebc894a0, traced=0)
> at subr_syscall.c:134
> #11 0xffffffff80d5f1db in Xfast_syscall ()
> at /usr/src/sys/amd64/amd64/exception.S:396
> #12 0x0000000801c8d94a in ?? ()
> Previous frame inner to this frame (corrupt stack?)
> Current language: auto; currently minimal
> (kgdb)
>
>
> Thanks,
> Palle
>
# kgdb kernel /var/crash/vmcore.2
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd"...
Unread portion of the kernel message buffer:
panic: tcp_detach: INP_TIMEWAIT && INP_DROPPED && tp != NULL
cpuid = 16
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe183d9e97e0
kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe183d9e9890
vpanic() at vpanic+0x126/frame 0xfffffe183d9e98d0
kassert_panic() at kassert_panic+0x139/frame 0xfffffe183d9e9940
tcp_usr_detach() at tcp_usr_detach+0xf9/frame 0xfffffe183d9e9970
sofree() at sofree+0x1f1/frame 0xfffffe183d9e99a0
soclose() at soclose+0x3a0/frame 0xfffffe183d9e99f0
_fdrop() at _fdrop+0x29/frame 0xfffffe183d9e9a10
closef() at closef+0x1e2/frame 0xfffffe183d9e9aa0
closefp() at closefp+0x9d/frame 0xfffffe183d9e9ae0
amd64_syscall() at amd64_syscall+0x25a/frame 0xfffffe183d9e9bf0
Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe183d9e9bf0
--- syscall (6, FreeBSD ELF64, sys_close), rip = 0x801c8d94a, rsp = 0x7ffff91c8668, rbp = 0x7ffff91c8680 ---
KDB: enter: panic
Uptime: 18h57m59s
Dumping 23085 out of 98263 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%
Reading symbols from /boot/kernel/nullfs.ko.symbols...done.
Loaded symbols for /boot/kernel/nullfs.ko.symbols
Reading symbols from /boot/kernel/zfs.ko.symbols...done.
Loaded symbols for /boot/kernel/zfs.ko.symbols
Reading symbols from /boot/kernel/opensolaris.ko.symbols...done.
Loaded symbols for /boot/kernel/opensolaris.ko.symbols
Reading symbols from /boot/kernel/ng_bridge.ko.symbols...done.
Loaded symbols for /boot/kernel/ng_bridge.ko.symbols
Reading symbols from /boot/kernel/netgraph.ko.symbols...done.
Loaded symbols for /boot/kernel/netgraph.ko.symbols
Reading symbols from /boot/kernel/ng_eiface.ko.symbols...done.
Loaded symbols for /boot/kernel/ng_eiface.ko.symbols
Reading symbols from /boot/kernel/ng_ether.ko.symbols...done.
Loaded symbols for /boot/kernel/ng_ether.ko.symbols
Reading symbols from /boot/kernel/accf_data.ko.symbols...done.
Loaded symbols for /boot/kernel/accf_data.ko.symbols
Reading symbols from /boot/kernel/accf_http.ko.symbols...done.
Loaded symbols for /boot/kernel/accf_http.ko.symbols
Reading symbols from /boot/kernel/ums.ko.symbols...done.
Loaded symbols for /boot/kernel/ums.ko.symbols
Reading symbols from /boot/kernel/ng_socket.ko.symbols...done.
Loaded symbols for /boot/kernel/ng_socket.ko.symbols
Reading symbols from /boot/kernel/fdescfs.ko.symbols...done.
Loaded symbols for /boot/kernel/fdescfs.ko.symbols
#0 doadump (textdump=1) at pcpu.h:219
219 __asm("movq %%gs:%1,%0" : "=r" (td)
(kgdb) bt
#0 doadump (textdump=1) at pcpu.h:219
#1 0xffffffff8094b337 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:451
#2 0xffffffff8094b845 in vpanic (fmt=<value optimized out>, ap=<value optimized out>)
at /usr/src/sys/kern/kern_shutdown.c:758
#3 0xffffffff8094b6d9 in kassert_panic (fmt=<value optimized out>) at /usr/src/sys/kern/kern_shutdown.c:646
#4 0xffffffff80b1ee59 in tcp_usr_detach (so=<value optimized out>) at /usr/src/sys/netinet/tcp_usrreq.c:202
#5 0xffffffff809cd291 in sofree (so=0xfffff801dd302000) at /usr/src/sys/kern/uipc_socket.c:747
#6 0xffffffff809cdb00 in soclose (so=<value optimized out>) at /usr/src/sys/kern/uipc_socket.c:849
#7 0xffffffff808fe659 in _fdrop (fp=0xfffff802a593db40, td=0x0) at file.h:343
#8 0xffffffff80901092 in closef (fp=0xfffff802a593db40, td=0xfffff80eebc894a0)
at /usr/src/sys/kern/kern_descrip.c:2338
#9 0xffffffff808feb5d in closefp (fdp=0xfffff80b20cce000, fd=<value optimized out>, fp=0xfffff802a593db40,
td=0xfffff80eebc894a0, holdleaders=<value optimized out>) at /usr/src/sys/kern/kern_descrip.c:1194
#10 0xffffffff80d7bc3a in amd64_syscall (td=0xfffff80eebc894a0, traced=0) at subr_syscall.c:134
#11 0xffffffff80d5f1db in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:396
#12 0x0000000801c8d94a in ?? ()
Previous frame inner to this frame (corrupt stack?)
Current language: auto; currently minimal
(kgdb) f 8
#8 0xffffffff80901092 in closef (fp=0xfffff802a593db40, td=0xfffff80eebc894a0)
at /usr/src/sys/kern/kern_descrip.c:2338
2338 return (fdrop(fp, td));
(kgdb) help
List of classes of commands:
aliases -- Aliases of other commands
breakpoints -- Making program stop at certain points
data -- Examining data
files -- Specifying and examining files
internals -- Maintenance commands
obscure -- Obscure features
running -- Running the program
stack -- Examining the stack
status -- Status inquiries
support -- Support facilities
tracepoints -- Tracing of program execution without stopping the program
user-defined -- User-defined commands
Type "help" followed by a class name for a list of commands in that class.
Type "help" followed by command name for full documentation.
Command name abbreviations are allowed if unambiguous.
(kgdb) disassemble
Dump of assembler code for function closef:
0xffffffff80900eb0 <closef+0>: push %rbp
0xffffffff80900eb1 <closef+1>: mov %rsp,%rbp
0xffffffff80900eb4 <closef+4>: push %r15
0xffffffff80900eb6 <closef+6>: push %r14
0xffffffff80900eb8 <closef+8>: push %r13
0xffffffff80900eba <closef+10>: push %r12
0xffffffff80900ebc <closef+12>: push %rbx
0xffffffff80900ebd <closef+13>: sub $0x58,%rsp
0xffffffff80900ec1 <closef+17>: mov %rsi,%r12
0xffffffff80900ec4 <closef+20>: mov %rdi,%r14
0xffffffff80900ec7 <closef+23>: cmpw $0x1,0x20(%r14)
0xffffffff80900ecd <closef+29>: jne 0xffffffff80901077 <closef+455>
0xffffffff80900ed3 <closef+35>: test %r12,%r12
0xffffffff80900ed6 <closef+38>: je 0xffffffff80901077 <closef+455>
0xffffffff80900edc <closef+44>: mov 0x8(%r12),%rax
0xffffffff80900ee1 <closef+49>: mov 0x428(%rax),%rcx
0xffffffff80900ee8 <closef+56>: testb $0x1,0xb0(%rcx)
0xffffffff80900eef <closef+63>: je 0xffffffff80900f50 <closef+160>
0xffffffff80900ef1 <closef+65>: mov 0x18(%r14),%rcx
0xffffffff80900ef5 <closef+69>: movw $0x0,-0x62(%rbp)
0xffffffff80900efb <closef+75>: movq $0x0,-0x78(%rbp)
0xffffffff80900f03 <closef+83>: movq $0x0,-0x70(%rbp)
0xffffffff80900f0b <closef+91>: movw $0x2,-0x64(%rbp)
0xffffffff80900f11 <closef+97>: mov 0x428(%rax),%rax
0xffffffff80900f18 <closef+104>: movq $0xffffffff81557f68,-0x58(%rbp)
0xffffffff80900f20 <closef+112>: mov %rcx,-0x50(%rbp)
0xffffffff80900f24 <closef+116>: mov %rax,-0x48(%rbp)
0xffffffff80900f28 <closef+120>: movl $0x2,-0x40(%rbp)
0xffffffff80900f2f <closef+127>: lea -0x78(%rbp),%rax
0xffffffff80900f33 <closef+131>: mov %rax,-0x38(%rbp)
0xffffffff80900f37 <closef+135>: movl $0x40,-0x30(%rbp)
0xffffffff80900f3e <closef+142>: mov 0x8(%rcx),%rdi
0xffffffff80900f42 <closef+146>: lea -0x58(%rbp),%rsi
0xffffffff80900f46 <closef+150>: callq 0xffffffff80ea8870 <VOP_ADVLOCK_APV>
0xffffffff80900f4b <closef+155>: mov 0x8(%r12),%rax
0xffffffff80900f50 <closef+160>: mov 0x50(%rax),%rbx
0xffffffff80900f54 <closef+164>: test %rbx,%rbx
0xffffffff80900f57 <closef+167>: je 0xffffffff80901077 <closef+455>
0xffffffff80900f5d <closef+173>: mov 0x48(%rax),%r15
0xffffffff80900f61 <closef+177>: add $0x40,%r15
0xffffffff80900f65 <closef+181>: xor %esi,%esi
0xffffffff80900f67 <closef+183>: mov $0xffffffff810042e9,%rdx
0xffffffff80900f6e <closef+190>: mov $0x906,%ecx
0xffffffff80900f73 <closef+195>: mov %r15,%rdi
0xffffffff80900f76 <closef+198>: callq 0xffffffff80952ba0 <_sx_xlock>
0xffffffff80900f7b <closef+203>: mov 0x20(%rbx),%rbx
0xffffffff80900f7f <closef+207>: mov 0x8(%r12),%rax
0xffffffff80900f84 <closef+212>: cmp 0x50(%rax),%rbx
---Type <return> to continue, or q <return> to quit---
0xffffffff80900f88 <closef+216>: je 0xffffffff80901063 <closef+435>
0xffffffff80900f8e <closef+222>: lea -0x58(%rbp),%r13
0xffffffff80900f92 <closef+226>: nopw %cs:0x0(%rax,%rax,1)
0xffffffff80900fa0 <closef+240>: mov 0x10(%rbx),%rax
0xffffffff80900fa4 <closef+244>: testb $0x1,0xb0(%rax)
0xffffffff80900fab <closef+251>: je 0xffffffff80901050 <closef+416>
0xffffffff80900fb1 <closef+257>: incl 0x4(%rbx)
0xffffffff80900fb4 <closef+260>: mov $0xffffffff810042e9,%rsi
0xffffffff80900fbb <closef+267>: mov $0x90e,%edx
0xffffffff80900fc0 <closef+272>: mov %r15,%rdi
0xffffffff80900fc3 <closef+275>: callq 0xffffffff80952f90 <_sx_xunlock>
0xffffffff80900fc8 <closef+280>: movw $0x0,-0x62(%rbp)
0xffffffff80900fce <closef+286>: movq $0x0,-0x78(%rbp)
0xffffffff80900fd6 <closef+294>: movq $0x0,-0x70(%rbp)
0xffffffff80900fde <closef+302>: movw $0x2,-0x64(%rbp)
0xffffffff80900fe4 <closef+308>: mov 0x18(%r14),%rax
0xffffffff80900fe8 <closef+312>: mov 0x10(%rbx),%rcx
0xffffffff80900fec <closef+316>: movq $0xffffffff81557f68,-0x58(%rbp)
0xffffffff80900ff4 <closef+324>: mov %rax,-0x50(%rbp)
0xffffffff80900ff8 <closef+328>: mov %rcx,-0x48(%rbp)
0xffffffff80900ffc <closef+332>: movl $0x2,-0x40(%rbp)
0xffffffff80901003 <closef+339>: lea -0x78(%rbp),%rcx
0xffffffff80901007 <closef+343>: mov %rcx,-0x38(%rbp)
0xffffffff8090100b <closef+347>: movl $0x40,-0x30(%rbp)
0xffffffff80901012 <closef+354>: mov 0x8(%rax),%rdi
0xffffffff80901016 <closef+358>: mov %r13,%rsi
0xffffffff80901019 <closef+361>: callq 0xffffffff80ea8870 <VOP_ADVLOCK_APV>
0xffffffff8090101e <closef+366>: xor %esi,%esi
0xffffffff80901020 <closef+368>: mov $0xffffffff810042e9,%rdx
0xffffffff80901027 <closef+375>: mov $0x917,%ecx
0xffffffff8090102c <closef+380>: mov %r15,%rdi
0xffffffff8090102f <closef+383>: callq 0xffffffff80952ba0 <_sx_xlock>
0xffffffff80901034 <closef+388>: decl 0x4(%rbx)
0xffffffff80901037 <closef+391>: jne 0xffffffff80901050 <closef+416>
0xffffffff80901039 <closef+393>: cmpl $0x0,0x8(%rbx)
0xffffffff8090103d <closef+397>: je 0xffffffff80901050 <closef+416>
0xffffffff8090103f <closef+399>: movl $0x0,0x8(%rbx)
0xffffffff80901046 <closef+406>: mov %rbx,%rdi
0xffffffff80901049 <closef+409>: callq 0xffffffff80954a40 <wakeup>
0xffffffff8090104e <closef+414>: xchg %ax,%ax
0xffffffff80901050 <closef+416>: mov 0x20(%rbx),%rbx
0xffffffff80901054 <closef+420>: mov 0x8(%r12),%rax
0xffffffff80901059 <closef+425>: cmp 0x50(%rax),%rbx
0xffffffff8090105d <closef+429>: jne 0xffffffff80900fa0 <closef+240>
0xffffffff80901063 <closef+435>: mov $0xffffffff810042e9,%rsi
0xffffffff8090106a <closef+442>: mov $0x91f,%edx
0xffffffff8090106f <closef+447>: mov %r15,%rdi
0xffffffff80901072 <closef+450>: callq 0xffffffff80952f90 <_sx_xunlock>
0xffffffff80901077 <closef+455>: mov $0xffffffff,%eax
---Type <return> to continue, or q <return> to quit---
0xffffffff8090107c <closef+460>: lock xadd %eax,0x28(%r14)
0xffffffff80901082 <closef+466>: cmp $0x1,%eax
0xffffffff80901085 <closef+469>: jne 0xffffffff809010a5 <closef+501>
0xffffffff80901087 <closef+471>: mov %r14,%rdi
0xffffffff8090108a <closef+474>: mov %r12,%rsi
0xffffffff8090108d <closef+477>: callq 0xffffffff808fe630 <_fdrop>
0xffffffff80901092 <closef+482>: mov %eax,%ebx
0xffffffff80901094 <closef+484>: mov %ebx,%eax
0xffffffff80901096 <closef+486>: add $0x58,%rsp
0xffffffff8090109a <closef+490>: pop %rbx
0xffffffff8090109b <closef+491>: pop %r12
0xffffffff8090109d <closef+493>: pop %r13
0xffffffff8090109f <closef+495>: pop %r14
0xffffffff809010a1 <closef+497>: pop %r15
0xffffffff809010a3 <closef+499>: pop %rbp
0xffffffff809010a4 <closef+500>: retq
0xffffffff809010a5 <closef+501>: xor %ebx,%ebx
0xffffffff809010a7 <closef+503>: test %eax,%eax
0xffffffff809010a9 <closef+505>: jne 0xffffffff80901094 <closef+484>
0xffffffff809010ab <closef+507>: add $0x28,%r14
0xffffffff809010af <closef+511>: xor %ebx,%ebx
0xffffffff809010b1 <closef+513>: mov $0xffffffff80ebcddb,%rdi
0xffffffff809010b8 <closef+520>: xor %eax,%eax
0xffffffff809010ba <closef+522>: mov %r14,%rsi
0xffffffff809010bd <closef+525>: callq 0xffffffff8094b5a0 <kassert_panic>
0xffffffff809010c2 <closef+530>: jmp 0xffffffff80901094 <closef+484>
End of assembler dump.
More information about the freebsd-net
mailing list