[Bug 237655] Non-deterministic panic with pf

[bugzilla-noreply at freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=237655

            Bug ID: 237655
           Summary: Non-deterministic panic with pf
           Product: Base System
           Version: CURRENT
          Hardware: Any
                OS: Any
            Status: New
          Severity: Affects Some People
          Priority: ---
         Component: kern
          Assignee: bugs at FreeBSD.org
          Reporter: ngie at FreeBSD.org

The last few test runs have been failing as follows, with panics in strncmp()
managing (what I assume is epair or tun) interfaces via the ioctl handler.

There are some questionable LoR messages also printed out in the pf code about
lock holding, but this is likely not the root cause.

>From https://ci.freebsd.org/job/FreeBSD-head-amd64-test/11014/consoleText :

sys/netpfil/pf/pfsync:basic  ->  lock order reversal: (sleepable after
non-sleepable)
 1st 0xfffff8003a43f070 pfsync (pfsync) @
/usr/src/sys/netpfil/pf/if_pfsync.c:1402
 2nd 0xffffffff820c48c0 in_multi_sx (in_multi_sx) @
/usr/src/sys/netinet/in_mcast.c:1251
stack backtrace:
#0 0xffffffff80c47773 at witness_debugger+0x73
#1 0xffffffff80c474bd at witness_checkorder+0xa7d
#2 0xffffffff80be7ed8 at _sx_xlock+0x68
#3 0xffffffff80d65271 at in_joingroup+0x31
#4 0xffffffff82839086 at pfsyncioctl+0x6e6
#5 0xffffffff80d60116 at in_control+0x376
#6 0xffffffff80ce168b at ifioctl+0x57b
#7 0xffffffff80c4c6ba at kern_ioctl+0x28a
#8 0xffffffff80c4c3bd at sys_ioctl+0x15d
#9 0xffffffff810b2e16 at amd64_syscall+0x276
#10 0xffffffff8108b5fd at fast_syscall_common+0x101
passed  [2.316s]
sys/netpfil/pf/pfsync:defer  ->  passed  [2.248s]
sys/netpfil/pf/rdr:basic  ->  Apr 29 18:28:59  kernel: nd6_dad_timer: cancel
DAD on epair3a because of ND6_IFF_IFDISABLED.

passed  [4.153s]
sys/netpfil/pf/route_to:v4  ->  passed  [3.195s]
sys/netpfil/pf/route_to:v6  ->  Apr 29 18:29:07  kernel: nd6_dad_timer: called
with non-tentative address fe80:7::39:3ff:fe4c:500a(epair4a)

Apr 29 18:29:07  kernel: nd6_dad_timer: called with non-tentative address
fe80:3::35:96ff:fe61:640b(epair3b)

Apr 29 18:29:07  kernel: nd6_dad_timer: called with non-tentative address
fe80:4::39:3ff:fe4c:500b(epair4b)

Apr 29 18:29:07  kernel: nd6_dad_timer: called with non-tentative address
fe80:5::35:96ff:fe61:640a(epair3a)

passed  [3.181s]
sys/netpfil/pf/set_skip:set_skip_group  ->  passed  [0.097s]
sys/netpfil/pf/set_skip:set_skip_group_lo  ->  passed  [0.113s]
sys/netpfil/pf/set_tos:v4  ->  passed  [8.702s]
sys/netpfil/pf/synproxy:synproxy  ->  passed  [0.161s]


Fatal trap 9: general protection fault while in kernel mode
cpuid = 0; apic id = 00
instruction pointer     = 0x20:0xffffffff80ccb525
stack pointer           = 0x28:0xfffffe0030ec9740
frame pointer           = 0x28:0xfffffe0030ec9740
code segment            = base 0x0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 69669 (ifconfig)
trap number             = 9
panic: general protection fault
cpuid = 0
time = 1556562559
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0030ec9450
vpanic() at vpanic+0x19d/frame 0xfffffe0030ec94a0
panic() at panic+0x43/frame 0xfffffe0030ec9500
trap_fatal() at trap_fatal+0x394/frame 0xfffffe0030ec9560
trap() at trap+0x6c/frame 0xfffffe0030ec9670
calltrap() at calltrap+0x8/frame 0xfffffe0030ec9670
--- trap 0x9, rip = 0xffffffff80ccb525, rsp = 0xfffffe0030ec9740, rbp =
0xfffffe0030ec9740 ---
strncmp() at strncmp+0x15/frame 0xfffffe0030ec9740
ifunit_ref() at ifunit_ref+0x51/frame 0xfffffe0030ec9780
ifioctl() at ifioctl+0x508/frame 0xfffffe0030ec9850
kern_ioctl() at kern_ioctl+0x28a/frame 0xfffffe0030ec98c0
sys_ioctl() at sys_ioctl+0x15d/frame 0xfffffe0030ec9990
amd64_syscall() at amd64_syscall+0x276/frame 0xfffffe0030ec9ab0
fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0030ec9ab0
--- syscall (54, FreeBSD ELF64, sys_ioctl), rip = 0x80048531a, rsp =
0x7fffffffe458, rbp = 0x7fffffffe4c0 ---
KDB: enter: panic
[ thread pid 69669 tid 100143 ]
Stopped at      kdb_enter+0x3b: movq    $0,kdb_why
db:0:kdb.enter.panic> show pcpu
cpuid        = 0
dynamic pcpu = 0xb84800
curthread    = 0xfffff80004f305a0: pid 69669 tid 100143 "ifconfig"
curpcb       = 0xfffffe0030ec9b80
fpcurthread  = 0xfffff80004f305a0: pid 69669 "ifconfig"
idlethread   = 0xfffff80003272000: tid 100003 "idle: cpu0"
curpmap      = 0xfffff8003e6fb130
tssp         = 0xffffffff821cd320
commontssp   = 0xffffffff821cd320
rsp0         = 0xfffffe0030ec9b80
gs32p        = 0xffffffff821d3f58
ldt          = 0xffffffff821d3f98
tss          = 0xffffffff821d3f88
tlb gen      = 455364
curvnet      = 0xfffff8000307aec0
spin locks held:
db:0:kdb.enter.panic> alltrace

Tracing command ifconfig pid 69669 tid 100143 td 0xfffff80004f305a0 (CPU 0)
kdb_enter() at kdb_enter+0x3b/frame 0xfffffe0030ec9450
vpanic() at vpanic+0x1ba/frame 0xfffffe0030ec94a0
panic() at panic+0x43/frame 0xfffffe0030ec9500
trap_fatal() at trap_fatal+0x394/frame 0xfffffe0030ec9560
trap() at trap+0x6c/frame 0xfffffe0030ec9670
calltrap() at calltrap+0x8/frame 0xfffffe0030ec9670
--- trap 0x9, rip = 0xffffffff80ccb525, rsp = 0xfffffe0030ec9740, rbp =
0xfffffe0030ec9740 ---
strncmp() at strncmp+0x15/frame 0xfffffe0030ec9740
ifunit_ref() at ifunit_ref+0x51/frame 0xfffffe0030ec9780
ifioctl() at ifioctl+0x508/frame 0xfffffe0030ec9850
kern_ioctl() at kern_ioctl+0x28a/frame 0xfffffe0030ec98c0
sys_ioctl() at sys_ioctl+0x15d/frame 0xfffffe0030ec9990
amd64_syscall() at amd64_syscall+0x276/frame 0xfffffe0030ec9ab0
fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0030ec9ab0
--- syscall (54, FreeBSD ELF64, sys_ioctl), rip = 0x80048531a, rsp =
0x7fffffffe458, rbp = 0x7fffffffe4c0 ---

-- 
You are receiving this mail because:
You are the assignee for the bug.