FreeBSD-main-amd64-test - Build #28023 - Still Failing

FreeBSD-main-amd64-test - Build #28023 (815ef05284d184d83613adcf60def403607be6a0) - Still Failing

Build information: https://ci.FreeBSD.org/job/FreeBSD-main-amd64-test/28023/
Full change log: https://ci.FreeBSD.org/job/FreeBSD-main-amd64-test/28023/changes
Full build log: https://ci.FreeBSD.org/job/FreeBSD-main-amd64-test/28023/console

Status explanation:
"Failure" - the build is suspected being broken by the following changes
"Still Failing" - the build has not been fixed by the following changes and
                  this is a notification to note that these changes have
                  not been fully tested by the CI system

Change summaries:
(Those commits are likely but not certainly responsible)

25cc459286a02b646751541ccde5a33319471c73 by emaste:
shm: Zero struct kinfo_file in sysctl handler

ce9aff829e02c9a21c04eae77a45f2193d1ed5a1 by mhorne:
hwpmc_amd: fix amd_get_msr() MSR offset for newer counter bases

7e68af7ce2c1b892954df415774fe59fd2f1b62f by asomers:
fusefs: redo vnode attribute locking

72472e52e310ec348949a3a67d3fa17e33fb8e50 by glebius:
carp: retire ioctl(2) API

985ac741384ec65463669edee5e1d90dee4c895a by glebius:
systat: remove kvm(3) support for -netstat mode

a68e3a8ae8401fe3ba6c0a85bbd3de87bd2e36f2 by glebius:
systm.h: don't declare socket and inpcb globally

c0462c2deafdcfe885e8d6f91b529d8cbddc6014 by glebius:
tcp: make sack_filter.c compilable without _WANT_TCPCB

815ef05284d184d83613adcf60def403607be6a0 by glebius:
netinet: remove _WANT_INPCB and _WANT_TCPCB



The end of the build log:

[...truncated 4.53 MiB...]
#10 0xffffffff80bdaac6 at userland_sysctl+0x1b6
#11 0xffffffff80bda8d5 at sys___sysctl+0x65
#12 0xffffffff8112ad61 at amd64_syscall+0x451
#13 0xffffffff810f9edb at fast_syscall_common+0xf8
uma_zalloc_debug: zone "MAP ENTRY" with 1 sleep inhibitors
stack backtrace:
#0 0xffffffff80c4404c at witness_debugger+0x6c
#1 0xffffffff80c45959 at witness_warn+0x4c9
#2 0xffffffff80f99964 at uma_zalloc_debug+0x34
#3 0xffffffff80f994b7 at uma_zalloc_arg+0x27
#4 0xffffffff80fb37b1 at vm_map_entry_clone+0x1a1
#5 0xffffffff80fad76b at vm_map_clip_end+0xcb
#6 0xffffffff80faf5a2 at vm_map_wire_locked+0x142
#7 0xffffffff80faf407 at vm_map_wire+0x67
#8 0xffffffff80fa6605 at vslock+0x75
#9 0xffffffff80bda6aa at sysctl_wire_old_buffer+0x4a
#10 0xffffffff80e19023 at sysctl_ip6_mcast_filters+0x93
#11 0xffffffff80bdb05c at sysctl_root_handler_locked+0x9c
#12 0xffffffff80bda3bf at sysctl_root+0x22f
#13 0xffffffff80bdaac6 at userland_sysctl+0x1b6
#14 0xffffffff80bda8d5 at sys___sysctl+0x65
#15 0xffffffff8112ad61 at amd64_syscall+0x451
#16 0xffffffff810f9edb at fast_syscall_common+0xf8
vm_map_clip_start: map 0xfffff80217f12248 entry 0xfffff801ffb06c60 start 0x3b8e46424000 with 1 sleep inhibitors
stack backtrace:
#0 0xffffffff80c4404c at witness_debugger+0x6c
#1 0xffffffff80c45959 at witness_warn+0x4c9
#2 0xffffffff80fad5e2 at vm_map_clip_start+0x42
#3 0xffffffff80faf58c at vm_map_wire_locked+0x12c
#4 0xffffffff80faf407 at vm_map_wire+0x67
#5 0xffffffff80fa6605 at vslock+0x75
#6 0xffffffff80bda6aa at sysctl_wire_old_buffer+0x4a
#7 0xffffffff80e19023 at sysctl_ip6_mcast_filters+0x93
#8 0xffffffff80bdb05c at sysctl_root_handler_locked+0x9c
#9 0xffffffff80bda3bf at sysctl_root+0x22f
#10 0xffffffff80bdaac6 at userland_sysctl+0x1b6
#11 0xffffffff80bda8d5 at sys___sysctl+0x65
#12 0xffffffff8112ad61 at amd64_syscall+0x451
#13 0xffffffff810f9edb at fast_syscall_common+0xf8
uma_zalloc_debug: zone "MAP ENTRY" with 1 sleep inhibitors
stack backtrace:
#0 0xffffffff80c4404c at witness_debugger+0x6c
#1 0xffffffff80c45959 at witness_warn+0x4c9
#2 0xffffffff80f99964 at uma_zalloc_debug+0x34
#3 0xffffffff80f994b7 at uma_zalloc_arg+0x27
#4 0xffffffff80fb37b1 at vm_map_entry_clone+0x1a1
#5 0xffffffff80fad66b at vm_map_clip_start+0xcb
#6 0xffffffff80faf58c at vm_map_wire_locked+0x12c
#7 0xffffffff80faf407 at vm_map_wire+0x67
#8 0xffffffff80fa6605 at vslock+0x75
#9 0xffffffff80bda6aa at sysctl_wire_old_buffer+0x4a
#10 0xffffffff80e19023 at sysctl_ip6_mcast_filters+0x93
#11 0xffffffff80bdb05c at sysctl_root_handler_locked+0x9c
#12 0xffffffff80bda3bf at sysctl_root+0x22f
#13 0xffffffff80bdaac6 at userland_sysctl+0x1b6
#14 0xffffffff80bda8d5 at sys___sysctl+0x65
#15 0xffffffff8112ad61 at amd64_syscall+0x451
#16 0xffffffff810f9edb at fast_syscall_common+0xf8
vm_map_clip_end: map 0xfffff80217f12248 entry 0xfffff801ffb06c60 end 0x3b8e46425000 with 1 sleep inhibitors
stack backtrace:
#0 0xffffffff80c4404c at witness_debugger+0x6c
#1 0xffffffff80c45959 at witness_warn+0x4c9
#2 0xffffffff80fad6e2 at vm_map_clip_end+0x42
#3 0xffffffff80faf5a2 at vm_map_wire_locked+0x142
#4 0xffffffff80faf407 at vm_map_wire+0x67
#5 0xffffffff80fa6605 at vslock+0x75
#6 0xffffffff80bda6aa at sysctl_wire_old_buffer+0x4a
#7 0xffffffff80e19023 at sysctl_ip6_mcast_filters+0x93
#8 0xffffffff80bdb05c at sysctl_root_handler_locked+0x9c
#9 0xffffffff80bda3bf at sysctl_root+0x22f
#10 0xffffffff80bdaac6 at userland_sysctl+0x1b6
#11 0xffffffff80bda8d5 at sys___sysctl+0x65
#12 0xffffffff8112ad61 at amd64_syscall+0x451
#13 0xffffffff810f9edb at fast_syscall_common+0xf8
uma_zalloc_debug: zone "MAP ENTRY" with 1 sleep inhibitors
stack backtrace:
#0 0xffffffff80c4404c at witness_debugger+0x6c
#1 0xffffffff80c45959 at witness_warn+0x4c9
#2 0xffffffff80f99964 at uma_zalloc_debug+0x34
#3 0xffffffff80f994b7 at uma_zalloc_arg+0x27
#4 0xffffffff80fb37b1 at vm_map_entry_clone+0x1a1
#5 0xffffffff80fad76b at vm_map_clip_end+0xcb
#6 0xffffffff80faf5a2 at vm_map_wire_locked+0x142
#7 0xffffffff80faf407 at vm_map_wire+0x67
#8 0xffffffff80fa6605 at vslock+0x75
#9 0xffffffff80bda6aa at sysctl_wire_old_buffer+0x4a
#10 0xffffffff80e19023 at sysctl_ip6_mcast_filters+0x93
#11 0xffffffff80bdb05c at sysctl_root_handler_locked+0x9c
#12 0xffffffff80bda3bf at sysctl_root+0x22f
#13 0xffffffff80bdaac6 at userland_sysctl+0x1b6
#14 0xffffffff80bda8d5 at sys___sysctl+0x65
#15 0xffffffff8112ad61 at amd64_syscall+0x451
#16 0xffffffff810f9edb at fast_syscall_common+0xf8
vm_map_clip_start: map 0xfffff80217f12248 entry 0xfffff801c93c2d80 start 0x3b8e46424000 with 1 sleep inhibitors
stack backtrace:
#0 0xffffffff80c4404c at witness_debugger+0x6c
#1 0xffffffff80c45959 at witness_warn+0x4c9
#2 0xffffffff80fad5e2 at vm_map_clip_start+0x42
#3 0xffffffff80faf58c at vm_map_wire_locked+0x12c
#4 0xffffffff80faf407 at vm_map_wire+0x67
#5 0xffffffff80fa6605 at vslock+0x75
#6 0xffffffff80bda6aa at sysctl_wire_old_buffer+0x4a
#7 0xffffffff80e19023 at sysctl_ip6_mcast_filters+0x93
#8 0xffffffff80bdb05c at sysctl_root_handler_locked+0x9c
#9 0xffffffff80bda3bf at sysctl_root+0x22f
#10 0xffffffff80bdaac6 at userland_sysctl+0x1b6
#11 0xffffffff80bda8d5 at sys___sysctl+0x65
#12 0xffffffff8112ad61 at amd64_syscall+0x451
#13 0xffffffff810f9edb at fast_syscall_common+0xf8
uma_zalloc_debug: zone "MAP ENTRY" with 1 sleep inhibitors
stack backtrace:
#0 0xffffffff80c4404c at witness_debugger+0x6c
#1 0xffffffff80c45959 at witness_warn+0x4c9
#2 0xffffffff80f99964 at uma_zalloc_debug+0x34
#3 0xffffffff80f994b7 at uma_zalloc_arg+0x27
#4 0xffffffff80fb37b1 at vm_map_entry_clone+0x1a1
#5 0xffffffff80fad66b at vm_map_clip_start+0xcb
#6 0xffffffff80faf58c at vm_map_wire_locked+0x12c
#7 0xffffffff80faf407 at vm_map_wire+0x67
#8 0xffffffff80fa6605 at vslock+0x75
#9 0xffffffff80bda6aa at sysctl_wire_old_buffer+0x4a
#10 0xffffffff80e19023 at sysctl_ip6_mcast_filters+0x93
#11 0xffffffff80bdb05c at sysctl_root_handler_locked+0x9c
#12 0xffffffff80bda3bf at sysctl_root+0x22f
#13 0xffffffff80bdaac6 at userland_sysctl+0x1b6
#14 0xffffffff80bda8d5 at sys___sysctl+0x65
#15 0xffffffff8112ad61 at amd64_syscall+0x451
#16 0xffffffff810f9edb at fast_syscall_common+0xf8
vm_map_clip_end: map 0xfffff80217f12248 entry 0xfffff801c93c2d80 end 0x3b8e46425000 with 1 sleep inhibitors
stack backtrace:
#0 0xffffffff80c4404c at witness_debugger+0x6c
#1 0xffffffff80c45959 at witness_warn+0x4c9
#2 0xffffffff80fad6e2 at vm_map_clip_end+0x42
#3 0xffffffff80faf5a2 at vm_map_wire_locked+0x142
#4 0xffffffff80faf407 at vm_map_wire+0x67
#5 0xffffffff80fa6605 at vslock+0x75
#6 0xffffffff80bda6aa at sysctl_wire_old_buffer+0x4a
#7 0xffffffff80e19023 at sysctl_ip6_mcast_filters+0x93
#8 0xffffffff80bdb05c at sysctl_root_handler_locked+0x9c
#9 0xffffffff80bda3bf at sysctl_root+0x22f
#10 0xffffffff80bdaac6 at userland_sysctl+0x1b6
#11 0xffffffff80bda8d5 at sys___sysctl+0x65
#12 0xffffffff8112ad61 at amd64_syscall+0x451
#13 0xffffffff810f9edb at fast_syscall_common+0xf8
uma_zalloc_debug: zone "MAP ENTRY" with 1 sleep inhibitors
stack backtrace:
#0 0xffffffff80c4404c at witness_debugger+0x6c
#1 0xffffffff80c45959 at witness_warn+0x4c9
#2 0xffffffff80f99964 at uma_zalloc_debug+0x34
#3 0xffffffff80f994b7 at uma_zalloc_arg+0x27
#4 0xffffffff80fb37b1 at vm_map_entry_clone+0x1a1
#5 0xffffffff80fad76b at vm_map_clip_end+0xcb
#6 0xffffffff80faf5a2 at vm_map_wire_locked+0x142
#7 0xffffffff80faf407 at vm_map_wire+0x67
#8 0xffffffff80fa6605 at vslock+0x75
#9 0xffffffff80bda6aa at sysctl_wire_old_buffer+0x4a
#10 0xffffffff80e19023 at sysctl_ip6_mcast_filters+0x93
#11 0xffffffff80bdb05c at sysctl_root_handler_locked+0x9c
#12 0xffffffff80bda3bf at sysctl_root+0x22f
#13 0xffffffff80bdaac6 at userland_sysctl+0x1b6
#14 0xffffffff80bda8d5 at sys___sysctl+0x65
#15 0xffffffff8112ad61 at amd64_syscall+0x451
#16 0xffffffff810f9edb at fast_syscall_common+0xf8
epair0a: link state changed to DOWN
epair0b: link state changed to DOWN
failed: atf-check failed; see the output of the test for details  [5.042s]
sys/netinet6/redirect:valid_redirect  ->  epair0a: Ethernet address: 92:0f:fa:ea:73:c6
epair0b: Ethernet address: 96:f2:79:30:f3:fd
epair0a: link state changed to UP
epair0b: link state changed to UP
panic: CURVNET_SET at /usr/src/sys/netinet6/nd6_nbr.c:1680 nd6_queue_timer() curvnet=0 vnet=0xfffff8000d121a40
cpuid = 1
time = 1773341283
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe008bc5dc00
vpanic() at vpanic+0x136/frame 0xfffffe008bc5dd30
panic() at panic+0x43/frame 0xfffffe008bc5dd90
nd6_queue_timer() at nd6_queue_timer+0x207/frame 0xfffffe008bc5de10
softclock_call_cc() at softclock_call_cc+0x19b/frame 0xfffffe008bc5dec0
softclock_thread() at softclock_thread+0xc6/frame 0xfffffe008bc5def0
fork_exit() at fork_exit+0x82/frame 0xfffffe008bc5df30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe008bc5df30
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---
KDB: enter: panic
[ thread pid 2 tid 100031 ]
Stopped at      kdb_enter+0x33: movq    $0,0x15ece52(%rip)
db:0:kdb.enter.panic> show pcpu
cpuid        = 1
dynamic pcpu = 0xfffffe008d3bdd80
curthread    = 0xfffff80102422780: pid 2 tid 100031 critnest 1 "clock (0)"
curpcb       = 0xfffff80102422cd0
fpcurthread  = none
idlethread   = 0xfffff8010240c000: tid 100004 "idle: cpu1"
self         = 0xffffffff82a11000
curpmap      = 0xffffffff81da1d00
tssp         = 0xffffffff82a11384
rsp0         = 0xfffffe008bc5e000
kcr3         = 0x8000000002944002
ucr3         = 0xffffffffffffffff
scr3         = 0x203278b16
gs32p        = 0xffffffff82a11404
ldt          = 0xffffffff82a11444
tss          = 0xffffffff82a11434
curvnet      = 0
spin locks held:
db:0:kdb.enter.panic>  reset
Uptime: 52m19s
+ rc=0
+ echo 'bhyve return code = 0'
bhyve return code = 0
+ sudo /usr/sbin/bhyvectl '--vm=testvm-main-amd64-28023' --destroy
+ sh -ex freebsd-ci/scripts/test/extract-meta.sh
+ METAOUTDIR=meta-out
+ rm -fr meta-out
+ mkdir meta-out
+ tar xvf meta.tar -C meta-out
x ./
x ./disable-dtrace-tests.sh
x ./auto-shutdown
x ./run.sh
x ./disable-zfs-tests.sh
x ./disable-notyet-tests.sh
x ./run-kyua.sh
+ rm -f 'test-report.*'
+ mv 'meta-out/test-report.*' .
mv: rename meta-out/test-report.* to ./test-report.*: No such file or directory
+ report=test-report.xml
+ [ -e freebsd-ci/jobs/FreeBSD-main-amd64-test/xfail-list -a -e test-report.xml ]
+ rm -f disk-cam
+ jot 5
+ rm -f disk1
+ rm -f disk2
+ rm -f disk3
+ rm -f disk4
+ rm -f disk5
+ rm -f disk-test.img
[PostBuildScript] - [INFO] Executing post build scripts.
[FreeBSD-main-amd64-test] $ /bin/sh -xe /tmp/jenkins8432049838143243878.sh
+ ./freebsd-ci/artifact/post-link.py
Post link: {'job_name': 'FreeBSD-main-amd64-test', 'commit': '815ef05284d184d83613adcf60def403607be6a0', 'branch': 'main', 'target': 'amd64', 'target_arch': 'amd64', 'link_type': 'latest_tested'}
"Link created: main/latest_tested/amd64/amd64 -> ../../815ef05284d184d83613adcf60def403607be6a0/amd64/amd64\n"
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Checking for post-build
Performing post-build step
Checking if email needs to be generated
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Sending mail from default account using System Admin e-mail address