FreeBSD-main-amd64-test - Build #19874 - Still Failing

FreeBSD-main-amd64-test - Build #19874 (e383c423c492781bd7e7a0de9dfe433e4d6a4eed) - Still Failing

Build information: https://ci.FreeBSD.org/job/FreeBSD-main-amd64-test/19874/
Full change log: https://ci.FreeBSD.org/job/FreeBSD-main-amd64-test/19874/changes
Full build log: https://ci.FreeBSD.org/job/FreeBSD-main-amd64-test/19874/console

Status explanation:
"Failure" - the build is suspected being broken by the following changes
"Still Failing" - the build has not been fixed by the following changes and
                  this is a notification to note that these changes have
                  not been fully tested by the CI system

Change summaries:
(Those commits are likely but not certainly responsible)

5bbfa333cfc435c16cdfee20d84b17954972f610 by allanjude:
ipresend(1): Make the build slightly less broken.

d9bb798725cfce9c72b80440659b48e8668eb10d by allanjude:
openssl: Fix detection of ARMv7 and ARM64 CPU features

c0601c3e0162a7ccf6160b32cfa7596bba7bb869 by cy:
RELNOTES: Add comment about WiFi 6 support

22b1cbec8e9e0ad3311257e99ded781fedd5c871 by cy:
Revert "RELNOTES: Add comment about WiFi 6 support"

e383c423c492781bd7e7a0de9dfe433e4d6a4eed by cy:
RELNOTES: Add comment about WiFi 6 support to wpa



The end of the build log:

[...truncated 3.92 MB...]
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5d9db50
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5d9db70
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5d9dbb0
sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe00a5d9dc00
sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00a5d9dc10
_sleep() at _sleep+0x2dd/frame 0xfffffe00a5d9dcb0
audit_read() at audit_read+0x9c/frame 0xfffffe00a5d9dce0
devfs_read_f() at devfs_read_f+0xe5/frame 0xfffffe00a5d9dd40
dofileread() at dofileread+0x81/frame 0xfffffe00a5d9dd90
sys_read() at sys_read+0xc0/frame 0xfffffe00a5d9de00
amd64_syscall() at amd64_syscall+0x6b4/frame 0xfffffe00a5d9df30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00a5d9df30
--- syscall (3, FreeBSD ELF64, sys_read), rip = 0x3863b97af15a, rsp = 0x7fffff7d91e8, rbp = 0x7fffff7d9230 ---

Tracing command sh pid 725 tid 100090 td 0xfffffe00775ad3a0
sched_switch() at sched_switch+0x559/frame 0xfffffe0063b57a10
mi_switch() at mi_switch+0x157/frame 0xfffffe0063b57a30
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0063b57a70
sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0063b57ac0
sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0063b57ad0
_sleep() at _sleep+0x2dd/frame 0xfffffe0063b57b70
kern_wait6() at kern_wait6+0x474/frame 0xfffffe0063b57c00
sys_wait4() at sys_wait4+0x7d/frame 0xfffffe0063b57e00
amd64_syscall() at amd64_syscall+0x6b4/frame 0xfffffe0063b57f30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0063b57f30
--- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x11df5a80f33a, rsp = 0x7fffffe2e198, rbp = 0x7fffffe2e250 ---

Tracing command sh pid 738 tid 100100 td 0xfffffe00099881e0
sched_switch() at sched_switch+0x559/frame 0xfffffe00085ffa10
mi_switch() at mi_switch+0x157/frame 0xfffffe00085ffa30
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00085ffa70
sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe00085ffac0
sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00085ffad0
_sleep() at _sleep+0x2dd/frame 0xfffffe00085ffb70
kern_wait6() at kern_wait6+0x474/frame 0xfffffe00085ffc00
sys_wait4() at sys_wait4+0x7d/frame 0xfffffe00085ffe00
amd64_syscall() at amd64_syscall+0x6b4/frame 0xfffffe00085fff30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00085fff30
--- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x21ca7f21433a, rsp = 0x7fffff576d78, rbp = 0x7fffff576e30 ---

Tracing command sh pid 745 tid 100110 td 0xfffffe009db5d020
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5daca10
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5daca30
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5daca70
sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe00a5dacac0
sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00a5dacad0
_sleep() at _sleep+0x2dd/frame 0xfffffe00a5dacb70
kern_wait6() at kern_wait6+0x474/frame 0xfffffe00a5dacc00
sys_wait4() at sys_wait4+0x7d/frame 0xfffffe00a5dace00
amd64_syscall() at amd64_syscall+0x6b4/frame 0xfffffe00a5dacf30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00a5dacf30
--- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x184ebed6633a, rsp = 0x7fffff66a8b8, rbp = 0x7fffff66a970 ---

Tracing command kyua pid 746 tid 100108 td 0xfffffe00775d1ac0
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5db6a10
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5db6a30
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5db6a70
sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe00a5db6ac0
sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00a5db6ad0
_sleep() at _sleep+0x2dd/frame 0xfffffe00a5db6b70
kern_wait6() at kern_wait6+0x474/frame 0xfffffe00a5db6c00
sys_wait4() at sys_wait4+0x7d/frame 0xfffffe00a5db6e00
amd64_syscall() at amd64_syscall+0x6b4/frame 0xfffffe00a5db6f30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00a5db6f30
--- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0xa8569e1a33a, rsp = 0x7fffff2e77f8, rbp = 0x7fffff2e7830 ---

Tracing command ng_queue pid 52036 tid 100127 td 0xfffffe009db5a560
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5d8ed80
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5d8eda0
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5d8ede0
_sleep() at _sleep+0x2e7/frame 0xfffffe00a5d8ee80
ngthread() at ngthread+0x88/frame 0xfffffe00a5d8eef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a5d8ef30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a5d8ef30
--- trap 0xc, rip = 0xc0c6ffa1d9a, rsp = 0x7fffff603f48, rbp = 0x7fffff604040 ---

Tracing command ng_queue pid 52036 tid 100865 td 0xfffffe00a6a29900
sched_switch() at sched_switch+0x559/frame 0xfffffe0063bc5d80
mi_switch() at mi_switch+0x157/frame 0xfffffe0063bc5da0
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0063bc5de0
_sleep() at _sleep+0x2e7/frame 0xfffffe0063bc5e80
ngthread() at ngthread+0x88/frame 0xfffffe0063bc5ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe0063bc5f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0063bc5f30
--- trap 0x16, rip = 0x2bf05366d9a, rsp = 0x604a88, rbp = 0x604aa0 ---

Tracing command kyua pid 99383 tid 101034 td 0xfffffe00aa6793a0
sched_switch() at sched_switch+0x559/frame 0xfffffe00aa912520
mi_switch() at mi_switch+0x157/frame 0xfffffe00aa912540
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00aa912580
_sleep() at _sleep+0x2e7/frame 0xfffffe00aa912620
bufwait() at bufwait+0x84/frame 0xfffffe00aa912660
breadn_flags() at breadn_flags+0x1cb/frame 0xfffffe00aa9126d0
vfs_bio_getpages() at vfs_bio_getpages+0x354/frame 0xfffffe00aa9127c0
ffs_getpages() at ffs_getpages+0x77/frame 0xfffffe00aa9127e0
vnode_pager_getpages() at vnode_pager_getpages+0x37/frame 0xfffffe00aa912820
vm_pager_get_pages() at vm_pager_get_pages+0x5f/frame 0xfffffe00aa912870
vm_page_grab_valid() at vm_page_grab_valid+0x4cc/frame 0xfffffe00aa912980
vm_page_grab_valid_unlocked() at vm_page_grab_valid_unlocked+0xa5/frame 0xfffffe00aa9129c0
exec_map_first_page() at exec_map_first_page+0xa7/frame 0xfffffe00aa9129f0
kern_execve() at kern_execve+0x45c/frame 0xfffffe00aa912d80
sys_execve() at sys_execve+0x5a/frame 0xfffffe00aa912e00
amd64_syscall() at amd64_syscall+0x6b4/frame 0xfffffe00aa912f30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00aa912f30
--- syscall (59, FreeBSD ELF64, sys_execve), rip = 0xa8569e1741a, rsp = 0x7fffff2e7368, rbp = 0x7fffff2e7370 ---

Tracing command g_mp_kt pid 44124 tid 100827 td 0xfffffe00a6a26720
sched_switch() at sched_switch+0x559/frame 0xfffffe00a6c04da0
mi_switch() at mi_switch+0x157/frame 0xfffffe00a6c04dc0
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a6c04e00
_sleep() at _sleep+0x2e7/frame 0xfffffe00a6c04ea0
g_multipath_kt() at g_multipath_kt+0x275/frame 0xfffffe00a6c04ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a6c04f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a6c04f30
--- trap 0xc, rip = 0x349732587d9a, rsp = 0x7fffffc0afa8, rbp = 0x7fffffc0afc0 ---

Tracing command aiod4 pid 1395 tid 100092 td 0xfffffe00775ac560
sched_switch() at sched_switch+0x559/frame 0xfffffe0063b4dd60
mi_switch() at mi_switch+0x157/frame 0xfffffe0063b4dd80
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0063b4ddc0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe0063b4de00
_sleep() at _sleep+0x2ce/frame 0xfffffe0063b4dea0
aio_daemon() at aio_daemon+0x305/frame 0xfffffe0063b4def0
fork_exit() at fork_exit+0x80/frame 0xfffffe0063b4df30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0063b4df30
--- trap 0xc, rip = 0x627bfd93d9a, rsp = 0x7ffffffdd278, rbp = 0x7ffffffdd290 ---

Tracing command aiod8 pid 1399 tid 100111 td 0xfffffe009db5c900
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5da7d60
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5da7d80
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5da7dc0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00a5da7e00
_sleep() at _sleep+0x2ce/frame 0xfffffe00a5da7ea0
aio_daemon() at aio_daemon+0x305/frame 0xfffffe00a5da7ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a5da7f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a5da7f30
--- trap 0xc, rip = 0x25796bc09d9a, rsp = 0x7fffff84c338, rbp = 0x7fffff84c350 ---

Tracing command aiod18 pid 1409 tid 100122 td 0xfffffe009dbc7e40
sched_switch() at sched_switch+0x559/frame 0xfffffe00773e6d60
mi_switch() at mi_switch+0x157/frame 0xfffffe00773e6d80
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00773e6dc0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00773e6e00
_sleep() at _sleep+0x2ce/frame 0xfffffe00773e6ea0
aio_daemon() at aio_daemon+0x305/frame 0xfffffe00773e6ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00773e6f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00773e6f30
--- trap 0xc, rip = 0xa8569e19d9a, rsp = 0x7fffff2e7968, rbp = 0x7fffff2e7970 ---

Tracing command aiod23 pid 1414 tid 100123 td 0xfffffe009db5ac80
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5d93d60
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5d93d80
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5d93dc0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00a5d93e00
_sleep() at _sleep+0x2ce/frame 0xfffffe00a5d93ea0
aio_daemon() at aio_daemon+0x305/frame 0xfffffe00a5d93ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a5d93f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a5d93f30
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---

Tracing command soaiod1 pid 1430 tid 100134 td 0xfffffe009dbf11e0
sched_switch() at sched_switch+0x559/frame 0xfffffe009d974d70
mi_switch() at mi_switch+0x157/frame 0xfffffe009d974d90
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe009d974dd0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe009d974e10
_sleep() at _sleep+0x2ce/frame 0xfffffe009d974eb0
soaio_kproc_loop() at soaio_kproc_loop+0x125/frame 0xfffffe009d974ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe009d974f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009d974f30
--- trap 0xc, rip = 0x3d1e13b88d9a, rsp = 0x7fffff9aaba8, rbp = 0x7fffff9aabc0 ---

Tracing command soaiod2 pid 1431 tid 100133 td 0xfffffe009dbf1900
sched_switch() at sched_switch+0x559/frame 0xfffffe009d979d70
mi_switch() at mi_switch+0x157/frame 0xfffffe009d979d90
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe009d979dd0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe009d979e10
_sleep() at _sleep+0x2ce/frame 0xfffffe009d979eb0
soaio_kproc_loop() at soaio_kproc_loop+0x125/frame 0xfffffe009d979ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe009d979f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009d979f30
--- trap 0xc, rip = 0xa7f31379d9a, rsp = 0x7fffff76dfa8, rbp = 0x7fffff76dfc0 ---

Tracing command soaiod3 pid 1432 tid 100132 td 0xfffffe009dbf2020
sched_switch() at sched_switch+0x559/frame 0xfffffe009d97ed70
mi_switch() at mi_switch+0x157/frame 0xfffffe009d97ed90
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe009d97edd0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe009d97ee10
_sleep() at _sleep+0x2ce/frame 0xfffffe009d97eeb0
soaio_kproc_loop() at soaio_kproc_loop+0x125/frame 0xfffffe009d97eef0
fork_exit() at fork_exit+0x80/frame 0xfffffe009d97ef30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009d97ef30
--- trap 0xc, rip = 0x18cb7d0add9a, rsp = 0x7fffffd93718, rbp = 0x7fffffd93730 ---

Tracing command soaiod4 pid 1433 tid 100131 td 0xfffffe009dbf2740
sched_switch() at sched_switch+0x559/frame 0xfffffe009d983d70
mi_switch() at mi_switch+0x157/frame 0xfffffe009d983d90
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe009d983dd0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe009d983e10
_sleep() at _sleep+0x2ce/frame 0xfffffe009d983eb0
soaio_kproc_loop() at soaio_kproc_loop+0x125/frame 0xfffffe009d983ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe009d983f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009d983f30
--- trap 0xc, rip = 0x32a41494ad9a, rsp = 0x7fffffdec268, rbp = 0x7fffffdec280 ---
db:0:kdb.enter.panic> dump
Dumping 460 out of 4058 MB:..4%..11%..21%..32%..42%..53%..63%..73%..84%..91%
Dump complete
db:0:kdb.enter.panic> reset
+ rc=0
+ echo 'bhyve return code = 0'
bhyve return code = 0
+ sudo /usr/sbin/bhyvectl '--vm=testvm-main-amd64-19874' --destroy
+ sh -ex freebsd-ci/scripts/test/extract-meta.sh
+ METAOUTDIR=meta-out
+ rm -fr meta-out
+ mkdir meta-out
+ tar xvf meta.tar -C meta-out
x ./
x ./disable-zfs-tests.sh
x ./run.sh
x ./run-kyua.sh
x ./disable-dtrace-tests.sh
x ./auto-shutdown
x ./disable-notyet-tests.sh
+ rm -f 'test-report.*'
+ mv 'meta-out/test-report.*' .
mv: rename meta-out/test-report.* to ./test-report.*: No such file or directory
+ report=test-report.xml
+ [ -e freebsd-ci/jobs/FreeBSD-main-amd64-test/xfail-list -a -e test-report.xml ]
+ rm -f disk-cam
+ jot 5
+ rm -f disk1
+ rm -f disk2
+ rm -f disk3
+ rm -f disk4
+ rm -f disk5
+ rm -f disk-test.img
[PostBuildScript] - [INFO] Executing post build scripts.
[FreeBSD-main-amd64-test] $ /bin/sh -xe /tmp/jenkins3458854588271175877.sh
+ ./freebsd-ci/artifact/post-link.py
Post link: {'job_name': 'FreeBSD-main-amd64-test', 'commit': 'e383c423c492781bd7e7a0de9dfe433e4d6a4eed', 'branch': 'main', 'target': 'amd64', 'target_arch': 'amd64', 'link_type': 'latest_tested'}
"Link created: main/latest_tested/amd64/amd64 -> ../../e383c423c492781bd7e7a0de9dfe433e4d6a4eed/amd64/amd64\n"
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Checking for post-build
Performing post-build step
Checking if email needs to be generated
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Sending mail from default account using System Admin e-mail address