FreeBSD-main-amd64-test - Build #19981 - Failure

From: <>
Date: Fri, 03 Dec 2021 19:10:44 UTC
FreeBSD-main-amd64-test - Build #19981 (ae04d30451056f16096cba7d8debcb15dac275d7) - Failure

Build information:
Full change log:
Full build log:

Status explanation:
"Failure" - the build is suspected being broken by the following changes
"Still Failing" - the build has not been fixed by the following changes and
                  this is a notification to note that these changes have
                  not been fully tested by the CI system

Change summaries:
(Those commits are likely but not certainly responsible)

5f034a005435815486844bf3b3fed6b7e4ecdd09 by allanjude:
fstab.5: spell encryption correctly

ae04d30451056f16096cba7d8debcb15dac275d7 by glebius:
ng_l2tp: use callout_reset() instead of ng_callout()

The end of the build log:

[...truncated 3.95 MB...]
sched_switch() at sched_switch+0x559/frame 0xfffffe0063b72a60
mi_switch() at mi_switch+0x157/frame 0xfffffe0063b72a80
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0063b72ac0
sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0063b72b10
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12/frame 0xfffffe0063b72b50
_cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x19a/frame 0xfffffe0063b72bd0
kern_select() at kern_select+0xa6f/frame 0xfffffe0063b72dc0
sys_select() at sys_select+0x56/frame 0xfffffe0063b72e00
amd64_syscall() at amd64_syscall+0x6b4/frame 0xfffffe0063b72f30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0063b72f30
--- syscall (93, FreeBSD ELF64, sys_select), rip = 0x28b1fa, rsp = 0x7fffffb86878, rbp = 0x7fffffb88a10 ---

Tracing command auditd pid 679 tid 100095 td 0xfffffe00767d2c80
sched_switch() at sched_switch+0x559/frame 0xfffffe0063bd6b50
mi_switch() at mi_switch+0x157/frame 0xfffffe0063bd6b70
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0063bd6bb0
sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0063bd6c00
sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0063bd6c10
_sleep() at _sleep+0x2dd/frame 0xfffffe0063bd6cb0
audit_read() at audit_read+0x9c/frame 0xfffffe0063bd6ce0
devfs_read_f() at devfs_read_f+0xe5/frame 0xfffffe0063bd6d40
dofileread() at dofileread+0x81/frame 0xfffffe0063bd6d90
sys_read() at sys_read+0xc0/frame 0xfffffe0063bd6e00
amd64_syscall() at amd64_syscall+0x6b4/frame 0xfffffe0063bd6f30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0063bd6f30
--- syscall (3, FreeBSD ELF64, sys_read), rip = 0x359a9767f45a, rsp = 0x7ffffff787e8, rbp = 0x7ffffff78830 ---

Tracing command atf-sh pid 19134 tid 101214 td 0xfffffe00aa7f51e0 (CPU 1)
cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00097cde00
ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe00097cde10
trap() at trap+0x56/frame 0xfffffe00097cdf20
nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00097cdf20
--- trap 0x13, rip = 0x441bd1130141, rsp = 0x7fffffb50ad0, rbp = 0x7fffffb50b70 ---

Tracing command sh pid 731 tid 100089 td 0xfffffe00767b33a0
sched_switch() at sched_switch+0x559/frame 0xfffffe0063b8ba10
mi_switch() at mi_switch+0x157/frame 0xfffffe0063b8ba30
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0063b8ba70
sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0063b8bac0
sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0063b8bad0
_sleep() at _sleep+0x2dd/frame 0xfffffe0063b8bb70
kern_wait6() at kern_wait6+0x474/frame 0xfffffe0063b8bc00
sys_wait4() at sys_wait4+0x7d/frame 0xfffffe0063b8be00
amd64_syscall() at amd64_syscall+0x6b4/frame 0xfffffe0063b8bf30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0063b8bf30
--- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x3fd4fd3d763a, rsp = 0x7fffff439af8, rbp = 0x7fffff439bb0 ---

Tracing command sh pid 744 tid 100102 td 0xfffffe009db5f000
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5da3a10
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5da3a30
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5da3a70
sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe00a5da3ac0
sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00a5da3ad0
_sleep() at _sleep+0x2dd/frame 0xfffffe00a5da3b70
kern_wait6() at kern_wait6+0x474/frame 0xfffffe00a5da3c00
sys_wait4() at sys_wait4+0x7d/frame 0xfffffe00a5da3e00
amd64_syscall() at amd64_syscall+0x6b4/frame 0xfffffe00a5da3f30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00a5da3f30
--- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x2fc95853663a, rsp = 0x7fffff57a0f8, rbp = 0x7fffff57a1b0 ---

Tracing command sh pid 751 tid 100091 td 0xfffffe00767b2560
sched_switch() at sched_switch+0x559/frame 0xfffffe0063b81a10
mi_switch() at mi_switch+0x157/frame 0xfffffe0063b81a30
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0063b81a70
sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0063b81ac0
sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0063b81ad0
_sleep() at _sleep+0x2dd/frame 0xfffffe0063b81b70
kern_wait6() at kern_wait6+0x474/frame 0xfffffe0063b81c00
sys_wait4() at sys_wait4+0x7d/frame 0xfffffe0063b81e00
amd64_syscall() at amd64_syscall+0x6b4/frame 0xfffffe0063b81f30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0063b81f30
--- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x2619b2bce63a, rsp = 0x7fffff3a1638, rbp = 0x7fffff3a16f0 ---

Tracing command kyua pid 752 tid 100092 td 0xfffffe00767b1e40
sched_switch() at sched_switch+0x559/frame 0xfffffe0063b7ca10
mi_switch() at mi_switch+0x157/frame 0xfffffe0063b7ca30
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0063b7ca70
sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0063b7cac0
sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0063b7cad0
_sleep() at _sleep+0x2dd/frame 0xfffffe0063b7cb70
kern_wait6() at kern_wait6+0x474/frame 0xfffffe0063b7cc00
sys_wait4() at sys_wait4+0x7d/frame 0xfffffe0063b7ce00
amd64_syscall() at amd64_syscall+0x6b4/frame 0xfffffe0063b7cf30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0063b7cf30
--- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x1bb94c2fd63a, rsp = 0x7fffffc8c938, rbp = 0x7fffffc8c970 ---

Tracing command ng_queue pid 70632 tid 100118 td 0xfffffe009dbe4ac0
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5df8d80
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5df8da0
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5df8de0
_sleep() at _sleep+0x2e7/frame 0xfffffe00a5df8e80
ngthread() at ngthread+0x88/frame 0xfffffe00a5df8ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a5df8f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a5df8f30
--- trap 0xc, rip = 0x30f26d67109a, rsp = 0x7fffff6e0a28, rbp = 0x7fffff6e0b20 ---

Tracing command ng_queue pid 70632 tid 100882 td 0xfffffe00a6a65020
sched_switch() at sched_switch+0x559/frame 0xfffffe00a6c27d80
mi_switch() at mi_switch+0x157/frame 0xfffffe00a6c27da0
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a6c27de0
_sleep() at _sleep+0x2e7/frame 0xfffffe00a6c27e80
ngthread() at ngthread+0x88/frame 0xfffffe00a6c27ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a6c27f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a6c27f30
--- trap 0xc, rip = 0xa07cf42109a, rsp = 0x403f48, rbp = 0x403f60 ---

Tracing command g_mp_kt pid 62718 tid 100096 td 0xfffffe00767d2560
sched_switch() at sched_switch+0x559/frame 0xfffffe0063bd1da0
mi_switch() at mi_switch+0x157/frame 0xfffffe0063bd1dc0
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0063bd1e00
_sleep() at _sleep+0x2e7/frame 0xfffffe0063bd1ea0
g_multipath_kt() at g_multipath_kt+0x275/frame 0xfffffe0063bd1ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe0063bd1f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0063bd1f30
--- trap 0xc, rip = 0x969da26b09a, rsp = 0x7fffff772ab8, rbp = 0x7fffff772bb0 ---

Tracing command aiod2 pid 19987 tid 100108 td 0xfffffe00767b5020
sched_switch() at sched_switch+0x559/frame 0xfffffe00765f6d60
mi_switch() at mi_switch+0x157/frame 0xfffffe00765f6d80
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00765f6dc0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00765f6e00
_sleep() at _sleep+0x2ce/frame 0xfffffe00765f6ea0
aio_daemon() at aio_daemon+0x305/frame 0xfffffe00765f6ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00765f6f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00765f6f30
--- trap 0xc, rip = 0x2aa52be3509a, rsp = 0x7fffffc05cf8, rbp = 0x7fffffc05d10 ---

Tracing command aiod4 pid 19989 tid 100116 td 0xfffffe009db60560
sched_switch() at sched_switch+0x559/frame 0xfffffe00765ced60
mi_switch() at mi_switch+0x157/frame 0xfffffe00765ced80
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00765cedc0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00765cee00
_sleep() at _sleep+0x2ce/frame 0xfffffe00765ceea0
aio_daemon() at aio_daemon+0x305/frame 0xfffffe00765ceef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00765cef30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00765cef30
--- trap 0xc, rip = 0x11899fa6809a, rsp = 0x7ffffff463b8, rbp = 0x7ffffff463d0 ---

Tracing command aiod18 pid 20003 tid 100112 td 0xfffffe009db621e0
sched_switch() at sched_switch+0x559/frame 0xfffffe00765e2d60
mi_switch() at mi_switch+0x157/frame 0xfffffe00765e2d80
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00765e2dc0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00765e2e00
_sleep() at _sleep+0x2ce/frame 0xfffffe00765e2ea0
aio_daemon() at aio_daemon+0x305/frame 0xfffffe00765e2ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00765e2f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00765e2f30
--- trap 0xc, rip = 0x2a51737cf09a, rsp = 0x7fffffca56d8, rbp = 0x7fffffca56f0 ---

Tracing command aiod32 pid 20017 tid 100129 td 0xfffffe00767d33a0
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5d85d60
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5d85d80
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5d85dc0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00a5d85e00
_sleep() at _sleep+0x2ce/frame 0xfffffe00a5d85ea0
aio_daemon() at aio_daemon+0x305/frame 0xfffffe00a5d85ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a5d85f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a5d85f30
--- trap 0, rip = 0, rsp = 0, rbp = 0x1bbdbc1be671 ---

Tracing command soaiod1 pid 20024 tid 100133 td 0xfffffe009dbe6740
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5dbcd70
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5dbcd90
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5dbcdd0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00a5dbce10
_sleep() at _sleep+0x2ce/frame 0xfffffe00a5dbceb0
soaio_kproc_loop() at soaio_kproc_loop+0x125/frame 0xfffffe00a5dbcef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a5dbcf30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a5dbcf30
--- trap 0xc, rip = 0x40859b009a, rsp = 0x7fffff662988, rbp = 0x7fffff6629a0 ---

Tracing command soaiod2 pid 20025 tid 100132 td 0xfffffe00a6a50000
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5dc1d70
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5dc1d90
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5dc1dd0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00a5dc1e10
_sleep() at _sleep+0x2ce/frame 0xfffffe00a5dc1eb0
soaio_kproc_loop() at soaio_kproc_loop+0x125/frame 0xfffffe00a5dc1ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a5dc1f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a5dc1f30
--- trap 0xc, rip = 0x1724ae88609a, rsp = 0x7fffffa22fe8, rbp = 0x7fffffa23000 ---

Tracing command soaiod3 pid 20026 tid 100131 td 0xfffffe00a6a50720
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5dc6d70
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5dc6d90
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5dc6dd0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00a5dc6e10
_sleep() at _sleep+0x2ce/frame 0xfffffe00a5dc6eb0
soaio_kproc_loop() at soaio_kproc_loop+0x125/frame 0xfffffe00a5dc6ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a5dc6f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a5dc6f30
--- trap 0xc, rip = 0x3bf17f4a409a, rsp = 0x7fffffe3a028, rbp = 0x7fffffe3a040 ---

Tracing command soaiod4 pid 20027 tid 100130 td 0xfffffe00a6a50e40
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5dcbd70
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5dcbd90
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5dcbdd0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00a5dcbe10
_sleep() at _sleep+0x2ce/frame 0xfffffe00a5dcbeb0
soaio_kproc_loop() at soaio_kproc_loop+0x125/frame 0xfffffe00a5dcbef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a5dcbf30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a5dcbf30
--- trap 0xc, rip = 0x3e6d7c35909a, rsp = 0x7fffff43ee28, rbp = 0x7fffff43ee40 ---
db:0:kdb.enter.panic> dump
Dumping 463 out of 4060 MB:..4%..11%..21%..32%..42%..52%..63%..73%..83%..94%
Dump complete
db:0:kdb.enter.panic> reset
+ rc=0
+ echo 'bhyve return code = 0'
bhyve return code = 0
+ sudo /usr/sbin/bhyvectl '--vm=testvm-main-amd64-19981' --destroy
+ sh -ex freebsd-ci/scripts/test/
+ METAOUTDIR=meta-out
+ rm -fr meta-out
+ mkdir meta-out
+ tar xvf meta.tar -C meta-out
x ./
x ./
x ./
x ./
x ./
x ./
x ./auto-shutdown
+ rm -f test-report.txt test-report.xml
+ mv 'meta-out/test-report.*' .
mv: rename meta-out/test-report.* to ./test-report.*: No such file or directory
+ report=test-report.xml
+ [ -e freebsd-ci/jobs/FreeBSD-main-amd64-test/xfail-list -a -e test-report.xml ]
+ rm -f disk-cam
+ jot 5
+ rm -f disk1
+ rm -f disk2
+ rm -f disk3
+ rm -f disk4
+ rm -f disk5
+ rm -f disk-test.img
[PostBuildScript] - [INFO] Executing post build scripts.
[FreeBSD-main-amd64-test] $ /bin/sh -xe /tmp/
+ ./freebsd-ci/artifact/
Post link: {'job_name': 'FreeBSD-main-amd64-test', 'commit': 'ae04d30451056f16096cba7d8debcb15dac275d7', 'branch': 'main', 'target': 'amd64', 'target_arch': 'amd64', 'link_type': 'latest_tested'}
"Link created: main/latest_tested/amd64/amd64 -> ../../ae04d30451056f16096cba7d8debcb15dac275d7/amd64/amd64\n"
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Checking for post-build
Performing post-build step
Checking if email needs to be generated
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Sending mail from default account using System Admin e-mail address