FreeBSD-main-amd64-test - Build #20008 - Still Failing

From: <>
Date: Mon, 06 Dec 2021 08:37:56 UTC
FreeBSD-main-amd64-test - Build #20008 (1d92e81fa47925dab1fa3779b8e8e2e35d3c14ce) - Still Failing

Build information:
Full change log:
Full build log:

Status explanation:
"Failure" - the build is suspected being broken by the following changes
"Still Failing" - the build has not been fixed by the following changes and
                  this is a notification to note that these changes have
                  not been fully tested by the CI system

Change summaries:
(Those commits are likely but not certainly responsible)

1d92e81fa47925dab1fa3779b8e8e2e35d3c14ce by imp:
cam/iosched: fix off by one error

The end of the build log:

[...truncated 3.95 MB...]
--- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x2b58e9e1663a, rsp = 0x7fffff682018, rbp = 0x7fffff6820d0 ---

Tracing command sh pid 751 tid 100102 td 0xfffffe000998a1e0
sched_switch() at sched_switch+0x559/frame 0xfffffe00085ffa10
mi_switch() at mi_switch+0x157/frame 0xfffffe00085ffa30
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00085ffa70
sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe00085ffac0
sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00085ffad0
_sleep() at _sleep+0x2dd/frame 0xfffffe00085ffb70
kern_wait6() at kern_wait6+0x474/frame 0xfffffe00085ffc00
sys_wait4() at sys_wait4+0x7d/frame 0xfffffe00085ffe00
amd64_syscall() at amd64_syscall+0x6b4/frame 0xfffffe00085fff30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00085fff30
--- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x9ff824a863a, rsp = 0x7fffff92f6f8, rbp = 0x7fffff92f7b0 ---

Tracing command kyua pid 752 tid 100093 td 0xfffffe00767b3720
sched_switch() at sched_switch+0x559/frame 0xfffffe0063b77a10
mi_switch() at mi_switch+0x157/frame 0xfffffe0063b77a30
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe0063b77a70
sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0063b77ac0
sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0063b77ad0
_sleep() at _sleep+0x2dd/frame 0xfffffe0063b77b70
kern_wait6() at kern_wait6+0x474/frame 0xfffffe0063b77c00
sys_wait4() at sys_wait4+0x7d/frame 0xfffffe0063b77e00
amd64_syscall() at amd64_syscall+0x6b4/frame 0xfffffe0063b77f30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0063b77f30
--- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x2bd3e315063a, rsp = 0x7fffff21ec18, rbp = 0x7fffff21ec50 ---

Tracing command ng_queue pid 52033 tid 100118 td 0xfffffe009db61e40
sched_switch() at sched_switch+0x559/frame 0xfffffe00765c1d80
mi_switch() at mi_switch+0x157/frame 0xfffffe00765c1da0
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00765c1de0
_sleep() at _sleep+0x2e7/frame 0xfffffe00765c1e80
ngthread() at ngthread+0x88/frame 0xfffffe00765c1ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00765c1f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00765c1f30
--- trap 0xc, rip = 0x3039e2b6d09a, rsp = 0x7fffff93c698, rbp = 0x7fffff93c790 ---

Tracing command ng_queue pid 52033 tid 100882 td 0xfffffe00a6b243a0
sched_switch() at sched_switch+0x559/frame 0xfffffe00a6c27d80
mi_switch() at mi_switch+0x157/frame 0xfffffe00a6c27da0
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a6c27de0
_sleep() at _sleep+0x2e7/frame 0xfffffe00a6c27e80
ngthread() at ngthread+0x88/frame 0xfffffe00a6c27ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a6c27f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a6c27f30
--- trap 0xc, rip = 0x1b3f25b4709a, rsp = 0x403f48, rbp = 0x403f60 ---

Tracing command g_mp_kt pid 44119 tid 100106 td 0xfffffe00767d3740
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5d9eda0
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5d9edc0
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5d9ee00
_sleep() at _sleep+0x2e7/frame 0xfffffe00a5d9eea0
g_multipath_kt() at g_multipath_kt+0x275/frame 0xfffffe00a5d9eef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a5d9ef30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a5d9ef30
--- trap 0xc, rip = 0x324a5dbcb09a, rsp = 0x7fffffa3a3a8, rbp = 0x7fffffa3a3c0 ---

Tracing command aiod22 pid 1419 tid 100126 td 0xfffffe009db65740
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5dd5d60
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5dd5d80
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5dd5dc0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00a5dd5e00
_sleep() at _sleep+0x2ce/frame 0xfffffe00a5dd5ea0
aio_daemon() at aio_daemon+0x305/frame 0xfffffe00a5dd5ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a5dd5f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a5dd5f30
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---

Tracing command aiod28 pid 1425 tid 100130 td 0xfffffe00a6a22000
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5dc1d60
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5dc1d80
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5dc1dc0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00a5dc1e00
_sleep() at _sleep+0x2ce/frame 0xfffffe00a5dc1ea0
aio_daemon() at aio_daemon+0x305/frame 0xfffffe00a5dc1ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a5dc1f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a5dc1f30
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---

Tracing command aiod29 pid 1426 tid 100131 td 0xfffffe009dbe2740
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5dbcd60
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5dbcd80
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5dbcdc0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00a5dbce00
_sleep() at _sleep+0x2ce/frame 0xfffffe00a5dbcea0
aio_daemon() at aio_daemon+0x305/frame 0xfffffe00a5dbcef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a5dbcf30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a5dbcf30
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---

Tracing command aiod32 pid 1429 tid 100133 td 0xfffffe009dbe1900
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5db2d60
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5db2d80
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5db2dc0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00a5db2e00
_sleep() at _sleep+0x2ce/frame 0xfffffe00a5db2ea0
aio_daemon() at aio_daemon+0x305/frame 0xfffffe00a5db2ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a5db2f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a5db2f30
--- trap 0x700, rip = 0xe00, rsp = 0x1800, rbp = 0x200 ---

Tracing command soaiod1 pid 1436 tid 100136 td 0xfffffe00a6a26740
sched_switch() at sched_switch+0x559/frame 0xfffffe009d9b4d70
mi_switch() at mi_switch+0x157/frame 0xfffffe009d9b4d90
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe009d9b4dd0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe009d9b4e10
_sleep() at _sleep+0x2ce/frame 0xfffffe009d9b4eb0
soaio_kproc_loop() at soaio_kproc_loop+0x125/frame 0xfffffe009d9b4ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe009d9b4f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009d9b4f30
--- trap 0xc, rip = 0x3715385cc09a, rsp = 0x7fffff938fb8, rbp = 0x7fffff938fd0 ---

Tracing command soaiod2 pid 1437 tid 100135 td 0xfffffe00a6a59000
sched_switch() at sched_switch+0x559/frame 0xfffffe009d9b9d70
mi_switch() at mi_switch+0x157/frame 0xfffffe009d9b9d90
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe009d9b9dd0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe009d9b9e10
_sleep() at _sleep+0x2ce/frame 0xfffffe009d9b9eb0
soaio_kproc_loop() at soaio_kproc_loop+0x125/frame 0xfffffe009d9b9ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe009d9b9f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009d9b9f30
--- trap 0xc, rip = 0xe7dcfb3009a, rsp = 0x7fffff31ec58, rbp = 0x7fffff31ec70 ---

Tracing command soaiod3 pid 1438 tid 100134 td 0xfffffe009dbe11e0
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5dadd70
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5dadd90
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5daddd0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00a5dade10
_sleep() at _sleep+0x2ce/frame 0xfffffe00a5dadeb0
soaio_kproc_loop() at soaio_kproc_loop+0x125/frame 0xfffffe00a5dadef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a5dadf30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a5dadf30
--- trap 0xc, rip = 0x23ee4f3a309a, rsp = 0x7fffffa7b638, rbp = 0x7fffffa7b650 ---

Tracing command soaiod4 pid 1439 tid 100109 td 0xfffffe00767d21e0
sched_switch() at sched_switch+0x559/frame 0xfffffe00a5d8fd70
mi_switch() at mi_switch+0x157/frame 0xfffffe00a5d8fd90
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a5d8fdd0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00a5d8fe10
_sleep() at _sleep+0x2ce/frame 0xfffffe00a5d8feb0
soaio_kproc_loop() at soaio_kproc_loop+0x125/frame 0xfffffe00a5d8fef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a5d8ff30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a5d8ff30
--- trap 0xc, rip = 0xb3bc530c09a, rsp = 0x7fffff988158, rbp = 0x7fffff988170 ---

Tracing command sh pid 83852 tid 100898 td 0xfffffe00a6a5d020
sched_switch() at sched_switch+0x559/frame 0xfffffe00a6cd4b80
mi_switch() at mi_switch+0x157/frame 0xfffffe00a6cd4ba0
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a6cd4be0
sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe00a6cd4c30
sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00a6cd4c40
_sleep() at _sleep+0x2dd/frame 0xfffffe00a6cd4ce0
pipe_read() at pipe_read+0x4b6/frame 0xfffffe00a6cd4d40
dofileread() at dofileread+0x81/frame 0xfffffe00a6cd4d90
sys_read() at sys_read+0xc0/frame 0xfffffe00a6cd4e00
amd64_syscall() at amd64_syscall+0x6b4/frame 0xfffffe00a6cd4f30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00a6cd4f30
--- syscall (3, FreeBSD ELF64, sys_read), rip = 0x16a20f51845a, rsp = 0x7fffff399288, rbp = 0x7fffff399410 ---

Tracing command inetd pid 83954 tid 101311 td 0xfffffe00ac575c80
sched_switch() at sched_switch+0x559/frame 0xfffffe00ac8b6ab0
mi_switch() at mi_switch+0x157/frame 0xfffffe00ac8b6ad0
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00ac8b6b10
sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe00ac8b6b60
sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00ac8b6b70
_cv_wait_sig() at _cv_wait_sig+0x165/frame 0xfffffe00ac8b6bd0
kern_select() at kern_select+0xa95/frame 0xfffffe00ac8b6dc0
sys_select() at sys_select+0x56/frame 0xfffffe00ac8b6e00
amd64_syscall() at amd64_syscall+0x6b4/frame 0xfffffe00ac8b6f30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00ac8b6f30
--- syscall (93, FreeBSD ELF64, sys_select), rip = 0x159bb33614da, rsp = 0x7fffffafa448, rbp = 0x7fffffafb2e0 ---

Tracing command KTLS pid 47103 tid 100114 td 0xfffffe009db63ac0
sched_switch() at sched_switch+0x559/frame 0xfffffe00765d5d00
mi_switch() at mi_switch+0x157/frame 0xfffffe00765d5d20
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00765d5d60
_sleep() at _sleep+0x2e7/frame 0xfffffe00765d5e00
ktls_work_thread() at ktls_work_thread+0x19c/frame 0xfffffe00765d5ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00765d5f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00765d5f30
--- trap 0xc, rip = 0x9c7a77ce09a, rsp = 0x7fffff702d68, rbp = 0x7fffff702db0 ---

Tracing command KTLS pid 47103 tid 100850 td 0xfffffe00a6a77ac0
sched_switch() at sched_switch+0x559/frame 0xfffffe00a6c45d00
mi_switch() at mi_switch+0x157/frame 0xfffffe00a6c45d20
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a6c45d60
_sleep() at _sleep+0x2e7/frame 0xfffffe00a6c45e00
ktls_work_thread() at ktls_work_thread+0x19c/frame 0xfffffe00a6c45ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a6c45f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a6c45f30
--- trap 0xc, rip = 0x17aab0ca449a, rsp = 0x102ea8, rbp = 0x102ef0 ---

Tracing command KTLS pid 47103 tid 100851 td 0xfffffe00a6a79740
sched_switch() at sched_switch+0x559/frame 0xfffffe00a6c59d40
mi_switch() at mi_switch+0x157/frame 0xfffffe00a6c59d60
sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00a6c59da0
_sleep() at _sleep+0x2e7/frame 0xfffffe00a6c59e40
ktls_alloc_thread() at ktls_alloc_thread+0x204/frame 0xfffffe00a6c59ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a6c59f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a6c59f30
--- trap 0xc, rip = 0x1a5db4e6f09a, rsp = 0x604a88, rbp = 0x604aa0 ---
db:0:kdb.enter.panic> dump

Dump failed. Partition too small (about 2859MB were needed this time).
Cannot dump: unknown error (error=7).
db:0:kdb.enter.panic> reset
+ rc=0
+ echo 'bhyve return code = 0'
bhyve return code = 0
+ sudo /usr/sbin/bhyvectl '--vm=testvm-main-amd64-20008' --destroy
+ sh -ex freebsd-ci/scripts/test/
+ METAOUTDIR=meta-out
+ rm -fr meta-out
+ mkdir meta-out
+ tar xvf meta.tar -C meta-out
x ./
x ./
x ./
x ./
x ./
x ./
x ./auto-shutdown
+ rm -f 'test-report.*'
+ mv 'meta-out/test-report.*' .
mv: rename meta-out/test-report.* to ./test-report.*: No such file or directory
+ report=test-report.xml
+ [ -e freebsd-ci/jobs/FreeBSD-main-amd64-test/xfail-list -a -e test-report.xml ]
+ rm -f disk-cam
+ jot 5
+ rm -f disk1
+ rm -f disk2
+ rm -f disk3
+ rm -f disk4
+ rm -f disk5
+ rm -f disk-test.img
[PostBuildScript] - [INFO] Executing post build scripts.
[FreeBSD-main-amd64-test] $ /bin/sh -xe /tmp/
+ ./freebsd-ci/artifact/
Post link: {'job_name': 'FreeBSD-main-amd64-test', 'commit': '1d92e81fa47925dab1fa3779b8e8e2e35d3c14ce', 'branch': 'main', 'target': 'amd64', 'target_arch': 'amd64', 'link_type': 'latest_tested'}
"Link created: main/latest_tested/amd64/amd64 -> ../../1d92e81fa47925dab1fa3779b8e8e2e35d3c14ce/amd64/amd64\n"
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Checking for post-build
Performing post-build step
Checking if email needs to be generated
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Sending mail from default account using System Admin e-mail address