FreeBSD-main-amd64-test - Build #19783 - Still Failing

From: <>
Date: Fri, 12 Nov 2021 01:33:32 UTC
FreeBSD-main-amd64-test - Build #19783 (44744f75386e2102584acbca24fbe67de16051ca) - Still Failing

Build information:
Full change log:
Full build log:

Status explanation:
"Failure" - the build is suspected being broken by the following changes
"Still Failing" - the build has not been fixed by the following changes and
                  this is a notification to note that these changes have
                  not been fully tested by the CI system

Change summaries:
(Those commits are likely but not certainly responsible)

44744f75386e2102584acbca24fbe67de16051ca by rmacklem:
nfscl: Add a LayoutError RPC for NFSv4.2 pNFS mounts

The end of the build log:

[...truncated 3.82 MB...]
_sleep() at _sleep+0x294/frame 0xfffffe0063b98b70
kern_wait6() at kern_wait6+0x474/frame 0xfffffe0063b98c00
sys_wait4() at sys_wait4+0x7d/frame 0xfffffe0063b98e00
amd64_syscall() at amd64_syscall+0x5cd/frame 0xfffffe0063b98f30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0063b98f30
--- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x80120a15a, rsp = 0x7fffffffe7f8, rbp = 0x7fffffffe8a0 ---

Tracing command sh pid 664 tid 100092 td 0xfffffe00775ade40
sched_switch() at sched_switch+0x578/frame 0xfffffe0063b52a10
mi_switch() at mi_switch+0x155/frame 0xfffffe0063b52a30
sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0063b52a70
sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe0063b52ac0
sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe0063b52ad0
_sleep() at _sleep+0x294/frame 0xfffffe0063b52b70
kern_wait6() at kern_wait6+0x474/frame 0xfffffe0063b52c00
sys_wait4() at sys_wait4+0x7d/frame 0xfffffe0063b52e00
amd64_syscall() at amd64_syscall+0x5cd/frame 0xfffffe0063b52f30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0063b52f30
--- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x80120a15a, rsp = 0x7fffffffe7f8, rbp = 0x7fffffffe8a0 ---

Tracing command kyua pid 665 tid 100109 td 0xfffffe009db60020
sched_switch() at sched_switch+0x578/frame 0xfffffe00773eda10
mi_switch() at mi_switch+0x155/frame 0xfffffe00773eda30
sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00773eda70
sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe00773edac0
sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00773edad0
_sleep() at _sleep+0x294/frame 0xfffffe00773edb70
kern_wait6() at kern_wait6+0x474/frame 0xfffffe00773edc00
sys_wait4() at sys_wait4+0x7d/frame 0xfffffe00773ede00
amd64_syscall() at amd64_syscall+0x5cd/frame 0xfffffe00773edf30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00773edf30
--- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x80154815a, rsp = 0x7fffffffdc58, rbp = 0x7fffffffdc90 ---

Tracing command g_mp_kt pid 43941 tid 100152 td 0xfffffe009dbf9e40
sched_switch() at sched_switch+0x578/frame 0xfffffe0063bdcda0
mi_switch() at mi_switch+0x155/frame 0xfffffe0063bdcdc0
sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0063bdce00
_sleep() at _sleep+0x29e/frame 0xfffffe0063bdcea0
g_multipath_kt() at g_multipath_kt+0x265/frame 0xfffffe0063bdcef0
fork_exit() at fork_exit+0x80/frame 0xfffffe0063bdcf30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0063bdcf30
--- trap 0xc, rip = 0x801209bba, rsp = 0x7fffffffe848, rbp = 0x7fffffffe940 ---

Tracing command sh pid 54472 tid 101084 td 0xfffffe00aa6a6020
sched_switch() at sched_switch+0x578/frame 0xfffffe00aaa6ba10
mi_switch() at mi_switch+0x155/frame 0xfffffe00aaa6ba30
sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00aaa6ba70
sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe00aaa6bac0
sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00aaa6bad0
_sleep() at _sleep+0x294/frame 0xfffffe00aaa6bb70
kern_wait6() at kern_wait6+0x474/frame 0xfffffe00aaa6bc00
sys_wait4() at sys_wait4+0x7d/frame 0xfffffe00aaa6be00
amd64_syscall() at amd64_syscall+0x5cd/frame 0xfffffe00aaa6bf30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00aaa6bf30
--- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x80120a15a, rsp = 0x7fffffffdae8, rbp = 0x7fffffffdb90 ---

Tracing command aiod17 pid 1238 tid 100104 td 0xfffffe00775b1020
sched_switch() at sched_switch+0x578/frame 0xfffffe0063b93d60
mi_switch() at mi_switch+0x155/frame 0xfffffe0063b93d80
sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0063b93dc0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe0063b93e00
_sleep() at _sleep+0x282/frame 0xfffffe0063b93ea0
aio_daemon() at aio_daemon+0x305/frame 0xfffffe0063b93ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe0063b93f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0063b93f30
--- trap 0xc, rip = 0x8011a6bba, rsp = 0x7fffffffe928, rbp = 0x7fffffffe940 ---

Tracing command aiod27 pid 1248 tid 100123 td 0xfffffe009db9d720
sched_switch() at sched_switch+0x578/frame 0xfffffe0063bb9d60
mi_switch() at mi_switch+0x155/frame 0xfffffe0063bb9d80
sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0063bb9dc0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe0063bb9e00
_sleep() at _sleep+0x282/frame 0xfffffe0063bb9ea0
aio_daemon() at aio_daemon+0x305/frame 0xfffffe0063bb9ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe0063bb9f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0063bb9f30
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---

Tracing command aiod29 pid 1250 tid 100125 td 0xfffffe009db9d000
sched_switch() at sched_switch+0x578/frame 0xfffffe0063bb4d60
mi_switch() at mi_switch+0x155/frame 0xfffffe0063bb4d80
sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe0063bb4dc0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe0063bb4e00
_sleep() at _sleep+0x282/frame 0xfffffe0063bb4ea0
aio_daemon() at aio_daemon+0x305/frame 0xfffffe0063bb4ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe0063bb4f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0063bb4f30
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---

Tracing command aiod31 pid 1252 tid 100127 td 0xfffffe00775a2720
sched_switch() at sched_switch+0x578/frame 0xfffffe00a5dbbd60
mi_switch() at mi_switch+0x155/frame 0xfffffe00a5dbbd80
sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00a5dbbdc0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00a5dbbe00
_sleep() at _sleep+0x282/frame 0xfffffe00a5dbbea0
aio_daemon() at aio_daemon+0x305/frame 0xfffffe00a5dbbef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a5dbbf30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a5dbbf30
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---

Tracing command soaiod1 pid 1260 tid 100133 td 0xfffffe009dba1740
sched_switch() at sched_switch+0x578/frame 0xfffffe00a5de3d70
mi_switch() at mi_switch+0x155/frame 0xfffffe00a5de3d90
sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00a5de3dd0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00a5de3e10
_sleep() at _sleep+0x282/frame 0xfffffe00a5de3eb0
soaio_kproc_loop() at soaio_kproc_loop+0x168/frame 0xfffffe00a5de3ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a5de3f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a5de3f30
--- trap 0xc, rip = 0x8011edbba, rsp = 0x7fffffffd538, rbp = 0x7fffffffd550 ---

Tracing command soaiod2 pid 1261 tid 100132 td 0xfffffe009dbee000
sched_switch() at sched_switch+0x578/frame 0xfffffe00a5de8d70
mi_switch() at mi_switch+0x155/frame 0xfffffe00a5de8d90
sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00a5de8dd0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00a5de8e10
_sleep() at _sleep+0x282/frame 0xfffffe00a5de8eb0
soaio_kproc_loop() at soaio_kproc_loop+0x168/frame 0xfffffe00a5de8ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a5de8f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a5de8f30
--- trap 0xc, rip = 0x8011edbba, rsp = 0x7fffffffd538, rbp = 0x7fffffffd550 ---

Tracing command soaiod3 pid 1262 tid 100131 td 0xfffffe009dbee720
sched_switch() at sched_switch+0x578/frame 0xfffffe00a5dedd70
mi_switch() at mi_switch+0x155/frame 0xfffffe00a5dedd90
sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00a5deddd0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00a5dede10
_sleep() at _sleep+0x282/frame 0xfffffe00a5dedeb0
soaio_kproc_loop() at soaio_kproc_loop+0x168/frame 0xfffffe00a5dedef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a5dedf30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a5dedf30
--- trap 0xc, rip = 0x8011edbba, rsp = 0x7fffffffd538, rbp = 0x7fffffffd550 ---

Tracing command soaiod4 pid 1263 tid 100130 td 0xfffffe009dbeee40
sched_switch() at sched_switch+0x578/frame 0xfffffe00a5df2d70
mi_switch() at mi_switch+0x155/frame 0xfffffe00a5df2d90
sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00a5df2dd0
sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xfffffe00a5df2e10
_sleep() at _sleep+0x282/frame 0xfffffe00a5df2eb0
soaio_kproc_loop() at soaio_kproc_loop+0x168/frame 0xfffffe00a5df2ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a5df2f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a5df2f30
--- trap 0xc, rip = 0x8011edbba, rsp = 0x7fffffffd538, rbp = 0x7fffffffd550 ---

Tracing command atf-check pid 54521 tid 100990 td 0xfffffe00aa5d5740
sched_switch() at sched_switch+0x578/frame 0xfffffe00aa809a10
mi_switch() at mi_switch+0x155/frame 0xfffffe00aa809a30
sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00aa809a70
sleepq_catch_signals() at sleepq_catch_signals+0x266/frame 0xfffffe00aa809ac0
sleepq_wait_sig() at sleepq_wait_sig+0x9/frame 0xfffffe00aa809ad0
_sleep() at _sleep+0x294/frame 0xfffffe00aa809b70
kern_wait6() at kern_wait6+0x474/frame 0xfffffe00aa809c00
sys_wait4() at sys_wait4+0x7d/frame 0xfffffe00aa809e00
amd64_syscall() at amd64_syscall+0x5cd/frame 0xfffffe00aa809f30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00aa809f30
--- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x80134215a, rsp = 0x7fffffffe658, rbp = 0x7fffffffe690 ---

Tracing command python3.8 pid 54523 tid 100928 td 0xfffffe00aa560740 (CPU 0)
kdb_enter() at kdb_enter+0x37/frame 0xfffffe00a6d4c990
vpanic() at vpanic+0x1b8/frame 0xfffffe00a6d4c9f0
panic() at panic+0x43/frame 0xfffffe00a6d4ca50
trap_fatal() at trap_fatal+0x387/frame 0xfffffe00a6d4cab0
trap_pfault() at trap_pfault+0x99/frame 0xfffffe00a6d4cb10
trap() at trap+0x2a7/frame 0xfffffe00a6d4cc20
calltrap() at calltrap+0x8/frame 0xfffffe00a6d4cc20
--- trap 0xc, rip = 0xffffffff80e1dd17, rsp = 0xfffffe00a6d4ccf0, rbp = 0xfffffe00a6d4cd20 ---
tcp_newtcpcb() at tcp_newtcpcb+0x127/frame 0xfffffe00a6d4cd20
tcp_usr_attach() at tcp_usr_attach+0x103/frame 0xfffffe00a6d4cd50
socreate() at socreate+0x1cf/frame 0xfffffe00a6d4cda0
kern_socket() at kern_socket+0xb8/frame 0xfffffe00a6d4ce00
amd64_syscall() at amd64_syscall+0x5cd/frame 0xfffffe00a6d4cf30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00a6d4cf30
--- syscall (97, FreeBSD ELF64, sys_socket), rip = 0x80076a47a, rsp = 0x7fffffff5ba8, rbp = 0x7fffffff5c80 ---

Tracing command KTLS pid 46923 tid 100113 td 0xfffffe009db5e3a0
sched_switch() at sched_switch+0x578/frame 0xfffffe00773d9d00
mi_switch() at mi_switch+0x155/frame 0xfffffe00773d9d20
sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00773d9d60
_sleep() at _sleep+0x29e/frame 0xfffffe00773d9e00
ktls_work_thread() at ktls_work_thread+0x19c/frame 0xfffffe00773d9ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe00773d9f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00773d9f30
--- trap 0xc, rip = 0x8011a9bba, rsp = 0x7fffffffeb18, rbp = 0x7fffffffeb30 ---

Tracing command KTLS pid 46923 tid 100832 td 0xfffffe00a6a16900
sched_switch() at sched_switch+0x578/frame 0xfffffe00a6c1dd00
mi_switch() at mi_switch+0x155/frame 0xfffffe00a6c1dd20
sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe00a6c1dd60
_sleep() at _sleep+0x29e/frame 0xfffffe00a6c1de00
ktls_work_thread() at ktls_work_thread+0x19c/frame 0xfffffe00a6c1def0
fork_exit() at fork_exit+0x80/frame 0xfffffe00a6c1df30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00a6c1df30
--- trap 0x16, rip = 0x801406bba, rsp = 0x7fffdf7f9a88, rbp = 0x7fffdf7f9aa0 ---

Tracing command KTLS pid 46923 tid 100833 td 0xfffffe00a6a0d1e0
sched_switch() at sched_switch+0x578/frame 0xfffffe009d98dd40
mi_switch() at mi_switch+0x155/frame 0xfffffe009d98dd60
sleepq_switch() at sleepq_switch+0x119/frame 0xfffffe009d98dda0
_sleep() at _sleep+0x29e/frame 0xfffffe009d98de40
ktls_alloc_thread() at ktls_alloc_thread+0x204/frame 0xfffffe009d98def0
fork_exit() at fork_exit+0x80/frame 0xfffffe009d98df30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe009d98df30
--- trap 0x16, rip = 0x801406bba, rsp = 0x7fffdfbfba88, rbp = 0x7fffdfbfbaa0 ---
db:0:kdb.enter.panic> dump
Dumping 420 out of 4058 MB:..4%..12%..23%..31%..42%..54%..61%..73%..84%..92%
Dump complete
db:0:kdb.enter.panic> reset
+ rc=0
+ echo 'bhyve return code = 0'
bhyve return code = 0
+ sudo /usr/sbin/bhyvectl '--vm=testvm-main-amd64-19783' --destroy
+ sh -ex freebsd-ci/scripts/test/
+ METAOUTDIR=meta-out
+ rm -fr meta-out
+ mkdir meta-out
+ tar xvf meta.tar -C meta-out
x ./
x ./
x ./
x ./
x ./auto-shutdown
x ./
x ./
+ rm -f 'test-report.*'
+ mv 'meta-out/test-report.*' .
mv: rename meta-out/test-report.* to ./test-report.*: No such file or directory
+ report=test-report.xml
+ [ -e freebsd-ci/jobs/FreeBSD-main-amd64-test/xfail-list -a -e test-report.xml ]
+ rm -f disk-cam
+ jot 5
+ rm -f disk1
+ rm -f disk2
+ rm -f disk3
+ rm -f disk4
+ rm -f disk5
+ rm -f disk-test.img
[PostBuildScript] - [INFO] Executing post build scripts.
[FreeBSD-main-amd64-test] $ /bin/sh -xe /tmp/
+ ./freebsd-ci/artifact/
Post link: {'job_name': 'FreeBSD-main-amd64-test', 'commit': '44744f75386e2102584acbca24fbe67de16051ca', 'branch': 'main', 'target': 'amd64', 'target_arch': 'amd64', 'link_type': 'latest_tested'}
"Link created: main/latest_tested/amd64/amd64 -> ../../44744f75386e2102584acbca24fbe67de16051ca/amd64/amd64\n"
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Checking for post-build
Performing post-build step
Checking if email needs to be generated
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Sending mail from default account using System Admin e-mail address