FreeBSD-main-amd64-test - Build #28343 - Still Failing

From: <jenkins-admin_at_FreeBSD.org>
Date: Mon, 20 Apr 2026 21:22:08 UTC
FreeBSD-main-amd64-test - Build #28343 (324b3c2892a53e5fd9b1d8795a31a59ee835c5de) - Still Failing

Build information: https://ci.FreeBSD.org/job/FreeBSD-main-amd64-test/28343/
Full change log: https://ci.FreeBSD.org/job/FreeBSD-main-amd64-test/28343/changes
Full build log: https://ci.FreeBSD.org/job/FreeBSD-main-amd64-test/28343/console

Status explanation:
"Failure" - the build is suspected being broken by the following changes
"Still Failing" - the build has not been fixed by the following changes and
                  this is a notification to note that these changes have
                  not been fully tested by the CI system

Change summaries:
(Those commits are likely but not certainly responsible)

505f381cb96ee25bcf0df11cfaf15a601d969500 by jhb:
nvmf_che: Don't leak a socket if an error occurs finalizing the socket

e56687bb2d32ee4b1774300f3960427ca56b34bb by jhb:
ctld: Return from the inner "main" loop on EINTR

7bb2b3801554a58039ed9d1fd05b65ce24c6c661 by jhb:
ctld: Support multiple physical ports in a target

4b79ee8eb139696c1fe845d25a56439d02ab4131 by jhb:
ctld: Update nchildren directly in wait_for_children

9b844b495e8e63439ffe523757ac7444a16317af by kib:
rtld: fix processing of preloaded z_initfirst objects

950fd59955e10429c1325c327f0141a7e97fcfe5 by kib:
membarrier(2): use atomic for lockless read of curproc->p_flag2

aaa10396a997bafc28ce6551e92563620caf3df4 by kib:
membarrier(2): implement the MEMBARRIER_CMD_GET_REGISTRATIONS command

324b3c2892a53e5fd9b1d8795a31a59ee835c5de by kib:
membarrier.2: document MEMBARRIER_CMD_GET_REGISTRATIONS



The end of the build log:

[...truncated 4.17 MiB...]
epair0a: link state changed to DOWN
epair0b: link state changed to DOWN
epair1a: link state changed to DOWN
epair1b: link state changed to DOWN
passed  [0.458s]
sys/netinet/output:output_raw_flowid_mpath_success  ->  epair0a: Ethernet address: 58:9c:fc:10:d7:fd
epair0b: Ethernet address: 58:9c:fc:10:0e:21
epair0a: link state changed to UP
epair0b: link state changed to UP
epair1a: Ethernet address: 58:9c:fc:10:25:12
epair1b: Ethernet address: 58:9c:fc:10:30:bf
epair1a: link state changed to UP
epair1b: link state changed to UP
lo1: link state changed to UP
lo2: link state changed to UP
epair0a: link state changed to DOWN
epair0b: link state changed to DOWN
epair1a: link state changed to DOWN
epair1b: link state changed to DOWN
passed  [0.964s]
sys/netinet/output:output_raw_success  ->  epair0a: Ethernet address: 58:9c:fc:10:d7:fd
epair0b: Ethernet address: 58:9c:fc:10:0e:21
epair0a: link state changed to UP
epair0b: link state changed to UP
epair0a: link state changed to DOWN
epair0b: link state changed to DOWN
passed  [0.226s]
sys/netinet/output:output_tcp_flowid_mpath_success  ->  epair0a: Ethernet address: 58:9c:fc:10:d7:fd
epair0b: Ethernet address: 58:9c:fc:10:0e:21
epair0a: link state changed to UP
epair0b: link state changed to UP
epair1a: Ethernet address: 58:9c:fc:10:25:12
epair1b: Ethernet address: 58:9c:fc:10:30:bf
epair1a: link state changed to UP
epair1b: link state changed to UP
lo1: link state changed to UP
lo2: link state changed to UP
epair0a: link state changed to DOWN
epair0b: link state changed to DOWN
epair1a: link state changed to DOWN
epair1b: link state changed to DOWN
passed  [3.354s]
sys/netinet/output:output_tcp_setup_success  ->  epair0a: Ethernet address: 58:9c:fc:10:d7:fd
epair0b: Ethernet address: 58:9c:fc:10:0e:21
epair0a: link state changed to UP
epair0b: link state changed to UP
epair0a: link state changed to DOWN
epair0b: link state changed to DOWN
passed  [0.306s]
sys/netinet/output:output_udp_flowid_mpath_success  ->  epair0a: Ethernet address: 58:9c:fc:10:d7:fd
epair0b: Ethernet address: 58:9c:fc:10:0e:21
epair0a: link state changed to UP
epair0b: link state changed to UP
epair1a: Ethernet address: 58:9c:fc:10:25:12
epair1b: Ethernet address: 58:9c:fc:10:30:bf
epair1a: link state changed to UP
epair1b: link state changed to UP
lo1: link state changed to UP
lo2: link state changed to UP
epair0a: link state changed to DOWN
epair0b: link state changed to DOWN
epair1a: link state changed to DOWN
epair1b: link state changed to DOWN
passed  [8.892s]
sys/netinet/output:output_udp_setup_success  ->  epair0a: Ethernet address: 58:9c:fc:10:d7:fd
epair0b: Ethernet address: 58:9c:fc:10:0e:21
epair0a: link state changed to UP
epair0b: link state changed to UP
epair0a: link state changed to DOWN
epair0b: link state changed to DOWN
passed  [1.379s]
sys/netinet/raw:input  ->  lo0: link state changed to UP
passed  [0.065s]
sys/netinet/raw:reconnect  ->  lo0: link state changed to UP
passed  [0.057s]
sys/netinet/redirect:valid_redirect  ->  epair0a: Ethernet address: 58:9c:fc:10:d7:fd
epair0b: Ethernet address: 58:9c:fc:10:0e:21
epair0a: link state changed to UP
epair0b: link state changed to UP
epair0a: promiscuous mode enabled
epair0a: promiscuous mode disabled
epair0b: link state changed to DOWN
epair0a: link state changed to DOWN
passed  [2.111s]
sys/netinet/so_reuseport_lb_test:basic_ipv4  ->  Limiting tcp reset response from 7827 to 211 packets/sec
Limiting tcp reset response from 7953 to 202 packets/sec
passed  [2.059s]
sys/netinet/so_reuseport_lb_test:basic_ipv6  ->  Limiting tcp reset response from 7100 to 196 packets/sec
Limiting tcp reset response from 7200 to 199 packets/sec
passed  [2.277s]
sys/netinet/so_reuseport_lb_test:bind_without_listen  ->  passed  [0.009s]
sys/netinet/so_reuseport_lb_test:concurrent_add  ->  passed  [2.603s]
sys/netinet/so_reuseport_lb_test:connect_bound  ->  passed  [0.007s]
sys/netinet/so_reuseport_lb_test:connect_not_bound  ->  passed  [0.007s]
sys/netinet/so_reuseport_lb_test:connect_udp  ->  passed  [0.209s]
sys/netinet/so_reuseport_lb_test:connect_udp6  ->  passed  [0.209s]
sys/netinet/so_reuseport_lb_test:double_listen_ipv4  ->  passed  [0.007s]
sys/netinet/so_reuseport_lb_test:double_listen_ipv6  ->  passed  [0.006s]
sys/netinet/socket_afinet:socket_afinet  ->  passed  [0.006s]
sys/netinet/socket_afinet:socket_afinet_bind_connected_port  ->  passed  [0.010s]
sys/netinet/socket_afinet:socket_afinet_bind_ok  ->  passed  [0.006s]
sys/netinet/socket_afinet:socket_afinet_bind_zero  ->  passed  [0.006s]
sys/netinet/socket_afinet:socket_afinet_bindany  ->  passed  [0.007s]
sys/netinet/socket_afinet:socket_afinet_multibind  ->  passed  [0.194s]
sys/netinet/socket_afinet:socket_afinet_poll_no_rdhup  ->  passed  [0.007s]
sys/netinet/socket_afinet:socket_afinet_poll_rdhup  ->  passed  [0.006s]
sys/netinet/socket_afinet:socket_afinet_stream_reconnect  ->  Limiting tcp reset response from 2459 to 216 packets/sec
passed  [0.006s]
sys/netinet/tcp_connect_port_test:basic_ipv4  ->  Limiting tcp reset response from 9404 to 212 packets/sec
Limiting tcp reset response from 9347 to 215 packets/sec
Limiting tcp reset response from 9586 to 214 packets/sec
Limiting tcp reset response from 9691 to 188 packets/sec
Limiting tcp reset response from 9529 to 216 packets/sec
Limiting tcp reset response from 9500 to 212 packets/sec
passed  [6.829s]
sys/netinet/tcp_connect_port_test:basic_ipv6  ->  Limiting tcp reset response from 9131 to 185 packets/sec
Limiting tcp reset response from 7870 to 190 packets/sec
Limiting tcp reset response from 7905 to 214 packets/sec
Limiting tcp reset response from 7906 to 188 packets/sec
Limiting tcp reset response from 7859 to 194 packets/sec
Limiting tcp reset response from 7859 to 201 packets/sec
Limiting tcp reset response from 7822 to 211 packets/sec
Limiting tcp reset response from 7815 to 212 packets/sec
Limiting tcp reset response from 7843 to 197 packets/sec
passed  [8.314s]
TCP HPTS started 2 ((unbounded)) swi interrupt threads
sys/netinet/tcp_hpts_test.py:TestTcpHpts::test_concurrent_operations  ->  passed  [0.513s]
sys/netinet/tcp_hpts_test.py:TestTcpHpts::test_cpu_assignment  ->  passed  [0.508s]
sys/netinet/tcp_hpts_test.py:TestTcpHpts::test_deferred_requests  ->  Kernel page fault with the following non-sleepable locks held:
exclusive rw test-inp (test-inp) r = 0 (0xfffff8007e6af030) locked @ /usr/src/sys/netinet/tcp_hpts.c:1275
Kernel page fault with 1 sleep inhibitors
stack backtrace:
#0 0xffffffff80c54c3c at witness_debugger+0x6c
#1 0xffffffff80c56549 at witness_warn+0x4c9
#2 0xffffffff8113ccdc at trap_pfault+0x8c
#3 0xffffffff8110c398 at calltrap+0x8
#4 0xffffffff8315f866 at __ktest_deferred_requests+0x996
#5 0xffffffff830e3abd at run_test+0x2ad
#6 0xffffffff80e6df1c at nl_receive_message+0x11c
#7 0xffffffff80e6d865 at nl_taskqueue_handler+0x3e5
#8 0xffffffff80c46912 at taskqueue_run_locked+0x1c2
#9 0xffffffff80c47803 at taskqueue_thread_loop+0xd3
#10 0xffffffff80b899d2 at fork_exit+0x82
#11 0xffffffff8110d3be at fork_trampoline+0xe


Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address	= 0xc8
fault code		= supervisor read data, page not present
instruction pointer	= 0x20:0xffffffff8316a3f1
stack pointer	        = 0x28:0xfffffe008c15dab0
frame pointer	        = 0x28:0xfffffe008c15dbd0
code segment		= base 0x0, limit 0xfffff, type 0x1b
			= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags	= interrupt enabled, resume, IOPL = 0
current process		= 0 (netlink_socket (PID)
rdi: fffff800033b2600 rsi: 0000000000000001 rdx: ffffffff8226a4f8
rcx: 0000000000000001  r8: ffffffff81ad2ae0  r9: 0000000000000001
rax: 0000000000000000 rbx: 0000000000002000 rbp: fffffe008c15dbd0
r10: 0000000000000000 r11: 0000000000000001 r12: fffff8007e6af000
r13: 0000000000000000 r14: fffff8007e6af048 r15: 0000000000001388
trap number		= 12
panic: page fault
cpuid = 0
time = 1776720116
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe008c15d7e0
vpanic() at vpanic+0x136/frame 0xfffffe008c15d910
panic() at panic+0x43/frame 0xfffffe008c15d970
trap_pfault() at trap_pfault+0x422/frame 0xfffffe008c15d9e0
calltrap() at calltrap+0x8/frame 0xfffffe008c15d9e0
--- trap 0xc, rip = 0xffffffff8316a3f1, rsp = 0xfffffe008c15dab0, rbp = 0xfffffe008c15dbd0 ---
tcp_hptsi() at tcp_hptsi+0x6f1/frame 0xfffffe008c15dbd0
__ktest_deferred_requests() at __ktest_deferred_requests+0x996/frame 0xfffffe008c15dc60
run_test() at run_test+0x2ad/frame 0xfffffe008c15dd00
nl_receive_message() at nl_receive_message+0x11c/frame 0xfffffe008c15dd40
nl_taskqueue_handler() at nl_taskqueue_handler+0x3e5/frame 0xfffffe008c15de40
taskqueue_run_locked() at taskqueue_run_locked+0x1c2/frame 0xfffffe008c15dec0
taskqueue_thread_loop() at taskqueue_thread_loop+0xd3/frame 0xfffffe008c15def0
fork_exit() at fork_exit+0x82/frame 0xfffffe008c15df30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe008c15df30
--- trap 0xc, rip = 0x9711543a32a, rsp = 0x97116d2df78, rbp = 0x97116d2dfa0 ---
KDB: enter: panic
[ thread pid 0 tid 107605 ]
Stopped at      kdb_enter+0x33: movq    $0,0x15dd2e2(%rip)
db:0:kdb.enter.panic> show pcpu
cpuid        = 0
dynamic pcpu = 0xdadcc0
curthread    = 0xfffff800534c2000: pid 0 tid 107605 critnest 1 "netlink_socket (PID"
curpcb       = 0xfffff800534c2550
fpcurthread  = none
idlethread   = 0xfffff8010240c780: tid 100003 "idle: cpu0"
self         = 0xffffffff82a10000
curpmap      = 0xffffffff81da2d20
tssp         = 0xffffffff82a10384
rsp0         = 0xfffffe008c15e000
kcr3         = 0x8000000002946002
ucr3         = 0xffffffffffffffff
scr3         = 0xbec40e6d
gs32p        = 0xffffffff82a10404
ldt          = 0xffffffff82a10444
tss          = 0xffffffff82a10434
curvnet      = 0xfffff800030e7ec0
spin locks held:
db:0:kdb.enter.panic>  reset
Uptime: 50m36s
+ rc=0
+ echo 'bhyve return code = 0'
bhyve return code = 0
+ sudo /usr/sbin/bhyvectl '--vm=testvm-main-amd64-28343' --destroy
+ sh -ex freebsd-ci/scripts/test/extract-meta.sh
+ METAOUTDIR=meta-out
+ rm -fr meta-out
+ mkdir meta-out
+ tar xvf meta.tar -C meta-out
x ./
x ./auto-shutdown
x ./run-kyua.sh
x ./disable-notyet-tests.sh
x ./disable-zfs-tests.sh
x ./disable-dtrace-tests.sh
x ./run.sh
+ rm -f 'test-report.*'
+ mv 'meta-out/test-report.*' .
mv: rename meta-out/test-report.* to ./test-report.*: No such file or directory
+ report=test-report.xml
+ [ -e freebsd-ci/jobs/FreeBSD-main-amd64-test/xfail-list -a -e test-report.xml ]
+ rm -f disk-cam
+ jot 5
+ rm -f disk1
+ rm -f disk2
+ rm -f disk3
+ rm -f disk4
+ rm -f disk5
+ rm -f disk-test.img
[PostBuildScript] - [INFO] Executing post build scripts.
[FreeBSD-main-amd64-test] $ /bin/sh -xe /tmp/jenkins13086410328444592437.sh
+ ./freebsd-ci/artifact/post-link.py
Post link: {'job_name': 'FreeBSD-main-amd64-test', 'commit': '324b3c2892a53e5fd9b1d8795a31a59ee835c5de', 'branch': 'main', 'target': 'amd64', 'target_arch': 'amd64', 'link_type': 'latest_tested'}
"Link created: main/latest_tested/amd64/amd64 -> ../../324b3c2892a53e5fd9b1d8795a31a59ee835c5de/amd64/amd64\n"
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Checking for post-build
Performing post-build step
Checking if email needs to be generated
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Sending mail from default account using System Admin e-mail address