From nobody Sun Aug 21 17:39:08 2022 X-Original-To: dev-ci@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4M9jR1557Yz4ZLFB for ; Sun, 21 Aug 2022 17:39:09 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4M9jR12X0Fz3NXF; Sun, 21 Aug 2022 17:39:09 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins.ci.freebsd.org (jenkins.ci.freebsd.org [IPv6:2610:1c1:1:607c::16:16]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4M9jR11S6MzJQ5; Sun, 21 Aug 2022 17:39:09 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Date: Sun, 21 Aug 2022 17:39:08 +0000 (GMT) From: jenkins-admin@FreeBSD.org To: glebius@FreeBSD.org, dev-ci@FreeBSD.org Cc: jenkins-admin@FreeBSD.org Message-ID: <1079372259.6483.1661103549208@jenkins.ci.freebsd.org> In-Reply-To: <1768592314.6477.1661069987118@jenkins.ci.freebsd.org> References: <1768592314.6477.1661069987118@jenkins.ci.freebsd.org> Subject: FreeBSD-main-amd64-test - Build #21910 - Failure List-Id: Continuous Integration Build and Test Results List-Archive: https://lists.freebsd.org/archives/dev-ci List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-ci@freebsd.org X-BeenThere: dev-ci@freebsd.org MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_6482_314551221.1661103548885" X-Jenkins-Job: FreeBSD-main-amd64-test X-Jenkins-Result: FAILURE List-ID: FreeBSD CI Build Notifications Precedence: bulk ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1661103549; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references:list-id; bh=G5xE3B76hFHLPDEr1QXhDm/57j9txPVHhJ5Rs09Gysw=; b=HgXLJrJEwLDvrbkcE2f1NB4qt4U8gxK8/BinFf1zeNQ/4wYiX9eENraG4TYpNydGRRJw9X nRGOxS4WniqJvHqEM8i1D+SwYKr+myi//zmrQFT76Z1sWZOqt/lggpiLab4zyQqgpKY0VH A8JiI5Vpa/5Ft8Q/ju+6X/6v3DfFdWeA3J5fXRMSk+lhiJ9nTUxZLjX3F9gKk1yaV5gSIk cVMOFQA7OqW8Hvd7QYqvLUJ5i0YveY0Rim+GE2ZF9l+FIcSj0s/SWKICS13JQmuC3S8e9P YJraDMAQtTMkkNgTwIVsRsykvgtNZ4txTNQkbq/Y+qmfhS3ehJWbVXUwcW2Hlw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1661103549; a=rsa-sha256; cv=none; b=ZYQFVJCSHfGzDUMA7BQmR44dNT+34FRqlwGm5rexLkDEpRgB3j8Bi+vbMxhMr6+j8YHwrX BvQJal7rKtNB7h5W2a1wEU7Zpw/yQvhmL+ydPkj4dczsT3L2JqzGJbCWGE5YIykfa2rCIK PAISG160eBKn3+CJEgcJJYDKDl2iph4XI8HF85/ygDi9+vxmy9XKXzhGPhUbFEN7h8V/wP NYD8Q1kJjJKXv+2lD12OKxPDCfZ7QMdncJKRfB4BJ2tVez25hcJxRIyLbtOwKtlTa3ZDZF BT8ccPAcj4HodNoS77H8WDtOCMG+YSUbdLbtnHGMD9nv8vC8wjnW/IaKU5YGXg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N ------=_Part_6482_314551221.1661103548885 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable FreeBSD-main-amd64-test - Build #21910 (9d54812421274e490dc5f0fe4722ab8d35d= 9b258) - Failure Build information: https://ci.FreeBSD.org/job/FreeBSD-main-amd64-test/21910= / Full change log: https://ci.FreeBSD.org/job/FreeBSD-main-amd64-test/21910/c= hanges Full build log: https://ci.FreeBSD.org/job/FreeBSD-main-amd64-test/21910/co= nsole Status explanation: "Failure" - the build is suspected being broken by the following changes "Still Failing" - the build has not been fixed by the following changes and this is a notification to note that these changes have not been fully tested by the CI system Change summaries: (Those commits are likely but not certainly responsible) 6498153665fddbcf3efa94a230933a3b8834240b by glebius: ip_reass: don't drain all vnets on a vnet destroy 9d54812421274e490dc5f0fe4722ab8d35d9b258 by glebius: snmp_mibII: use net.inet.ip.fragttl sysctl for ipReasmTimeout The end of the build log: [...truncated 4.59 MB...] vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 vmnet0: Ethernet address: 58:9c:fc:10:e8:51 passed [10.033s] sys/net/if_epair:params -> epair75a: Ethernet address: 02:99:7f:95:84:0a epair75b: Ethernet address: 02:99:7f:95:84:0b epair75a: link state changed to UP epair75b: link state changed to UP epair75a: link state changed to DOWN epair75b: link state changed to DOWN passed [0.024s] sys/net/if_gif:basic -> epair75a: Ethernet address: 02:2a:9b:2d:51:0a epair75b: Ethernet address: 02:2a:9b:2d:51:0b epair75a: link state changed to UP epair75b: link state changed to UP gif0: link state changed to UP gif0: link state changed to UP epair75a: link state changed to DOWN epair75b: link state changed to DOWN passed [0.202s] sys/net/if_lagg_test:create -> tap0: Ethernet address: 58:9c:fc:00:2c:7f tap1: Ethernet address: 58:9c:fc:10:ff:e3 lagg0: link state changed to DOWN lock order reversal: 1st 0xffffffff81fa32b0 in_control (in_control, sx) @ /usr/src/sys/netinet/= in.c:362 2nd 0xfffff80006bb2028 if_lagg sx (if_lagg sx, sx) @ /usr/src/sys/net/if_l= agg.c:1269 lock order in_control -> if_lagg sx attempted at: #0 0xffffffff80c5c9ed at witness_checkorder+0xbfd #1 0xffffffff80bf63b3 at _sx_xlock+0x63 #2 0xffffffff8291eb3e at lagg_init+0x2e #3 0xffffffff80d1ee4e at ether_ioctl+0x1be #4 0xffffffff80dab402 at in_control+0x9c2 #5 0xffffffff80d19134 at ifioctl+0x7f4 #6 0xffffffff80c62752 at kern_ioctl+0x202 #7 0xffffffff80c6249a at sys_ioctl+0x12a #8 0xffffffff810c88b6 at amd64_syscall+0x6a6 #9 0xffffffff8109bd1b at fast_syscall_common+0xf8 lock order reversal: 1st 0xffffffff81fa3338 in_multi_sx (in_multi_sx, sx) @ /usr/src/sys/netine= t/in_mcast.c:1206 2nd 0xfffff80006bb2028 if_lagg sx (if_lagg sx, sx) @ /usr/src/sys/net/if_l= agg.c:1686 lock order in_multi_sx -> if_lagg sx attempted at: #0 0xffffffff80c5c9ed at witness_checkorder+0xbfd #1 0xffffffff80bf63b3 at _sx_xlock+0x63 #2 0xffffffff8291ed54 at lagg_ioctl+0xd4 #3 0xffffffff80d1857e at if_addmulti+0x2be #4 0xffffffff80db102b at in_joingroup_locked+0x27b #5 0xffffffff80db0d82 at in_joingroup+0x42 #6 0xffffffff80dab447 at in_control+0xa07 #7 0xffffffff80d19134 at ifioctl+0x7f4 #8 0xffffffff80c62752 at kern_ioctl+0x202 #9 0xffffffff80c6249a at sys_ioctl+0x12a #10 0xffffffff810c88b6 at amd64_syscall+0x6a6 #11 0xffffffff8109bd1b at fast_syscall_common+0xf8 Kernel page fault with the following non-sleepable locks held: exclusive sleep mutex if_cloners lock (if_cloners lock) r =3D 0 (0xffffffff= 81fa2268) locked @ /usr/src/sys/net/if_clone.c:421 stack backtrace: #0 0xffffffff80c5ce25 at witness_debugger+0x65 #1 0xffffffff80c5df8a at witness_warn+0x3fa #2 0xffffffff810c7a16 at trap_pfault+0x86 #3 0xffffffff8109b408 at calltrap+0x8 #4 0xffffffff82922436 at vnet_lagg_uninit+0x26 #5 0xffffffff80d56890 at vnet_destroy+0x160 #6 0xffffffff80bab8a0 at prison_deref+0xa80 #7 0xffffffff80c4f0da at taskqueue_run_locked+0xaa #8 0xffffffff80c50182 at taskqueue_thread_loop+0xc2 #9 0xffffffff80ba2680 at fork_exit+0x80 #10 0xffffffff8109c46e at fork_trampoline+0xe Fatal trap 12: page fault while in kernel mode cpuid =3D 0; apic id =3D 00 fault virtual address=09=3D 0x70 fault code=09=09=3D supervisor read data, page not present instruction pointer=09=3D 0x20:0xffffffff80d1cda7 stack pointer=09 =3D 0x28:0xfffffe0084dddd60 frame pointer=09 =3D 0x28:0xfffffe0084dddd80 code segment=09=09=3D base 0x0, limit 0xfffff, type 0x1b =09=09=09=3D DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags=09=3D interrupt enabled, resume, IOPL =3D 0 current process=09=09=3D 0 (thread taskq) trap number=09=09=3D 12 panic: page fault cpuid =3D 0 time =3D 1661103548 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0084ddd= b20 vpanic() at vpanic+0x151/frame 0xfffffe0084dddb70 panic() at panic+0x43/frame 0xfffffe0084dddbd0 trap_fatal() at trap_fatal+0x387/frame 0xfffffe0084dddc30 trap_pfault() at trap_pfault+0xab/frame 0xfffffe0084dddc90 calltrap() at calltrap+0x8/frame 0xfffffe0084dddc90 --- trap 0xc, rip =3D 0xffffffff80d1cda7, rsp =3D 0xfffffe0084dddd60, rbp = =3D 0xfffffe0084dddd80 --- if_clone_detach() at if_clone_detach+0x27/frame 0xfffffe0084dddd80 vnet_lagg_uninit() at vnet_lagg_uninit+0x26/frame 0xfffffe0084dddda0 vnet_destroy() at vnet_destroy+0x160/frame 0xfffffe0084ddddd0 prison_deref() at prison_deref+0xa80/frame 0xfffffe0084ddde40 taskqueue_run_locked() at taskqueue_run_locked+0xaa/frame 0xfffffe0084dddec= 0 taskqueue_thread_loop() at taskqueue_thread_loop+0xc2/frame 0xfffffe0084ddd= ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe0084dddf30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0084dddf30 --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- KDB: enter: panic [ thread pid 0 tid 100018 ] Stopped at kdb_enter+0x32: movq $0,0x12a3b13(%rip) db:0:kdb.enter.panic> show pcpu cpuid =3D 0 dynamic pcpu =3D 0x9c2e40 curthread =3D 0xfffffe00112be000: pid 0 tid 100018 critnest 1 "thread ta= skq" curpcb =3D 0xfffffe00112be520 fpcurthread =3D none idlethread =3D 0xfffffe001122c3a0: tid 100003 "idle: cpu0" self =3D 0xffffffff82610000 curpmap =3D 0xffffffff81e90bd0 tssp =3D 0xffffffff82610384 rsp0 =3D 0xfffffe0084dde000 kcr3 =3D 0x8000000002569007 ucr3 =3D 0xffffffffffffffff scr3 =3D 0x1b9216fe2 gs32p =3D 0xffffffff82610404 ldt =3D 0xffffffff82610444 tss =3D 0xffffffff82610434 curvnet =3D 0xfffff8011bc2ab40 spin locks held: db:0:kdb.enter.panic> reset + rc=3D0 + echo 'bhyve return code =3D 0' bhyve return code =3D 0 + sudo /usr/sbin/bhyvectl '--vm=3Dtestvm-main-amd64-21910' --destroy + sh -ex freebsd-ci/scripts/test/extract-meta.sh + METAOUTDIR=3Dmeta-out + rm -fr meta-out + mkdir meta-out + tar xvf meta.tar -C meta-out x ./ x ./auto-shutdown x ./disable-dtrace-tests.sh x ./run-kyua.sh x ./run.sh x ./disable-notyet-tests.sh x ./disable-zfs-tests.sh + rm -f test-report.txt test-report.xml + mv 'meta-out/test-report.*' . mv: rename meta-out/test-report.* to ./test-report.*: No such file or direc= tory + report=3Dtest-report.xml + [ -e freebsd-ci/jobs/FreeBSD-main-amd64-test/xfail-list -a -e test-report= .xml ] + rm -f disk-cam + jot 5 + rm -f disk1 + rm -f disk2 + rm -f disk3 + rm -f disk4 + rm -f disk5 + rm -f disk-test.img [PostBuildScript] - [INFO] Executing post build scripts. [FreeBSD-main-amd64-test] $ /bin/sh -xe /tmp/jenkins6548770707542509070.sh + ./freebsd-ci/artifact/post-link.py Post link: {'job_name': 'FreeBSD-main-amd64-test', 'commit': '9d54812421274= e490dc5f0fe4722ab8d35d9b258', 'branch': 'main', 'target': 'amd64', 'target_= arch': 'amd64', 'link_type': 'latest_tested'} "Link created: main/latest_tested/amd64/amd64 -> ../../9d54812421274e490dc5= f0fe4722ab8d35d9b258/amd64/amd64\n" Recording test results ERROR: Step =E2=80=98Publish JUnit test result report=E2=80=99 failed: No t= est report files were found. Configuration error? Checking for post-build Performing post-build step Checking if email needs to be generated Email was triggered for: Failure - Any Sending email for trigger: Failure - Any Sending mail from default account using System Admin e-mail address ------=_Part_6482_314551221.1661103548885--