From nobody Tue Sep 05 00:16:56 2023 X-Original-To: freebsd-ports@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RfmKP1Wr2z4sJGh for ; Tue, 5 Sep 2023 00:17:13 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic306-21.consmr.mail.gq1.yahoo.com (sonic306-21.consmr.mail.gq1.yahoo.com [98.137.68.84]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RfmKN0TS3z4RrD for ; Tue, 5 Sep 2023 00:17:11 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=lOuTJXRD; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.68.84 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1693873030; bh=TkifR2Kfcj7/lsIOfFKJZ8QWUhIFupzPfk8sNHNrgyE=; h=From:Subject:Date:Cc:To:References:From:Subject:Reply-To; b=lOuTJXRDZfd4rsIPGtrIv/HFlqcLz+rJn3vwY8zs7vQ54AqOK9vOAknK5rrl+GmihC7gBcGAPCdwE2QTe0IcM/E65n1rN0SkJNIjMQ0jHGQowHAa5px72pxHc2D0GaoaAYdHvIZy/8m3gi3PyvNCJQwJ/LwQz9T1VrEgl9cHgC7LhSussjyXlIQkwXvfQv5jOE7iD0Igadwp424lCJROLLIGzM15sZHhk1bLdnJiH1iG9LohNyQdr9puzz/5NzEUdg1taf0fySWcC6AMXm+VVq7zFBuxVe457iwVFi8GuYkmYff1Me2IyV82x9fTkk5pBjxpnUYe44JUuH8Xeh8cJQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1693873030; bh=XgKTTq422XAsXNgaqC/dr7LRWs/X0aeMyNg+CZyeOKI=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=BJ7Hk3NTbRWEA/A6MvCgN7dSUhAhzQD0P2b40v0Tt2q8uqk1ZLbnn9914euO994A/C+0rS3vDGG3ilKLA+LzgtDYKfLqbltS//RryaEpDv8fU1JO4HB3rfgqOVR+nS+0QjpmxmJaaqMyefQnvuWzN9BTIS1QnKeFEdkm5TwQ4vtnqTzzp3hm3bbctKkzDB0e4Tz7IlIziLfhRQ+eqIZGX/4bMmg1euu1eZSD4cVsHt7oLoIas8K9zwGjYSvCDeBcCcHaEU/4vfwNw4N/LwqrhjYRcjatriAltPaBHOpmKv11Jj3w7TZO0qYkFVYvpzOot+JEKuvwlZEmlJe6M9x56g== X-YMail-OSG: 6amqwAYVM1l0p3SMKzGbD_Crql69oy3LRt18qIVctoMJpg6KT0IYtwuWg2affeo NlGtdX_ddNyx2iK3i4nzPV3xFXbM2tjkvUd2qCpxXrXNDp2v4wsOZUsMcVQt4VepEeINWO9Kdop0 s7CwcS6BR8.4hr06nS3WzpqRdas7L00nveLsGFITdwackugXVbflk2yl_qJLdsKwryEuCeV3vvPD 80Sxp.7vJu.XNCHMJt7lK6uhIvH_ZqeywDvIzeYol0V0CzJ4eBncL3J7Z_rqG2MxJ7AZfMbcGPl5 KIUqtY7WkOmqnrJKpODmavFskDWMkrxc.ZKUlLPiUMncWVBjUt.gSZUEDqtlr3GjhLOihLfUT3IH TfZFeD2dfP9TQDcNpg9KxdJjpDuYV74PLfI3Ttdaoli89.YYhfJ5eCkVI5fkaKYiyXBsvV4AWktU gFbt89IplweVW6vMyRx9pR2HbJrRTEC6IJvAsF4Xdt7nuz1BW.VvtlsmY7GOzXV8S1aMv9z44Pkr jkPbbnBXu0Zz9Z3IBTEW5vDuexZf1E0DNTn411HgQx7gTNyig00j6W8igEAN5uLxz4FM2CsRyb97 xh_0L79Qqq.ae1Mi7J0WNYgAH5UYiLb28UVwRsfSY5VUcPM_sznJ8kf5nhkZ6DOQStuSVqV2RRNw Oc_Usg4oVUQFsWaurM9.Ud_DZmvlWW7Mg6Qk.s8bPpXn8OXCjcLZ5EyVLHJmtbcIWzSkNT4gOf__ Z5imINUtVyEw79oXOGuc_4DsTwIdKHhSZuwi1ZOFyP2k3QF9LAhOOukHGRemPUUhoFesmKNWG7hi zKNXNzJJWiNokELiz7RKPGqDdhYGCTM5geOfYb1PlY.3wIoR8zEjbssFWIZAvOCCRT48sStvMvxQ Lf0O4pV7Jq0o67s8lw2tifwlIypYVTMRmURPAaJ3ZxA16IyaVzN06JJhEWqsQ1if5l0srOko6GR1 vVUe7TUnAROVHrZsMQ4znSG9SyVag1364Gb9YVQWmCxT4O8EPdEfpQv44Qx_Hwz1.wHWccqlnbA_ Xhef7D3LAC.Ulb.RUPeLNHRHn.wb2Z1oL_VQ8p876xIrWJIAtL7sIB8stn7iMRFNaceZuVqCllY9 MG9iQJBJP1zscyxn_L32rNtbQH9eRXV5rpPszjb1zOj96jma29owYQMEGgQzE3ZPpBCbozQOwNVy 0aHmRdzzCPlUYwozRYU_BzFyND6X9siYmwsYs.2RmeoWKeErmgcqbMHSPOqJnyIv5GOya4B59a0T Z_ymv2uC9_0P9jgS.d21PJSwAQtN_gsy8uVBIKrNXDpMjaesWD8lba196411xVDfz9yhQLEPTE8v XkRLbU9HyrSLDB2RIcj4eMIpQkcJDZ0qH66d6yoqQ8wZA7Xqicn2eE2Tv.YV2hBMx.qlXgRVxiSc mZn3fpFZLkGvyQ09VaVAJt6WrUKqfZtKyCzHBew.XsavtJ3VgBppPm8_ULJGh9v96R_JoaLRmtzo wzdRH5FsZ8u27cWd.krUuKktm.5.QDbiSWZypa9LPWiGgXR9Pk7xFSqwwPJ5GXASsUJfM0Z_yOPT QEbJW1FZg.jd_f9lNhCZr0SxSCt3frVaXKZQpJaSVdr84oIO.aAgRuzRdVfnXRPXMukhoTq8SlMO KIHlsrovo7X6RIpzN_SmNyI428vfAZwvtxHD0kVAMtDbSOAMQcafArWzXeMBY5T.vsGnCIOwF.Hr XNC94XJkogQ5xvJHn4hkLvV0X9ys1jCfmHSH6RujuKWuWq4e2med0GckdUm16OPFOM36LIPCvNce 2xeuWuiIJfT4YE02t_F1R.x13eV3bD3jrPoO.NshxrDaFqJjRin7dtM9eUdfZ6e5Yall4FhE8IPS RNvXxE0Z4ARxO9ILr53IAUi4SjxZfJDil2ehZ8wHV1fBXUCXF0FC7A6D.nTbxQWRVnOXwWBD4A4m v8bKsQXIhxmPaIl5hYp6ATNc1eeTesglLhZ0kaWRtCEgPB8Dj20VdYM_.sltElWcQpk4pCeNvQ.S zLCV5qHwWWOD2cyBFziL1nDt71EQAlbB96pf9WfNh3kk8ccW7KWFvNBk84CJ6imCbHHqWJj4ClpL rN_2vJpUPYtNFzfgJ7SfZ9x.ow.WoeISluQZkDZ_xjrW8fwRrCFmII846QZFC6N._VTOCqxNDQ5Y bcJt2vVfDtL2r3yt16UCRihIzOsx.q_LzoOpK9sK_LjJrNuaXZBRP1umAThlH8hRjUYy6GNlPPf1 jZ3ei.pisJM4cZgxMh6y1qaC3azecIXQIvaAA8FKkJ6pjYurBmH3LFSgD1ALyXGXAtME1w_RlS99 9SGjU9OKNqqk- X-Sonic-MF: X-Sonic-ID: 17fb4b19-44f2-4c48-a368-2c81ed101ab9 Received: from sonic.gate.mail.ne1.yahoo.com by sonic306.consmr.mail.gq1.yahoo.com with HTTP; Tue, 5 Sep 2023 00:17:10 +0000 Received: by hermes--production-gq1-6b7c87dcf5-m4lb7 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 47d44e3f1c1eb6468b8e2f5aaa239dad; Tue, 05 Sep 2023 00:17:07 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Porting software to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-ports List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-ports@freebsd.org X-BeenThere: freebsd-ports@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: FYI: ^T use during poudriere bulk vs. /bin/sh operation: I got a "Unsafe ckmalloc() call" Abort trap that left a mess Message-Id: <6B7D2081-B152-400F-A294-11F039077A9F@yahoo.com> Date: Mon, 4 Sep 2023 17:16:56 -0700 Cc: "jilles@freebsd.org" , Bryan Drewery To: Current FreeBSD , FreeBSD Mailing List X-Mailer: Apple Mail (2.3731.700.6) References: <6B7D2081-B152-400F-A294-11F039077A9F.ref@yahoo.com> X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.998]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_EQ_ADDR_SOME(0.00)[]; BLOCKLISTDE_FAIL(0.00)[98.137.68.84:server fail]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MLMMJ_DEST(0.00)[freebsd-ports@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.84:from]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.84:from]; FREEMAIL_FROM(0.00)[yahoo.com]; DKIM_TRACE(0.00)[yahoo.com:+]; TO_DN_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4RfmKN0TS3z4RrD During a (zfs based) poudriere bulk -a run a ^T got a: Unsafe ckmalloc() call Abort trap (core dumped) from /bin/sh . AT that point it had built something like 350 ports into packages in .building/ . After the last finish was reported for the already building ports, I killed 2 stuck poudriere related processes (the rest were gone already) and then I used: # poudriere jail -jmain-amd64-bulk_a -k [00:06:03] Unmounting file systems umount: unmount of = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/usr/tests = failed: Device busy umount: unmount of = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/usr/share = failed: Device busy umount: unmount of = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/usr/lib32 = failed: Device busy umount: unmount of = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/rescue = failed: Device busy umount: unmount of = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/usr/tests = failed: Device busy umount: unmount of = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/usr/share = failed: Device busy umount: unmount of = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/usr/lib32 = failed: Device busy umount: unmount of = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/rescue = failed: Device busy chflags: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/rescue/rcorder:= Read-only file system chflags: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/rescue/mdconfig= : Read-only file system chflags: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/rescue/mdmfs: = Read-only file system . . . chflags: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/usr/tests/lib: = Read-only file system chflags: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/usr/tests: = Read-only file system rm: /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/rescue: = Device busy rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/usr/lib32: = Device busy rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/usr/share: = Device busy rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/usr/tests: = Device busy rm: /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/usr: = Directory not empty rm: /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref: = Directory not empty rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/../40/dev: = Device busy rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/../40/var/db/po= rts: Device busy rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/../40/var/db: = Directory not empty rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/../40/var/empty= : Operation not permitted rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/../40/var: = Directory not empty rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/../40/rescue: = Device busy rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/../40/sbin/init= : Operation not permitted rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/../40/sbin: = Directory not empty rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/../40/libexec/l= d-elf.so.1: Operation not permitted rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/../40/libexec/l= d-elf32.so.1: Operation not permitted rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/../40/libexec: = Directory not empty rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/../40/packages:= Device busy rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/../40/proc: = Device busy rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/../40/distfiles= : Device busy rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/../40/compat/li= nux/proc: Device busy rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/../40/compat/li= nux: Directory not empty rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/../40/compat: = Directory not empty rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/../40/lib/libth= r.so.3: Operation not permitted rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/../40/lib/libc.= so.7: Operation not permitted rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/../40/lib/libcr= ypt.so.5: Operation not permitted rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/../40/lib: = Directory not empty rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/../40/usr/src: = Device busy rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/../40/usr/bin/c= rontab: Operation not permitted rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/../40/usr/bin/p= asswd: Operation not permitted rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/../40/usr/bin/c= hpass: Operation not permitted rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/../40/usr/bin/s= u: Operation not permitted rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/../40/usr/bin/l= ogin: Operation not permitted rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/../40/usr/bin: = Directory not empty rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/../40/usr/tests= : Device busy rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/../40/usr/ports= : Device busy rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/../40/usr/lib/l= ibrt.so.1: Operation not permitted rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/../40/usr/lib: = Directory not empty rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/../40/usr/lib32= : Device busy rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/../40/usr/share= : Device busy rm: = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/../40/usr: = Directory not empty rm: /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/../40: = Directory not empty . . . rm: /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/ref/../: = Invalid argument After that "df -m" shows lots of "Mounted on"s matching the patterns: /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/*/rescue /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/*/usr/lib32 /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/*/usr/share /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/*/usr/tests /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/*/var/db/ports /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/*/packages /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/*/usr/src /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/*/usr/ports /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/*/distfiles /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/*/dev /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/*/dev/fd = /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/*/compat/linux/proc= /usr/local/poudriere/data/.m/main-amd64-bulk_a-default/*/proc No clue if I'll ever reproduce it. The odd part of the context was I was testing bulk -J128 (no ALLOW_MAKE_JOBS use) on a ThreadRipper 1950X, so 32 hardware threads. This was part of attempting to see if I get any examples of deadlocks or corruptions or such that some earlier imports of openzfs had produced for others. For reference: # uname -apKU FreeBSD amd64-ZFS 15.0-CURRENT FreeBSD 15.0-CURRENT amd64 1500000 #118 = main-n265152-f49d6f583e9d-dirty: Mon Sep 4 14:26:56 PDT 2023 = root@amd64_ZFS:/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.a= md64/sys/GENERIC-NODBG amd64 amd64 1500000 1500000 My attribution to ^T handling is unverified: I did not find the sh.core file. It is just what the timing looked like. =3D=3D=3D Mark Millard marklmi at yahoo.com