From nobody Mon May 01 21:21:26 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q9GP40B9lz492BM for ; Mon, 1 May 2023 21:21:44 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic301-22.consmr.mail.gq1.yahoo.com (sonic301-22.consmr.mail.gq1.yahoo.com [98.137.64.148]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q9GP2320Bz3Qfh for ; Mon, 1 May 2023 21:21:42 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b="ioo/gDBX"; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.64.148 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1682976100; bh=zux7QH1t8qvzq6qDMuP24l22VAwqouhbdhExbhPJ4DU=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=ioo/gDBXuLGuu/bKRlJazmbiYoHrQ70/MU50OC7w3/EureLkZ9/jbHhoFScEBb9wXFLmX2hMw+sSbzPKJvWJ9SDWC3qhEKMfhbI5rC4h/VmS3ORg4U9+0hTMXF/95TWb043Q5fuhxsFgX6fQPvDFCkaOUo4HRE/6BH9K7GoAgoJ8Y53cneQ0PiTHaciPNaC5cQIIyLCE1WIfP7dfF02gKRlIGoTT2oevlYXLL8UOy0q9BAZwuuVgRJjmeedNefZ3Puz05I1xfMKTgmsvg3S+biGiOCbpoz6GK+2FRAV67X9qmAq+1zHeh1Ok73wOiRWPvtgPvgn0DefoCE4YrlhM/Q== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1682976100; bh=wTY/7LVnvXTDGO5RN8EVAKQspxL2Hv+jD6GsNRvFEaM=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=ZwnjuzUs8vA6SKxSaqSv6ONH+YjB3z9nljn7rFYmsR2trA5yNydLiSJLBcxyC7ujbYys5tx5kqXuxLpuddoixyffHDPrmSOceUkXzBPVsUuw/MFEpeZDoqcWvTr6JcLLv1T0xOXIg/4lrJ09L4GBd0s/2kst6SnclofNLaZuXl432aotV4YolmkGKE61S9UBWuZUMQmeR9s7T8MJt5lFwMhR4+k6JuzMwqguwC31d5IETH3YtKzQpnTtnEICXzc/LUulraTdpx4a/dVIs1R6gIkFnoA6WQkL3sdOGoxmY99eNeQDTZm5NZNKFPtaHzsT9WXxOd73EwOzREs35EAtCg== X-YMail-OSG: wQqzf84VM1lRmRofstmjdy0Q0U._o88s3sqEZ__MzeTFIOm1KbrkZXt4AFhtV78 toQNh8VTc75JEjc8bpSeSWOi4lN6dAvyp8PyZQA7YoVUgB4YHM.bTxXfaqjvfEb_mdxqCw_Hwst7 PeQfdaWomGy2e1H3k1JOFzDARqFfsLpWIc5kxSN8Tg18.Z5GSSbLGhc33XgiLfU.t2tZQBe8fz8H IjAsmH33sBSCVGnwNSKdGzQD9KjaEK0jytdlv4tbdI2J.J.1O6GR66SGC3PS6k0Uv.GBN.7IQpDT DoHl9hO0E9gy8BGS66aEGK1SOoy0u3HxZ64PRmJUNiqtQyIDi3dZp3ltvTPN0Xx9u9weyRwQv.ng xR3GJ_Crx5p_GRGcUpNMf6.8MUxIgtBqWsZKNeQJdFvaVezilFm6C52KoatfQOzsTEyIxdQ6mr_E btTumEaTKWiD7x4XlBSYBRwznF_5tDzgRLhoDzVLKFGs0n3XydlKAVo2AI9EiU85K36BVDBDMbT8 Nur8V5Jx1WSOq5rLruCodIqVH_LbQXbxLVxlfEKRBtjY2BDra_pgm8APFOwgsG1nchSYZfYZKd65 egRTFTgqAv7QKLgyfU4hrwsz5hKCSewoGSG0ZsYfkOPA3Fzjkc5Grox1NwkHN7j7G8rbXYCMZLVC Oh_RaNdZI6pCaIKjVxOAUwHNFxnmYmxQTPzrDTbs5VNZXK.y0RPfMfSirCPBGs89jn_CGUwlF_1n SgMeOLU3iQQbIPZ6e24pwBx2A21UnEvDBVMGd1aL8akrKsmIOP5Buy2uDbGqYj9Pv7LBYALUTzRp hPQionw.6nXR41XsJkiZFlbLQxoc0LYL1d0O6xyj3Ok74k2P_hDmMynGjdkC1D9Ae_VRWYOtuGik X7Jj00QNkebhWImVN0zDrFPnfv5ralyK755trDn9gYKBf7b7a_X70MeqFcEZtAzo9JwmcWnc1ngU LgtlEhdNANFbCbS4R4IpMrP6aW6shY5lUxHLm5j727CfQ4FMay5mPpNLVO0OLHN34HZuJyNUjisN 0zKCsiBqzimsLncUE6PuI9y.FZrHQzhpm3kVplgUO6kUrBVTUdm1Ro5OjvUKEI1pQ4DnQRRUg0b3 BcGRMgQ68CTLK1wFoM8NlPGZwZrP_duJcXE6XA_aoZLxlBnKGJlvVPiLUeqqPHVLkshLpnVc5eHh djlhc2.xkVc4pflhtnwcmW2_FoUZ_1WG3L.M7Tr_YETVk3SzCCLixueuivQ4r8PMJCx4W_FM4enV d8xTNWaipcF8k8nBHuBmmJ1XTxXcQgB9JJqZeYUeq9Nm9OpOa5uTELsjmaL4QrZeEVFfdZgcf.Vj 5z0_fHQrCw9AABmnY0NO7c0h.BJgIcu1NQrhZ2OEj3p7A.iTvlTJX2pTi.J.HaxHWf.R7HwJYv26 2zKLqv_0_WDJ.8jYDsCPBDFGgscH0LnJmprbdCDd0Krcmh37fs59a87VMYUQsU49yuzm7_LkEwmQ yDSemiBXenHs9a63Q4BbB3u6R2qxA7pFpcyaWaFUgLM8oVK8W3UqJs3aG30TEAwaf5WDXdA.xHRi iipktYZ5sGxuexc1vgFqrZmYAdH0YcNArCe.6UjhpD_rlssSK76gvF7R11Dcy_b_KX2WIBbk4LSd 1uzZVvPIQAVQADuUAnHJ.VD0daWxi.JD0B.wKAOISFdqLk6GsWxfQdgOMVgks7XvlPfWI8uFLGqJ SQa3ZTbMD5sp9DNtlndGQWqqRNsirHhdL8VT.g6q88gdacExcKF6YtKvma.JEbmxkFV_uROxgkYd i3yWuft03B_VtxoMj20HE3u_ZqJi.irm6zm_A9yXyBXgSGVSu56eAiacxS0S1mxrnsgTd8DUxXES q0N6GFBINYXh45CjSIZYH_m90zubpM08xS.K10_rHk0Nva5AJKTuymK42Tk276vMAjPC41lB9Z.S 85B3FWcpcQvQuXGJ6HKhoKJJmps_rOUAsUZ8rVnkLGd0uqx1UaS59cBKvqWdeG0zOjhITT9YsDwp CeOVxuqjh8AiG.6IoFEfBSfepQbDqu.2IK86GwAuOG0WTZIrr9aQyYtMgu5bGJCNBHV8r.9Ogv1O lEx0ndeRYl1s7KdvKeMjiM5qyfuXaQUNVIzsnI_XRFITKbkj9vSyWtsaNwF6bM9kOq3wBd2wNEfW U8PNZuLaBHT4ZaYLLcy5ORSyd41FOdQNoIWfUOtoffgZy3mkURO2vcWIdeo.wMGgse9lANZ9abIf U6DzoaKLcenJ_5Tf0XMB5nadCQ.h1U.Ve1K39nVw4jiSiF330cwwo.bKvtX9Y2xbZZwo4RndO_hV 1J4k- X-Sonic-MF: X-Sonic-ID: 8f881a25-07ff-4f4f-8d0b-a3b62dbac332 Received: from sonic.gate.mail.ne1.yahoo.com by sonic301.consmr.mail.gq1.yahoo.com with HTTP; Mon, 1 May 2023 21:21:40 +0000 Received: by hermes--production-gq1-546798879c-l2qgj (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 23455dc20bc6a2c6fc14c38aa33e2c08; Mon, 01 May 2023 21:21:37 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: armv8.2-A+ tuned FreeBSD kernels vs. poudriere bulk and USB3 media: tx->tx_quiesce_done_cv related blocking of processes? From: Mark Millard In-Reply-To: <8F883286-D713-4632-9575-5813E885D125@yahoo.com> Date: Mon, 1 May 2023 14:21:26 -0700 Cc: FreeBSD Hackers , freebsd-arm Content-Transfer-Encoding: 7bit Message-Id: References: <7AE28A5B-109E-4C26-9D70-BCA5D49CD79D@yahoo.com> <02DC03AE-E082-4FB5-AA0D-396F64CC23CB@yahoo.com> <8F883286-D713-4632-9575-5813E885D125@yahoo.com> To: Mateusz Guzik X-Mailer: Apple Mail (2.3731.400.51.1.1) X-Spamd-Result: default: False [-1.70 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-0.202]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MIME_TRACE(0.00)[0:+]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; TO_DN_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[yahoo.com]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.148:from]; RCVD_TLS_LAST(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org] X-Rspamd-Queue-Id: 4Q9GP2320Bz3Qfh X-Spamd-Bar: - X-ThisMailContainsUnwantedMimeParts: N On May 1, 2023, at 12:57, Mark Millard wrote: > On Apr 30, 2023, at 18:57, Mark Millard wrote: > >> On Apr 30, 2023, at 17:44, Mateusz Guzik wrote: >> >>> can you redo zfs test with: >>> sysctl vfs.zfs.per_txg_dirty_frees_percent=5 >> >> Sure. >> >> Result summary: Seems to have avoided the sustained periods >> of low load average activity. Much better for the context. >> >> >> Context: Original ZFS USB3 media. World or Kernel in use >> had been built (non-debug style) using: >> >> -mcpu=cortex-a78C+flagm+nofp16fml >> >> >> Steps for this test . . . >> >> # poudriere pkgclean -A -jmain-CA78C >> . . . >> >> # sysctl vfs.zfs.per_txg_dirty_frees_percent=5 >> vfs.zfs.per_txg_dirty_frees_percent: 30 -> 5 >> >> # grep USE_TMPFS= /usr/local/etc/poudriere.conf >> # EXAMPLE: USE_TMPFS="wrkdir data" >> USE_TMPFS="data" >> #USE_TMPFS=all >> >> # poudriere bulk -jmain-CA78C -w -f ~/origins/CA78C-origins.txt >> . . . >> >> At 15 minutes into the build: >> 46 ports in 1st 15 minutes. Load average stayed reasonable >> for the configuration. >> >> At 30 minutes into the build: >> 102 ports in 1st 30 minutes. Load average still reasonable >> for the configuration. >> >> Looks good compared to before. >> >> >> I've no clue what optimal would be for the context, but >> >> vfs.zfs.per_txg_dirty_frees_percent=5 >> >> is vastly better for the context than the default 30 was. >> >> Thanks. >> >> >> I'm going to stop the test and do the conversion to the >> U2 960GB Optane media in the USB3 adaptor and then >> compare USE_TMPFS=data vs. USE_TMPFS=all --but using your >> vfs.zfs.per_txg_dirty_frees_percent=5 assignment. >> > > Took a while to actually get around to stopping the test. > It got 186 of the ports built in the 1st hour. (A from > scratch build, starting with building pkg.) > > I finally have started the U2 960GB Optane based tests, > currently USE_TMPFS=data . The initial activity looks > like it might build about as many ports as the earlier > USE_TMPFS=all test (for different media, > vfs.zfs.per_txg_dirty_frees_percent being 30). > > . . . waiting . . . > > It got 222 of the ports built in the 1st hour, again > starting with pkg. That compares to 262 for the earlier > USE_TMPFS=all test. > > None of these ports form the first hour are large, long > running port builds, none using large scale amounts of > storage space for its builder. (As I build things, rust > for example, uses 17GiBytes+ of file system space, more > than half of the size of the RAM in the Windows Dev Kit > 2023 for USE_TEMPFS=all just for file system content.) > > Now for a USE_TMPFS=all build test with > vfs.zfs.per_txg_dirty_frees_percent being 5 . I may try > letting that run to completion. (The configuration has > 118 GiBytes of swap for paging activity.) USE_TMPFS=all with the U2 960GB Optane USB3 media: It built 270 ports in the first hour. I'm going to let it run to completion (assuming that it does in under, say, 16 hours total for the overall 480 ports). I'll see what it is like when llvm15, llvm16, rust, and possibly gcc13 or such will likely build in overlapping time frames. (I do not build LTO style and, for gcc* do do not even use a bootstrap style. My usage of compiler toolchains between toolchain builds is minor compared to the toolchain builds themselves for the most part and my time preferences for builds are fairly strongly biased to less wait time.) === Mark Millard marklmi at yahoo.com