From nobody Thu Mar 23 04:09:22 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PhsLG3dJmz40VhH for ; Thu, 23 Mar 2023 04:09:42 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic306-21.consmr.mail.gq1.yahoo.com (sonic306-21.consmr.mail.gq1.yahoo.com [98.137.68.84]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PhsLF1Rwqz3Brq for ; Thu, 23 Mar 2023 04:09:41 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=IIu7GuDm; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.68.84 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1679544578; bh=pB3PRf8WWhiNLcCu+Ix0rJK8MB9kjPIL4BWbyIxddQA=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=IIu7GuDmyeicPGWrkNQADuvmlM+fQmx1UkjDdiHu1zbTivLgRbY8P2wQBH7LIjyC+9bSlvp2UIMAX3aVbJNSVZwAao7QR3jcr/yYLQlWWcoYYUaHEw6a8U67n3xQgRr8YqyqD8rVk1KnVEqigCliNMrrAr1KS/AzowUgOXqiPxA61yp/NRo3fSG7T2QlDlzjCkolo/QXQkGUIE/6XuODIKqteuPvsTFAGsWDuYbDejhmgVX2hPKEUGel0fneLjjUbQGnkDhZSL/dR4vhAn0teBPQyHllthFkPEkw0A0gHuRObqq1IpalrYkz9/WYuCK3fxuKKIICk+ZLkvsnU+ZXgA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1679544578; bh=c73l7JdaCsaM10SQ2TgnRM5llYog5DKajFB0Ozl6ad6=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=gsFt7Qr76wFo8ejjjkY/8NF+Pw6xrgt/3A+BWfiCruA5pRwU2dpi+z9UnIK/uLoT3aG4tUsJZNnOGp8u6IWJco3pYVnWGIO6Y9eXk8MKBMpDeOT5WxWbQK9mga4hfids3dRmHZCLDy1m63qYoQyY3Fc8slffZ16xQNDIoyBfRQXrZ6IRQUKQZNTdmqVTHm1gi+6vceZ4d52C+7TaP4yZvmSQQ4RvbklIjoJnw+RpT+RiSoDbbLlZ4u0COm+SKg6YassljCQjH0fFkVozVkHkl40QwAX8N2FVccYBlJh+wDGqhrBPiIOl9NH98FvcsNy2JYuuau/nH69rKKxrUOQrow== X-YMail-OSG: 3Hziu68VM1nqL7ueyD3vsI4TB2gv52BioDOWZbyT3KIZRECyJ3eN.n4QH7naEdp uKHF08I438xzYo66iiOgrBN_YKUrL7zV1calPTurcDNVkLNqsZxPFfrEI4wgFi78_nyCxdXq3_KB M4A5ujy_uWZrEYv4NbDbsFzIXQuhpr00rTrg.gSJ3csCzTqOpHpGUTPO3POVFUVqUgWeWgg9ln6Q RTx_odQmscaHswmZXnCD7F1LDnHODUl3JLwo5kRWOArQsSV7xCcC7hnTnG.97KI.kRJ8j8PNxPIl NjWf3TxPfyC3B3oNc67gdXQSxROVcVV8cTQ1fHsKW4izQKKm47Q.Sb4NGPZNNZsvWRrPAtQt26C2 IfdoYmAI1FcoyaxJtjsJzaDfN5F9pcAL4d2yh7FEftpXd5FhTkGCkB0UlYCM48pJAJ2Oyd1lTj6b FVNtQuD7xxG4WfR3pafhbStWn7525Jm5e_KdWzKAf0Do3vLRpekFKDWfTRCvIrCtveuoJf7BLtfZ if8C8.CM_4j9hTnaYfOqwc8Gu.JoFSven4Q52EvbT3rXon06xI1U46BqVorbVa2.fjmmcDInHgv3 aJnndpGjNrRr68nkfR717Jf4ik.yJqkj6Y2JBWaO0n_mHsbQEJsJaqtH7K0tA69lkqDKMjBH.H4U ni0s72t2cSSadiAHZwMhYugHOBNHN4NpCVEDi46XwOcCEHw3EDo0lZ0QT5FQ9Tw8D5jAzldfkDzM hzxcMHOXJqYIC1yuBp86ILOThYq4SNlEQXGSM_LQtnjZXNFuK14SCNs6DuwEinkJa.Ty4lT5pf3b xSjJD6A4cR7_ndobYSCqrJ_CA8M5FPeykzwtEnnvIyy6Jq3274iW5IgcIXvRL1QFn9FciTr3p2bk K4LwsJjkKQ8CW6fZhOMzgCa35siJKrSHt_UfLY3sCCP.zop.gm4kxBBazV2pELLDn6CCSAEhQ6aB ZKlK4Fg_4FYTfKTh4apMeYhSYvBVT1X3G5.Qy7C2BZ.DhJz.is3tLNV9o.zKx7GDpkxEU.fbxmLr QkN9yr6GsphSDQ5SEn.93By1rqVUrLMAQkBu0_LW9ojor4xB04x9Gc9K3FJR84bIhekXhljv8mbG vOWVOJWJe5ct7S46umkxRyhZgkZI_IRSi_zHr3ShBQSG3KSBRq2XvbIRTZ6yDjR3ZHH12nBG0jsQ hJWMu55Pim9ZY3vwsnTabYuk38IJOOea9MHts_tSoOu0bK773x00vRWH4NGGCv3NkEEzQ6zHCJQz hg0W4bnkqUJ9O7hO7y4YtH01USz8u.BTrEbBaQfsOrAGy5fawn.ZTqIZ111HqTfr.2ONlexDOSpP K19VH66gfdHRZl6vbmkI4iWLPJPHJ8KLznNRimGfCFkMxB71gZbyTfQARQPmgB9QDRmb8.lg5M_j Bp19oqSgzRIMnzekLID0pz2d8CmS2FLunfW54euz0ccw_l5ni8.scAqm17PF3TSgYEV.wqn6D0kH CqPdHHh.dHdrqbrsrOiXiHWVoHmK_FtiXVYgkYHxDmmbQ1J__dYAeYde_z0OfiLijKwTz_e2TM4P 5vQkrvWPPL0KCQD3t3.JJcscWzvg7LDhLRjtAi4GeRP23qa_7zJkV.URwnDMlrPHuUlQzwW30gxQ 0a21EiU6DTdwA0gjOJujGaqOOEoNQofPabZiK57.BYRckLi3Utx_LFfwTEym_3blhBJjaFDYdnJ_ fTo_loC56Hft4K2btBnn0Q42VAyvuFcQpClAMMLXWUDuaWmewxH9.Cs6QIsI_QRRsgAQQ5.XwjOP QUAW1b9Bg_48qd3roRerrEbSA.HYW23FQD_FOZwinmLp1WWv0KEjEoI0gr8XBhSgTu0Ib0rIW3wF cnmVBOf0NIqC29tQnRxI8.wX.UUDeoiQPZnamkkH2jnGU.LE_kNK6yED9t9RepuavsQgE4kPXSVn 03VctIqzhuRDdpXn3Ja0C4Oegkm4Jq5TzjOO7HzfqkErvGXxVqLgBISlBIvhYR0xvW7oc7_PRDnq mBO8OMs0EoxY90epE1Z5t3_sdzCyus9bT7x8W4isyw0WukKeZXUdZzZHDoVNmTPFVc8Bwu24FbQJ 3.KhXUvE5aV2Iz7ge6..mU.C8vIUbLV.XI.WpiyCl4LgpWNrrmgaC0Ce6E_c44_QGSsaS4Tr8krX 8KPlr_9IYi9l.NjE.VJIcvENc8ylSxOwLxP4GdIGLNIVlU96F7DB6qhFxWpW_mNOvgJmwsuzd8BC _zejEt929YoUALUX1hls4Ygk1ToVQUcPsimQtVw99WZlDoUSXDEWrw3kzre8kDaG4op5e4GNG_SB f X-Sonic-MF: X-Sonic-ID: 4e028b8f-853c-42ec-a813-dcc0da67ecb9 Received: from sonic.gate.mail.ne1.yahoo.com by sonic306.consmr.mail.gq1.yahoo.com with HTTP; Thu, 23 Mar 2023 04:09:38 +0000 Received: by hermes--production-bf1-777648578f-lk5ld (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID c140b7de6075f5be5081a3e6c61a88fe; Thu, 23 Mar 2023 04:09:34 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: Periodic rant about SCHED_ULE From: Mark Millard In-Reply-To: Date: Wed, 22 Mar 2023 21:09:22 -0700 Cc: FreeBSD Hackers Content-Transfer-Encoding: quoted-printable Message-Id: <6822D3CE-789E-4FB0-BB62-BEEEA65DB72F@yahoo.com> References: <6BD317F2-7EDD-45C0-9DC9-5B94C1BBB8E1.ref@yahoo.com> <6BD317F2-7EDD-45C0-9DC9-5B94C1BBB8E1@yahoo.com> <952d9795-19dc-8ad1-bb75-5c556ca6795a@m5p.com> <78EF511D-BAF1-495F-BAC9-03AC1B8FD56A@yahoo.com> <0196A767-5DC2-4DA3-BA5E-62A358CBCF64@yahoo.com> To: George Mitchell X-Mailer: Apple Mail (2.3731.400.51.1.1) X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TAGGED_RCPT(0.00)[freebsd]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.84:from]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; TO_DN_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[yahoo.com]; RCVD_TLS_LAST(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.84:from] X-Rspamd-Queue-Id: 4PhsLF1Rwqz3Brq X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N [I added a -j32 buildworld buildkernel with SCHED_4BSD and dnetc-in-use comparison, to the other ThreadRipper 1950X examples. SCHED_4BSD does take notably less time than SCHED_ULE when dnetc is also active: still a good match to the simple round-robin for this building activity. I will note that the 1950X UEFI/firmware is not configured present itself as NUMA but the FreeBSD kernels in use are NUMA capable as built.] On Mar 22, 2023, at 19:44, Mark Millard wrote: > On Mar 22, 2023, at 18:08, Mark Millard wrote: >=20 >> On Mar 22, 2023, at 18:03, Mark Millard wrote: >>=20 >>> On Mar 22, 2023, at 16:17, Mark Millard wrote: >>>=20 >>>> On Mar 22, 2023, at 15:39, Mark Millard wrote: >>>>=20 >>>>> On Mar 22, 2023, at 13:34, Mark Millard wrote: >>>>>=20 >>>>>> On Mar 22, 2023, at 12:40, George Mitchell = wrote: >>>>>>=20 >>>>>>> On 3/22/23 15:21, Mark Millard wrote: >>>>>>>> George Mitchell wrote on >>>>>>>> Date: Wed, 22 Mar 2023 17:36:39 UTC : >>>>>>>> [...] >>>>>>>>> Here are the very complicated instructions for reproducing the = problem: >>>>>>>>> 1. Install and start misc/dnetc from ports. >>>>>>>> Installing is likely easy, as likely would be building >>>>>>>> with default options (if any). I know nothing about >>>>>>>> starting misc/dnetc so that is research. (Possibly >>>>>>>> trivial, although if it has alternatives to control >>>>>>>> then I'd need to match that context too.) >>>>>>>=20 >>>>>>> service dnetc start >>>>>>=20 >>>>>> I built and installed misc/dnetc and got a binary >>>>>> blob that clearly was not built in my environment: >>>>>>=20 >>>>>> # file /usr/local/distributed.net/dnetc >>>>>> /usr/local/distributed.net/dnetc: ELF 64-bit LSB executable, = x86-64, version 1 (FreeBSD), statically linked, for FreeBSD 10.1 = (1001515), FreeBSD-style, stripped >>>>>>=20 >>>>>> Way older FreeBSD vintage than the locally available toolchains >>>>>> would normally build. Some might be cautious about such a thing. >>>>>>=20 >>>>>> The man page reported that: >>>>>>=20 >>>>>> QUOTE >>>>>> If you have never run the client before, it will initiate the = menu-driven >>>>>> configuration. Save and quit when done, the configuration file = will be >>>>>> saved in the same directory as the client. Now, simply restart = the >>>>>> client. =46rom that point on it will use the saved configuration. >>>>>> END QUOTE >>>>>>=20 >>>>>> I've not seen what the configuration asks about yet. >>>>>=20 >>>>> I went through the configuration, basically just looking >>>>> at it, other than providing an E-mail address. Then . . . >>>>>=20 >>>>> $ sudo service dnetc start >>>>> Password: >>>>> Cannot 'start' dnetc. Set dnetc_enable to YES in /etc/rc.conf or = use 'onestart' instead of 'start'. >>>>>=20 >>>>> $ sudo service dnetc onestart >>>>>=20 >>>>> I just let it run without any extra competing activity, other >>>>> than I had my patched version of top running. It records and >>>>> reports various maximum-observed (MaxObs) figures, here >>>>> the load averages being relevant. >>>>>=20 >>>>> Top showed that dnetc started 32 processes, one per hardware >>>>> thread. Mostly I saw: 100% nice and 0% idle. >>>>>=20 >>>>> Letting it run and then looking at the load averages (and >>>>> their matching MaxObs figures) after something like 60+ min >>>>> (not carefully timed: was doing other things) showed: >>>>>=20 >>>>> load averages: 31.97, 31.88, 31.66 MaxObs: 32.12, 31.97, = 31.66 >>>>>=20 >>>>> (Note: The machine had been up for over 2.75 days before >>>>> starting this and had not been building much of anything >>>>> during that time.) >>>>>=20 >>>>> I've not yet experimented with having other, significant >>>>> competing activity. >>>>>=20 >>>>>>>>> 2. Run "make buildworld". >>>>>>>> So on the 32 hardware-thread (16 cores) amd64 machine that >>>>>>>> I have access to, the test is to only have buildworld use >>>>>>>> about one hardware thread, no matter what else is going on. >>>>>>>> I never would have guessed that the steps would not involve >>>>>>>> more like -j$(sysctl -n hw.ncpu) (so around -j32 in this >>>>>>>> context). So it is good that you provided your note or >>>>>>>> I'd not know if I'd done similarly or not when trying such. >>>>>>>> [Note: -j1 and lack of -j are not strictly equivalent in >>>>>>>> how make operates. As I remember, the distinction makes >>>>>>>> a notable difference in the number of subprocesses created >>>>>>>> directly by make (one per action "line" vs. one for the >>>>>>>> whole block?). So even using -j1 might make a difference >>>>>>>> vs. what you specified. I'd have to test to see.] >>>>>>>=20 >>>>>>> I am literally running "make buildworld" with no additional = options. >>>>>>=20 >>>>>> So required for repeating your results, but likely making >>>>>> such results not be interesting relative to how I normally >>>>>> deal with buildworld buildkernel and the likel, no matter >>>>>> if there is other activity in an overlapping time frame or >>>>>> not: my time preferences are too strong to wait for a single >>>>>> hardware thread to do my normal builds, even with no >>>>>> competing activity on the builder. >>>>>>=20 >>>>>>>>> Standard out conveniently reports how long it took (wall = clock). >>>>>>>> But nothing in your instructions indicate about how >>>>>>>> to get an idea much progress dnetc made during the >>>>>>>> various tests? [...] >>>>>>>=20 >>>>>>> Honestly, I've never worried about this part. But dnetc logs = its >>>>>>> progress in /usr/local/distributed.net/dnetc.txt, though not in = terms >>>>>>> that are easy to relate to real-world progress. Oddly, when I = run >>>>>>> "make buildworld," I'm primarily interested in getting the world = built. >>>>>>> Perhaps others feel differently. >>>>>>=20 >>>>>> Off topic for the specifics of the actual benchmark >>>>>> that you run: >>>>>>=20 >>>>>> Then why not use of -jN ? In my context, any buildworld >>>>>> using -j1 or no -j at all takes a huge amount of time >>>>>> longer than letting it use all the hardware threads (or >>>>>> so). (I've avoided having any I/O bound contexts for >>>>>> such.) It does not take additional load on the system >>>>>> for that to be true --including on the 4-core small arm >>>>>> boards when I happen to buildworld on such (rare). >>>>>>=20 >>>>>>=20 >>>>>>>> [...] >>>>>>>> FYI: I've never built with and run the alternate >>>>>>>> scheduler so if there is any appropriate background >>>>>>>> for that that would not be obvious on finding basic >>>>>>>> instructions, it would be appropriate to provide >>>>>>>> such notes. >>>>>>>> [...] >>>>>>>=20 >>>>>>> You have to build a new kernel, using a config file in which you = have >>>>>>> replaced "options SCHED_ULE" with "options SCHED_4BSD". -- = George >>>>>>=20 >>>>>> Thanks for the notes. >>>>>>=20 >>>>>> I've not decided if I'll do anything with the binary >>>>>> blob or not. >>>>>=20 >>>>=20 >>>> FYI: >>>>=20 >>>> It is not your specific experiment, but I started my >>>> "extra load" experimenst with . . . >>>>=20 >>>> I started a -j32 buildworld buildkernel with dnetc still >>>> running. I'm generally seeing around 55% Active and 42% >>>=20 >>> Note "Active": user, sorry. >>>=20 >>>> nice, < 2% system (it was building libllvm at this point). >>>> At that time: >>>>=20 >>>> load averages: 64.41, 60.52, 49.81 MaxObs: 64.47, 60.52, = 49.81 >>>>=20 >>>=20 >>> Contrasting results for some obj-lib32 build activity: >>> much more variety of User, nice, and system, including >>> times with < 5% user, 90+% nice. But not typical overall. >>> But lots of time roughly around 50%/50% or 35%/60%. There >>> were times with 15+% system. >>>=20 >>> Somewhat after buildkernel started: >>>=20 >>> load averages: 69.15, 64.12, 58.72 MaxObs: 75.98, 64.12, 58.72 >>>=20 >>> Harder to summarize, so overall timing reports from the >>> buildworld and buildkernel stages. >>>=20 >>>=20 >>> buildworld: >>>=20 >>> -------------------------------------------------------------- >>> ... World build completed on Wed Mar 22 16:37:57 PDT 2023 >>> ... World built in 2615 seconds, ncpu: 32, make -j32 >>> -------------------------------------------------------------- >>>=20 >>>=20 >>> buildkernel: >>>=20 >>> -------------------------------------------------------------- >>> ... Kernel build for GENERIC-NODBG completed on Wed Mar 22 16:43:10 = PDT 2023 >>> -------------------------------------------------------------- >>> ... Kernel(s) GENERIC-NODBG built in 311 seconds, ncpu: 32, make = -j32 >>> -------------------------------------------------------------- >>>=20 >>> Afterwards: >>>=20 >>> load averages: 36.08, 53.14, 55.79 MaxObs: 75.98, 65.77, 59.84 >>>=20 >>>=20 >>> I then did (not all in the same window): >>>=20 >>> $ sudo service dnetc onestop >>> # rm -fr /usr/obj/BUILDs/main-amd64-nodbg-clang-alt/usr/ >>>=20 >>> before another -j32 buildworld buildkernel (no dnetc). The >>> reuslts for this were: >>>=20 >>>=20 >>> buildworld: >>>=20 >>> -------------------------------------------------------------- >>> ... World build completed on Wed Mar 22 17:39:19 PDT 2023 >>> ... World built in 1240 seconds, ncpu: 32, make -j32 >>> -------------------------------------------------------------- >>>=20 >>> (compared to the 2615 for dnetc also in use) >>>=20 >>>=20 >>> buildkernel: >>>=20 >>> -------------------------------------------------------------- >>> ... Kernel build for GENERIC-NODBG completed on Wed Mar 22 17:41:17 = PDT 2023 >>> -------------------------------------------------------------- >>> ... Kernel(s) GENERIC-NODBG built in 118 seconds, ncpu: 32, make = -j32 >>> -------------------------------------------------------------- >>>=20 >>> (compared to the 311 for dnetc also in use) >>=20 >> I forgot to show the MaxObs load averages for the no-dnetc >> context: >>=20 >> MaxObs: 39.77, 32.15, 25.75 >>=20 >>> Experiments without -j32 will take a lot longer, even >>> without dnetc in use. I'm not sure there will be such >>> results today. >>>=20 >>=20 >=20 > I decided to do some more of the less time consuming > testing. SCHED_4BSD, no dnetc, -j32 buildworld buildkernel : >=20 >=20 > buildworld: >=20 > -------------------------------------------------------------- > ... World build completed on Wed Mar 22 19:16:35 PDT 2023 > ... World built in 1235 seconds, ncpu: 32, make -j32 > -------------------------------------------------------------- >=20 > (compared to 1240 for SCHED_ULE) >=20 > So: no significant difference. >=20 >=20 > buildkernel (SCHED_4BSD building a SCHED_4BSD): >=20 > -------------------------------------------------------------- > ... Kernel build for GENERIC-NODBG-SCHED_4BSD completed on Wed Mar 22 = 19:18:34 PDT 2023 > -------------------------------------------------------------- > ... Kernel(s) GENERIC-NODBG-SCHED_4BSD built in 119 seconds, ncpu: = 32, make -j32 > -------------------------------------------------------------- >=20 > (compared to 118 for SCHED_ULE building a SCHED_ULE) >=20 > So: no significant difference. I again forgot to show MaxObs load averages (for the above): MaxObs: 39.23, 31.58, 24.30 > I'll try it with dnetc also active. >=20 I still have no good indication of dnetc progress to allow comparison of the combination. So the below focuses on buildworld buildkernel . I expect that the comparative results suggest a buildworld/buildkernel vs. dnetc progress tradeoff, not that I can well quantify it. The below are with dnetc also active. load averages, MaxObs: 73.03, 65.48, 56.30 (I remembered this time!) buildworld: -------------------------------------------------------------- ... World build completed on Wed Mar 22 20:15:56 PDT 2023 ... World built in 1667 seconds, ncpu: 32, make -j32 -------------------------------------------------------------- (compared to 2615 for SCHED_ULE with dnetc and to 1240 or so for no dnetc) buildkernel: -------------------------------------------------------------- ... Kernel build for GENERIC-NODBG-SCHED_4BSD completed on Wed Mar 22 = 20:18:34 PDT 2023 -------------------------------------------------------------- ... Kernel(s) GENERIC-NODBG-SCHED_4BSD built in 158 seconds, ncpu: 32, = make -j32 -------------------------------------------------------------- (compared to 311 for SCHED_ULE with dnetc and to 118 or so for no dnetc) With dnetc active, it does not take being near -j1 (or no -j) for buildworld buildkernel to take noticably less time: -j32 (the number of hardware threads, 16 cores) also takes noticeably less time. buildworld buildkernel in this context seems to be a good match to SCHED_4BSD and its round-robin. (I make no general claim to SCHED_4BSD being better across a large range of contexts.) I've not decided if I'll try anything like a -j1 or no -j alternative. Without dnetc active, SCHED_ULE and SCHED_4BSD did not make much of a distinction. For how I use the builder machines, the scheduler choice is not suggested to be significant for my system-build activities. I've not tested port building in poudriere-devel for how I configure such. But nothing suggests to me to expect a significant distinction between the 2 schedulers for my way of working for building packages from ports. =3D=3D=3D Mark Millard marklmi at yahoo.com