From nobody Thu Mar 23 22:06:14 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PjKDn3RG5z40PMX for ; Thu, 23 Mar 2023 22:06:33 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic316-55.consmr.mail.gq1.yahoo.com (sonic316-55.consmr.mail.gq1.yahoo.com [98.137.69.31]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PjKDm6ZYTz4M5B for ; Thu, 23 Mar 2023 22:06:32 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1679609190; bh=o9S3qdcS9aQat0rgRhxOPvSULl7LgGuDCSajQbuknXk=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=DSSofkhLB3KyXb7aLKnK5mkgG03E8TINsQgIrP1vbpUr9LoqzmqHv0j05UzbO4v1bRLNIe9mV2OTJtU+yB+vKS1vu0/wevCDZwmLV8XjoI3mBiwvNEpcrCO12Zwgb4Z+HnZTw4PS+0IuobJFYS22yfk6tt6X7bhv1EPxePMHLKMAYm96iM0yfMK3b9MPyIYT/S+lq51I12HADgewMVmu7F9mPxrD8u9UpyX33Jv5f+BP6idwY98QXtGTBjj2GjtmunH16dukCqmfyPaLICpNOXkNpsjOtFhcAU8iSmq75UvqT6XCA24T0xPfiG18oTlpuNLJFn+m8UIFcf1HrgkWAQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1679609190; bh=4sUvc6GbfBlKbhkZuKiciRaZN47yiTbICKuzUaGfprR=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=lmXMuDBTkQxS8moRrPJJZNCAXSFoo75luPNkqbF+jV/5as8/hY/FnW40Q7e05vngje7adHQWcIpumcYAqbgzUZmPk/L4BYZE0+N19yHHxT66a3DNBD2OQFJ7ur/rmZNL1vIUK7YF4thBW7WOBXPcOkxTGylN0LfGdEe6W9F1lC/aSeMY0ejuh1Byf4zNtdyqfx0hlwqnetNsCS3Aic058uQmaeSjZkOFfOBGfpp0KLSw1gTrzU2kFLfbpoqxxaIiaEzQ3Dn+1xObIE3B75pxMITdyIv8WvFU+LmEfOLFEDtW7E+XFWXrQbsSQgQMdGs84MHtaxIpmQcpplX2U4p+0w== X-YMail-OSG: xMUEmI4VM1ko0QeUII3r2.2H1nNo4OK985S4ApmbHJqCqmk4mb63Q_agm_h5Jpk SNKNIS5URAGK1Wz14l9QcK9Nxtnzqbc86rq6KdBxtbmKu96YTK72xM1UGARr7EDUYzTs8rtsFyva JaAMOaPe8q43ynnzvhg3gVt1JqyC2qnAeqZdvlQz0kNx_IVEsabEXyHyZe4weXokFKAQ4mD8j9O8 uOKECghaC_Je.mTV4eOoHxKUnmRUb7zuJK.9PH1V.xrVtZWKBbnturYGQTWaE3kRZX9PM32MLGwj RSdJYiffzMjxWh1xKTXE_E8FBsKNIX8BSXpbni9jOpCeBQ3gcldrMlqbEvyGws6xRQxNktcz9MzO X1Cl7FqpZHhqeecX.WAhCqhYhYC8RjwQ3RcJ5Gpm3T3C03z7l3cyKrK6P5eSZvAZnPoVSw7E4Mke X9fAnpCB0aDw9FEX_CbX7QXBJQZjcolSi2gY7yxgyQxx6eKYXQt36pJYJ7SCRsifCMM_uUd5H.OC c.DbY_EFfixzXy0Bj4owVS9bP.0Am20i0rQY7ncWzt4dQ.k7jIWBOCXxQo3LqMCCACED0UXK7SjB Rgjax_c8tPLkIU78BdpD_UFeR13uts3Qm9Mc5vWVPlwu2UGXp1UkHgizIcAs50w7czwSc6Z3QnQe RcEBnuPC4zlaphKYx_MPxfAYd5HG8iYAHinfIMXQ.GswHhHrIAh34QhhVc6d6trrjBs.7dY_YKv2 qyvUc_2v5WYZaeTrpHxHwm.yrNFnGL3q3lYBXIQwWKIIbxLHso8lkv66qiAGVCib9Jipnh_y9UGK M2JoP0aKwrCHz48vlXfA7gv4onbWVJMZPFUKxa.eDYNxubwnwUYQ_j1dDiw5f2rXkoLUKQly553A p1LMD7JmG7rnfooBoo3Y6THQYgY2mAw0XVEQfzLujrhGAkf_.8GarUYOUTSsCTA.L5luQMZ_fev. ZqQ_pIgSjNPiGp_xCbUB1cGDoslmZHdgNfB2bi2wJOiMEOz2ah9ydPdwe7NOBBx6G8Jtgw.FlCPc 0kt7v6xf51b28FoHp17eQT17mdKSH7FGJFOKxrIosb7HfRF1RCDyxF6mkgrji0znEfC7sHz4ljhV yikv9Urz81foe7CBkngdBYIOv3kMWTe9ivg1en3hrZ9Irv5xK9OwdZykPf9S_pE..nZyeHe4GJkD VtLzwsNR__knY8bDVqiPfhrq7ke95azQpMrH4WEcyi4QI7l4fiTo0Dp9GVl3_RrZdDb0rMewDFuu soN7qWRhH4SJu0gAgVELYPeusNeZBydIyqA2bq7pUxqoXgbtn3bwiRiUg0qUEaKmCPoxoxyFbxzg wESchkCwbXtYUdQ.CDXXP1yI4SKY0OsBSN_miJbWS80H3nvYeV1jS3c73KUEM597pCCeCHMbEASO sA7bsBVfWITKB3rp6kaeKdntONAYzjhmNnKlr9HgLzNlJiNaM8GvzFcCkUt9_gYx1_J8HlxsNa78 ssFk2pAhGdD.CYf9PzI7ISnrIot7XSxnbEtX5NpHI33fKksG5ydKwzIJcYm4vrPJFpBIG8a9U5wX Tw3ik2fW9dnpTg.WE2aclkhTwAAZz6rcDOBVCpfdC6RGOxmmnO4vXomXR2Wwww8C_DEDHok6pCSP pQXdtB147UkIfkXRgm4pF.chp19QtvdZfuGjy9YBhdcL3Q31FiUCfY5MF8ehC.euNldtMouiOFre gdoaduPYwRo8a3V2mOIJqWsf.Wlmwb2sZv5t4op5LVmaPBkFVvOQOJfd_0kUYxkDy2WHCWRW87jc ELoYNQo8FG7xb2UHxkBFDI3eo6jiwa2nSrDqwtfAfVt9mjBy022rd5MIWz9CcU468R9bgi_rU9TF aSb5fbjTRKF6MdqldTU25YgORkK6psOIvJEwCkr3ybR.nJkqXCy2zsDm7QcBQJEWxeQd8rB5eyh. .sS8Hb.DSaipAVs82ccvLyBADpThGt_xqhpFGx1Nn7ExEJe5riyD9oieykXlsvyKheylsDv21kwM PwzVoPtQSHPlWINqaq2cJBFo.XSAeBk_08q25cpTHuEe0Rg72LiEMFqpuCyMew6shOD1rt9UfYx. J2QXbwxAoMC2ULq4xvrKAHRg6lY2iIvHoSN_aBNKLpUgqbd7o6v7.f2fBc_GDpJlOOvOGGATXv0S DnmERnvNpgG0bx2iSCBR4Gg0a8g03JHvya9SnsBBfoFJtKQUhAKXwCqgj4U7Nz9Gdurzil8Bd0Uu t7.TxjfNSu7FXZnh935KDajIt1670bMgPR7BCH8MrzXE20mHuf0nE5SOUZMWzDjHz6WmrEuzRwUJ d0EZf X-Sonic-MF: X-Sonic-ID: 54ad577e-67b6-4c3b-9df4-753a5a82d7a9 Received: from sonic.gate.mail.ne1.yahoo.com by sonic316.consmr.mail.gq1.yahoo.com with HTTP; Thu, 23 Mar 2023 22:06:30 +0000 Received: by hermes--production-ne1-759c9b8c64-l4sxl (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID bfb0c18e226a0970b281309809bebda1; Thu, 23 Mar 2023 22:06:26 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: Periodic rant about SCHED_ULE From: Mark Millard In-Reply-To: Date: Thu, 23 Mar 2023 15:06:14 -0700 Cc: FreeBSD Hackers Content-Transfer-Encoding: quoted-printable Message-Id: <907C8A16-D271-4C32-8DF2-A719CCC76FE0@yahoo.com> References: <24F6D88F-3F15-48FB-AA5A-AFD4B77A1D39.ref@yahoo.com> <24F6D88F-3F15-48FB-AA5A-AFD4B77A1D39@yahoo.com> To: Warner Losh X-Mailer: Apple Mail (2.3731.400.51.1.1) X-Rspamd-Queue-Id: 4PjKDm6ZYTz4M5B X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On Mar 23, 2023, at 13:53, Warner Losh wrote: >=20 >> On Thu, Mar 23, 2023, 9:46 PM Mark Millard wrote: >> Warner Losh wrote on >> Date: Wed, 22 Mar 2023 22:57:08 UTC : >>=20 >> > On Wed, Mar 22, 2023 at 1:41=E2=80=AFPM George Mitchell = >> > wrote: >> >=20 >> > > service dnetc start >> > > I am literally running "make buildworld" with no additional = options. >> > > >> > > >> > So what are the results for make buildworld -j $(sysctl -n hw.ncpu = )? >>=20 >>=20 >> Note: My experiments have been in this -j $(sysctl -n hw.ncpu ) >> realm. >>=20 >> > ULE scales much better, but when there's too little to do it can = make poor >> > choices. >> >=20 >> > ULE is better locked and don't fall over on high core count systems = like >> > BSD does at moderate load. >>=20 >> (I'm presuming the above is not about the specifics >> of the effectively different interpretations of the >> likes of having extra "nice 20" activity by the two >> schedulers for the examples related to the original >> "rant", other than the -jN issue.) >>=20 >> Any idea on what scale "high core count systems" need to be >> for what sort of "moderate load" to end up with signficant >> differences? >=20 > Sched_bsd is basically unusable on my 64 core 128 thread machine with = make -j 150 (nice or no). So, well beyond what I've access to, even if 32 core 64 thread would also show such issues. > With ULE I don't notice. That's not to say ule can't be better (me not = noticing is hardly scientific), but I tried sched bsd when I got the = thread ripper and found the machine too unresponsive when I was doing = large builds...=20 So the classification is based on responsiveness. Good to know. (I assume you had avoided having interactive processes end up with their kernel stacks swapped out. That is a separate sort of issue.) > But I wasn't playing video on this box... so maybe I hit a local = optimal point... No X11 or such invovled in my context. Mostly ssh over EtherNet. I'm rarely at the (video) console (no serial console). I did not see responsiveness issues in the ssh sessions during my activities with the 2 schedulers. (So I'd not explicitly commented about such at the time.) Thanks for reporting the example. > Warner >=20 >> What sort of context(s) show ULE scaling much >> better? On the 16 core ThreadRipper 1950X (32 hardware >> threads) I've really only demonstrated the "nice 20" >> distinction as significant between the schedulers so far. >> (I do not have acccess to anything with more hardware threads.) >>=20 >> Note: I've not (yet?) been looking at having just a little >> more than the number of hardware threads active (no nice >> involvement). >=20 Side note: In order to be sure to avoid interactive processes ending up with their kernel threads swapped out, I use in /etc/sysctl.conf (which prevents far more from ending up that way): # # Together this pair avoids swapping out the process kernel stacks. # This avoids processes for interacting with the system from being # hung-up by such. vm.swap_enabled=3D0 vm.swap_idle_enabled=3D0 While I have such everywhere, only some machines and a rare type of use on them is actually likely need to above in my context. After adding the above, I've never had the loss of access problem again. But nothing I've done would indicate much about X11 use reasonableness, for example. =3D=3D=3D Mark Millard marklmi at yahoo.com