From nobody Fri Apr 15 22:41:25 2022 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id A9C292DC6B2 for ; Fri, 15 Apr 2022 22:41:36 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic312-24.consmr.mail.gq1.yahoo.com (sonic312-24.consmr.mail.gq1.yahoo.com [98.137.69.205]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4KgBC34TZCz3lNj for ; Fri, 15 Apr 2022 22:41:35 +0000 (UTC) (envelope-from marklmi@yahoo.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1650062488; bh=9yQABf2MIAufJRDE8hAwavmYruzmFfJQ/P8sX65VF6w=; h=From:Subject:Date:Cc:To:References:From:Subject:Reply-To; b=X3q2SRYgsdD0eXN/WJX2tsPzucWqzcg94NfJH+VEmBn25HuQxL2md96+RusPVq8A6hGlGptH0xX5jEjtYKcBV9+R3EzYyR71EIHpJ4z8uNG/MJRxhjNLnDHhYm3NdJB4uPcLuEge8v6XKpMQ6P48b6MrZfLXDBOJWL2uDrzSVcYTtEkUV/0+wzq3efUI0M1OHofdbqVWwN9O6ZAPGpi3U0JanVa7kpBzcwspcH/Wl5eza4UwfjxxeBlELRGvnX2+N38Yij9fvsrdcT+ve9vH5u0L7iN9Be3yIu7GI2wPdMTBikLRne1ge0NXj4d3yBjBKvcOBTpTuzqxSPj5buwG7Q== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1650062488; bh=q5fNQlC8eFVIny7RupqfpxWvzUyYP6CKciSbzyJ9kgn=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=kKc7mFI4bpWMeryQ5m7lRQehHJUrSk1gGCG03ihg35XkeJ4oXRcwZt/nFBZ8tQt85E2mSMju58RMBVfg1ZcbRsQlwHJAfbAmoyUHr1jZyFJhdIwyB/HLHF0sRX1B1Z9MmhMpI3VkZKWcyoqEioqpHWAtX5CvgLVHAi/vDIVHz8m+OMr7V4WrreuBFiSMsXFsHmV7uW168KGiPa4cpBB+cfdLJEhVE/E2iD7iZ89VFsJNM3SSIZ+Iy5cYM3hP8iY2z2PP6vCdKGyYA4hilYglwx7kTNuqZ6Tyt8LD/PvOn7dcyXJV1cBkrDnb/9WZY5R89kZ6lz2g10hlXmgcw1yNtA== X-YMail-OSG: lnYWTtwVM1nNtA_SBfkUp_3FXMT9dvjOfWPjKpoSc4IsZ3pYJc4pEzgLUBtHNVh WFgRMcoSGkcOH7BOxFoGmSbSQiUL83Xyc2FtnZmah2VU_G81Andge3W72Z6Zy9exP9l9xS.G8TgQ eL4F3Ayi0w8tecQAtIkxHEBP.1yz5RduPREbK7ilrJ8eCsU9gY.97tAP2FN7VbHlPtX2b5kY6Dii 24O5dFr30EHZumk89C0qMxlW0X1RWkKNTHSAheUQk..ZrCLxvjypU4LgdkUdPciAyjtFnABtTE2m ZXJf2HV2mEOHCxc27fWf8p9IKqp3VOdO5Y0t6emEBRzIz1T.vommXuj52IeDmfclkvGlPxgHAYzy yWri7MiFj8XfyFtE05AsZ8uNS1qnekrDxGnwCB5OVu.sdFBD5CgB4ziBcoAVfmeKiLKxpO87r1V4 fGV5jVsvL7ziM_EJIdWg4gAVnjj5CakfyS3djc.5m62EVc0jXjWyMDBnjMqwAn6tiCP3cZQT7i5w d5LrCNQsv5SwvIb54NRKpDm_8kxYg1KCYVvY6y2zIVYj7rzaW8sL9wOdidf52.OkrKo9xB_Z8ykO xIcL0Z_7Thn2NulP_.o1zjfmdZyamMsJOwB8W3FV3eTZv8bcK5wUCtu1xsEinLt69CKDFJbNziZw otKu_xnD5XRSabeatRlRSuoTUXH99kz_Lc3MQEP0jHMCaUemtfc5n3q.0REentFN6R4PIXvhvIVA seyrQAxw1QtRdnDGyroF74irzUmamvNrJi32vX4.ItLlh1.FExf1MMtWFtyaJ3pys9iCp716USzC TLuuegCLIL0w5Z4RGmTqmcSBb8203doWaxiiQuqcbjqJv7diIuyXejkuAvShpNkryWuLVmEGPIgi HWOkWfoGbpeZG27THaKJqzL2TT.6Fhc7bNNxnA_VOzVNQFmGwYWGoEXlP3XPTJ7YxThok8543.ax svRsJtu.U2BUDDnDS5Vaskjnn6JvX9xXrTA3qLmWC9k__pcBQj8Wy9gS27nba.sx12ZkXER4iHep cPLli6xikS6e7CflNxfbmKitd4YhYvwHsoPUHrw21GjD_Ouw2MCZqojAqPFm2fY8euqyucC0.Tff 7Y3dU897Ao8XhfvPA1l2DTKOFaeKf7eEiIpZLZhTyoCcnGDG2ZnukhYaAvL_mUnIA75IOqYmMMYv wWxH01EBQjLI89BJdtINARt3ubIdlYST3vA0SYKl9Aw3Knbke12TYmD4D1FtYETCgp.snF9vmZIN GfD40mkV3bfWWyIHPCeYv63yuLBDxMlNONuMSoykJH2GQexr_6PrjyLtBbXztbWcZXnsUfR7MNtb ciFfiYRwlMYjPBxBrMLZaYN10i2angWIbIE8bXkmS726PeJrBsjNzXmM2URUvPn4eyLarGKhSb9w HS2TAoYMC0D59EC0.ygbSjDx0HMgllF_5qsyskPc7sVS5xqfmr3PAnIybnwSsJSmzEp60GucVJrU IfwP6r37r6R8BJcyanXsgB8XQ2_11a4A.9.Nx.u_fnjBOssLx4mamGTFsEwyqQO8MdAM6bCclhAo TIJatdfP5z9D9aOelO7juYlhSCFEtwNKqHCobRmXkVxvQRX6z5DC9vykzExB77F_ub3SM0j9D5wC qPEfjEmRvs1jG1GEnGIMX1N0WGPkEjnyNYgkxFGT8Oabl.Ggqm_JdNYM5rX9Y7uaqV2.S1.AVbFs QorGAZNp5BYgGG9Y5NG5b_1KvxOSvPAz35T4pokCvifmmFsY6VYfeZUbyYC4SJ9z4vTWHOY5HreF LY05sMaXp4Fye1GDvGbddEbwLb7t7nISpe2AqW2iYOT1jysH1Exs392zYJ3wfXaxnrEbUYaXVRph 55qpvncsX5T0rNS77WuaDUOuMOrFKU7m2ZIGQM39N3CxEjItTrA4CiPgSAFmtcnRR7DypvKOhZ1Y WHjjMF_sLOzETb5WzscMaKWGiz0sjDI0gHX2Rgnd9qosJbbt_EJWkoSdz3tUPcCRa1wjELaV4dlY b5868QW1Z4TR0MAQ3DfRPn9EU.NK2dwaJGf7cOJubaSbsdxzE8foukOTB8_0BPyrTLI7y1I4qHOS .lEHhJdP_aUIR5Glzksn4Is0JjzM5u2.Ir9nd.bZKpu24yYCVIpkjNOAPQhR.cFHQtDOw11ib35j HH4EmwotbHD.YztgcIIB0i03Tz8275XrBbfqwRDnZXBZJw84ymRFEMOeJZv.pfFeWgMmoD0WJ3ox Na85u9qTmuZZZrA3rFCBpwEf5J8A6n9IvD82f1bdxcHtXJZJeeW.PYmAvISZYL4C65HMEUF4LuME NKER4RhV3oqMfa6orA4pfLdZvorOFQ32Ia14I1XjTWVHxz6aWyo_NA5RxYYxvolRV16l4AGYdHyI SfrW_Aq8F2kyU9x4- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic312.consmr.mail.gq1.yahoo.com with HTTP; Fri, 15 Apr 2022 22:41:28 +0000 Received: by hermes--canary-production-ne1-c7c4f6977-cwn8d (VZM Hermes SMTP Server) with ESMTPA ID 3bc377034c61a81805b69fa2f37babc5; Fri, 15 Apr 2022 22:41:27 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.13\)) Subject: Re: Can not build kernel on 1GB VM Message-Id: <64D7342D-A0FA-4E05-B883-CCEC6DA79515@yahoo.com> Date: Fri, 15 Apr 2022 15:41:25 -0700 Cc: Mark Johnston To: freebsd07@wayne47.com, FreeBSD Hackers X-Mailer: Apple Mail (2.3654.120.0.1.13) References: <64D7342D-A0FA-4E05-B883-CCEC6DA79515.ref@yahoo.com> X-Rspamd-Queue-Id: 4KgBC34TZCz3lNj X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=X3q2SRYg; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.205 as permitted sender) smtp.mailfrom=marklmi@yahoo.com X-Spamd-Result: default: False [-3.50 / 15.00]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-1.00)[-0.998]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.205:from]; MLMMJ_DEST(0.00)[freebsd-hackers]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.205:from]; RCVD_COUNT_TWO(0.00)[2] X-ThisMailContainsUnwantedMimeParts: N From: Michael Wayne =20 Date: Fri, 15 Apr 2022 15:17:43 -0400 : > On Fri, Apr 15, 2022 at 11:40:02AM -0700, Mark Millard wrote: > > From: Michael Wayne =20 > > Date: Fri, 15 Apr 2022 13:49:53 -0400 : > >=20 > > > I'm trying to upgrade the machine to 12.3 and having swap = failures. >=20 > I reduced the swapspace back to 1 GB. It's only ever really hit during=20= > builds. >=20 > I set > vm.pageout_oom_seq=3D120 > vm.pfault_oom_attempts=3D-1 >=20 > There was no improvement. I still see processes getting killed due > to no swap space despite only 7-8 MB being reported used. It sorta > feels like it's not really able to use swap at all. >=20 > Note that everything worked fine on 11.x, this is a new issue on 12. It is really unfortunate that the 3 or 4 conditions that initiate the OOM kill activity are not reported as being the specific initiator of the activity in 12.x . (Mostly fixed in sufficiently modern contexts. 2 of the conditions are very similar and tend to be treated as 1, leading to 3 instead of 4. The other 2 have detail-specific wording these days.) I went looking back in time and 12.1-RELEASE-p3 has logic for vm.pageout_oom_seq (2015) and vm.pfault_oom_attempts (2019-Sep) from what I can tell. If using vm.pageout_oom_seq=3D120 made it take longer before the OOM activity, then further increases could be appropriate. I've never had to use more than 120 but I know one person used something like 1200 on a low-end arm Small Board Computer (1 GiByte RAM, microsd card media in use for swap, as I remember). I do not know that, say, 600 would not have worked, however. That vm.pfault_oom_attempts=3D-1 did not stop the issue should eliminate "a thread waited too long to allocate a page" (modern message) as I understand from looking at the code. That is despite your report of: QUOTE Apr 15 12:11:26 g1 kernel: swap_pager: indefinite wait buffer: = bufobj: 0, blkno: 240593, size: 4096 Apr 15 12:11:35 g1 kernel: swap_pager: indefinite wait buffer: = bufobj: 0, blkno: 236224, size: 16384 Apr 15 12:11:37 g1 kernel: swap_pager: indefinite wait buffer: = bufobj: 0, blkno: 245, size: 12288 Apr 15 12:11:46 g1 kernel: swap_pager: indefinite wait buffer: = bufobj: 0, blkno: 240593, size: 4096 Apr 15 12:11:55 g1 kernel: swap_pager: indefinite wait buffer: = bufobj: 0, blkno: 236224, size: 16384 Apr 15 12:11:57 g1 kernel: swap_pager: indefinite wait buffer: = bufobj: 0, blkno: 245, size: 12288 END QUOTE types of notices. If vm.pageout_oom_seq=3D120 made no noticable increase in how far things got (how long it ran) before the OOM activity, it is possible that you reach one of the 2 conditions that are treated as VM_OOM_SWAPZ. In such a case, either increasing the RAM space available or doing kern.maxswzone tuning may well be the only options to do the 12.3 build from the existing 12.1-RELEASE-p3 context. I've no hint to give for kern.maxswzone tuning. There is the possibility of creating a 12.3 /boot/kernel/ area from the likes of: = http://ftp3.freebsd.org/pub/FreeBSD/releases/amd64/12.3-RELEASE/kernel.txz= and possibly also creating/replacing the matching /usr/lib/debug/boot/kernel/ debug information (if you keep such): = http://ftp3.freebsd.org/pub/FreeBSD/releases/amd64/12.3-RELEASE/kernel-dbg= .txz This would avoid the kernel build. ( base.txz use is not as reasonable as it would replace configuration files with default versions and the like. ) I've CC'd Mark Johnston who has sometimes been able to help with these types of problems and how to avoid/control them. He is also the one that has improved the messaging in more modern FreeBSD versions. I include a reference to your original message for Mark J. in case he has time to look, since the content has been stripped in the message I'm replying to: = https://lists.freebsd.org/archives/freebsd-hackers/2022-April/001018.html =3D=3D=3D Mark Millard marklmi at yahoo.com