From nobody Sat Jun 11 22:03:26 2022 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 88074857CF1 for ; Sat, 11 Jun 2022 22:03:39 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic315-55.consmr.mail.gq1.yahoo.com (sonic315-55.consmr.mail.gq1.yahoo.com [98.137.65.31]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4LLBfx62Nzz3MBl for ; Sat, 11 Jun 2022 22:03:37 +0000 (UTC) (envelope-from marklmi@yahoo.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1654985010; bh=gHSNcZ1bzQxC1C5xhvLBmw49ce4c6R3aLyrDB/c2Vjg=; h=From:Subject:Date:References:To:In-Reply-To:From:Subject:Reply-To; b=ComyDIIOS0Jt+O/deI/f6+6rZRI+YIYGD8oo5QHC/I+m2+BIBxaGZh10C6FrkzrD7kzR2jLOm5zkXUGVAZbV2cuGN36rJ/xTVOJwAhpIlgiXj4Cken9qVgxB5ynNHhHt0gumBpOZdj9NaM0WoWTO3IP33fHmRN0OjS3l5P7kjU7d6ISRgqkFXHYNAoefhoCCR+bZHZGBHXGZLSCU08SxZVeOqeHBzIQKfykmb+3pt5vkh9Yj0Wuj8o+weaA6qP+5rInXShmyMfLG0yCIypR6GlBQZxQ1SaJlGpkY70Vokh4kaBnoF2JOc2gtneLUZrI6f9pBfApu9GhZ6OxpE2OdnA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1654985010; bh=s+IN+ldFmTQMvInWV9+v+a0L6RW9iVGapoTj+bZWINP=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=Ei7fdICkvxH+7HA4a4HD7nqQmDmfdXb8Nk9Jd7+e1O32JhWqLkMeXc/rDdwS6MQvJXJnGKE6qqa5XFHE3xaeICwy/7xjxfn0piO2b9oPP3FqedYsVuMjuWC5lh2GeWrPZo/Eozh+984qDkPC3YqkWZc6MCrmLXETCSe8tEhpfsma74IfXMN22XCw4YZi5SlbKktZyKkx3l/fX+yXNUDJGnkRK+sHnWtw4Wz9T9kBtXKL6nzpVxpcdn7/+dpiG0mT2KrAFtssLOEiPfTeD9smSr38tAGrX/nUcE+11a/4It72bqBErQEyk3gr3Qe0zlV189eTYsEl0Z+MK1bCJkwNjg== X-YMail-OSG: JKkhpVAVM1m_UAmtdsWXjb4I4u0QBBhUYfgR6fhcmE4Aoj67JUWkXFTWsICU3VZ S6mwji.iyz274nN1Fk0qciz.nuqzIkSS4ZBp1rf6k3jnuFvuqHB6.8lrvk8ldcvgz3_qxC2IwEcf 8cprCvBPC.2JZFlm1DjkLhQv4xGAl9Sx9aGnxdLLZQ.4BaXI5xu52__IBhRhGt.QcVaYDALROcNC l8A218EhNyj64E07Sb0q9Ig8ssYw.3P9zNHjg_nb15jdi5OeT1T5PWf1SZddLYvx1dz93BsI2AP0 Tr6L7L7blyfoRLSy4IKXMy3bZrW.omFoXailHPzvjvVFPJkXCv4A_a.hXtDazxo7COy0m8F_N2wk _onXnPLu7uM_bR1as16bEcMTe0Ow8A3ejkaUrnZJgAUSfB0rt8BQTnOud2wFx98Ilj7.PkKyek9o Ppcj5yrx1UbJ0SPXDxjJLBzvjRmOEOnFf4XDUZWeUGUr3QIEI76WjHpffEKhXy2new0RQdcgbiLX 7yklRT5x7mAcVWaBdGj9vD.mUlZ.i68lNA2r1gR6UGLzmlfBKNkDufq7Qfe_uce0gcAhzcrkaOSl ZsT1nSDHBl_5DfUiFT7ktF6CxYHKonUhwNm3MoD9T7DP0T6nMnBoRHYGp9eE49kfBay_qiuy5VGW nze5jD8xV_D58VsErgDwlYsKm2sD6lGGj8PJBAv.ycGFUSMcR.k_ERIUqpq2ztsw5P5dH8QNgAG4 gOpHR7Z5fUVwI0Z88UaJCCxj5iQz1rHTFkGCHEY.FhPmzbOHb4B6TIAjaOMFiZQMAld4RGnoHo4L dNLRGJZnZeYrQB1grhH.MbvoRzR6sE2pWpcIjO84.PVL18kNY6zkJ4vZrTrLB2iRDKbb5ClTJgmR hwMT1H3g7ufHrC6pEgeqDPPxOe5JJ07mdWdb6WBC.7vKuRJ1aKf43wxs9EsE5GQepFSSr2rBu93U nJ7SjQfxaHKb9..sKL.P4RZIvibz3.qVcUDNQQhzI4737AxEcUjsstls2kdUpbdC4QOeq3fGNLjp gzta_iOOSpeJVvEAXR3c.wZ_vAhNXewTG8SklV1DBLYgqHUx4upAo710YMBy8r8NNK8o6kzcys5y vVHkpQAq0NkObVNGmIgwiLghLCxnyTXjamLNhfS0EN6VrjGRYn7s_SGd7ePU_2_ljgR9Lk6ORE44 6cdpXsB3IyOLeqQUC5xSTmYTsCWEtl5wI.HxXHBReoUtd8TpZwSnmr3Nf7SjeZiT6.qKTQGJFLxS 9sVJ30se6KBHruf8.62WblD.AbLwmT4Q4cDTBV8.k4lUQsN4EdZMmaFDhLK5N2cLPOnaSxuDHLdm .ldvN2_KYtLoht7wAnfMD1I12n6PVB_dI4CW0Lptlif0guBF.Hx5j5y1PoUMfZ0.MaeEKge3XL2n Fx6HQCqoLgoFfT5SQHgYllLCY9bmT3NsdR9bfnACKC8QPbnlX3TtJivgbrd0LE527kTppEGeA690 QEx8THAGDtvtpM3Vz2nbTeAN_CP18qhad8p1ykdK8t0wkBxcvCVE2xZjPmBpIvJdsiAx2t9nNc7Y f055NkCqnsulV43j4y3FGno2tA5h_.7LzPNn1TIoF2k1e0LVChuRyJLcG1tCOg7a3khMPjHWfYk3 Ct38EOjxRZqbv57_oMJayCr_92fkLXOSu0LQSouSluOZzrX1Jm.TcXNIFlQihbevjJ0A_LzQ9Bxy 3FLnmI3YG7QIvb8jzyOsrxazX_p2Be5AEUVLb6yOmiZFxwKDRMvvCh1xIk1_T5k0LcNdkiwjioKH JDIdqRFtO3rzLijVKjLJV0uF7H16m1rfWhnfKNNCxO3YFW4lR_UFZNx_1zzKGYOg043EzlR4TZce G9pm_p0H0A2xtGQSHWId0CYoebvKgcCzBYOm5dn.0bFufcoBUGTjxAB5EzT4239wOIQIMuIC_IPB SI9r0kSpSKTCg1bDP8ZgTDumlK8AE_Bw6mXj5mzxKptWqiFAVWHoblp0JnJ7CXfJ_YI6t9OuVLxi tcrKaYpLXT5blqMpWFThEa_0.jjWOAj6NHd0W9BR.HhWYrK5NpwQ9Tb3oinAfvc4GLw25FSpvFcG TEv5zXEm38wTSMRsS8S6v_gllLDJoNlzV0lwtpgAHAE15IHuCrqGJR5n1C14oCiNfYdG3sz8KfX9 OGF3XtN3utSJxOolc2Y5Q2UgDYznh2BQMaOkkIZ.cKE4aUD8Jq3meAdSsZJx1aVfb1v4Ztur03qA 8VAKvsgHO6.k4eQgwbCPFeTXzGfuMe8fg7pOUBfhUDafm7w7WqVhfjWOX3qTWWEhw_0q2caOMeCl 9_sAK3hQkPjwr X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic315.consmr.mail.gq1.yahoo.com with HTTP; Sat, 11 Jun 2022 22:03:30 +0000 Received: by hermes--canary-production-ne1-799d7bd497-d2s2t (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 76c2580bb58dc5f20940b8590d31106f; Sat, 11 Jun 2022 22:03:28 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.13\)) Subject: (Retitled!) /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p (example) USE_TMPFS="data" tmpfs usage for poudriere Date: Sat, 11 Jun 2022 15:03:26 -0700 References: <4997AB05-8CD4-4A2C-AAB3-34F6DB2CE325.ref@yahoo.com> <4997AB05-8CD4-4A2C-AAB3-34F6DB2CE325@yahoo.com> <82ffa6bb-5826-62b6-2b82-a735b183272f@grosbein.net> <2F1A9EE0-FB9E-4BBF-AF76-4E5AD372FCC1@yahoo.com> <69FDC72E-F1DB-449F-A9C1-25827895E6B3@yahoo.com> To: Bryan Drewery , FreeBSD Hackers In-Reply-To: <69FDC72E-F1DB-449F-A9C1-25827895E6B3@yahoo.com> Message-Id: X-Mailer: Apple Mail (2.3654.120.0.1.13) X-Rspamd-Queue-Id: 4LLBfx62Nzz3MBl X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=ComyDIIO; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.65.31 as permitted sender) smtp.mailfrom=marklmi@yahoo.com X-Spamd-Result: default: False [-1.54 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; SUBJECT_HAS_EXCLAIM(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.98)[-0.980]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; NEURAL_SPAM_SHORT(0.93)[0.935]; NEURAL_HAM_LONG(-1.00)[-0.999]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.31:from]; MLMMJ_DEST(0.00)[freebsd-hackers]; RCVD_COUNT_TWO(0.00)[2] X-ThisMailContainsUnwantedMimeParts: N On 2022-Jun-11, at 13:50, Mark Millard wrote: > [I've deleted prior parts of the exchange.] >=20 > I finally have a hypothesis with some evidence about what > primarily contributes to the 400 or so "1M-blocks Used" in: >=20 > # df -mi /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/ > Filesystem 1M-blocks Used Avail Capacity iused ifree %iused Mounted = on > tmpfs 1024 400 623 39% 103659 3828501 3% = /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p >=20 > --where the poudriere-devel is using USE_TMPFS=3D"data" for a > "bulk -a -c" (that has been running for a little over 2 weeks > now and is past 1/3 done). >=20 > Taking a quick estimate of the file count in . . ./ref/.p/var/cache/ > I get: >=20 > # ls -Tla = /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/var/cache/ | = wc > 82146 821452 7942485 >=20 > Taking 82146*4096 I get: 336,470,016 > Taking 336,470,016 / 1024 / 1024 I get around: 320 Mi >=20 > By contrast: >=20 > # du -xsAm = /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/var/cache/ > 46 = /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/var/cache/ >=20 > But: >=20 > # du -xsm = /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/var/cache/ > 295 = /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/var/cache/ >=20 > and: >=20 > # du -xsm /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/ > 319 /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/ I see that a little relevant material you have not been able to see because of what I omitted from earlier messages --messages not sent to you at the time. In my earlier messages I'd shown: # du -xsAm /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p 101 /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p # du -xsm /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p 68 /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p so -xsm resulsts were smaller than -xsAm results and way smaller than the "df -m" results. The 101-ish -xsAm result for .p/ has stayed fairly consistent. It appears that the swapoff activity mentioned below changed the -xsm results by forcing tmpfs material back into RAM. This ends up explaining the original 403 MiBytes vs. 101 MiBytes question that was mentioned in the original message subject. I mention this as it would appear to be important to trying to replicate or otherwise test the issue: Context matters for interpreting "du -xsm" results for tmpfs from what I can tell. > This is all after having done a swapoff for the 2 swap partitions > and a swap on for both and then waiting a while with the "bulk -a -c" > still running throughout and after. (So under 82 MiBytes is in > use in swap space at this point.) The above seems to be what lead to my "du -xsm" result being fairly close to the "df -m" result. > poudriere's tmpfs usage reporting does not seem to cover this > area's tmpfs usage. But, at least for "bulk -a -c" kinds of > activity, it ends up being the majority of the tmpfs usage for > USE_TMPFS=3D"data" types of configuration. >=20 > For reference: >=20 > # ls -Tla = /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/var/cache/ | = more > total 301234 > drwxr-xr-x 2 root wheel 5257152 May 28 02:31:26 2022 . > drwxr-xr-x 4 root wheel 128 May 28 01:56:42 2022 .. > -rw-r--r-- 1 root wheel 2 May 28 02:26:40 2022 = builder_active%01 > -rw-r--r-- 1 root wheel 2 May 28 02:31:24 2022 = builder_active%02 > -rw-r--r-- 1 root wheel 2 May 28 02:31:26 2022 = builder_active%03 > -rw-r--r-- 1 root wheel 2 May 28 02:31:26 2022 = builder_active%04 > -rw-r--r-- 1 root wheel 18 May 28 01:56:51 2022 = originspec-pkgname%accessibility_accerciser > -rw-r--r-- 1 root wheel 19 May 28 01:56:51 2022 = originspec-pkgname%accessibility_at-spi2-atk > -rw-r--r-- 1 root wheel 20 May 28 01:56:51 2022 = originspec-pkgname%accessibility_at-spi2-core > -rw-r--r-- 1 root wheel 11 May 28 01:56:51 2022 = originspec-pkgname%accessibility_atk > -rw-r--r-- 1 root wheel 13 May 28 01:56:51 2022 = originspec-pkgname%accessibility_atkmm > -rw-r--r-- 1 root wheel 17 May 28 01:56:51 2022 = originspec-pkgname%accessibility_caribou > . . . > -rw-r--r-- 1 root wheel 17 May 28 01:56:55 2022 = pkgname-originspec%zutils-1.11 > -rw-r--r-- 1 root wheel 15 May 28 02:03:32 2022 = pkgname-originspec%zxfer-1.1.7 > -rw-r--r-- 1 root wheel 14 May 28 02:03:00 2022 = pkgname-originspec%zxid-1.42_1 > -rw-r--r-- 1 root wheel 19 May 28 02:04:03 2022 = pkgname-originspec%zxing-cpp-1.3.0 > -rw-r--r-- 1 root wheel 12 May 28 01:59:41 2022 = pkgname-originspec%zydis-3.1.0 > -rw-r--r-- 1 root wheel 18 May 28 01:57:13 2022 = pkgname-originspec%zynaddsubfx-3.0.6,2 > -rw-r--r-- 1 root wheel 9 May 28 02:02:07 2022 = pkgname-originspec%zyre-2.0.1 > -rw-r--r-- 1 root wheel 14 May 28 01:59:41 2022 = pkgname-originspec%zziplib-0.13.72_1 > -rw-r--r-- 1 root wheel 14 May 28 02:03:00 2022 = pkgname-originspec%zzuf-0.13_1 > -rw-r--r-- 1 root wheel 13 May 28 01:56:45 2022 = ports_metadata%top_git_hash > -rw-r--r-- 1 root wheel 4 May 28 01:56:49 2022 = ports_metadata%top_unclean =3D=3D=3D Mark Millard marklmi at yahoo.com