From nobody Thu Dec 21 18:36:10 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SwzfQ47xrz555qv for ; Thu, 21 Dec 2023 18:36:30 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic317-20.consmr.mail.gq1.yahoo.com (sonic317-20.consmr.mail.gq1.yahoo.com [98.137.66.146]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SwzfP0K49z3b6C for ; Thu, 21 Dec 2023 18:36:28 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=fJ8WU1T2; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.66.146 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1703183786; bh=6JpI267D+zflN+Twokyek4iAWDWPeoykjSMKaQkFAQg=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=fJ8WU1T29omlD6KiMjerZnp/ujiraIrQzXV1htOsZo/RckFa2W92YxJ1KlZl+pFYn8YdBE0uiN7iUCPcRjWaFJEnQeD7Yn9Y12QuBuOIkyoCOYpcEo9A/PsWHz9ytIaQVVBzaTzwuVgRAJRCxm712sxa+2UMW/et6VSVs66mrR8B7EK4f8XXgMMnwX2X1Q4uYkCQ0u98f0sYwzcN2ZvjAme4ZS2YG7k3x2IjXBrRDP+vksiUrfqx7AzKXFXh4tBf6/PfYyH5Evez2FH76QHGGZxfctBLJJgn56hMPhB4JK2Leb5dmj2ltJc0PnZ0kjcM53BbG0oRmS6oLqOCK3wpZA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1703183786; bh=o2Usr4ffIz7+LoccAA/Y3Mu2vWETWQxG21cbRw34vbF=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=bmi/oIkqhoqcFWroGtQGOT0kI0UDCZxxiaUU6iiCs/6Nn2tJaf5Ar57KBbiyiIIaC7GaItnKUp6lcnCHGAf+HHV+FNAP0jQcgRIYkg3V7D3ki2HojnjDd/GxcUMdJSKrL5bZkCQ9bcCpWvM4PBguIuTb/OHY7YUwAOqCDQMlzB5hVbvfrSKrWuRmbbcAwGaqXKLjrPo6hqXPZrD1AVUXRNDHziXnkTnt9xVmXcgM8uh0BujyDRadEz+JRJCpi9GVgja6/Ks22LY2jzyh+QyuO3SfhY7eEfykucKe2scDLhMN6r0x2zSZDfxSzEC3fChyIdZXLUE3h+M1+mdHAaJXZw== X-YMail-OSG: fldAhmcVM1kAABYM4YObFXon8T0gtPIz8D_xE9QDmHMUEwD9mOhxYFu6iMRYAMK dH6ttnxiTRPprpZaUIyljQl7hdassvFQh99JKn0dqQBgyPXHQVVxfOQwFqQetkli70fG_FBexzZG 28.YqD5ZsCVtIXEM60Tkm4A2yOakb1RbUdfBK6FKtkJ5glQnh6zqNOz3jEQ6C8Yx_EuyXruPFn0M lXLzNHB.wS_T4yuVTCdJL3CERo3.uveTqH.QInqiXS7rjhOPXbnf9_3fevxJXif2HaxePmfpmQ7a hQe5gv7lrzJteBrcJSrz9OxSBpNNxkvY6jjgQUOQ2rmO4VaLV.TDzh5dVQr849GNLj2ceRZO3A7I mX3E_NfGw8ZupXGmBiS6XsR3GQflcbWL8ZVkESU6y9uqSLTyVpqzrCKBv6Hd.nEKISkgzJv9Fjez 68QNRIVqHFBio1wc7FhnZqcuWPHFItFyCWkw_Nfc0hjCGOCSZ19.0QhCMP_hPiwyeej_XI2dhI.y IMSZt79j0BOjDyAby6OnME01E2Gqu7yl5hq2fmeOp7zaIOEjLDZhBX0lKkF_aMyvzK63Fv62RSNm AWmtHG.GsCxCIJAekeqlnHijFSc45qh8B0Z0rJ_Op1wNdLZHt7TuYcAba.K2R91Xr6iw5DyD6M8F 2KjkYyENMcMRyUakJrFb41nESV_xLY1jd69SgE5uKuDmbUkXesUQ3QCk0g.0ZTPrE1Gu6XNV5bVP 30mz7xSbKCoTbXojGFjbftBwuvPQ9D94T2HBSZ.wr4xLxjipNhI8Zmi4_BYdl7vBT_j4QZ30DUt_ Q9zPdRutWL4qwP865EZVMiW6p_kfJEOSZIMbqMkUHLCh7TPzuECIIdW7KHfP4EkcBiaXGcj79q4W wlNkKAfawwdo1RPsW7zneJydpcqbj394yRv7oSw1mJ1O8CEg5wb516nKqDuXyhMOVZoXySWCkTwY ICbgma10u2BfuMov53X6pQFBut27tOnFZQzcaa5itpaaTH1e.rLLY..ngzrD655KaNxyYsvA6OH1 3ivKFp287U4.l1IPN2kRiy_NWbmzrEp9bk14HH9hOs1Sl70Nfuha4Vpot.aj6le6Cwds0hhGdsnE jf7Wh6txmvU1E1_r4h0Ay3Lu4MX9MhPOCU96Wq4B9ycFEIOgBKDmFq2t5q71oH.5knQ1SkVRdaH3 OaD9c_ZuOQKZYEKF3b_8IL8ZzQYxRoBPFQOCwQ1hajsAwaXr5xL.IURW4dTNiI5BzEpcTAwUV.Dx EsyCORZdPg39FBt4Sjk9bQ9dhWqM_Jwed.iucgXTWoMS1buFZoBavWrIRSaVzHD3ZpakSXFe57ri UrmyIX0nkVAq4Jn3STNSHrH4vCro4vXJjg9wNYVIVWdj3ShQ_qo7Vw3GX4vrzy2LwCDBk9E9k._P h_MlFxNBLhHYyLS6l1zlaXf7a2rZzUkerXZkVJYD02zgMkHeuCES3NMWddrK6w1jWpEhQqCbyPI9 qvmFT5DqNleAH5hekfL_Dl8E7pu5kLKeGDOmYRrRAyQFYJWUao4ITsb4.luRVVLa33.ZlY8I33Qw LCYm4sGBcve7EzJR9hjsLKhQxiHLRsuV7o7Ucy.LOLV9i4vf87Y_602iAO7xW0uQ545gm2Jo_gak zNlYvRMj0mijU5vDwz_0EMut_AvtDYOerPKFmjnh6fH1cp16f_fhvY4OcODvLXnNLUW4yKPc3Iui sPBc4s5bYwy8EuCxsc535B9Z4wfbkjVKx3ieRhEX8IIKId2M4_lype0lo.eKy.11Q2pMMo8WwDIO gJWBnrrKWxlhHgcQC88N6.BurHprybXt1R19C4mS5HGGBeXTe5rXaACQNFnNM9tohG0dhrAJIerz UgXT4Tf4M0BgPXxfqmwuy3hziASUgt5cglbDkf5z4OQe1Y0helNZl603OaiOsQ2KQwivjz9ZTPPC hmKbPHE2uoK8CegjvVIkZTH0tdZuKedxZmSpYPQW34XPWqfcpnnKSxeClEZ8dJPYYmqy86g2HM7O mLZ96c15nErAgeVQz_27sI5T9XSbyol6OugiTyNcJUsg40wB59UIp8ifOiQso8ECoYtb.bIKLPwa tGmK5jowcU.DYpW158RHOMWfhT5zK.PXLdPShNO5bMUd5ZTWKXDOxNKnFX4czLVAtejJggDHuAST dYnUfS.OuQn_ijTHlc2ocfklMvR2grTNhEkWcd8TEH7sfAHoPAtOwo4gHddJ1.1bBiBenLcDIGdv ZbqJmv_gfIV6OBUlPSGd4XKwbBEyc6VlLlKX.SBj2WOOhYabGVzc9lcH9.JI.f4AcN0e0vLaqiHj V_g-- X-Sonic-MF: X-Sonic-ID: dee42a02-7368-41c5-8458-948758c7150d Received: from sonic.gate.mail.ne1.yahoo.com by sonic317.consmr.mail.gq1.yahoo.com with HTTP; Thu, 21 Dec 2023 18:36:26 +0000 Received: by hermes--production-gq1-6949d6d8f9-7dnvp (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID f721cea4849e239702ff1ab86c68f112; Thu, 21 Dec 2023 18:36:21 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.300.61.1.2\)) Subject: Re: measuring swap partition speed Message-Id: Date: Thu, 21 Dec 2023 10:36:10 -0800 To: void , freebsd-fs@freebsd.org X-Mailer: Apple Mail (2.3774.300.61.1.2) References: X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[98.137.66.146:from]; MID_RHS_MATCH_FROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.66.146:from]; DKIM_TRACE(0.00)[yahoo.com:+]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_TO(0.00)[f-m.fm,freebsd.org]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4SwzfP0K49z3b6C X-Spamd-Bar: --- void wrote on Date: Thu, 21 Dec 2023 15:50:52 UTC : > On Wed, Dec 20, 2023 at 07:48:14PM -0800, Mark Millard wrote: >=20 > ># swapoff /dev/label/growfs_swap > ># dd if=3D/dev/urandom of=3D/dev/da0s2b bs=3D8k count=3D250000 = conv=3Dsync status=3Dprogress > >^C478830592 bytes (479 MB, 457 MiB) transferred 22.001s, 22 MB/s > >60557+0 records in > >60556+0 records out > >496074752 bytes transferred in 22.790754 secs (21766491 bytes/sec) >=20 > 22MB/s is usable, I think. In my context, I'd be satisfied with that. > My context differs from yours slightly in that yours is SSD and mine > is spinning rust. I do not have access to spinning rust to test for comparison. Others likely do. > This is unusable: > # dd if=3D/dev/urandom of=3D/dev/da0p4 bs=3D8k count=3D250000 = conv=3Dsync status=3Dprogress > ^C11862016 bytes (12 MB, 11 MiB) transferred 40.063s, 296 kB/s My point is that the performance seems to be strongly tied to the media type contributions to the performance. There is no general problem with partition swap based performance. (But the type of test was set up to match yours, not to be realistic for paging activity.) The paging access pattern likely ends up doing lots of seek activity, making for lots of accumulation of latencies. It also is likely is a mix of read and write activity. Mixes of small reads/writes to fairly random palces tends to worsen performance compared to sequential. Paging is not a good match to large sequential writes as the only activity. Perhaps someone with fio background can fully specify how to run a noticeably more realistic benchmark for making swap perforamnce judgments, perhaps monitored via gstat during its operation. > because it's way too slow. Swap never gets fully reclaimed, > thrashing happens, loads of other followon effects happen.=20 >=20 > The same partition formatted as ufs reports 113 MB/s. Multiple swap = partitions > have been tested, then converted to ufs. Results are the same. This uses the file system caching and larger than 8K writes --and only writes, not a realistic mixing of reads and writes or the more random distribution of where the reads and writes would be form/to. conv=3Dsync does not prevent the caching effect for sequential activity as I understand. I suggest using: # gstat -spod to get an idea when the actual I/Os are like in each of whatever relevant contexts are of interest (actual operation with the swap performance issue and benchmarking). So far as I can tell only you can provide such information, as the issue is not readily repeatable by others. =46rom a broader view, actual-operation examples from "gstat -spod" output might be of more general interest for your type of context. > There are no reported errors in smartctl. Long smartctl tests run = monthly. >=20 > 5 Reallocated_Sector_Ct PO--CK 100 100 050 - 0 > 9 Power_On_Hours -O--CK 001 001 000 - 48992 > 196 Reallocated_Event_Count -O--CK 100 100 000 - 0 > 197 Current_Pending_Sector -O--CK 100 100 000 - 0 > 198 Offline_Uncorrectable ----CK 100 100 000 - 0 >=20 > I can't find any hardware problem here. Possible workarounds, bearing = in mind=20 > I'm not versant in C so it's not like I can fix this myself in code: >=20 > 1. swap as swapfile and not partition [a] (1) is subject to "trivial and unavoidable deadlocks". After suffering such, I avoid always avoid this form. > 2. swap as nfs [b] I've never used nfs for this but it likely has the same issue as (1). > 3. swapoff & swapon script running every minute [c] If this ways works for bringing everything into RAM, it seems to be an approximation of not having swap in the first place and would be subbject to (4). > 4. just turn all swap off and reboot after crashing (undesirable) (I tend to have active SWAP partion(s) that has about 3.8*RAM space because of doing a form of high load average "poudriere bulk" runs.) [I have multiple SWAP partitions because of using the same media in various machines that have widely different amounts of RAM. I form a total active swap space that is appropriate to the RAM present for the boot. Other than that I'd use just one partition.] > 5. use another OS that doesn't have this problem You omit the alternative of using media for the swap/paging space that avoids the problem. There are such around. Is there a blocking issue for going the direction of also having a separate swap media that has helpful characteristics? I will note that the RPi4B shares its USB3 bandwidth across the 2 USBC ports: they are not independent channels. Having sustained I/O that competes for the bandwidth can be a bottleneck issue of itself. A similar point can happen at the media level when the swap space I/O and other I/O are to the same media. (For spinning rust, that includes more time spent seeking: additional latency.) > [a] not tried yet, and i hope it works. Legacy info suggests swap as = partition is usually > faster than filesystem-based swap. But the reverse might be the case = here. >=20 > [b] also not tried. This, I imagine, would be filesystem only (I'm = unsure a zfs volume can > be exported to look like a mountable partition to the client) >=20 > [c] https://github.com/Freaky/swapflush.git - usually works but maybe = i need to run it every=20 > minute instead of every five mins. For testing, this script was = disabled. >=20 > Any additional suggestions on how to overcome this problem gratefully = received. =3D=3D=3D Mark Millard marklmi at yahoo.com