From nobody Thu Dec 21 18:56:21 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Sx05V0LVJz547sk for ; Thu, 21 Dec 2023 18:56:30 +0000 (UTC) (envelope-from SRS0=yuxr=IA=klop.ws=ronald-lists@realworks.nl) Received: from smtp-relay-int.realworks.nl (smtp-relay-int.realworks.nl [194.109.157.24]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Sx05T4tyYz3g0f for ; Thu, 21 Dec 2023 18:56:29 +0000 (UTC) (envelope-from SRS0=yuxr=IA=klop.ws=ronald-lists@realworks.nl) Authentication-Results: mx1.freebsd.org; none Date: Thu, 21 Dec 2023 19:56:21 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=klop.ws; s=rw2; t=1703184982; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=PbsUI1AQLsB8wt9I3atnKWK/iaGpiuO6MxizQ7F6tZE=; b=LCvaDAI8rbTbVaJaao87f4q3KsucHyj+YZdDdJFguL/gAPLKgqVfZcMzZoiNTh7OvkKsYX QeIddO6sdyn4lq+jbKIGuUbE+fI2obY8gML1fvOYUQpf970xVJ9xbXHI5E9/C0+7WAjO+B 8UpZftgmi6eigUaK8BHkJwilMOrFUVq8BXTTzz2CTvd24lGiAflhSibKCwyx4IMuCIOsEB BOZrZosvBT6BITkChh6Kty+inK0FnS7Y4Gulkx+AUwyEt/tG2SenG7UhVjWNAOQajV0IkH 5yhVezG8OWO6TXEx4zdOYtQGknnxBjksKsYz1AuyRlR5sbzYd1jwJuEJazmBAA== From: Ronald Klop To: void Cc: freebsd-fs@freebsd.org Message-ID: <1439231787.9333.1703184981914@localhost> In-Reply-To: References: Subject: Re: measuring swap partition speed List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_9332_1497520887.1703184981868" X-Mailer: Realworks (683.53) Importance: Normal X-Priority: 3 (Normal) X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:3265, ipnet:194.109.0.0/16, country:NL] X-Spamd-Bar: ---- X-Rspamd-Queue-Id: 4Sx05T4tyYz3g0f ------=_Part_9332_1497520887.1703184981868 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit A bit weird that a simple dd is so slow. Just a quick thought. Are your partitions aligned properly? Is other IO going on on the same time? What does gstat say about %util, queue length and all the other stats while running the dd? Or "iostat -x -d 1". Could you try if another disk has the same issues? How is your disk connected? USB-to-SATA-adapter? Any output of dmesg? Regards, Ronald. Van: void Datum: donderdag, 21 december 2023 16:50 Aan: freebsd-fs@freebsd.org Onderwerp: Re: measuring swap partition speed > > On Wed, Dec 20, 2023 at 07:48:14PM -0800, Mark Millard wrote: > > ># swapoff /dev/label/growfs_swap > ># dd if=/dev/urandom of=/dev/da0s2b bs=8k count=250000 conv=sync status=progress > >^C478830592 bytes (479 MB, 457 MiB) transferred 22.001s, 22 MB/s > >60557+0 records in > >60556+0 records out > >496074752 bytes transferred in 22.790754 secs (21766491 bytes/sec) > > 22MB/s is usable, I think. In my context, I'd be satisfied with that. > My context differs from yours slightly in that yours is SSD and mine > is spinning rust. > > This is unusable: > # dd if=/dev/urandom of=/dev/da0p4 bs=8k count=250000 conv=sync status=progress > ^C11862016 bytes (12 MB, 11 MiB) transferred 40.063s, 296 kB/s > > because it's way too slow. Swap never gets fully reclaimed, > thrashing happens, loads of other followon effects happen. > The same partition formatted as ufs reports 113 MB/s. Multiple swap partitions > have been tested, then converted to ufs. Results are the same. > > There are no reported errors in smartctl. Long smartctl tests run monthly. > > 5 Reallocated_Sector_Ct PO--CK 100 100 050 - 0 > 9 Power_On_Hours -O--CK 001 001 000 - 48992 > 196 Reallocated_Event_Count -O--CK 100 100 000 - 0 > 197 Current_Pending_Sector -O--CK 100 100 000 - 0 > 198 Offline_Uncorrectable ----CK 100 100 000 - 0 > > I can't find any hardware problem here. Possible workarounds, bearing in mind I'm not versant in C so it's not like I can fix this myself in code: > > 1. swap as swapfile and not partition [a] > 2. swap as nfs [b] > 3. swapoff & swapon script running every minute [c] > 4. just turn all swap off and reboot after crashing (undesirable) > 5. use another OS that doesn't have this problem > > [a] not tried yet, and i hope it works. Legacy info suggests swap as partition is usually > faster than filesystem-based swap. But the reverse might be the case here. > > [b] also not tried. This, I imagine, would be filesystem only (I'm unsure a zfs volume can > be exported to look like a mountable partition to the client) > > [c] https://github.com/Freaky/swapflush.git - usually works but maybe i need to run it every minute instead of every five mins. For testing, this script was disabled. > > Any additional suggestions on how to overcome this problem gratefully received. > > -- > > > > ------=_Part_9332_1497520887.1703184981868 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit
A bit weird that a simple dd is so slow.

Just a quick thought.
Are your partitions aligned properly?
Is other IO going on on the same time?
What does gstat say about %util, queue length and all the other stats while running the dd? Or "iostat -x -d 1".

Could you try if another disk has the same issues?

How is your disk connected? USB-to-SATA-adapter? Any output of dmesg?

Regards,
Ronald.

 

Van: void <void@f-m.fm>
Datum: donderdag, 21 december 2023 16:50
Aan: freebsd-fs@freebsd.org
Onderwerp: Re: measuring swap partition speed

On Wed, Dec 20, 2023 at 07:48:14PM -0800, Mark Millard wrote:

># swapoff /dev/label/growfs_swap
># dd if=/dev/urandom of=/dev/da0s2b bs=8k count=250000 conv=sync status=progress
>^C478830592 bytes (479 MB, 457 MiB) transferred 22.001s, 22 MB/s
>60557+0 records in
>60556+0 records out
>496074752 bytes transferred in 22.790754 secs (21766491 bytes/sec)

22MB/s is usable, I think. In my context, I'd be satisfied with that.
My context differs from yours slightly in that yours is SSD and mine
is spinning rust.

This is unusable:
# dd if=/dev/urandom of=/dev/da0p4 bs=8k count=250000 conv=sync status=progress
^C11862016 bytes (12 MB, 11 MiB) transferred 40.063s, 296 kB/s   

because it's way too slow. Swap never gets fully reclaimed,
thrashing happens, loads of other followon effects happen.
The same partition formatted as ufs reports 113 MB/s. Multiple swap partitions
have been tested, then converted to ufs. Results are the same.

There are no reported errors in smartctl. Long smartctl tests run monthly.

5 Reallocated_Sector_Ct   PO--CK   100   100   050    -    0
9 Power_On_Hours          -O--CK   001   001   000    -    48992
196 Reallocated_Event_Count -O--CK   100   100   000    -    0
197 Current_Pending_Sector  -O--CK   100   100   000    -    0
198 Offline_Uncorrectable   ----CK   100   100   000    -    0

I can't find any hardware problem here. Possible workarounds, bearing in mind I'm not versant in C so it's not like I can fix this myself in code:

1. swap as swapfile and not partition [a]
2. swap as nfs [b]
3. swapoff & swapon script running every minute [c]
4. just turn all swap off and reboot after crashing (undesirable)
5. use another OS that doesn't have this problem

[a] not tried yet, and i hope it works. Legacy info suggests swap as partition is usually
     faster than filesystem-based swap. But the reverse might be the case here.

[b] also not tried. This, I imagine, would be filesystem only (I'm unsure a zfs volume can
     be exported to look like a mountable partition to the client)

[c] https://github.com/Freaky/swapflush.git - usually works but maybe i need to run it every      minute instead of every five mins. For testing, this script was disabled.

Any additional suggestions on how to overcome this problem gratefully received.

-- 
 

  ------=_Part_9332_1497520887.1703184981868--