From nobody Sat Dec 16 18:16:31 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SsvRy66mWz54Fw2 for ; Sat, 16 Dec 2023 18:16:46 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com [IPv6:2a00:1450:4864:20::52a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SsvRx0dmcz3Xss for ; Sat, 16 Dec 2023 18:16:45 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20230601.gappssmtp.com header.s=20230601 header.b=ZHAkwU3X; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2a00:1450:4864:20::52a) smtp.mailfrom=wlosh@bsdimp.com; dmarc=none Received: by mail-ed1-x52a.google.com with SMTP id 4fb4d7f45d1cf-54f4f7d082cso2089101a12.0 for ; Sat, 16 Dec 2023 10:16:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1702750603; x=1703355403; darn=freebsd.org; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :from:to:cc:subject:date:message-id:reply-to; bh=3qTF0WEgZbR6SiCvm8YgHUloxekYZtkA4xlSrz9y9+U=; b=ZHAkwU3XsCA10BQdapMfxheZKHKD8Y7gjkOdBtoX64HFDAR3sqHCnbDBMBhgh1IXLE 9KPHj+MU3xdLUhhV6O2O1VS3d5PfX+jA1+aDciuN1FkT4DkdE43q8p7Tqvpp02rOiMIw SrIuEkbcVfdWlyBexeEJ7f8NmCYxERruxmtt+FUa4wa9JEjAcq0gCloRCLqLbDC3pZbc gvL6NfxqYTkNHFAa6YUCLcpwGDGd1AsNp5LRTyYeIy8z8rFX8zrl53L10/sEfj6YYjp0 kef4g3rs85tWukeYc8FFMRUS2Ehya2coh+D5bg8HT/u5ZmLeXHC4q5a5miNucScgGj+S fH5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702750603; x=1703355403; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=3qTF0WEgZbR6SiCvm8YgHUloxekYZtkA4xlSrz9y9+U=; b=qW7KjM0Zb2u3UBHbUV5tU8k1Cs02y8hQpVLgn6hZIrKiVVgywC9wnGoZciUYACPOQE tZmSC/fHkwV+XneoZcwEJoCXAq2ql/MwoME4T1stLKNVdYb5yDkER8W1SoPK7wuDM39N 9oSaoJbpnHZZTqRktAydAhOK4NXDNnJiT1ilRvtpljLdaYBqA4HEgG7au2VSJIw5LEFM fH69zfPjEJqoQYHoYQSazbpRANDU7Ct3ivQW3dYHY02l+MoaWTKtGBuC7NtulHu8mGIs U2u+CR8X3bZ2t730JUw/eiMATmZYM+IQTBvw3DR9vUBzZ9LKUEf7X+nZ5RoqXy6wsGOT xX7A== X-Gm-Message-State: AOJu0Yy02ka/m1MNKfLYNuuLIhueJ9dhp82egEmNI3dm/WwLttBlEdCt 2twYgoryOIQJ8bGhWvvgp1x1Dse4ahSK/oS5mNXqha4Yv80XAALf X-Google-Smtp-Source: AGHT+IHc75cb1IrTyun1lQJP3DYOzeqKAqOZ841wi/Em+78QirFEbfJnR3+jKw7CLV51WDSfWAr3ofeHud4bEzumijw= X-Received: by 2002:a17:906:5188:b0:a23:4262:b6d3 with SMTP id y8-20020a170906518800b00a234262b6d3mr22249ejk.100.1702750602501; Sat, 16 Dec 2023 10:16:42 -0800 (PST) List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Sat, 16 Dec 2023 11:16:31 -0700 Message-ID: Subject: Re: measuring swap partition speed To: freebsd-fs@freebsd.org Content-Type: multipart/alternative; boundary="000000000000cdb043060ca4845f" X-Spamd-Result: default: False [-3.00 / 15.00]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-0.999]; NEURAL_HAM_MEDIUM(-1.00)[-0.999]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20230601.gappssmtp.com:s=20230601]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::52a:from]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_COUNT_ONE(0.00)[1]; R_SPF_NA(0.00)[no SPF record]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; DMARC_NA(0.00)[bsdimp.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20230601.gappssmtp.com:+]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com] X-Rspamd-Queue-Id: 4SsvRx0dmcz3Xss X-Spamd-Bar: -- --000000000000cdb043060ca4845f Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sat, Dec 16, 2023 at 5:24=E2=80=AFAM void wrote: > On Fri, Dec 15, 2023 at 08:41:10AM -0700, Warner Losh wrote: > > > What's the underlying hardware? > > my apologies, I should have supplied this detail initially. > It's a spinning rust TOSHIBA MQ01ABD100 > > > So the good news, kinda, is that if this is spinning rust, your swap > > partitions are > > on the fastest part of the disk. I'd expect 1 that's 12G would work > better > > than 6 that > > are 2G since you'd have less head thrash. Parllelism with multiple swap > > partitions > > works best when they are on separate spindles. > > ok that's easily fixable. It was originally one big 12G swapspace. > The reason it's split into 6 is for testing purposes. I've noticed when > swap gets used there's the marked slowdown even if the process using > it has paused. So I've found that using swapoff then swapon puts these > processes back into active memory and the system sluggishness goes, > for a short while until swap gets used again. With the swapspace split, > I can swapoff/on each partition individually. If there's 2GB free mem > and 3GB on swap, I can swapoff a couple of partitions, swapon them again > then swapoff/on the relatively full partition to get total swap use > back to 0. > I'm not sure that makes a lot of sense. Having one big partition with swap in use just means there's processes in the system that aren't paged into memory. That's totally fine most of the time. It makes for a larger RAM working set for everything else. At least that's the theory.. When you're exceeding the amount of RAM by a lot, weird things are going to happen... sounds like you are trying to cope as best you can with memory shortages... The best way around that is either to get more ram, or put less load on the system. > >The bad news is that your disk may be fine. I'd expect that as the ZFS > >partition fills up, > >the seek size will increase as greater distances have to be traversed to > >get back > >to the swap space. There's a sweet spot of a few tens of GB that drives > >usually can seek > >far faster than longer throws... > > (not an SSD) > > TYVM for the fio tip. Am building it now. > Great! Be careful with it: it writes to partitions. > I'm wondering really if I should reprovision this rpi4 for UFS rather > than zfs. Have had great positive experience with zfs as data (ie, bootin= g > ufs, zfs is a non-bootable single disk or array), less than ideal with > zfs-on-root. There's periodic backups and not much call for this device t= o > utilise zfs functionality that can't be provided on hardware with more > horsepower. > You can try that. I think you'll find that ZFS puts more memory pressure on your system. But the RPi4 likely has enough RAM to not be terrible (though given your swap issues, maybe not). UFS might be better, especially if you don't need the features of ZFS for this system. Warner > it must be said though that FreeBSD is *very stable* on this hardware. > > -- > > --000000000000cdb043060ca4845f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Sat, Dec 16, 2023 at 5:24=E2=80=AF= AM void <void@f-m.fm> wrote:
On Fri, Dec 15, 2023 = at 08:41:10AM -0700, Warner Losh wrote:

> What's the underlying hardware?

my apologies, I should have supplied this detail initially.
It's a spinning rust TOSHIBA MQ01ABD100

> So the good news, kinda, is that if this is spinning rust, your swap > partitions are
> on the fastest part of the disk. I'd expect 1 that's 12G would= work better
> than 6 that
> are 2G since you'd have less head thrash. Parllelism with multiple= swap
> partitions
> works best when they are on separate spindles.

ok that's easily fixable. It was originally one big 12G swapspace.
The reason it's split into 6 is for testing purposes. I've noticed = when
swap gets used there's the marked slowdown even if the process using it has paused. So I've found that using swapoff then swapon puts these<= br> processes back into active memory and the system sluggishness goes,
for a short while until swap gets used again. With the swapspace split,
I can swapoff/on each partition individually. If there's 2GB free mem and 3GB on swap, I can swapoff a couple of partitions, swapon them again then swapoff/on the relatively full partition to get total swap use
back to 0.

I'm not sure that makes = a lot of sense. Having one big partition with
swap in use just me= ans there's processes in the system that aren't
paged int= o memory. That's totally fine most of the time. It makes for
= a larger RAM working set for everything else. At least that's the theor= y..
When you're exceeding the amount of RAM by a lot, weird t= hings are
going to happen... sounds like you are trying to cope a= s best you can
with memory shortages...=C2=A0 The best way around= that is either to get more
ram, or put less load on the system.<= br>
=C2=A0
>The bad news is that your disk may be fine. I'd expect that as the = ZFS
>partition fills up,
>the seek size will increase as greater distances have to be traversed t= o
>get back
>to the swap space. There's a sweet spot of a few tens of GB that dr= ives
>usually can seek
>far faster than longer throws...

(not an SSD)

TYVM for the fio tip. Am building it now.

Great! Be careful with it: it writes to partitions.
=C2=A0=
I'm wondering really if I should reprovision this rpi4 for UFS rather than zfs. Have had great positive experience with zfs as data (ie, booting<= br> ufs, zfs is a non-bootable single disk or array), less than ideal with
zfs-on-root. There's periodic backups and not much call for this device= to
utilise zfs functionality that can't be provided on hardware with more<= br> horsepower.

You can try that. I think y= ou'll find that ZFS puts more memory=C2=A0 pressure
on your s= ystem. But the RPi4 likely has enough RAM to not be terrible (though
<= div>given your swap issues, maybe not). UFS might be better, especially if<= /div>
you don't need the features of ZFS for this system.

Warner
=C2=A0
it must be said though that FreeBSD is *very stable* on this hardware.

--

--000000000000cdb043060ca4845f--