From nobody Wed Feb 02 07:58:19 2022 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 9693719AF251 for ; Wed, 2 Feb 2022 07:58:31 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ua1-x929.google.com (mail-ua1-x929.google.com [IPv6:2607:f8b0:4864:20::929]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JpZ1L5LvRz4bQR for ; Wed, 2 Feb 2022 07:58:30 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-ua1-x929.google.com with SMTP id b16so17634874uaq.4 for ; Tue, 01 Feb 2022 23:58:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20210112.gappssmtp.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=s8zu7WlJC9dEqH/8BcEVlUMKRXMhiRI7M4KL8i8TN4U=; b=i0Rm3+UhIziNdoljUrRMz00YmgKSdWDXJhnn2CjfSg779Dt/KAUasFFnxgblru/RnL 8JRoT+EOAOzzKOoIkL9XRv9o+hvpW4iGR0drjcpx3N4RJkN3wHlodoaSFBrGoAfYYzvh JnLIw6SDI/2DSYoC4kgov9B7f0otAs1y7lf9ls38AerWmHGsd9mXalcgWzqcOf1UPwzn hwU6M7exQDZ6HAJWK7HWJomy5ckLmGe68Dd/l18G/JY+2WZkLXhyhiJti+qM/UefKQDX Q1dEKdxmkgScamRZuYlHWU9KlvncaxaRn6mqdMEFfKfhfltIPsZm+2B11e6nFTn4vaIC U2eA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=s8zu7WlJC9dEqH/8BcEVlUMKRXMhiRI7M4KL8i8TN4U=; b=mEsWfZcmEx7FhvwNLM5aqWxb8sSJ2RwZ2KZkg1q8XucRq2zQtnrISmnlsBC5Xz0ksS l1BVypGm76AyYxvVDFh1PeIk2ao2/6EzGTbutzCgfgOkys8C0A9EMdGqsG9YxNRg9S/z bfcs8ChgzG/WXfOmzYD4KLkFGPzm4KdpsFJpooX9OXkbp14jNW2lltwxJT4yfahxuf/T MrURYAuRKXBiWvjogoJwtiiAPcQB1196jHeqgnQUoRB4E1DDXe/wuB87ZpmWdiRfELU5 zkE4mV9KFk1ooxOoErNYf8BwSLIJ2c/w5c91ZKnWsNCnnIQjemGK91fb8JeGk74ffYMD sYNQ== X-Gm-Message-State: AOAM533xiRpSjycD7xcQvuLEw44LB4xK2Uxr0VHpK34gb2Zp/zkQA6R6 b7IImqmv4QoxJ/mPtUL/Ftzo1r4yrlGYPRv3vzoS3A== X-Google-Smtp-Source: ABdhPJxqqcq9I8UxOKXI1XheR+T4FkCOPOrYWqNagQ/fEpGi4MzLuFslX1Mv5hIJknzaJta1fCMs9bsZ6g0vDpk8YjU= X-Received: by 2002:ab0:6294:: with SMTP id z20mr10905765uao.11.1643788710161; Tue, 01 Feb 2022 23:58:30 -0800 (PST) List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Wed, 2 Feb 2022 00:58:19 -0700 Message-ID: Subject: Re: bio re-ordering To: Peter Jeremy Cc: Konstantin Belousov , FreeBSD FS , "freebsd-geom@FreeBSD.org" Content-Type: multipart/alternative; boundary="00000000000027c10305d704624f" X-Rspamd-Queue-Id: 4JpZ1L5LvRz4bQR X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20210112.gappssmtp.com header.s=20210112 header.b=i0Rm3+Uh; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::929) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-3.00 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20210112.gappssmtp.com:s=20210112]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20210112.gappssmtp.com:+]; NEURAL_HAM_SHORT(-1.00)[-0.996]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::929:from]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; MLMMJ_DEST(0.00)[freebsd-fs]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; R_SPF_NA(0.00)[no SPF record]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_COUNT_TWO(0.00)[2]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; FREEMAIL_CC(0.00)[gmail.com,freebsd.org] X-ThisMailContainsUnwantedMimeParts: N --00000000000027c10305d704624f Content-Type: text/plain; charset="UTF-8" On Wed, Feb 2, 2022, 12:49 AM Peter Jeremy wrote: > Thanks all for the very prompt responses. > > On 2022-Jan-28 22:32:02 -0700, Warner Losh wrote: > >I think that ufs relies on two ordering primitives, both marked with > >BIO_ORDERED today. > >That's what most of the drivers key off of. We always set BIO_ORDERED on > >all the BIO_FLUSH > >events as far as I Can tell. > > Thanks for that warning. I don't think geom_gate understands either > B_BARRIER or BIO_ORDERED. I shall have a closer look. > It needs to understand BIO_ORDERED. >to it. b*barrierwrite() sets this, and that's used in the ffs_alloc code. > > In my case, I'm interested in ZFS, rather than UFS and it doesn't seem > to set B_BARRIER or BIO_ORDERED or indirectly. > I went hunting ZFS for this year's ago and in the pre OpenZFS code they were used, but there were three layers of indirection that obscured it. ZFS doesn't use the buffer cache, so B_BARRIER isn't relevant. I'll see if I can find it with the new code. But if it never sets BIO_ORDERED, drivers are already reordering things. That's all any other driver in the tree worries about... Warner -- > Peter Jeremy > --00000000000027c10305d704624f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Wed, Feb 2, 2022, 12:49 AM Peter Jeremy <peterj@freebsd.org> wrote:
<= blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px= #ccc solid;padding-left:1ex">Thanks all for the very prompt responses.

On 2022-Jan-28 22:32:02 -0700, Warner Losh <imp@bsdimp.com> wrote: >I think that ufs relies on two ordering primitives, both marked with >BIO_ORDERED today.
>That's what most of the drivers key off of. We always set BIO_ORDER= ED on
>all the BIO_FLUSH
>events as far as I Can tell.

Thanks for that warning.=C2=A0 I don't think geom_gate understands eith= er
B_BARRIER or BIO_ORDERED.=C2=A0 I shall have a closer look.

It needs to unde= rstand BIO_ORDERED.


=
>to it. b*barrierwrite() sets this, and that's used in the ffs_alloc= code.

In my case, I'm interested in ZFS, rather than UFS and it doesn't s= eem
to set B_BARRIER or BIO_ORDERED or indirectly.
=

I went hunting ZFS for this y= ear's ago and in the pre OpenZFS code they were used, but there were th= ree layers of indirection that obscured it. ZFS doesn't use the buffer = cache, so B_BARRIER isn't relevant. I'll see if I can find it with = the new code.

But if it = never sets BIO_ORDERED, drivers are already reordering things. That's a= ll any other driver in the tree worries about...
Warner=C2=A0

--
Peter Jeremy
--00000000000027c10305d704624f--