Re: git: c01af41c3c8f - main - ata_da: add quirk to disable NCQ TRIM for Samsung 860/870 SSDs

On Mon, Feb 19, 2024, 4:29 AM cglogic <cglogic@protonmail.com> wrote:

> On Monday, February 19th, 2024 at 1:03 PM, Andriy Gapon <avg@FreeBSD.org>
> wrote:
>
> > On 19/02/2024 12:47, cglogic wrote:
> >
> > > Hello Andriy,
> > >
> > > I use ZFS with autotrim enabled on Samsung 860 PRO connected to Intel
> AHCI SATA controller for 9 years without any issue.
> >
> >
> > I think that it was released in 2018?
>
> You are right, it's 9 years old computer, SSD was added later.
>
> >
> > > Can I disable this quirk locally or have I revert the patch to
> continue use NCQ TRIM with this SSD?
> >
> >
> > I don't think that there is a way to disable the quirk, so you'll have
> to revert.
> > Note that ATA TRIM is still enabled, it's only NCQ TRIM that got
> disabled.
>
> From my understanding ATA TRIM is non-queued TRIM as opposed to NCQ TRIM.
> With only ATA TRIM enabled, drive will have increased latency when large
> amount of files deleted.
>
> However I'm not sure how ZFS handles TRIMs, maybe it groups several TRIMs
> and then sends when drive is idle.
> Another question is how this affects TRIM enabled swap partition.
>
> In general, if I'm correct here, disabling NCQ TRIM should not be an issue
> for desktop usage, except for large git repository updates, etc.
>

In general, it won't matter a ton. Unless you are seeing stalls because of
read/write I/O waiting behind trims that have to go to the drive one at a
time, the effects are in the noise. If you are seeing that, though, the
effects can be quite helpful in terms of latency... assuming the drive is
cooperative. The benefits of it vary a lot from drive to drive as well.
Some early drives still serialize the trims, so the latency benefit is not
so great. Some enterprise drives do trims instantly and keep some internal
state that is helped a lot by them. But if NCQ trims are unstable, we
likely should err on the side of disabling.

The errors that Andriy was seeing typically are cable or controller issues.
In his case, it was a controller issue (apparently, and surprisingly,
paired with a specific type of drive). Linux has a specific workaround for
the pair, which is unusual. But there's two issues at play.

The first issue is that these drives should have NCQ Trim turned off,
according to Linux's commit log. This may be firmware revision based or may
just be some people are luckier with newer firmware (it's hard to say from
the mixed reports, but they read more like luck than version for the
sampling I did). So Andriy's change is good. The second issue, and one I
was confused on earlier, is that Linux goes a step further and disables NCQ
entirely for these old controllers for 860 and 870 drives. This isn't too
surprising. We have a workaround for PMP for these controllers already....
One can disable NCQ entirely by setting the tags to 0... Given the
specificity of the problem, and the age of the controllers, I'm inclined to
not implement the Linux workaround for this specific pair.

Warner

>
> > > On Monday, February 19th, 2024 at 12:09 PM, Andriy Gapon
> avg@FreeBSD.org wrote:
> > >
> > > > The branch main has been updated by avg:
> > > >
> > > > URL:
> https://cgit.FreeBSD.org/src/commit/?id=c01af41c3c8fdd570764ff9b6bfbad6ac9ca1664
> > > >
> > > > commit c01af41c3c8fdd570764ff9b6bfbad6ac9ca1664
> > > > Author: Andriy Gapon avg@FreeBSD.org
> > > >
> > > > AuthorDate: 2024-02-19 10:08:12 +0000
> > > > Commit: Andriy Gapon avg@FreeBSD.org
> > > >
> > > > CommitDate: 2024-02-19 10:08:12 +0000
> > > >
> > > > ata_da: add quirk to disable NCQ TRIM for Samsung 860/870 SSDs
> > > >
> > > > NCQ TRIM for Samsung 860/870 SSDs results in data corruption on
> systems
> > > > with some SATA controllers.
> > > >
> > > > This can be easily reproduced using ZFS which uses TRIM and is able
> to
> > > > detect block content changes.
> > > >
> > > > Linux bug report for this issue:
> > > > https://bugzilla.kernel.org/show_bug.cgi?id=201693
> > > >
> > > > Since at present we can not limit a quirk based on the contorller /
> SIM,
> > > > apply the quirk in all cases.
> > > >
> > > > Reviewed by: imp
> > > > MFC after: 2 weeks
> > > > Differential Revision: https://reviews.freebsd.org/D43961
> > > > ---
> > > > sys/cam/ata/ata_da.c | 16 ++++++++++++++++
> > > > 1 file changed, 16 insertions(+)
> > > >
> > > > diff --git a/sys/cam/ata/ata_da.c b/sys/cam/ata/ata_da.c
> > > > index f5d3aeca9329..d4a591943307 100644
> > > > --- a/sys/cam/ata/ata_da.c
> > > > +++ b/sys/cam/ata/ata_da.c
> > > > @@ -727,6 +727,22 @@ static struct ada_quirk_entry ada_quirk_table[]
> =
> > > > { T_DIRECT, SIP_MEDIA_FIXED, "", "Samsung SSD 850", "" },
> > > > /quirks/ADA_Q_4K | ADA_Q_NCQ_TRIM_BROKEN
> > > > },
> > > > + {
> > > > + /
> > > > + * Samsung 860 SSDs
> > > > + * 4k optimised, NCQ TRIM broken (normal TRIM fine)
> > > > + /
> > > > + { T_DIRECT, SIP_MEDIA_FIXED, "", "Samsung SSD 860*", "" },
> > > > + /quirks/ADA_Q_4K | ADA_Q_NCQ_TRIM_BROKEN
> > > > + },
> > > > + {
> > > > + /
> > > > + * Samsung 870 SSDs
> > > > + * 4k optimised, NCQ TRIM broken (normal TRIM fine)
> > > > + /
> > > > + { T_DIRECT, SIP_MEDIA_FIXED, "", "Samsung SSD 870*", "" },
> > > > + /quirks/ADA_Q_4K | ADA_Q_NCQ_TRIM_BROKEN
> > > > + },
> > > > {
> > > > /
> > > > * Samsung SM863 Series SSDs (MZ7KM*)
> >
> >
> > --
> > Andriy Gapon
>