[Bug 297325] geli: BIO_DELETE passed through unmapped when authentication is enabled

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=297325

            Bug ID: 297325
           Summary: geli: BIO_DELETE passed through unmapped when
                    authentication is enabled
           Product: Base System
           Version: 15.1-RELEASE
          Hardware: Any
                OS: Any
            Status: New
          Severity: Affects Some People
          Priority: ---
         Component: kern
          Assignee: bugs@FreeBSD.org
          Reporter: dr.johannes.bruegmann@gmail.com

Created attachment 273518
  --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=273518&action=edit
dmesg output, disk / provider characteristics

I am a FreeBSD user, not a kernel developer. The observations are from my
machine and accurate; the code analysis was done with AI assistance and I
cannot
verify it myself. I am not able to run follow-up experiments beyond simple
commands.

## Summary

With `geli init -a hmac/sha256`, HMAC sectors are interleaved with data
sectors,
so logical and physical offsets differ (geli(8): only 89% of the provider is
usable at 4k/HMAC-SHA256). `g_eli_start()` translates offsets for reads and
writes, but passes `BIO_DELETE` through unchanged. TRIM therefore hits
unrelated
physical regions and destroys live data.

## Observed

FreeBSD 15.1-RELEASE-p2 amd64, root pool on geli `-a hmac/sha256 -s 4096`,
`zpool create -o ashift=12 -o autotrim=on`, WD_BLACK SN8100 8TB.

After ~84 power-on hours (install, one `pkg upgrade` of 47 base packages, a few
`bectl destroy`):

```
scan: scrub repaired 4.72M in 00:00:56 with 6157 errors
  nda0p1.eli  ONLINE  0 0 1.81K
errors: Permanent errors ... (6157 files, incl. /usr/src, package caches,
        /usr/lib/libprivateclang.so.19, /boot/kernel/if_ice.ko)
GEOM_ELI: nda0p1.eli: Failed to authenticate 4096 bytes at offset 137443987456
```

SMART clean: 0 media errors, empty error log, 0% used.

Most affected files were written once at install time and never touched again —
consistent with "their blocks were trimmed away", not with faulty writes.
Damage
grew sharply after operations that free many blocks at once.

## Analysis

`sys/geom/eli/g_eli_integrity.c` (lines 342, 398, 476) translates for
read/write:

```c
cbp->bio_offset = (bp->bio_offset / bp->bio_to->sectorsize) *
    sc->sc_bytes_per_sector;
```

`sys/geom/eli/g_eli.c`, `g_eli_start()` (~line 477) does not:

```c
case BIO_DELETE:
        if (!(sc->sc_flags & G_ELI_FLAG_NODELETE))
                break;              /* pass through */
...
case BIO_DELETE:
        cbp->bio_to = cp->provider;
        g_io_request(cbp, cp);      /* clone keeps logical offset/length */
```

The deviation grows linearly with the offset (~11%), so a TRIM for logical
137 GB lands >15 GB away — reliably on unrelated live data.

## Reproduce (expected)

1. `geli init -a hmac/sha256 -s 4096 <prov>` (no `-T`)
2. `zpool create -o ashift=12 -o autotrim=on tank <prov>.eli`
3. Write several GB, delete a large part so ZFS issues TRIM
4. `zpool scrub tank`

## Workaround

`geli init -T` (or `geli configure -T`) and `zpool set autotrim=off`.

## Suggested fix

Translate offset and length for `BIO_DELETE` when `G_ELI_FLAG_AUTH` is set, or
reject `BIO_DELETE` with `EOPNOTSUPP` in that case. At minimum geli(8) should
warn that `-a` must not be combined with TRIM pass-through.


## Side note

On an authenticated provider, never-written sectors cannot be read at all (no
valid HMAC). `zpool create` on a freshly initialised provider therefore fails
with "one or more devices is currently unavailable" until the provider has been
written once in full. A note in geli(8) would help.

-- 
You are receiving this mail because:
You are the assignee for the bug.