git: e8827f4094cb - main - g_vfs_done: Report when we switch on ENXIO conversion

From: Warner Losh <imp_at_FreeBSD.org>
Date: Sun, 24 Apr 2022 20:02:26 UTC
The branch main has been updated by imp:

URL: https://cgit.FreeBSD.org/src/commit/?id=e8827f4094cb9ec7c5a4dcf9ee144442c989cd0c

commit e8827f4094cb9ec7c5a4dcf9ee144442c989cd0c
Author:     Warner Losh <imp@FreeBSD.org>
AuthorDate: 2022-04-24 19:54:20 +0000
Commit:     Warner Losh <imp@FreeBSD.org>
CommitDate: 2022-04-24 20:01:08 +0000

    g_vfs_done: Report when we switch on ENXIO conversion
    
    On the 0 -> 1 transition of sc_enxio_active, report that we're doing
    this. This is a rare, but interesting, event. Convert to using atomics
    to set this field to prevent a rare race:
    
        In CAM, when we invalidate a device, one thread (T1) will start the
        process in error processing called from *dadone
        (cam_periph_error). This routine will queue work to xpt_async_td
        (T2) and indicate to *dadone to call biodone(ENXIO) for the bio. T2
        wakes up and basically waits to acquire the periph lock. T2 will do
        so when T1 drops the periph lock just before T1's call to
        biodone. T2 acquires the lock and calls biodone(ENXIO) on all
        pending bios. These two threads will race and we could lose the
        printf or get two in rare cases. Since we only touch sc_enxio_active
        in an error path that's infrequent, the extra atomic traffic will be
        rare but will ensure robustness.
    
    Sponsored by:           Netflix
    Reviewed by:            kib
    Differential Revision:  https://reviews.freebsd.org/D35037
---
 sys/geom/geom_vfs.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/sys/geom/geom_vfs.c b/sys/geom/geom_vfs.c
index beb5632f43d6..b6f5916c386a 100644
--- a/sys/geom/geom_vfs.c
+++ b/sys/geom/geom_vfs.c
@@ -143,8 +143,11 @@ g_vfs_done(struct bio *bip)
 	cp = bip->bio_from;
 	sc = cp->geom->softc;
 	if (bip->bio_error != 0 && bip->bio_error != EOPNOTSUPP) {
-		if ((bp->b_xflags & BX_CVTENXIO) != 0)
-			sc->sc_enxio_active = 1;
+		if ((bp->b_xflags & BX_CVTENXIO) != 0) {
+			if (atomic_cmpset_int(&sc->sc_enxio_active, 0, 1))
+				printf("g_vfs_done(): %s converting all errors to ENXIO\n",
+				    bip->bio_to->name);
+		}
 		if (sc->sc_enxio_active)
 			bip->bio_error = ENXIO;
 		g_print_bio("g_vfs_done():", bip, "error = %d",