From nobody Sat Dec 02 19:28:32 2023 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SjKjD5Zdhz53V7N; Sat, 2 Dec 2023 19:28:32 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SjKjD1v4vz4T7V; Sat, 2 Dec 2023 19:28:32 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1701545312; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=RynA8COGHREymDgz1la+VFUoFL/UCocWyoDNrJivP5c=; b=g2jOAFEGmA8bZV4IANnPYg5023eJ8/lAyv1d4HDHbqZA5YvonCtR4T/d9V8YSeljFil5EP oIA8gwIXKM6CyYDjRa7o+mRdojJFy7P4MtldAa6UuSn3ocbriFGl9qSbjYfB9Hlre1oeCV tNWm2e7e6b5cSaDzkjv/Hr6fbCXK6QjgaXRvnJzCudalHmwZv4u6OBhfVeyPGi7+4wjNZl ZUxtY8BmiY6qd7wGYWttKe8UR9hPTCzuFrLNh63nijdEOrSCyEolCFnyWrKEViZCWq+P6g zLbPSQJylwzvrLuql6qDmM7eFqcWkdJI6MgAg0HV1sJJkdvlqFCh4ajLThGCkQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1701545312; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=RynA8COGHREymDgz1la+VFUoFL/UCocWyoDNrJivP5c=; b=nUt1uNvOf61An9Dk6KUpc3xkcLmhq1yW3TxWbFEvqS/eYLB/FRJC7FzKwqNcSLpmC0NZNm 00Do5IF+3mGbQ29I6gUSFO8bhhlSeB4n0h7/25HgIVmyNX9JG4ab+jZeHehzKuokieri8O acOm4zx7Dqcyg/ak6DvOVkWrP7/dm4XeEzLbCdSj+3h02kOGIfFH9K43F90XCknR/7IlqC 0qtHzGk7AqG0fys1kEouVCH+Uml31mxwPpcRGHpcyr20lnR46df6K7QHZ812DIpkq5iR8X bc7gOwPJ6PI69ViHNnrm11Xq55uiipuTp6Ah2RWzP9udEsfHF0Cwu2HM3R1s6w== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1701545312; a=rsa-sha256; cv=none; b=qO3/3ySq7bcKrCFgpUKuZwzvQ1zhGdZ++M6w/sXZeb3GjStaVbg4RHX67yZOY9uoAcIo8V 4fWRkF7KejcW6fzxyIT8wsIpZOTYfd19FMtVNiN/h5A9uU4HocWqpLMLvqUv22BXyLcm/n wbiSWri6pW6FM36q91Re+LukDjeGuhUjVxiH7xA04i+8WE2EuOpeP+yrZ+DKxuTmCze3HP eqJOZ5plaMWc1Y9oSF8av751+U0JQm2l8csbE7ghgiqLQpecbkdC3FrKu4ZYZARviUy19w 3OYITBpbhCLBdsJUPMueM95Kg6vh6b8S8/y/FRbLbEmIR9Mrx68kjMNEt9wlmw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4SjKjD0z2vzDQq; Sat, 2 Dec 2023 19:28:32 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 3B2JSW58084995; Sat, 2 Dec 2023 19:28:32 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 3B2JSW5K084992; Sat, 2 Dec 2023 19:28:32 GMT (envelope-from git) Date: Sat, 2 Dec 2023 19:28:32 GMT Message-Id: <202312021928.3B2JSW5K084992@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mark Johnston Subject: git: 8f269234b700 - stable/14 - graid3: Fix teardown races List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-branches@freebsd.org X-BeenThere: dev-commits-src-branches@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: markj X-Git-Repository: src X-Git-Refname: refs/heads/stable/14 X-Git-Reftype: branch X-Git-Commit: 8f269234b7003d5cb8ddcd574f10896a2364e2c8 Auto-Submitted: auto-generated The branch stable/14 has been updated by markj: URL: https://cgit.FreeBSD.org/src/commit/?id=8f269234b7003d5cb8ddcd574f10896a2364e2c8 commit 8f269234b7003d5cb8ddcd574f10896a2364e2c8 Author: Mark Johnston AuthorDate: 2023-11-02 18:35:37 +0000 Commit: Mark Johnston CommitDate: 2023-12-02 19:25:51 +0000 graid3: Fix teardown races Port commit dc399583ba09 from g_mirror, which has an effectively identical startup sequence. This fixes a race that was occasionally causing panics during GEOM test suite runs on riscv. MFC after: 1 month (cherry picked from commit 33cb9b3c3a2260cc273713b0adcd95016ff8b675) --- sys/geom/raid3/g_raid3.c | 80 ++++++++++++++++++++++++++---------------------- sys/geom/raid3/g_raid3.h | 1 + 2 files changed, 45 insertions(+), 36 deletions(-) diff --git a/sys/geom/raid3/g_raid3.c b/sys/geom/raid3/g_raid3.c index 79bfd690a35e..0d4ecb704aa5 100644 --- a/sys/geom/raid3/g_raid3.c +++ b/sys/geom/raid3/g_raid3.c @@ -104,6 +104,7 @@ static int g_raid3_destroy_geom(struct gctl_req *req, struct g_class *mp, static g_taste_t g_raid3_taste; static void g_raid3_init(struct g_class *mp); static void g_raid3_fini(struct g_class *mp); +static void g_raid3_providergone(struct g_provider *pp); struct g_class g_raid3_class = { .name = G_RAID3_CLASS_NAME, @@ -112,7 +113,8 @@ struct g_class g_raid3_class = { .taste = g_raid3_taste, .destroy_geom = g_raid3_destroy_geom, .init = g_raid3_init, - .fini = g_raid3_fini + .fini = g_raid3_fini, + .providergone = g_raid3_providergone, }; static void g_raid3_destroy_provider(struct g_raid3_softc *sc); @@ -605,6 +607,34 @@ g_raid3_destroy_disk(struct g_raid3_disk *disk) disk->d_state = G_RAID3_DISK_STATE_NODISK; } +static void +g_raid3_free_device(struct g_raid3_softc *sc) +{ + KASSERT(sc->sc_refcnt == 0, + ("%s: non-zero refcount %u", __func__, sc->sc_refcnt)); + + if (!g_raid3_use_malloc) { + uma_zdestroy(sc->sc_zones[G_RAID3_ZONE_64K].sz_zone); + uma_zdestroy(sc->sc_zones[G_RAID3_ZONE_16K].sz_zone); + uma_zdestroy(sc->sc_zones[G_RAID3_ZONE_4K].sz_zone); + } + mtx_destroy(&sc->sc_queue_mtx); + mtx_destroy(&sc->sc_events_mtx); + sx_xunlock(&sc->sc_lock); + sx_destroy(&sc->sc_lock); + free(sc->sc_disks, M_RAID3); + free(sc, M_RAID3); +} + +static void +g_raid3_providergone(struct g_provider *pp) +{ + struct g_raid3_softc *sc = pp->private; + + if (--sc->sc_refcnt == 0) + g_raid3_free_device(sc); +} + static void g_raid3_destroy_device(struct g_raid3_softc *sc) { @@ -649,16 +679,9 @@ g_raid3_destroy_device(struct g_raid3_softc *sc) g_wither_geom(sc->sc_sync.ds_geom, ENXIO); G_RAID3_DEBUG(0, "Device %s destroyed.", gp->name); g_wither_geom(gp, ENXIO); + if (--sc->sc_refcnt == 0) + g_raid3_free_device(sc); g_topology_unlock(); - if (!g_raid3_use_malloc) { - uma_zdestroy(sc->sc_zones[G_RAID3_ZONE_64K].sz_zone); - uma_zdestroy(sc->sc_zones[G_RAID3_ZONE_16K].sz_zone); - uma_zdestroy(sc->sc_zones[G_RAID3_ZONE_4K].sz_zone); - } - mtx_destroy(&sc->sc_queue_mtx); - mtx_destroy(&sc->sc_events_mtx); - sx_xunlock(&sc->sc_lock); - sx_destroy(&sc->sc_lock); } static void @@ -1049,7 +1072,7 @@ g_raid3_scatter(struct bio *pbp) off_t atom, cadd, padd, left; int first; - sc = pbp->bio_to->geom->softc; + sc = pbp->bio_to->private; bp = NULL; if ((pbp->bio_pflags & G_RAID3_BIO_PFLAG_NOPARITY) == 0) { /* @@ -1118,7 +1141,7 @@ g_raid3_gather(struct bio *pbp) struct bio *xbp, *fbp, *cbp; off_t atom, cadd, padd, left; - sc = pbp->bio_to->geom->softc; + sc = pbp->bio_to->private; /* * Find bio for which we have to calculate data. * While going through this path, check if all requests @@ -1284,7 +1307,7 @@ g_raid3_regular_request(struct bio *cbp) g_topology_assert_not(); pbp = cbp->bio_parent; - sc = pbp->bio_to->geom->softc; + sc = pbp->bio_to->private; cbp->bio_from->index--; if (cbp->bio_cmd == BIO_WRITE) sc->sc_writes--; @@ -1431,7 +1454,7 @@ g_raid3_start(struct bio *bp) { struct g_raid3_softc *sc; - sc = bp->bio_to->geom->softc; + sc = bp->bio_to->private; /* * If sc == NULL or there are no valid disks, provider's error * should be set and g_raid3_start() should not be called at all. @@ -1780,7 +1803,7 @@ g_raid3_register_request(struct bio *pbp) int round_robin, verify; ndisks = 0; - sc = pbp->bio_to->geom->softc; + sc = pbp->bio_to->private; if ((pbp->bio_cflags & G_RAID3_BIO_CFLAG_REGSYNC) != 0 && sc->sc_syncdisk == NULL) { g_io_deliver(pbp, EIO); @@ -2008,9 +2031,7 @@ g_raid3_try_destroy(struct g_raid3_softc *sc) sc->sc_worker = NULL; } else { g_topology_unlock(); - g_raid3_destroy_device(sc); - free(sc->sc_disks, M_RAID3); - free(sc, M_RAID3); + g_raid3_free_device(sc); } return (1); } @@ -2344,6 +2365,8 @@ g_raid3_launch_provider(struct g_raid3_softc *sc) } pp->stripesize *= sc->sc_ndisks - 1; pp->stripeoffset *= sc->sc_ndisks - 1; + pp->private = sc; + sc->sc_refcnt++; sc->sc_provider = pp; g_error_provider(pp, 0); g_topology_unlock(); @@ -3089,9 +3112,7 @@ g_raid3_access(struct g_provider *pp, int acr, int acw, int ace) G_RAID3_DEBUG(2, "Access request for %s: r%dw%de%d.", pp->name, acr, acw, ace); - sc = pp->geom->softc; - if (sc == NULL && acr <= 0 && acw <= 0 && ace <= 0) - return (0); + sc = pp->private; KASSERT(sc != NULL, ("NULL softc (provider=%s).", pp->name)); dcr = pp->acr + acr; @@ -3160,6 +3181,7 @@ g_raid3_create(struct g_class *mp, const struct g_raid3_metadata *md) sc->sc_idle = 1; sc->sc_last_write = time_uptime; sc->sc_writes = 0; + sc->sc_refcnt = 1; for (n = 0; n < sc->sc_ndisks; n++) { sc->sc_disks[n].d_softc = sc; sc->sc_disks[n].d_no = n; @@ -3215,18 +3237,8 @@ g_raid3_create(struct g_class *mp, const struct g_raid3_metadata *md) if (error != 0) { G_RAID3_DEBUG(1, "Cannot create kernel thread for %s.", sc->sc_name); - if (!g_raid3_use_malloc) { - uma_zdestroy(sc->sc_zones[G_RAID3_ZONE_64K].sz_zone); - uma_zdestroy(sc->sc_zones[G_RAID3_ZONE_16K].sz_zone); - uma_zdestroy(sc->sc_zones[G_RAID3_ZONE_4K].sz_zone); - } - g_destroy_geom(sc->sc_sync.ds_geom); - mtx_destroy(&sc->sc_events_mtx); - mtx_destroy(&sc->sc_queue_mtx); - sx_destroy(&sc->sc_lock); g_destroy_geom(sc->sc_geom); - free(sc->sc_disks, M_RAID3); - free(sc, M_RAID3); + g_raid3_free_device(sc); return (NULL); } @@ -3252,8 +3264,6 @@ g_raid3_destroy(struct g_raid3_softc *sc, int how) struct g_provider *pp; g_topology_assert_not(); - if (sc == NULL) - return (ENXIO); sx_assert(&sc->sc_lock, SX_XLOCKED); pp = sc->sc_provider; @@ -3302,8 +3312,6 @@ g_raid3_destroy(struct g_raid3_softc *sc, int how) G_RAID3_DEBUG(4, "%s: Woken up %p.", __func__, &sc->sc_worker); sx_xlock(&sc->sc_lock); g_raid3_destroy_device(sc); - free(sc->sc_disks, M_RAID3); - free(sc, M_RAID3); return (0); } diff --git a/sys/geom/raid3/g_raid3.h b/sys/geom/raid3/g_raid3.h index 2b5c897867fd..3eecc02798a9 100644 --- a/sys/geom/raid3/g_raid3.h +++ b/sys/geom/raid3/g_raid3.h @@ -219,6 +219,7 @@ struct g_raid3_softc { int sc_idle; /* DIRTY flags removed. */ time_t sc_last_write; u_int sc_writes; + u_int sc_refcnt; /* Number of softc references. */ TAILQ_HEAD(, g_raid3_event) sc_events; struct mtx sc_events_mtx;