From nobody Mon Dec 06 15:56:45 2021 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 6020818C5F4F; Mon, 6 Dec 2021 15:56:49 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4J77Mz5Tr4z4bk9; Mon, 6 Dec 2021 15:56:47 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4EEA268CA; Mon, 6 Dec 2021 15:56:45 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1B6Fuj5l033943; Mon, 6 Dec 2021 15:56:45 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1B6Fujaf033942; Mon, 6 Dec 2021 15:56:45 GMT (envelope-from git) Date: Mon, 6 Dec 2021 15:56:45 GMT Message-Id: <202112061556.1B6Fujaf033942@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Warner Losh Subject: git: de8bb30885a4 - stable/13 - mpr: fix freeze / release mismatch in timeout code List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-branches@freebsd.org X-BeenThere: dev-commits-src-branches@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: imp X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: de8bb30885a4e4f43ef92c8af34557636f9e8216 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1638806208; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=uh5Hfsxcgk5nlhyrG37q9h1QXc7myTU+sXsGlSb+2XU=; b=Mlo7KrR6xyLGIuWGKzzhmhJgHSRt5OepUrsfXz+sXQ+hOCkvpH1e71FNsX8gKWnoLaExNM jDJtsW5nAaQ3oK5raAiCfjHsZu4Yy3RbVq6xA988sPSKHYiDD5cHWiWKSP3qGW1nDg77JG lzgLgWgNNTAe2uJxPl3sAsO/XTELcra4ETK6QS+hNS4i7FZRsS62z63PRJWyqBtLq4oOIR X2qvK6pfQkJ0YY8iQCizG49lR7iJKzqVzyYNQEYLeMpD1w0KRb1sLQIfCBOdebwwVTwzj3 D3RD36ng8cbTmFh44jtAVpEw34mfJj4ow1rWWLteT+1YK0nKniMFITAu200H7g== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1638806208; a=rsa-sha256; cv=none; b=rrZl/cr/UX0LRz01sN4iRqfLtZ5a+C/Ywzvl/+ScwL29ZDVgkwQmY2RZCLjVGjTZ3zdZDZ z/J88H1IpbECY1jpdR4uYxch9hHPLZFARqQ4AnD0iaj2IKkVkLHJSkVi0Ts4x/w9K5mYNP XN2PiQugCElpUcLRF68mpO/osvTWg1o0kvh88FHJWTHRc8ZOeHT0rmcYhWDSIerTKuDrvA +zA/Nxd3cXKSZxnKuDUixIqjmbDcuvFMxlu1Xup41WZv85HbQwKTQ5vd9LI/Iyrn5EVgC3 5qsObNJqg+8sz0FgAlqHvRyYgz8TGd9ea7yRoynILXvg3QtnE0WbLS1X+N/pQQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by imp: URL: https://cgit.FreeBSD.org/src/commit/?id=de8bb30885a4e4f43ef92c8af34557636f9e8216 commit de8bb30885a4e4f43ef92c8af34557636f9e8216 Author: Warner Losh AuthorDate: 2021-11-21 15:50:46 +0000 Commit: Warner Losh CommitDate: 2021-12-06 15:56:00 +0000 mpr: fix freeze / release mismatch in timeout code So, if we're processing a timeout, and we've sent an ABORT to the firmware for that timeout, but not yet received the response from the firmware, AND we get another timeout, we queue the timeout and freeze the queue. However, when we've finally processed them all, we only release the queue once. This causes all I/O to halt as the devq remains frozen forever. Instead, only freeze the queue when we start the process (eg set INRESET on the target). This will allow the release when all the timed out I/Os have finished ABORTing. Sponsored by: Netflix Reviewed by: mav Differential Revision: https://reviews.freebsd.org/D33054 (cherry picked from commit a8837c77efd0bb9d934657edf87a9a66baac7479) --- sys/dev/mpr/mpr_sas.c | 31 +++++++++++++++++++++---------- 1 file changed, 21 insertions(+), 10 deletions(-) diff --git a/sys/dev/mpr/mpr_sas.c b/sys/dev/mpr/mpr_sas.c index 09fa6ac4bc92..3a0bf5aca702 100644 --- a/sys/dev/mpr/mpr_sas.c +++ b/sys/dev/mpr/mpr_sas.c @@ -248,7 +248,8 @@ mprsas_free_tm(struct mpr_softc *sc, struct mpr_command *tm) * INRESET flag as well or scsi I/O will not work. */ if (tm->cm_ccb) { - mpr_dprint(sc, MPR_XINFO, "Unfreezing devq for target ID %d\n", + mpr_dprint(sc, MPR_XINFO | MPR_RECOVERY, + "Unfreezing devq for target ID %d\n", tm->cm_targ->tid); tm->cm_targ->flags &= ~MPRSAS_TARGET_INRESET; xpt_release_devq(tm->cm_ccb->ccb_h.path, 1, TRUE); @@ -1924,6 +1925,9 @@ mprsas_action_scsiio(struct mprsas_softc *sassc, union ccb *ccb) */ if (targ->flags & MPRSAS_TARGET_INRESET) { ccb->ccb_h.status = CAM_REQUEUE_REQ | CAM_DEV_QFRZN; + mpr_dprint(sc, MPR_XINFO | MPR_RECOVERY, + "%s: Freezing devq for target ID %d\n", + __func__, targ->tid); xpt_freeze_devq(ccb->ccb_h.path, 1); xpt_done(ccb); return; @@ -2513,8 +2517,8 @@ mprsas_scsiio_complete(struct mpr_softc *sc, struct mpr_command *cm) if ((sassc->flags & MPRSAS_QUEUE_FROZEN) == 0) { xpt_freeze_simq(sassc->sim, 1); sassc->flags |= MPRSAS_QUEUE_FROZEN; - mpr_dprint(sc, MPR_XINFO, "Error sending command, " - "freezing SIM queue\n"); + mpr_dprint(sc, MPR_XINFO | MPR_RECOVERY, + "Error sending command, freezing SIM queue\n"); } } @@ -2549,7 +2553,7 @@ mprsas_scsiio_complete(struct mpr_softc *sc, struct mpr_command *cm) if (sassc->flags & MPRSAS_QUEUE_FROZEN) { ccb->ccb_h.status |= CAM_RELEASE_SIMQ; sassc->flags &= ~MPRSAS_QUEUE_FROZEN; - mpr_dprint(sc, MPR_XINFO, + mpr_dprint(sc, MPR_XINFO | MPR_RECOVERY, "Unfreezing SIM queue\n"); } } @@ -2817,7 +2821,7 @@ mprsas_scsiio_complete(struct mpr_softc *sc, struct mpr_command *cm) if (sassc->flags & MPRSAS_QUEUE_FROZEN) { ccb->ccb_h.status |= CAM_RELEASE_SIMQ; sassc->flags &= ~MPRSAS_QUEUE_FROZEN; - mpr_dprint(sc, MPR_XINFO, "Command completed, unfreezing SIM " + mpr_dprint(sc, MPR_INFO, "Command completed, unfreezing SIM " "queue\n"); } @@ -3424,6 +3428,11 @@ mprsas_async(void *callback_arg, uint32_t code, struct cam_path *path, * the target until the reset has completed. The CCB holds the path which * is used to release the devq. The devq is released and the CCB is freed * when the TM completes. + * We only need to do this when we're entering reset, not at each time we + * need to send an abort (which will happen if multiple commands timeout + * while we're sending the abort). We do not release the queue for each + * command we complete (just at the end when we free the tm), so freezing + * it each time doesn't make sense. */ void mprsas_prepare_for_tm(struct mpr_softc *sc, struct mpr_command *tm, @@ -3439,13 +3448,15 @@ mprsas_prepare_for_tm(struct mpr_softc *sc, struct mpr_command *tm, target->tid, lun_id) != CAM_REQ_CMP) { xpt_free_ccb(ccb); } else { - mpr_dprint(sc, MPR_XINFO, - "%s: Freezing devq for target ID %d\n", - __func__, target->tid); - xpt_freeze_devq(ccb->ccb_h.path, 1); tm->cm_ccb = ccb; tm->cm_targ = target; - target->flags |= MPRSAS_TARGET_INRESET; + if ((target->flags & MPRSAS_TARGET_INRESET) == 0) { + mpr_dprint(sc, MPR_XINFO | MPR_RECOVERY, + "%s: Freezing devq for target ID %d\n", + __func__, target->tid); + xpt_freeze_devq(ccb->ccb_h.path, 1); + target->flags |= MPRSAS_TARGET_INRESET; + } } } }