From nobody Fri May 24 15:40:43 2024 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Vm8Q36gN0z5LH7S; Fri, 24 May 2024 15:40:43 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Vm8Q32K4Wz4vkX; Fri, 24 May 2024 15:40:43 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1716565243; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=jbfG8UrFbwaa2L3NLmcgDP1Rle+TQthwRv+7m8MZFwg=; b=uSQH+UoWGA/Q6S/sTG5Zw8zGmjKEIGwcs3ouvI5X3NDif+SV/LYCIzNCoMiqhCAlGoaob4 A1M+Ofdpr1CI8GpmGr4H0RLmrtODmZhQS4GQlQrB1VbzXYsYZmZnz46jlpF1t8ZKwoPhDc CloKjt4VWQ0+Z51oFVsvHqZZ2d9qFeWP82qH54ewWkXRpTDzHBsY3cN2klnzNQ51VNhTph QROXyIU1EEjPwVhPpH/iC8/6Uw7Mdm/jlPChnEkE2AduHBig+Y04xdQJLsKFoy5Hh18B+3 pY3TaArGqUJGvSYmw1sKbfCPEJEMY30VuMcDhm/pBpG8c1hOyrcOsn+NUu8Tuw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1716565243; a=rsa-sha256; cv=none; b=trpWg0Pm9UHAPTBoHyTge8scM9wNX9uuax8Mz6MMOFdQzppmAjob85SXmz96p4mzhasvCQ aHUq9a7/MU5XL9ZWPicOVq/gv2pzN6zTakej7e78Z8+3xosu9bOk7XInbbB0WpA+DNNRAM nEFeJjpFF07Etp/EnUWOXyf7PMjq78mDAN01Wi9SzBAYFvto8c5oLoL3yetph3YadBEISZ QNvfSlI0sUksNJ4j2zwKH7L1+bg8CCmxeFT7iGBTy0KxN8kkzb72Tf6UcU/cxEhwlmk7Gp Q3qDQ0p/D5o2iHDFIYxqvJ6rTXYT/1cjayqg81yNnlLRl2fvluhqirK2ZUlb3g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1716565243; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=jbfG8UrFbwaa2L3NLmcgDP1Rle+TQthwRv+7m8MZFwg=; b=yICYV5yPZlLoIH8npmNn3d4LBECkDZOyMf8Va3GRw7A8PvwBGQIeaxaE0pt8c+uIRxLp+1 sYLFTlOWp7IVDffxcfEX51bzUz0sHSiCex5+4i/lUB4UIP2C9luB0+zFm78MrjApaBRn5k HNiXJTvbpIIjHN6jcpegNxKMBptkQYv4M5+oEozgh60VVJOgVKE0K/Jg9vAveUbK/jcm5n QwKrprRNN3+EfKvQnT4o2+wBN/cZsU9q+FGP+kEvRASQejSQYaeiiDfeeim+7zfbzltDQy b2zE+6Gw83A90I9zADfaWN+fZVoVz6FLVFGWnyi39VVgpoJwFUV33kBqKtG2Sw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Vm8Q31tnmzf8l; Fri, 24 May 2024 15:40:43 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 44OFehCT012280; Fri, 24 May 2024 15:40:43 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 44OFeh1d012277; Fri, 24 May 2024 15:40:43 GMT (envelope-from git) Date: Fri, 24 May 2024 15:40:43 GMT Message-Id: <202405241540.44OFeh1d012277@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Warner Losh Subject: git: 99c14fb99ffc - main - cam: Drop periph lock when completing I/O with ENOMEM status List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: imp X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 99c14fb99ffc8fd601ad15bbe54a54fac8f4f203 Auto-Submitted: auto-generated The branch main has been updated by imp: URL: https://cgit.FreeBSD.org/src/commit/?id=99c14fb99ffc8fd601ad15bbe54a54fac8f4f203 commit 99c14fb99ffc8fd601ad15bbe54a54fac8f4f203 Author: Warner Losh AuthorDate: 2024-05-24 14:32:04 +0000 Commit: Warner Losh CommitDate: 2024-05-24 14:32:04 +0000 cam: Drop periph lock when completing I/O with ENOMEM status When biofinish calls g_io_deliver with an error of ENOMEM, that kicks off the slowdown protocol, forcing I/O to go through g_down rather than be directly dispatch. One of the side effects is that the I/O is resubmitted, so the start routines get called recursively, leading to a recursive lock panic. Rather than make the periph lock recursive, drop and reacquire the lock around such calls to biofinish. For nda, this happens only when we can't allocate space to construct a TRIM. For ada and da, this is only for certain ZONE operations. Sponsored by: Netflix Reviewed by: gallatin Differential Revision: https://reviews.freebsd.org/D45310 --- sys/cam/ata/ata_da.c | 9 +++++++++ sys/cam/nvme/nvme_da.c | 10 ++++++++++ sys/cam/scsi/scsi_da.c | 11 ++++++++++- 3 files changed, 29 insertions(+), 1 deletion(-) diff --git a/sys/cam/ata/ata_da.c b/sys/cam/ata/ata_da.c index d4a591943307..6e008cfc8d22 100644 --- a/sys/cam/ata/ata_da.c +++ b/sys/cam/ata/ata_da.c @@ -2518,7 +2518,16 @@ adastart(struct cam_periph *periph, union ccb *start_ccb) error = ada_zone_cmd(periph, start_ccb, bp, &queue_ccb); if ((error != 0) || (queue_ccb == 0)) { + /* + * g_io_deliver will recurisvely call start + * routine for ENOMEM, so drop the periph + * lock to allow that recursion. + */ + if (error == ENOMEM) + cam_periph_unlock(periph); biofinish(bp, NULL, error); + if (error == ENOMEM) + cam_periph_lock(periph); xpt_release_ccb(start_ccb); return; } diff --git a/sys/cam/nvme/nvme_da.c b/sys/cam/nvme/nvme_da.c index 3f6cf8702870..41c552e2780a 100644 --- a/sys/cam/nvme/nvme_da.c +++ b/sys/cam/nvme/nvme_da.c @@ -1077,7 +1077,17 @@ ndastart(struct cam_periph *periph, union ccb *start_ccb) trim = malloc(sizeof(*trim), M_NVMEDA, M_ZERO | M_NOWAIT); if (trim == NULL) { + /* + * We have to drop the periph lock when + * returning ENOMEM. g_io_deliver treats these + * request differently and will recursively call + * the start routine which causes us to get into + * ndastrategy with the periph lock held, + * leading to a panic when its acquired again. + */ + cam_periph_unlock(periph); biofinish(bp, NULL, ENOMEM); + cam_periph_lock(periph); xpt_release_ccb(start_ccb); ndaschedule(periph); return; diff --git a/sys/cam/scsi/scsi_da.c b/sys/cam/scsi/scsi_da.c index 0daaff9229b0..59745231bca5 100644 --- a/sys/cam/scsi/scsi_da.c +++ b/sys/cam/scsi/scsi_da.c @@ -3455,10 +3455,19 @@ more: queue_ccb = 0; - error = da_zone_cmd(periph, start_ccb, bp,&queue_ccb); + error = da_zone_cmd(periph, start_ccb, bp, &queue_ccb); if ((error != 0) || (queue_ccb == 0)) { + /* + * g_io_deliver will recurisvely call start + * routine for ENOMEM, so drop the periph + * lock to allow that recursion. + */ + if (error == ENOMEM) + cam_periph_unlock(periph); biofinish(bp, NULL, error); + if (error == ENOMEM) + cam_periph_lock(periph); xpt_release_ccb(start_ccb); return; }