From nobody Fri Feb 11 01:10:38 2022 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 98EC319A6169; Fri, 11 Feb 2022 01:10:40 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JvwXc1hlzz57ND; Fri, 11 Feb 2022 01:10:40 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1644541840; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=68kZJ+poXEZ421uYKeOJt7I8teFJG8jDdjNAkv501P8=; b=fU7b10RkdpjhoNa4bBU+JZtUO2bsDkG+YqmShF+/J4z0z8Eopwddv0megjf6/HIq5XXEo8 OtFbkWDEcbqhjgSXRxSqfKVdQTwvvfTCXWrNMMQGgiruiPl+3az5NvHR0/clRdXrq3xjHo XWXpYDhGkEgPNVuKyh5Aa9sXsStrDa3fT0znXyOv9Tr14OCxU2C2W1xx6SYpoEizIjsaTc nxmL2dVFgaW5lKLrZRIliIb7MWVYIjefgI69iF5JXRdfSDzJ24mNCGlYi+J3cTcMsiWeQF SIu0+e/XL5FCDsc5rpFW4n9ool5rXKJ3ooWcbX0o1C5X0b4S5oZl/t+kxq8YkQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 12E912F81; Fri, 11 Feb 2022 01:10:39 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 21B1AcnS073539; Fri, 11 Feb 2022 01:10:38 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 21B1Acs1073538; Fri, 11 Feb 2022 01:10:38 GMT (envelope-from git) Date: Fri, 11 Feb 2022 01:10:38 GMT Message-Id: <202202110110.21B1Acs1073538@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Alexander Motin Subject: git: 7adb9583bfac - stable/13 - GEOM: Relax direct dispatch for GEOM threads. List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-branches@freebsd.org X-BeenThere: dev-commits-src-branches@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mav X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 7adb9583bfac505f5a5bc79ab9635c7944be8f59 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1644541840; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=68kZJ+poXEZ421uYKeOJt7I8teFJG8jDdjNAkv501P8=; b=v4VU1dfo4gvN+YUFDQTDxC88P1VdActsrt1MIr2TSqc2FLiaSogG/F1TWeisBljiVyYsbS f1/o0KLmWpwbUYYMnWWoItnFXjIRTNjM2e/74VyLs6vaaqANsjekkgEfsSq3CjycpkiMho fOnNX2wcgRXmPRzHaGisyJgmrk5ySuT4L/C3MMbt6XRAJ7qa1bFZY5yKYK9JLczewDCOAw SdQYzWZnBB1wMP+vfl3wjiGpRFsj24MIKPjXZMFXpr8ZnLezYGO/jlCdETPK5SeWbIJytk 0waqB0aJ5GUZjZZzs+1Uju4nGD9olt+zdeh3mwI+mBYhRA/VPc1N+Bsvv3ve3A== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1644541840; a=rsa-sha256; cv=none; b=vi1Bpizs0G4aezqhDt1SZ/HF+RhTpaJJNttA3YboDaqx8WgesvPU+xfAlZC80o5iDSSAZx jFr93hgRDBHCmbhEwcJaJPXUPmGVQB7/PYomNvXUnh5AL6KSnE0Fg4hG+Kjio2TcNoAOpT LtUKfIZeaSRFvMP089OsMuFZfgpI5FpvBt115UlVVt2XDEDNjpG7V8nJc/83J2rvCykCga VwCJdvdAYz6i1aEkb3lTOkp98sggZDC0Ee1ib6VJ40yJRFfMFUfhjhjb/03/l5g0fKFlsx bqnywBo1G1A6vCLWMa0jZGI5nz1so/SMbKiVgvDy4sORMQJ0rXIWe80fEaoyUw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by mav: URL: https://cgit.FreeBSD.org/src/commit/?id=7adb9583bfac505f5a5bc79ab9635c7944be8f59 commit 7adb9583bfac505f5a5bc79ab9635c7944be8f59 Author: Alexander Motin AuthorDate: 2022-01-28 19:12:29 +0000 Commit: Alexander Motin CommitDate: 2022-02-11 00:50:45 +0000 GEOM: Relax direct dispatch for GEOM threads. The only cases when direct dispatch does not make sense is for I/O submission from down thread and for completion from up thread. In all other cases, if both consumer and producer are OK about it, we can save on context switches. MFC after: 2 weeks (cherry picked from commit ffc1cc95e78ec05a3e1a0aed869e33a44d9f6641) --- sys/geom/geom_int.h | 2 ++ sys/geom/geom_io.c | 4 ++-- sys/geom/geom_kern.c | 4 ++-- 3 files changed, 6 insertions(+), 4 deletions(-) diff --git a/sys/geom/geom_int.h b/sys/geom/geom_int.h index 9f2a011b23f0..67c46d715885 100644 --- a/sys/geom/geom_int.h +++ b/sys/geom/geom_int.h @@ -69,6 +69,8 @@ void g_io_schedule_up(struct thread *tp); /* geom_kern.c / geom_kernsim.c */ void g_init(void); +extern struct thread *g_up_td; +extern struct thread *g_down_td; extern int g_shutdown; extern int g_notaste; diff --git a/sys/geom/geom_io.c b/sys/geom/geom_io.c index 0ccbf00c1058..7c9dc4164767 100644 --- a/sys/geom/geom_io.c +++ b/sys/geom/geom_io.c @@ -561,7 +561,7 @@ g_io_request(struct bio *bp, struct g_consumer *cp) direct = (cp->flags & G_CF_DIRECT_SEND) != 0 && (pp->flags & G_PF_DIRECT_RECEIVE) != 0 && - !g_is_geom_thread(curthread) && + curthread != g_down_td && ((pp->flags & G_PF_ACCEPT_UNMAPPED) != 0 || (bp->bio_flags & BIO_UNMAPPED) == 0 || THREAD_CAN_SLEEP()) && pace == 0; @@ -653,7 +653,7 @@ g_io_deliver(struct bio *bp, int error) direct = (pp->flags & G_PF_DIRECT_SEND) && (cp->flags & G_CF_DIRECT_RECEIVE) && - !g_is_geom_thread(curthread); + curthread != g_up_td; if (direct) { /* Block direct execution if less then half of stack left. */ size_t st, su; diff --git a/sys/geom/geom_kern.c b/sys/geom/geom_kern.c index 4b7219591dce..429bd7a05a4b 100644 --- a/sys/geom/geom_kern.c +++ b/sys/geom/geom_kern.c @@ -61,8 +61,8 @@ MALLOC_DEFINE(M_GEOM, "GEOM", "Geom data structures"); struct sx topology_lock; static struct proc *g_proc; -static struct thread __read_mostly *g_up_td; -static struct thread __read_mostly *g_down_td; +struct thread __read_mostly *g_up_td; +struct thread __read_mostly *g_down_td; static struct thread __read_mostly *g_event_td; int __read_mostly g_debugflags;