From nobody Sat Jul 31 23:48:40 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 330A312D49A0 for ; Sat, 31 Jul 2021 23:49:54 +0000 (UTC) (envelope-from pmc@citylink.dinoex.sub.org) Received: from uucp.dinoex.org (uucp.dinoex.org [IPv6:2a0b:f840::12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "uucp.dinoex.sub.de", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Gcgwx26CDz3hrj for ; Sat, 31 Jul 2021 23:49:53 +0000 (UTC) (envelope-from pmc@citylink.dinoex.sub.org) Received: from uucp.dinoex.sub.de (uucp.dinoex.org [185.220.148.12]) by uucp.dinoex.org (8.16.1/8.16.0.50) with ESMTPS id 16VNnjPj005934 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Sun, 1 Aug 2021 01:49:45 +0200 (CEST) (envelope-from pmc@citylink.dinoex.sub.org) X-MDaemon-Deliver-To: X-Authentication-Warning: uucp.dinoex.sub.de: Host uucp.dinoex.org [185.220.148.12] claimed to be uucp.dinoex.sub.de Received: (from uucp@localhost) by uucp.dinoex.sub.de (8.16.1/8.16.0.50/Submit) with UUCP id 16VNnjH5005933 for freebsd-hackers@freebsd.org; Sun, 1 Aug 2021 01:49:45 +0200 (CEST) (envelope-from pmc@citylink.dinoex.sub.org) Received: from gate.intra.daemon.contact (gate-e [192.168.98.2]) by citylink.dinoex.sub.de (8.16.1/8.16.1) with ESMTP id 16VNnUeJ046825 for ; Sun, 1 Aug 2021 01:49:30 +0200 (CEST) (envelope-from peter@gate.intra.daemon.contact) Received: from gate.intra.daemon.contact (gate-e [192.168.98.2]) by gate.intra.daemon.contact (8.16.1/8.16.1) with ESMTPS id 16VNmeOa046613 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Sun, 1 Aug 2021 01:48:40 +0200 (CEST) (envelope-from peter@gate.intra.daemon.contact) Received: (from peter@localhost) by gate.intra.daemon.contact (8.16.1/8.16.1/Submit) id 16VNme32046612 for freebsd-hackers@freebsd.org; Sun, 1 Aug 2021 01:48:40 +0200 (CEST) (envelope-from peter) Date: Sun, 1 Aug 2021 01:48:40 +0200 From: Peter To: freebsd-hackers@freebsd.org Subject: (patch) Re: Periodic rant about SCHED_ULE Message-ID: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Milter: Spamilter (Reciever: uucp.dinoex.sub.de; Sender-ip: 185.220.148.12; Sender-helo: uucp.dinoex.sub.de;) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.4 (uucp.dinoex.org [185.220.148.12]); Sun, 01 Aug 2021 01:49:48 +0200 (CEST) X-Rspamd-Queue-Id: 4Gcgwx26CDz3hrj X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of pmc@citylink.dinoex.sub.org has no SPF policy when checking 2a0b:f840::12) smtp.mailfrom=pmc@citylink.dinoex.sub.org X-Spamd-Result: default: False [-0.45 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.997]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.40)[-0.401]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; TO_DN_NONE(0.00)[]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[4]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; NEURAL_SPAM_SHORT(0.05)[0.051]; DMARC_NA(0.00)[sub.org]; R_SPF_NA(0.00)[no SPF record]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:205376, ipnet:2a0b:f840::/32, country:DE]; RCVD_TLS_LAST(0.00)[]; MAILMAN_DEST(0.00)[freebsd-hackers] X-ThisMailContainsUnwantedMimeParts: N Hi all, let's hope I can post now. Somehow my subscriptions have been deleted and I cannot not post anymore to the lists, and *some* of them (e.g. "stable") have also ceased to deliver, since 3rd July. Neither can I find a notification about such change. Now I am trying to subsribe anew... hth (Such unsuspected changes are something that makes me angry) I was just seeing the discussion about Sched-ULE. I myself was once hit by a serious flaw in the performing of sched-ULE, so I went into the source and fixed that specific matter. It solves the issue that of multiple compute jobs the one with the least I/O gets starved; I don't know if there are side-effects or downsides. I don't notice a performance impact, but I have no idea how this might behave on e.g. networking routers or such. During fixing I found one or two other obvious bugs in the code. I tried to talk to the developer, but that communication somehow ended in mid-air. I don't know what to further do with this - a patch of mine lingers in the sendbug facility for some 12 years already without being reacted on, so it is pointless to send patches. And in this case I don't even know if this one does suit all conditions except solving my problem. (It's not about absolute performance, but about making things run evenly.) So here it is, here you have it and see to it. The thing is switchable in runtime, you get a new sysctl kern.sched.resume_preempted, "1" gives the default behaviour from distribution, "0" activates the patch. Maybe it helps for one or the other. Cheerio. diff --git a/sys/kern/sched_ule.c b/sys/kern/sched_ule.c index 50f037076f4..f0bb6b38db4 100644 --- a/sys/kern/sched_ule.c +++ b/sys/kern/sched_ule.c @@ -38,7 +38,7 @@ */ #include -__FBSDID("$FreeBSD$"); +__FBSDID("$FreeBSD: releng/12.2/sys/kern/sched_ule.c 355610 2019-12-11 15:15:21Z mav $"); #include "opt_hwpmc_hooks.h" #include "opt_sched.h" @@ -223,6 +223,7 @@ static int __read_mostly preempt_thresh = 0; static int __read_mostly static_boost = PRI_MIN_BATCH; static int __read_mostly sched_idlespins = 10000; static int __read_mostly sched_idlespinthresh = -1; +static int __read_mostly resume_preempted = 1; /* * tdq - per processor runqs and statistics. All fields are protected by the @@ -483,7 +484,10 @@ tdq_runq_add(struct tdq *tdq, struct thread *td, int flags) * This queue contains only priorities between MIN and MAX * realtime. Use the whole queue to represent these values. */ - if ((flags & (SRQ_BORROWING|SRQ_PREEMPTED)) == 0) { + if (((flags & SRQ_PREEMPTED) && resume_preempted) || + (flags & SRQ_BORROWING)) + pri = tdq->tdq_ridx; + else { pri = RQ_NQS * (pri - PRI_MIN_BATCH) / PRI_BATCH_RANGE; pri = (pri + tdq->tdq_idx) % RQ_NQS; /* @@ -494,8 +498,7 @@ tdq_runq_add(struct tdq *tdq, struct thread *td, int flags) if (tdq->tdq_ridx != tdq->tdq_idx && pri == tdq->tdq_ridx) pri = (unsigned char)(pri - 1) % RQ_NQS; - } else - pri = tdq->tdq_ridx; + } runq_add_pri(ts->ts_runq, td, pri, flags); return; } else @@ -3073,6 +3076,9 @@ SYSCTL_INT(_kern_sched, OID_AUTO, interact, CTLFLAG_RW, &sched_interact, 0, SYSCTL_INT(_kern_sched, OID_AUTO, preempt_thresh, CTLFLAG_RW, &preempt_thresh, 0, "Maximal (lowest) priority for preemption"); +SYSCTL_INT(_kern_sched, OID_AUTO, resume_preempted, CTLFLAG_RW, + &resume_preempted, 0, + "Reinsert preemted threads at queue-head"); SYSCTL_INT(_kern_sched, OID_AUTO, static_boost, CTLFLAG_RW, &static_boost, 0, "Assign static kernel priorities to sleeping threads"); SYSCTL_INT(_kern_sched, OID_AUTO, idlespins, CTLFLAG_RW, &sched_idlespins, 0,