From nobody Thu Feb 24 10:04:27 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id D9A3C19E0951; Thu, 24 Feb 2022 10:04:29 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4K47mW5pxbz3QRM; Thu, 24 Feb 2022 10:04:27 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1645697069; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=seM0zl/etWQAgk9yN41jDURxYbyHpbcXMl9dKyaNoYM=; b=TIGiUo0FiweVsMz4FW2syt8IR8ihwlHDTrWTrR3Rlb9doMIXkUlOLIbo2OujePey71MyCw aDJl0WB5NzWH2LbTmjWH/99W8VgngYsxGizLjuPO6rlmNmk4SPvrCwW8Sx8JeyiHsAeCRw 0VYsOsDa5yGB+z91ChTQaITfBs1xVUXsfNEFBNKU5m7xv+m/U9A9e/SJiCUMoBmJTsLzTL OMbGZI0Z3VU2L+5P1roW5LyXcvvtD+inHLW6RQIfBCRzAsnGmySy/pyypyrEll8Dsz3q4A zOdF30g9vzZAskcNeV8YdP6QOU8xwcq9AOz2RgyogpAtOUHGVMlZhsaXzs+sng== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9B1881F83; Thu, 24 Feb 2022 10:04:27 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 21OA4Rp4003730; Thu, 24 Feb 2022 10:04:27 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 21OA4Rvo003729; Thu, 24 Feb 2022 10:04:27 GMT (envelope-from git) Date: Thu, 24 Feb 2022 10:04:27 GMT Message-Id: <202202241004.21OA4Rvo003729@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Hans Petter Selasky Subject: git: 6bf4f9268fb2 - stable/13 - uma: Add UMA_ZONE_UNMANAGED List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: hselasky X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 6bf4f9268fb247248a9e4f33fba1940b46debac7 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1645697069; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=seM0zl/etWQAgk9yN41jDURxYbyHpbcXMl9dKyaNoYM=; b=P6YV/n8JC2kiF8oLXFT+3/k/fzQlHO5vAiuvTTGJy641+Nl6LLpR531rSW5koMqP4+xBMm UYeH/kYrrkP/h+CH+GlcwPC2EjfGvG/2xNrUtS8Y1Rhfoql3ol+47qEQpGa1FMertFmlC/ m/Ylp+sCvh0UfP90Oyr6EubzGINufiMnlypyPbGft5vWw/vbXkKr/KQKlicNE+yxEyEVXW MnDjx6wAHhdEq/od1x+8cu5OQZU02OeOrZqlXdnfXguvUJ7qGZ8w2B7MdydVGFa1Kx/PRc nxjC12MNinJ08OiCLU0x9DKS5b9wHqEu23z8brCWMQW7OUUHcbGtlV1YU6uUvg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1645697069; a=rsa-sha256; cv=none; b=rIQSfD7p0rAtWvtL4LIIjjawv7OW/zgmUuHopTGLQD65IK8/F5t2DAfNDhDvzaAaA9fdKw ZcBkJaDcw0fxgYOZQLg/+hSXIrXuNtZkSdH0Qy0tSu27ZUAVmDOucjCqHHSxpe2LSbfXI5 1bBxHFk/TW9zK9mQvupvrNgSeI+G059LVNlH7vqko4hTs3mjvzFjdDkeYu37GKjHIpMiaX hLcDR2NS/j9/o3nfuhnE4iky1GxCs5cF6y2unTplXDL7cZrdhZmrtvW4S+XviIunZfHad2 REgD3+Fygao+SX03XweKgUWB8k2XAhWqveGkCEbFO91zXXECye/4/m6/OjyS3g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by hselasky: URL: https://cgit.FreeBSD.org/src/commit/?id=6bf4f9268fb247248a9e4f33fba1940b46debac7 commit 6bf4f9268fb247248a9e4f33fba1940b46debac7 Author: Hans Petter Selasky AuthorDate: 2022-02-24 09:59:28 +0000 Commit: Hans Petter Selasky CommitDate: 2022-02-24 09:59:28 +0000 uma: Add UMA_ZONE_UNMANAGED Allow a zone to opt out of cache size management. In particular, uma_reclaim() and uma_reclaim_domain() will not reclaim any memory from the zone, nor will uma_timeout() purge cached items if the zone is idle. This effectively means that the zone consumer has control over when items are reclaimed from the cache. In particular, uma_zone_reclaim() will still reclaim cached items from an unmanaged zone. Reviewed by: hselasky, kib Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D34142 (cherry picked from commit 389a3fa693ef61c35e15b36f042fb24197b7afb1) --- share/man/man9/zone.9 | 8 +++++- sys/vm/uma.h | 4 +++ sys/vm/uma_core.c | 73 ++++++++++++++++++++++++--------------------------- 3 files changed, 46 insertions(+), 39 deletions(-) diff --git a/share/man/man9/zone.9 b/share/man/man9/zone.9 index f28d92d6797e..a3dce742aebe 100644 --- a/share/man/man9/zone.9 +++ b/share/man/man9/zone.9 @@ -25,7 +25,7 @@ .\" .\" $FreeBSD$ .\" -.Dd April 14, 2021 +.Dd February 15, 2022 .Dt UMA 9 .Os .Sh NAME @@ -326,6 +326,12 @@ The zone is for the VM subsystem. Items in this zone must be contiguous in physical address space. Items will follow normal alignment constraints and may span page boundaries between pages with contiguous physical addresses. +.It Dv UMA_ZONE_UNMANAGED +By default, UMA zone caches are shrunk to help resolve free page shortages. +Cached items that have not been used for a long period may also be freed from +zone. +When this flag is set, the system will not reclaim memory from the zone's +caches. .El .Pp Zones can be destroyed using diff --git a/sys/vm/uma.h b/sys/vm/uma.h index 5d473ba909b6..dc8bd07d299e 100644 --- a/sys/vm/uma.h +++ b/sys/vm/uma.h @@ -233,6 +233,10 @@ uma_zone_t uma_zcache_create(const char *name, int size, uma_ctor ctor, * These flags share space with UMA_ZFLAGs in uma_int.h. Be careful not to * overlap when adding new features. */ +#define UMA_ZONE_UNMANAGED 0x0001 /* + * Don't regulate the cache size, even + * under memory pressure. + */ #define UMA_ZONE_ZINIT 0x0002 /* Initialize with zeros */ #define UMA_ZONE_CONTIG 0x0004 /* * Physical memory underlying an object diff --git a/sys/vm/uma_core.c b/sys/vm/uma_core.c index eb3773904db6..d4c9906cc106 100644 --- a/sys/vm/uma_core.c +++ b/sys/vm/uma_core.c @@ -1168,10 +1168,12 @@ zone_timeout(uma_zone_t zone, void *unused) trim: /* Trim caches not used for a long time. */ - for (int i = 0; i < vm_ndomains; i++) { - if (bucket_cache_reclaim_domain(zone, false, false, i) && - (zone->uz_flags & UMA_ZFLAG_CACHE) == 0) - keg_drain(zone->uz_keg, i); + if ((zone->uz_flags & UMA_ZONE_UNMANAGED) == 0) { + for (int i = 0; i < vm_ndomains; i++) { + if (bucket_cache_reclaim_domain(zone, false, false, i) && + (zone->uz_flags & UMA_ZFLAG_CACHE) == 0) + keg_drain(zone->uz_keg, i); + } } } @@ -1680,24 +1682,6 @@ zone_reclaim(uma_zone_t zone, int domain, int waitok, bool drain) ZONE_UNLOCK(zone); } -static void -zone_drain(uma_zone_t zone, void *arg) -{ - int domain; - - domain = (int)(uintptr_t)arg; - zone_reclaim(zone, domain, M_NOWAIT, true); -} - -static void -zone_trim(uma_zone_t zone, void *arg) -{ - int domain; - - domain = (int)(uintptr_t)arg; - zone_reclaim(zone, domain, M_NOWAIT, false); -} - /* * Allocate a new slab for a keg and inserts it into the partial slab list. * The keg should be unlocked on entry. If the allocation succeeds it will @@ -3492,7 +3476,8 @@ uma_zalloc_debug(uma_zone_t zone, void **itemp, void *udata, int flags) zone->uz_ctor(item, zone->uz_size, udata, flags) != 0) { counter_u64_add(zone->uz_fails, 1); - zone->uz_fini(item, zone->uz_size); + if (zone->uz_fini != NULL) + zone->uz_fini(item, zone->uz_size); *itemp = NULL; return (error); } @@ -5140,6 +5125,21 @@ uma_zone_memory(uma_zone_t zone) return (sz * PAGE_SIZE); } +struct uma_reclaim_args { + int domain; + int req; +}; + +static void +uma_reclaim_domain_cb(uma_zone_t zone, void *arg) +{ + struct uma_reclaim_args *args; + + args = arg; + if ((zone->uz_flags & UMA_ZONE_UNMANAGED) == 0) + uma_zone_reclaim_domain(zone, args->req, args->domain); +} + /* See uma.h */ void uma_reclaim(int req) @@ -5150,23 +5150,23 @@ uma_reclaim(int req) void uma_reclaim_domain(int req, int domain) { - void *arg; + struct uma_reclaim_args args; bucket_enable(); - arg = (void *)(uintptr_t)domain; + args.domain = domain; + args.req = req; + sx_slock(&uma_reclaim_lock); switch (req) { case UMA_RECLAIM_TRIM: - zone_foreach(zone_trim, arg); - break; case UMA_RECLAIM_DRAIN: - zone_foreach(zone_drain, arg); + zone_foreach(uma_reclaim_domain_cb, &args); break; case UMA_RECLAIM_DRAIN_CPU: - zone_foreach(zone_drain, arg); + zone_foreach(uma_reclaim_domain_cb, &args); pcpu_cache_drain_safe(NULL); - zone_foreach(zone_drain, arg); + zone_foreach(uma_reclaim_domain_cb, &args); break; default: panic("unhandled reclamation request %d", req); @@ -5177,8 +5177,8 @@ uma_reclaim_domain(int req, int domain) * we visit again so that we can free pages that are empty once other * zones are drained. We have to do the same for buckets. */ - zone_drain(slabzones[0], arg); - zone_drain(slabzones[1], arg); + uma_zone_reclaim_domain(slabzones[0], UMA_RECLAIM_DRAIN, domain); + uma_zone_reclaim_domain(slabzones[1], UMA_RECLAIM_DRAIN, domain); bucket_zone_drain(domain); sx_sunlock(&uma_reclaim_lock); } @@ -5221,19 +5221,16 @@ uma_zone_reclaim(uma_zone_t zone, int req) void uma_zone_reclaim_domain(uma_zone_t zone, int req, int domain) { - void *arg; - - arg = (void *)(uintptr_t)domain; switch (req) { case UMA_RECLAIM_TRIM: - zone_trim(zone, arg); + zone_reclaim(zone, domain, M_NOWAIT, false); break; case UMA_RECLAIM_DRAIN: - zone_drain(zone, arg); + zone_reclaim(zone, domain, M_NOWAIT, true); break; case UMA_RECLAIM_DRAIN_CPU: pcpu_cache_drain_safe(zone); - zone_drain(zone, arg); + zone_reclaim(zone, domain, M_NOWAIT, true); break; default: panic("unhandled reclamation request %d", req);