From nobody Tue Mar 25 16:57:16 2025 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4ZMbgl0X9gz5rWTQ; Tue, 25 Mar 2025 16:57:23 +0000 (UTC) (envelope-from bz@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4ZMbgk6gq9z3c8D; Tue, 25 Mar 2025 16:57:22 +0000 (UTC) (envelope-from bz@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1742921842; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=4KqWxPterQVRvMxV29GomyvS+E1rqiDKGDXPwZPs0X4=; b=X4ralv7sdB7yd7F8aCV5njHTAVcImTxKwoSul/hwoQ67ZdfAbsrr6zVIIj+wQYpU1UKLoZ bXNaXnmhsW3XcihxznVuXW8s3Qn+i5KEw3rV1qQUhiA8193r8BhcS+afFEsZx5sE4tvI6e Y6eh56YUcQMTH/xhHer9/Fhh7DdwWPkW/Xsqm7j83GtbpTdrd+7qeWv7F6s6oNkxTWSzKW tF5b7e0IGtFMbfUe0tYCqO2QWHqhJYTEzi7CiF8l5rBUou6OA5Q79OL3iZk3skUVOF/qXV vvZpiZpFyoe1H6qkHUYOvmBDFku8xd1BVaxCOixAuUcUgq+/6zgw6P4WA+FdHg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1742921842; a=rsa-sha256; cv=none; b=r1yTCeBrycOfxCmCMVpRq+lRrXT/NUcmaUu+6XZ02/x3Sk71ViUZztjvx+swdRqx1zi92u GmeEIQPLOoSUDE0m1vx4jxJVjSpxfvItUuQVUwNY+ttWuo/J2iCtt/SXN++xBMmwKUBIOE Kegc6+ldnkTIf+vDJ/BadNeymK8YvxThMI2P2+C7X9HGcOj2xO9wuYP67ii0Eq+ZtCQXjY FpV224af1TCFePXgcv+HDqmvqwkfLeTUn6fOIiMM3mZ3cawwNFJKtEIb/WDvJcfXZs9rzr BFUpfPnlP19JKLbJfLVKqf2vg/+kxE8OieQId3hy+1aYgGFOTSALziMk5WiB6A== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1742921842; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=4KqWxPterQVRvMxV29GomyvS+E1rqiDKGDXPwZPs0X4=; b=ZmhGuYdtjF5bKk4H3DZkXQ5guK3+kqv+vjJLIujja2wpmq0EtFXwBBmC3AoV1BGNx/QSm4 1mL6ztT0FCth3ViWxWQza5OVP2aLHOS1OivXuyLjSVOwyy+1Eof75Ehe6/OwXR1oU/cbsd Q8pq/1pNJ75YFYux8wtOKkkcyrcWwKczPFfGxlRLumdefulk0xqUHMHmIf1vtkPmDa3vdp U6S3BJ+Z+gkEZY0BOvVC2TPLOqWbFk+mHpoKuZW8V2a5MzZkUQ45AGn4Fyj6VIdK8DNEp1 61SiGFeNyWgDKfiWSJ3oBKE+eDY88VEEbRqfCwMGRDBiY2zh1e21EtCd6QtWBw== Received: from mx-01.divo.sbone.de (mx-01.divo.sbone.de [IPv6:2003:a:140a:2200:6:594:fffe:19]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (prime256v1) client-digest SHA256) (Client CN "mx-01.divo.sbone.de", Issuer "E5" (verified OK)) (Authenticated sender: bz/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4ZMbgk4GT5z13dc; Tue, 25 Mar 2025 16:57:22 +0000 (UTC) (envelope-from bz@FreeBSD.org) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:4902:0:7404:2:1025]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256) (No client certificate requested) by mx-01.divo.sbone.de (Postfix) with ESMTPS id 6DF9DA64805; Tue, 25 Mar 2025 16:57:16 +0000 (UTC) Received: from content-filter.t4-02.sbone.de (content-filter.t4-02.sbone.de [IPv6:fde9:577b:c1a9:4902:0:7404:2:2742]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id C70AB2D029E0; Tue, 25 Mar 2025 16:57:18 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:4902:0:7404:2:1025]) by content-filter.t4-02.sbone.de (content-filter.t4-02.sbone.de [IPv6:fde9:577b:c1a9:4902:0:7404:2:2742]) (amavisd-new, port 10024) with ESMTP id RQt1bksMOtYR; Tue, 25 Mar 2025 16:57:16 +0000 (UTC) Received: from strong-rtwn0.sbone.de (strong-rtwn0.sbone.de [IPv6:fde9:577b:c1a9:4902:3e64:cfff:fe55:bc80]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id B09962D029D8; Tue, 25 Mar 2025 16:57:16 +0000 (UTC) Date: Tue, 25 Mar 2025 16:57:16 +0000 (UTC) From: "Bjoern A. Zeeb" To: Olivier Certner cc: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org Subject: Re: git: 718d1928f874 - main - LinuxKPI: make linux_alloc_pages() honor __GFP_NORETRY In-Reply-To: <202503250919.52P9JfOA032702@gitrepo.freebsd.org> Message-ID: References: <202503250919.52P9JfOA032702@gitrepo.freebsd.org> X-OpenPGP-Key-Id: 0x14003F198FEFA3E77207EE8D2B58B8F83CCF1842 List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed On Tue, 25 Mar 2025, Olivier Certner wrote: > The branch main has been updated by olce: > > URL: https://cgit.FreeBSD.org/src/commit/?id=718d1928f8748fe4429c011296f94f194d63c695 > > commit 718d1928f8748fe4429c011296f94f194d63c695 > Author: Mathieu > AuthorDate: 2024-11-14 00:24:02 +0000 > Commit: Olivier Certner > CommitDate: 2025-03-25 08:41:44 +0000 > > LinuxKPI: make linux_alloc_pages() honor __GFP_NORETRY > > This is to fix slowdowns with drm-kmod that get worse over time as > physical memory become more fragmented (and probably also depending on > other factors). > > Based on information posted in this bug report: > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=277476 > > By default, linux_alloc_pages() retries failed allocations by calling > vm_page_reclaim_contig() to attempt to free contiguous physical memory > pages. vm_page_reclaim_contig() does not always succeed and calling it > can be very slow even when it fails. When physical memory is very > fragmented, vm_page_reclaim_contig() can end up being called (and > failing) after every allocation attempt. This could cause very > noticeable graphical desktop hangs (which could last seconds). > > The drm-kmod code in question attempts to allocate multiple contiguous > pages at once but does not actually require them to be contiguous. It > can fallback to doing multiple smaller allocations when larger > allocations fail. It passes alloc_pages() the __GFP_NORETRY flag in this > case. What is the drm code in question? ttm_pool_alloc -> ttm_pool_alloc_page()? As all other uses of __GFP_NORETRY in 6.1 (ignoring drm_printf.c) seem to be in i915. > This patch makes linux_alloc_pages() fail early (without retrying) when > this flag is passed. > > [olce: The problem this patch fixes is longer and longer GUI freezes as > a machine's memory gets filled and becomes fragmented, when using amdgpu > from DRM kmod 5.15 and DRM kmod 6.1 (DRM kmod 5.10 is unaffected; newer > Linux kernel introduced an "optimization" by which a pool of pages is > filled preferentially with contiguous pages, which triggered the problem > for us). The original commit message above evokes freezes lasting > seconds, but I occasionally witnessed some lasting tens of minutes, > rendering a machine completely useless. > > The patch has been reviewed for its potential impacts to other LinuxKPI > parts and our existing DRM kmods' code. In particular, there is no > other user of __GFP_NORETRY/GFP_NORETRY with Linux's alloc_pages*() > functions in our tree or DRM kmod ports. Are you sure? i915_gem_object_get_pages_internal() in drm-6.1 at least seems to conditionally pass it down: 17 #define QUIET (__GFP_NORETRY | __GFP_NOWARN) ... 74 page = alloc_pages(gfp | (order ? QUIET : MAYFAIL), Seems it can deal with allocation failures, lowering order or returning -ENOMEM from the function so should be fine hopefully. > It has also been tested extensively, by me for months against 14-STABLE > and sporadically on -CURRENT on a RX580, and by several others as > reported below and as is visible in more details in the quoted bugzilla > PR and in the initial drm-kmod issue at > https://github.com/freebsd/drm-kmod/issues/302, on a variety of other > AMD GPUs (several RX580, RX570, Radeon Pro WX5100, Green Sardine 5600G, > Ryzen 9 4900H with embedded Renoir).] > > PR: 277476 > Reported by: Josef 'Jeff' Sipek > Reviewed by: olce > Tested by: many (olce, Pierre Pronchery, Evgenii Khramtsov, chaplina, rk) > MFC after: 2 weeks > Relnotes: yes > Sponsored by: The FreeBSD Foundation (review and part of testing) > --- > sys/compat/linuxkpi/common/include/linux/gfp.h | 4 ++-- > sys/compat/linuxkpi/common/src/linux_page.c | 3 ++- > 2 files changed, 4 insertions(+), 3 deletions(-) > > diff --git a/sys/compat/linuxkpi/common/include/linux/gfp.h b/sys/compat/linuxkpi/common/include/linux/gfp.h > index bd8fa1a18372..35dbe3e2a436 100644 > --- a/sys/compat/linuxkpi/common/include/linux/gfp.h > +++ b/sys/compat/linuxkpi/common/include/linux/gfp.h > @@ -43,7 +43,6 @@ > #define __GFP_NOWARN 0 > #define __GFP_HIGHMEM 0 > #define __GFP_ZERO M_ZERO > -#define __GFP_NORETRY 0 > #define __GFP_NOMEMALLOC 0 > #define __GFP_RECLAIM 0 > #define __GFP_RECLAIMABLE 0 > @@ -57,7 +56,8 @@ > #define __GFP_KSWAPD_RECLAIM 0 > #define __GFP_WAIT M_WAITOK > #define __GFP_DMA32 (1U << 24) /* LinuxKPI only */ > -#define __GFP_BITS_SHIFT 25 > +#define __GFP_NORETRY (1U << 25) /* LinuxKPI only */ > +#define __GFP_BITS_SHIFT 26 > #define __GFP_BITS_MASK ((1 << __GFP_BITS_SHIFT) - 1) > #define __GFP_NOFAIL M_WAITOK > > diff --git a/sys/compat/linuxkpi/common/src/linux_page.c b/sys/compat/linuxkpi/common/src/linux_page.c > index bece8c954bfd..b5a0d34a6ad7 100644 > --- a/sys/compat/linuxkpi/common/src/linux_page.c > +++ b/sys/compat/linuxkpi/common/src/linux_page.c > @@ -117,7 +117,8 @@ linux_alloc_pages(gfp_t flags, unsigned int order) > page = vm_page_alloc_noobj_contig(req, npages, 0, pmax, > PAGE_SIZE, 0, VM_MEMATTR_DEFAULT); > if (page == NULL) { > - if (flags & M_WAITOK) { > + if ((flags & (M_WAITOK | __GFP_NORETRY)) == > + M_WAITOK) { > int err = vm_page_reclaim_contig(req, > npages, 0, pmax, PAGE_SIZE, 0); > if (err == ENOMEM) > -- Bjoern A. Zeeb r15:7