From nobody Tue May 14 17:18:18 2024 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Vf33Y0nBkz5KhYr for ; Tue, 14 May 2024 17:18:33 +0000 (UTC) (envelope-from rlibby@gmail.com) Received: from mail-lj1-f171.google.com (mail-lj1-f171.google.com [209.85.208.171]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Vf33X35fdz3xGX; Tue, 14 May 2024 17:18:32 +0000 (UTC) (envelope-from rlibby@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=freebsd.org (policy=none); spf=pass (mx1.freebsd.org: domain of rlibby@gmail.com designates 209.85.208.171 as permitted sender) smtp.mailfrom=rlibby@gmail.com Received: by mail-lj1-f171.google.com with SMTP id 38308e7fff4ca-2e564cad1f1so51808911fa.0; Tue, 14 May 2024 10:18:32 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715707110; x=1716311910; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8OrAiOqbQ/d0kicoiP7drTyzy1bCPSiFb5tHARgnHbU=; b=HWdY2F+xGLvpywbbiKzvPAMhy+7Z9vk0Bc4QXB3NPG9bNfudpFHWZBSh/wkekAuoLe Z+DKu+C01da30zmdE7DiaFj0s6Y/agfiNf2hxQbssgXYSybvDEXGzj1fZIgUlvkrPYko h114oIb6O9421LjTfynEpA5AjHZ56L799T+Uaq4hk7Kmu3sskJOjK5u3yI8QuxxiJoRM aMLl5J7ysrkEVcKLZaPQOh2gZH0AZfAXmVk5WGo5nGJZgQ0Xdz6+8GzN3wKC5lPs3uul OtG9Noqe/t9+0nLwAdhKzSeYe5+m8WjtEGQNxbZihQ5y/3/6lQRYt0EH/jnhVuZKMV3Y 0cCg== X-Forwarded-Encrypted: i=1; AJvYcCWZ/H45nzIsHqnEEcXYQ/bHnSAcykrHTrApKD2V4O0InL07mU9wnEGCmiOt3iemSMBv6H7rao3Lw2aUustZCMlIFN981monqbBWtWpRQsAyjvcKuvE= X-Gm-Message-State: AOJu0YwBSvtmIgnIJaX9QyD9qlk3T8kI2n6X9pT7vllOstAZlkY7+21J wmXrcR01a7SrxdUTNjSX00zcobbx/YL9OgNNftaUueCnx0lZBjTqappxvK8C X-Google-Smtp-Source: AGHT+IGrw3z20FQJHH3cOzwAy8PYArwYYks6995PeI40VItoubzaKtdMtfz8MIlhbElfLeWJC0C61w== X-Received: by 2002:a2e:828b:0:b0:2e5:1dae:1789 with SMTP id 38308e7fff4ca-2e51fe5820emr81296821fa.22.1715707109945; Tue, 14 May 2024 10:18:29 -0700 (PDT) Received: from mail-lj1-f178.google.com (mail-lj1-f178.google.com. [209.85.208.178]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-2e4d0bbd6b3sm18786931fa.17.2024.05.14.10.18.29 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 14 May 2024 10:18:29 -0700 (PDT) Received: by mail-lj1-f178.google.com with SMTP id 38308e7fff4ca-2df83058d48so76127641fa.1; Tue, 14 May 2024 10:18:29 -0700 (PDT) X-Forwarded-Encrypted: i=1; AJvYcCUV6h6EFDKIqH4tY5n4kMsI4Tqg7rUzxkpVgU/67qtsBbFWWpzDr+FHnii3JnNe6YNwRMHAWtCJH+Ev8cPIT1QQtnpKzxPXs7ZtKQIcIyBTDljyXIU= X-Received: by 2002:a2e:8847:0:b0:2e2:2791:983e with SMTP id 38308e7fff4ca-2e51fd451femr83157071fa.13.1715707109519; Tue, 14 May 2024 10:18:29 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 References: <0a3ddc685e54a289ff5cff569a95cd29@Leidinger.net> <5c7357c3-5a10-4a8e-9245-8a5787c57f35@freebsd.org> <09dd93a8c9bf836f061413c30d0b14c6@Leidinger.net> In-Reply-To: From: Ryan Libby Date: Tue, 14 May 2024 10:18:18 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Graph of the FreeBSD memory fragmentation To: Alexander Leidinger Cc: =?UTF-8?Q?Bojan_Novkovi=C4=87?= , Current , alc@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: / X-Spamd-Result: default: False [-0.91 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-0.64)[-0.641]; NEURAL_SPAM_SHORT(0.63)[0.631]; FORGED_SENDER(0.30)[rlibby@freebsd.org,rlibby@gmail.com]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; DMARC_POLICY_SOFTFAIL(0.10)[freebsd.org : SPF not aligned (relaxed), No valid DKIM,none]; MIME_GOOD(-0.10)[text/plain]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; RCVD_VIA_SMTP_AUTH(0.00)[]; FREEFALL_USER(0.00)[rlibby]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ARC_NA(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[209.85.208.171:from,209.85.208.178:received]; MISSING_XM_UA(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_NEQ_ENVFROM(0.00)[rlibby@freebsd.org,rlibby@gmail.com]; FROM_HAS_DN(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.208.171:from]; RCVD_TLS_LAST(0.00)[]; MLMMJ_DEST(0.00)[current@freebsd.org]; RCVD_COUNT_THREE(0.00)[3]; R_DKIM_NA(0.00)[]; RCPT_COUNT_THREE(0.00)[4] X-Rspamd-Queue-Id: 4Vf33X35fdz3xGX On Tue, May 14, 2024 at 9:09=E2=80=AFAM Ryan Libby wro= te: > > On Tue, May 14, 2024 at 1:14=E2=80=AFAM Alexander Leidinger > wrote: > > > > Am 2024-05-14 03:54, schrieb Ryan Libby: > > > That was a long winded way of saying: the "UMA bucket" axis is > > > actually "vm phys free list order". > > > > > > That said, I find that dimension confusing because in fact there's > > > just one piece of information there, the average size of a free list > > > entry, and it doesn't actually depend on the free list order. The > > > graph could be 2D. > > > > It evolved into that... > > At first I had a 3 dimensional dataset and the first try was to plot it > > as is (3D). The outcome (as points) was not as good as I wanted it to > > be, and plotting as lines gave the wrong direction of lines. I massaged > > the plotting instructions until it looked good enough. I did not try a > > 2D plot. I agree, with different colors for each free list order a 2D > > plot may work too. If a 2D plot is better than a 3D plot in this case, > > depends on the mental model of the topic the viewer has. One size may > > not fit all. Feel free to experiment with other plotting styles. > > > > What I mean is that the 13 values in the depth dimension (now "freelist > size") are actually all showing the same information -- except for > integer truncation issues and having clamped the negative values at > -1000. Each index value for a given order completely determines the > values for the other orders at a given time point. > > In the patch (D40575) this is > return (1000 - > ((info.free_pages * 1000) / (1 << order) / info.free_blocks))= ; > but notice that free_pages and free_blocks don't depend on order, they > are computed across all free list entries, of all orders, and are the > same for a calculation for any order. So for example we could solve for > the average free list entry size by taking the value from order of 0: > index_0 =3D 1000 - 1000 / 1 * free_pages / free_blocks > avg_pages =3D free_pages / free_blocks =3D -(index_0 - 1000) / 10= 00 > and from that you can calculate all the other values. Or just display > it directly. I'd suggest try plotting log2(avg_pages). > > In other words, I think just considering one value per time point is > simpler and doesn't lose any information. > > > > The paper that defines this fragmentation index also says that "the > > > fragmentation index is only meaningful when an allocation fails". Ar= e > > > you actually seeing any contiguous allocations failures in your > > > measurements? > > > > I'm not aware of such. > > The index may only be meaningful for the purposes of the goal of the > > paper when there are such failures, but if you look at the graph and ho= w > > it changed when Bojan changed the guard pages, I see value in the graph > > for more than what the paper suggests. > > > > > Without that context, it seems like what the proposed sysctl reports > > > is indirectly just the average size of free list entries. We could > > > just report that. > > > > The calculation of the value is part of a bigger picture. The value > > returned is used by some other code to make decisions. > > > > Bye, > > Alexander. > > > > -- > > http://www.Leidinger.net Alexander@Leidinger.net: PGP 0x8F31830F9F2772B= F > > http://www.FreeBSD.org netchild@FreeBSD.org : PGP 0x8F31830F9F2772B= F > > Okay I see that D40772 uses it, but always passes order=3D9, and compares > it with threshold=3D300. I see that it is not "always" as the order is actually arch dependent. > Rearranging, it asks if the average free list > entry size is at least 1.4 MiB. ...on amd64. > > Personally I'd prefer to consider values that are easy to interpret > rather than an arbitrary index value. > > Ryan