From nobody Tue May 14 16:09:30 2024 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Vf1X839jHz5KbL0 for ; Tue, 14 May 2024 16:09:44 +0000 (UTC) (envelope-from rlibby@gmail.com) Received: from mail-lf1-f41.google.com (mail-lf1-f41.google.com [209.85.167.41]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Vf1X81Q2Bz4nhf; Tue, 14 May 2024 16:09:44 +0000 (UTC) (envelope-from rlibby@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-lf1-f41.google.com with SMTP id 2adb3069b0e04-51f45104ef0so6304463e87.3; Tue, 14 May 2024 09:09:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715702982; x=1716307782; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mv+QW6Xl0uUJ/JEdErxvOk1CwFK4gC37KCoNUlO5S9A=; b=of8RKS5X4fTB3kKavLyNZqGMVibTNLPJsVn9NeXP9MuQeTsMENzjScfXSow+Xve7fR zajafvlmpSt6dvUjYyVFgXhO83N3hx98ZFQSwvN2LVSKaLJjCWreI3/QFJg3voC8u+cj 0u1lECbx1bq6Sos+PTOp4QwRAz+C0dZAbPp+0Yg2TdLpuTYwWu+GfhJ6QjckWMBkuNYd RrXecAYcLxH+cu3kNlBoEznezKil5fZQJXTzc0hXR1GknuO7UZKjW8vuQ+HYfrllFthH 2oODirgmp1b+YmfWW91kjHPiRqtEgAyrVDu3oCmytJtp0JWgUUG9Lcr7PDJZZIyvEYfM sOQQ== X-Forwarded-Encrypted: i=1; AJvYcCWJg5075aLza6P9HCypafTdoDVzHy5vwKFyDffzcqDK3YVCvIuf+5tpQPErBgXls6C4YG5w96DFjqg2pVH/+oObsfPJLKC4jN/i2Lv5SaKSQTEYqBQ= X-Gm-Message-State: AOJu0Yzq90y6TFNKHwm/sk34HTTS5ur5oFv0g/XYd/BmAbbMMQMwgk4r gepfc5J8aRwVDUmMMsjkosqJb1+G7QbH8IyTNnsPFNKTqghY3CDMW55aVVmX X-Google-Smtp-Source: AGHT+IEXEhJjBpaaJAzkZCJU+72eNg72BC5ncQxYuaM5onSkHQi6H22Iv+NhhVB8NlTJ+HtdqRuUrg== X-Received: by 2002:a05:6512:12d6:b0:523:4230:9179 with SMTP id 2adb3069b0e04-52342309259mr3625922e87.40.1715702981762; Tue, 14 May 2024 09:09:41 -0700 (PDT) Received: from mail-lj1-f180.google.com (mail-lj1-f180.google.com. [209.85.208.180]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-521f35ba351sm2186883e87.63.2024.05.14.09.09.41 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 14 May 2024 09:09:41 -0700 (PDT) Received: by mail-lj1-f180.google.com with SMTP id 38308e7fff4ca-2e52181c228so45794551fa.0; Tue, 14 May 2024 09:09:41 -0700 (PDT) X-Forwarded-Encrypted: i=1; AJvYcCVMS2mEGFSF2mFyX70wcLsxjGtR2ZhQCXXlU5+rJeXBAHCaVy0STjLnIyaDNdMnH6nTnNHFy3s4TnJ539d+t+CXu9jpn5V9ol49pE5/y12XT3BJNu4= X-Received: by 2002:a2e:8187:0:b0:2de:d4ef:af19 with SMTP id 38308e7fff4ca-2e51fd4306dmr89686521fa.10.1715702981432; Tue, 14 May 2024 09:09:41 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 References: <0a3ddc685e54a289ff5cff569a95cd29@Leidinger.net> <5c7357c3-5a10-4a8e-9245-8a5787c57f35@freebsd.org> <09dd93a8c9bf836f061413c30d0b14c6@Leidinger.net> In-Reply-To: <09dd93a8c9bf836f061413c30d0b14c6@Leidinger.net> From: Ryan Libby Date: Tue, 14 May 2024 09:09:30 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Graph of the FreeBSD memory fragmentation To: Alexander Leidinger Cc: =?UTF-8?Q?Bojan_Novkovi=C4=87?= , Current , alc@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US] X-Rspamd-Queue-Id: 4Vf1X81Q2Bz4nhf On Tue, May 14, 2024 at 1:14=E2=80=AFAM Alexander Leidinger wrote: > > Am 2024-05-14 03:54, schrieb Ryan Libby: > > That was a long winded way of saying: the "UMA bucket" axis is > > actually "vm phys free list order". > > > > That said, I find that dimension confusing because in fact there's > > just one piece of information there, the average size of a free list > > entry, and it doesn't actually depend on the free list order. The > > graph could be 2D. > > It evolved into that... > At first I had a 3 dimensional dataset and the first try was to plot it > as is (3D). The outcome (as points) was not as good as I wanted it to > be, and plotting as lines gave the wrong direction of lines. I massaged > the plotting instructions until it looked good enough. I did not try a > 2D plot. I agree, with different colors for each free list order a 2D > plot may work too. If a 2D plot is better than a 3D plot in this case, > depends on the mental model of the topic the viewer has. One size may > not fit all. Feel free to experiment with other plotting styles. > What I mean is that the 13 values in the depth dimension (now "freelist size") are actually all showing the same information -- except for integer truncation issues and having clamped the negative values at -1000. Each index value for a given order completely determines the values for the other orders at a given time point. In the patch (D40575) this is return (1000 - ((info.free_pages * 1000) / (1 << order) / info.free_blocks)); but notice that free_pages and free_blocks don't depend on order, they are computed across all free list entries, of all orders, and are the same for a calculation for any order. So for example we could solve for the average free list entry size by taking the value from order of 0: index_0 =3D 1000 - 1000 / 1 * free_pages / free_blocks avg_pages =3D free_pages / free_blocks =3D -(index_0 - 1000) / 1000 and from that you can calculate all the other values. Or just display it directly. I'd suggest try plotting log2(avg_pages). In other words, I think just considering one value per time point is simpler and doesn't lose any information. > > The paper that defines this fragmentation index also says that "the > > fragmentation index is only meaningful when an allocation fails". Are > > you actually seeing any contiguous allocations failures in your > > measurements? > > I'm not aware of such. > The index may only be meaningful for the purposes of the goal of the > paper when there are such failures, but if you look at the graph and how > it changed when Bojan changed the guard pages, I see value in the graph > for more than what the paper suggests. > > > Without that context, it seems like what the proposed sysctl reports > > is indirectly just the average size of free list entries. We could > > just report that. > > The calculation of the value is part of a bigger picture. The value > returned is used by some other code to make decisions. > > Bye, > Alexander. > > -- > http://www.Leidinger.net Alexander@Leidinger.net: PGP 0x8F31830F9F2772BF > http://www.FreeBSD.org netchild@FreeBSD.org : PGP 0x8F31830F9F2772BF Okay I see that D40772 uses it, but always passes order=3D9, and compares it with threshold=3D300. Rearranging, it asks if the average free list entry size is at least 1.4 MiB. Personally I'd prefer to consider values that are easy to interpret rather than an arbitrary index value. Ryan