From nobody Wed Sep 28 16:34:57 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Md2CV6G9Kz4ckfb; Wed, 28 Sep 2022 16:35:02 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Md2CV5kMxz3q1q; Wed, 28 Sep 2022 16:35:02 +0000 (UTC) (envelope-from avg@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1664382902; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=veUpDSbeW91M2Fg3vEODh+uhv+/s1fobGeEQ0f5xU2s=; b=M38ARL5yBEUKc6hcYibduEzHnRO4NvTdoZH4xsLs/6twHBJrvTwITyYj0WT8PKB4FgsKEg 1I39kfgcsgCDS4WkkrCGe86GW/6vVCBkNc5dlqC4yhRMHfP4HboqtzTen0OrS42nW3AFBr wzqcepeOK4RVrQbnmRcLWuyoyhnArK34o5jjbxHAl8ebifxJHKc+o/Yo7DP+fZ4hegIseG AcvdfG+N49WZ8VHWXd9JweLxdCSRRBbBUFqGhYY2TflaOYQmOJ3J9CyTvXKVsKBHid/fsV ew3vGvOZu+5t8sag4Mrjpsk1NL6mey30wVQFgk4MkL/y9ZPkn6nKAk6QTIkWAw== Received: from [192.168.0.88] (unknown [195.64.148.76]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: avg/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4Md2CT6Zf2zsfj; Wed, 28 Sep 2022 16:35:01 +0000 (UTC) (envelope-from avg@FreeBSD.org) Message-ID: <008896fa-9b89-3114-f2a5-99dc9da5f600@FreeBSD.org> Date: Wed, 28 Sep 2022 19:34:57 +0300 List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Firefox/102.0 Thunderbird/102.3.0 Subject: Re: git: d3f96f661050 - main - Fix O(n^2) behavior in sysctl Content-Language: en-US To: Maxim Sobolev , Alan Somers Cc: src-committers , dev-commits-src-all@freebsd.org, dev-commits-src-main@freebsd.org References: <202209270004.28R04K1r086731@gitrepo.freebsd.org> From: Andriy Gapon In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1664382902; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=veUpDSbeW91M2Fg3vEODh+uhv+/s1fobGeEQ0f5xU2s=; b=C96kyd9W9Tcf67mEQXgN8HWPPOu48rIMacGor4cTRF33vDC+OLYgstYJYvRW5Uc8d/g3AC ECR+Kwuu6PV2EfhqY+bV2H1AYinDyZigLP86KrBHOzlQH5+HcupvE3iAQURSJLO0jqTPJP BqQQy1ZyGwgMTUfJ0M+QKnPRZlAwBb2HRMZAxKzaj5Vbup5CJXIbixbVYEur4XVjKLbcp8 9QuV/bGdMvVdUTiU0FRMMb3BotOxHOu7wADUt+Kb0I5inY0+NnhzmimZCKmmU7tbtZsbRz GFe+1i3oRHHaUt//icLvFVsHjeHgSvNqRw58RTdHMjnFHgmUklUb0UpTl4UCBw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1664382902; a=rsa-sha256; cv=none; b=SIghh21rkZYwaYEIHYDZ5VD6UjAHXILlKISvkaqkyViUsY9eKAqZsh82UXycpJ3xVUYvY3 ywOaF+fPg2wf1c8kcurFDoSsY0eB1RoqWilDZsPtbykAI3VA4K7YH5utaIfgWpjOMV4GXv 90hgjj+vJlaxChheNKfs0ogBdWUfEd4DqWR8kehQbjFowW18AEHlag+at+ITtjWeuQ1jM5 /ZnAH+lg4Bbfs7wwJQHlfvQnr7WzwQqxb75YTl0/177S+vFYiVTbRJfl9PgzzmdSlRR7yu +6EAkkQeOifcisn4XiQ0wWzNM5ZpmTDPGnZ0o/xCJHWj3eQ/jXFzoj6cX24pew== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N On 28/09/2022 19:18, Maxim Sobolev wrote: > This also brings a question as to whether sysctl is the right interface to pull > this data from the kernel in the first place? From my somewhat ignorant look > this approach is likely to be poised with all sorts of race conditions, such so > if configuration changes while you are pulling it out you'd get some > inconsistent view that is not here not there. Wouldn't it be easier to use some > other mechanism to pull configuration of all 1,000 datasets as one blob in one > or few system calls? Like read(2) from /dev/zfsstats or something like that? > Then you can iterate over it as much as you need in userland. I've learnt that we do not talk about the "sensors framework" in a good company, but one of the reasons it got killed was the "inadequate" sysctl mechanism for querying [allegedly massive amounts of] data. > On Tue, Sep 27, 2022, 3:04 AM Alan Somers > wrote: > > The branch main has been updated by asomers: > > URL: > https://cgit.FreeBSD.org/src/commit/?id=d3f96f661050e9bd21fe29931992a8b9e67ff189 > > commit d3f96f661050e9bd21fe29931992a8b9e67ff189 > Author:     Alan Somers > AuthorDate: 2022-09-07 14:12:49 +0000 > Commit:     Alan Somers > CommitDate: 2022-09-27 00:03:34 +0000 > >     Fix O(n^2) behavior in sysctl > >     Sysctl OIDs were internally stored in linked lists, triggering O(n^2) >     behavior when userland iterates over many of them.  The slowdown is >     noticeable for MIBs that have > 100 children (for example, vm.uma).  But >     it's unignorable for kstat.zfs when a pool has > 1000 datasets. > >     Convert the linked lists into RB trees.  This produces a ~25x speedup >     for listing kstat.zfs with 4100 datasets, and no measurable penalty for >     small dataset counts. > >     Bump __FreeBSD_version for the KPI change. > >     Sponsored by:   Axcient >     Reviewed by:    mjg >     Differential Revision: https://reviews.freebsd.org/D36500 >                              \ -- Andriy Gapon