From nobody Mon Oct 16 21:16:47 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4S8VKs3CNlz4xpN7 for ; Mon, 16 Oct 2023 21:16:49 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mail-oo1-xc2c.google.com (mail-oo1-xc2c.google.com [IPv6:2607:f8b0:4864:20::c2c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4S8VKr55ddz4PnG for ; Mon, 16 Oct 2023 21:16:48 +0000 (UTC) (envelope-from mjguzik@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=DtENZmL+; spf=pass (mx1.freebsd.org: domain of mjguzik@gmail.com designates 2607:f8b0:4864:20::c2c as permitted sender) smtp.mailfrom=mjguzik@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-oo1-xc2c.google.com with SMTP id 006d021491bc7-57b5ef5b947so3336249eaf.0 for ; Mon, 16 Oct 2023 14:16:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1697491008; x=1698095808; darn=freebsd.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :references:in-reply-to:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=89M72pPEYHpqokn+eRETfhCXO7VD9CJodXJJzn2X9dc=; b=DtENZmL+9BxdLbS8xnIRcQ1T7VNeIrMokaqrKBP5QqnuWH1Vk5Q4d39uMlsUs/gGDQ xmV+nORp01VBsAqcJkPP/F9aTmhof8MulohfYlPLcp4iEbVTIed8UTwNQbIafLj8ztSv gaJj3SrfIQa/QgCHvsvtn1tEkwQL9ZoUSirNOfhPL7H/XtreZmUnVDhNme1hPSFS5LTH BcGIDo/QMwLqyKLS/2JSAs2DjxChUgbICDIhThN1x6xYmwL3HggQL1jgo/vQhzvbToP5 jG0YjIHmNfuyaFZAprbX7YlDCeiNjYlD7CIp0ps3ixT2XLX7UpNoHLLVc6f2FG5h9Giy z4Rg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697491008; x=1698095808; h=content-transfer-encoding:cc:to:subject:message-id:date:from :references:in-reply-to:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=89M72pPEYHpqokn+eRETfhCXO7VD9CJodXJJzn2X9dc=; b=Sv0H4Jxxon/OHxbhAE4KcifMYhiHaDGZ5/mb+wGXgvFSMR9TYYlIOQx1ZqxlzD9wr1 zv9K2Dhhx3qSRqL9vmoONyLQ8eIw6LSwIvVkp+Ckl172iLLajLCR/QV1JO8GBX3zpfzZ KOxL8Kqdp83wNvI72kprM1cukba2JTjIi1Y2UKpKj5hvC28+WpJPGs7c+6og+6oPs1Y3 mFaA3qDzQ6zS4BcIerfmH2TUyVrTj1M4cF8mBlvt4uf1OpYkkX4DQIRP7WevJG6F3tHy 7t032slsX2sJRZYBe3lZliVPqOW0TBg1j3rXBE8NGXDYX7QPNSJHg+DsMUkd131PYKSs aH8Q== X-Gm-Message-State: AOJu0YwHCPtUPLKAjtFm82AB2b5aiR1p8frTO6P1IzeWPqMEcOosw4dH je+XkS0WIYfOHNsrCrcC2d0l7J/z0L5Zk9bZejE= X-Google-Smtp-Source: AGHT+IFh5KHovoF3BtQM/b4Ft5awq1H2e+ZTjMGgLvKOOAZlvtCNEbmW4qk9/Wnv/8IUs+omfzkh0sh0BLDz7lf80sI= X-Received: by 2002:a4a:df54:0:b0:56e:466c:7393 with SMTP id j20-20020a4adf54000000b0056e466c7393mr266069oou.5.1697491007888; Mon, 16 Oct 2023 14:16:47 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Received: by 2002:ac9:6684:0:b0:4f0:1250:dd51 with HTTP; Mon, 16 Oct 2023 14:16:47 -0700 (PDT) In-Reply-To: <84712139-f7d8-41b5-a122-7ab1b24ffd61@gmail.com> References: <95e9f65b-58c4-ef3a-a5d1-b794179e4252@gmail.com> <23c596e7-d710-c17d-a4e7-b21c99a4fe00@gmail.com> <5528e510-2d79-ebc9-b3f4-79d5fb80f38f@gmail.com> <84712139-f7d8-41b5-a122-7ab1b24ffd61@gmail.com> From: Mateusz Guzik Date: Mon, 16 Oct 2023 23:16:47 +0200 Message-ID: Subject: Re: kernel 100% CPU To: Graham Perrin Cc: FreeBSD CURRENT Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: --- X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; ARC_NA(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::c2c:from]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; DKIM_TRACE(0.00)[gmail.com:+]; TO_DN_ALL(0.00)[]; MID_RHS_MATCH_FROMTLD(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_TO(0.00)[gmail.com]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4S8VKr55ddz4PnG On 10/14/23, Graham Perrin wrote: > On 03/09/2023 20:25, Mateusz Guzik wrote: > >> =E2=80=A6 >> >> Sorry mate, neglected to specify: collect sysctl -a once you run into >> the problem. >> >> Once I look at that I'm probably going to ship some debug patches to >> narrow it down. > > > I had what might be the same issue this afternoon, for the first time in > weeks. With the slowdown, it took a few minutes for me to find your > email, unfortunately the symptoms subsided moments before finding it. > > So, I collected the information, but it's probably another missed > opportunity (the information collected too late). > > The busy period lasted for around six minutes: > > root@mowa219-gjp4-8570p-freebsd:~ # poudriere bulk -j main -J 3 -Ctv > devel/pkgconf > [00:00:00] Creating the reference jail... done > [00:00:01] Mounting system devices for main-default > [00:00:02] Warning: Using packages from previously failed, or > uncommitted, build: > /usr/local/poudriere/data/packages/main-default/.building > [00:00:02] Mounting ccache from: /usr/.ccache > [00:00:02] Mounting ports from: /usr/local/poudriere/ports/default > [00:00:02] Mounting packages from: > /usr/local/poudriere/data/packages/main-default > [00:00:02] Mounting distfiles from: /usr/ports/distfiles > [00:00:02] Copying /var/db/ports from: > /usr/local/etc/poudriere.d/main-options > [00:00:02] Appending to make.conf: /usr/local/etc/poudriere.d/make.conf > /etc/resolv.conf -> > /usr/local/poudriere/data/.m/main-default/ref/etc/resolv.conf > [00:00:02] Starting jail main-default > [00:00:02] Will build as nobody:nobody (65534:65534) > [00:00:05] Logs: > /usr/local/poudriere/data/logs/bulk/main-default/2023-10-14_16h16m30s > [00:00:05] Loading MOVED for > /usr/local/poudriere/data/.m/main-default/ref/usr/ports > [00:00:06] Ports supports: FLAVORS SELECTED_OPTIONS > [00:00:06] Inspecting ports tree for modifications to git checkout... yes > [00:06:39] Ports top-level git hash: e843b8293c (dirty) > =E2=80=A6 > sysctl output you pasted privately shows you are running a kernel form October 9th, fee14577d590. This predates a slew of fixes to vnlru I committed the day after. They fix some of the problems (not all of them!), but it should be good eno= ugh. tl;dr just update to newest main and it should be fine. --=20 Mateusz Guzik