From nobody Sun Sep 17 15:09:13 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RpWYL1FNHz4t2Gd for ; Sun, 17 Sep 2023 15:09:26 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com [IPv6:2a00:1450:4864:20::52b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RpWYL0WZMz4HKw for ; Sun, 17 Sep 2023 15:09:26 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ed1-x52b.google.com with SMTP id 4fb4d7f45d1cf-530ab2d9e89so1962505a12.2 for ; Sun, 17 Sep 2023 08:09:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1694963364; x=1695568164; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=bB7Bzhdc50PXZxTJMoqT7XB6LdMH3F2CQS9gQD4jYhM=; b=AP9TUnDGP5A5GIhvPTVEvox80taIeQZoQUaGA37s267vJTWqNc6HUQQzvE60sazzlL SXUJyKR/11xBFdwaqzdZRH48QZvfnKPaTXbx4xbSi+/pLrEZ0suxrip4N1wn5v1sfXpJ 3D0BybOgAEjfjjo3Zu9B1cX64hONyAsnw5WAzbIXO9OOW4YYWbw0JvtPFydafOXINWyg 8CafsmVQdSOWwxEqmdTkivf7q4h+QaE6cZa0Q5aUBkvwXhFFvJ8+a6oFK0cYIeXTjJak cPLVikS9CyDye2O4I4guB+3qqYAwKkou9mMoy5AUZb7DnCAxtsa9N6VZnpE3Vs2IxTKO sbaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694963364; x=1695568164; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=bB7Bzhdc50PXZxTJMoqT7XB6LdMH3F2CQS9gQD4jYhM=; b=jLty5WrP81/rfZ80icLKi6I9IZ+BaR4isGE6dbBWwywPM9ngo+gfbv17B57UCKiILt fQBF8hvrl+i9zeMrleP3XbK8oh0G/HzXG0qS6jGYCol4mQ7sQTPL892ImpUBg9hW8lSs MwAZ7zbdroDClj4clmW5CC64wZ3D1s6LYk8aGUtqgitx1fM8NNY30zfPGVecgdK0MfPJ Zuc4BeODJACVAsnVBdf1FVXsvFP8CLCfpLH8TDrwFn20/AYrKxfSR5FZBpwRfmown3ab nG82qX4WudUlRGY/YkKzMTRKkVv8rSlu4Fmvqql+GAf7mRM9P7EX01uBpRjbUW+SCmqE BaZg== X-Gm-Message-State: AOJu0YytZo/Ff7IsD1Q/0xwV1EUaXxxR5XlJb3k4OhGzlK6aGEHPPleF CMeeqNW0ENxEMF9Pw99sQpQpyGhv9g7smAorFF80wSqwta4V2wYX+J8= X-Google-Smtp-Source: AGHT+IFi4W7GLC3Cgb03p5GeiUVJ3C8Vzg4b3FTJngQBybmkqWJZW+M2zsT8d7s5H9awAZP8iKmduJ+LXPlom3ik11s= X-Received: by 2002:aa7:d905:0:b0:524:9564:4fee with SMTP id a5-20020aa7d905000000b0052495644feemr6158000edr.10.1694963364425; Sun, 17 Sep 2023 08:09:24 -0700 (PDT) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 References: <20230903123028.4ffceb705824f86d2efc21e3@dec.sakura.ne.jp> <20230903133328.54577b85b097da319ecde4ba@dec.sakura.ne.jp> <20230917093950.0dbe3eefb1c34d61dea8adef@dec.sakura.ne.jp> In-Reply-To: <20230917093950.0dbe3eefb1c34d61dea8adef@dec.sakura.ne.jp> From: Warner Losh Date: Sun, 17 Sep 2023 16:09:13 +0100 Message-ID: Subject: Re: Is there any plan for ZFS and timerfd updates on stable/14? To: Tomoaki AOKI Cc: Jake Freeland , FreeBSD-STABLE Mailing List Content-Type: multipart/alternative; boundary="0000000000003e854306058f6919" X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Queue-Id: 4RpWYL0WZMz4HKw --0000000000003e854306058f6919 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hello AOKI-san Thanks for keeping on top of this issue. I had some, but not all of these, staged in my testing tree. Testing worked on it a while ago. I've gone ahead and pushed that branch just now. I've staged the rest of these and will push them after some light testing. My usual workflow is disrupted by travel to EuroBSDcon 2023 and having too much fun here seeing old friends. Please let me know if there's anything else. There is one test regression that needs to be sorted out to get back to where we were before my push. I hope to have that done by the end of next week since I'm taking a few days off after the conference to see a little of Portugal. Warner On Sun, Sep 17, 2023 at 1:39=E2=80=AFAM Tomoaki AOKI wrote: > On Sun, 3 Sep 2023 13:33:28 +0900 > Tomoaki AOKI wrote: > > > On Sat, 2 Sep 2023 22:47:53 -0500 > > Jake Freeland wrote: > > > > > On Sat, Sep 2, 2023 at 10:40=E2=80=AFPM Warner Losh = wrote: > > > > > > > > > > > > > > > On Sat, Sep 2, 2023, 9:36 PM Jake Freeland < > jake@technologyfriends.net> > > > > wrote: > > > > > > > >> On Sat, Sep 2, 2023 at 10:31=E2=80=AFPM Tomoaki AOKI < > junchoon@dec.sakura.ne.jp> > > > >> wrote: > > > >> > > > >>> Hi. > > > >>> > > > >>> There are discussions about deadlocks issue of ZFS on > freebsd-current > > > >>> ML, starting from [1] last month. > > > >>> IIRC, at least some fixes (candidates?) are merged to main, but > not yet > > > >>> to stable/14. > > > >>> > > > >>> Upcoming (aleready released? or still rc3?) OpenZFS 2.2-release > seems > > > >>> to have most of them. So my 1st question is "Is there any plan to > > > >>> import vendor/openzfs/zfs-2.2-release into stable/14 BEFORE > BRANCHING > > > >>> releng/14? > > > >>> > > > >>> And one more. timerfd is added at last-minutes BEFORE stable/14 i= s > > > >>> branched, and already have not-yet-MFC'ed fixes [2], [3], [4] and > > > >>> Differential revision D41600 on Phablicator [5] related to memory > leaks > > > >>> and locks. > > > >>> Additionally, splitting out lib32 part to proper place is propose= d > > > >>> as D41640 [6]. Both [5] and [6] are accepted but not yet landed. > > > >>> Also, D41641 [7] proposes namespace pollution adjustments. This > can be > > > >>> optional? > > > >>> > > > >>> Memory leaks and improper locks can lead system to security issue= s > or > > > >>> deadlocks, so it would be benefical if landed and MFC'ed BEFORE > > > >>> releng/14 branches. > > > >>> > > > >>> Is there any plan to do so? At least, existing deadlocks should b= e > > > >>> considered as SHOW-STOPPER and resolved. > > > >>> > > > >> > > > >> The plan is to get all of those patches in before releng/14.0, I > believe. > > > >> > > > >> What are your thoughts, Warner? > > > >> > > > > > > > > Sounds like the reviews are done or nearly so. I've not had time to > look > > > > closely to be sure... I'd planned on making time Tuesday morning. > > > > > > > > > > Yes. All reviews are good to go. > > > > > > Jake Freeland > > > > Glad to know. Thanks! > > Looking forward to see them landed / MFC'ed before releng/14 branches. > > > > Regards. > > > > > > > > > > > > > > > > Warner > > > > > > > > > > > >> Thanks, > > > >> Jake Freeland > > > >> > > > >> > > > >>> > > > >>> I myself am bitten by several deadlocks on poudriere full builds > after > > > >>> upgrading base from stable/13 to stable/14, finally finished with > > > >>> increasing kern.maxvnodes after powercycle on each deadlock and > > > >>> continue. > > > >>> > > > >>> > > > >>> Thanks in advance! > > > >>> > > > >>> [1] > > > >>> > > > >>> > https://lists.freebsd.org/archives/freebsd-current/2023-August/004162.htm= l > > > >>> > > > >>> [2] > > > >>> > > > >>> > https://cgit.freebsd.org/src/commit/?id=3D02f534b57f84d6f4f97c337b05b383c= 8b3aaf18c > > > >>> > > > >>> [3] > > > >>> > > > >>> > https://cgit.freebsd.org/src/commit/?id=3D5eab523053db79b4bd4f926c7d7ac04= 444d9c1da > > > >>> > > > >>> [4] > > > >>> > > > >>> > https://cgit.freebsd.org/src/commit/?id=3Df4296cfb409a48de00bfa60e76f686c= 2b031876f > > > >>> > > > >>> [5] https://reviews.freebsd.org/D41600 > > > >>> > > > >>> [6] https://reviews.freebsd.org/D41640 > > > >>> > > > >>> [7] https://reviews.freebsd.org/D41641 > > > >>> > > > >>> -- > > > >>> Tomoaki AOKI > > > > -- > > Tomoaki AOKI > > Hi. Thanks for your hard work on it. > > I could confirm commits to main as below, but they are not yet > MFC'ed/MFS'/ed. > It would be nice these commits to be incorporated at worst on first RC. > Any plans for MFC and following MFS? > > commit 02f534b57f84d6f4f97c337b05b383c8b3aaf18c > timerfd: fix up a memory leak and missing locking > > commit 5eab523053db79b4bd4f926c7d7ac04444d9c1da > timerfd: compute fflags before calling falloc > > commit f4296cfb409a48de00bfa60e76f686c2b031876f > timerfd: convert timerfd_list_lock from sx to mtx > > commit a1f506156c4db885d3cc177c93e9c8a28d535d30 > timerfd: Define a locking regime > > commit 918966a27479b4fb7c4c8999c4926d83c2c081e5 > timerfd: Relocate 32-bit compat code > > commit fb5daae920bae84e3eec8175bf9e46304c3b2ae6 > timerfd: Namespace pollution adjustments > > > Note that first 3 commits are authord/committed by mjg@. > These are all I could confirm landed with commit histories of > sys/kern/sys_timerfd.c and sys/sys/timerfd.c, excluding > already-in-stable/14 one. > > I've found a related commit to sys/compat/linux/linux_event.c but > intentionally excluded it, as it's already MFC'ed and MFS'ed. > > And I found another request on dev-commits-src-main ML archive [1], > without any reply. > > Attached is the hand-merged patch to cherry pick them to stable/14, > created before last 3 commits landed on main. HTH. > > [1] > > https://lists.freebsd.org/archives/dev-commits-src-main/2023-September/01= 8407.html > > Regards. > > -- > Tomoaki AOKI > --0000000000003e854306058f6919 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hello AOKI-san

Thanks for keeping on to= p of this issue. I had some, but not all of these, staged in my testing tre= e.
Testing worked on it a while ago. I've gone ahead and push= ed that branch just now. I've staged the
rest of these and wi= ll push them after some light testing. My usual workflow is disrupted by tr= avel to
EuroBSDcon 2023 and having too much fun here seeing old f= riends.

Please let me know if there's anything= else. There is one test regression that needs to be sorted out
t= o get back to where we were before my push. I hope to have that done by the= end of next week since
I'm taking a few days off after the c= onference to see a little of Portugal.

Warner

On Sun, Sep 17, 2023 at 1:39=E2=80=AFAM Tomoaki AOKI <junchoon@dec.sakura.ne.jp> wrote:
On Sun, 3 Sep 2023 13:= 33:28 +0900
Tomoaki AOKI <junchoon@dec.sakura.ne.jp> wrote:

> On Sat, 2 Sep 2023 22:47:53 -0500
> Jake Freeland <jake@technologyfriends.net> wrote:
>
> > On Sat, Sep 2, 2023 at 10:40=E2=80=AFPM Warner Losh <imp@bsdimp.com> wrote:<= br> > >
> > >
> > >
> > > On Sat, Sep 2, 2023, 9:36 PM Jake Freeland <jake@technologyfriends.n= et>
> > > wrote:
> > >
> > >> On Sat, Sep 2, 2023 at 10:31=E2=80=AFPM Tomoaki AOKI <= ;junchoon@de= c.sakura.ne.jp>
> > >> wrote:
> > >>
> > >>> Hi.
> > >>>
> > >>> There are discussions about deadlocks issue of ZFS o= n freebsd-current
> > >>> ML, starting from [1] last month.
> > >>> IIRC, at least some fixes (candidates?) are merged t= o main, but not yet
> > >>> to stable/14.
> > >>>
> > >>> Upcoming (aleready released? or still rc3?) OpenZFS = 2.2-release seems
> > >>> to have most of them. So my 1st question is "Is= there any plan to
> > >>> import vendor/openzfs/zfs-2.2-release into stable/14= BEFORE BRANCHING
> > >>> releng/14?
> > >>>
> > >>> And one more. timerfd is added at last-minutes BEFOR= E stable/14 is
> > >>> branched, and already have not-yet-MFC'ed fixes = [2], [3], [4] and
> > >>> Differential revision D41600 on Phablicator [5] rela= ted to memory leaks
> > >>> and locks.
> > >>> Additionally, splitting out lib32 part to proper pla= ce is proposed
> > >>> as D41640 [6].=C2=A0 Both [5] and [6] are accepted b= ut not yet landed.
> > >>> Also, D41641 [7] proposes namespace pollution adjust= ments. This can be
> > >>> optional?
> > >>>
> > >>> Memory leaks and improper locks can lead system to s= ecurity issues or
> > >>> deadlocks, so it would be benefical if landed and MF= C'ed BEFORE
> > >>> releng/14 branches.
> > >>>
> > >>> Is there any plan to do so? At least, existing deadl= ocks should be
> > >>> considered as SHOW-STOPPER and resolved.
> > >>>
> > >>
> > >> The plan is to get all of those patches in before releng= /14.0, I believe.
> > >>
> > >> What are your thoughts, Warner?
> > >>
> > >
> > > Sounds like the reviews are done or nearly so. I've not = had time to look
> > > closely to be sure... I'd planned on making time Tuesday= morning.
> > >
> >
> > Yes. All reviews are good to go.
> >
> > Jake Freeland
>
> Glad to know. Thanks!
> Looking forward to see them landed / MFC'ed before releng/14 branc= hes.
>
> Regards.
>
> >
> >
> > >
> > > Warner
> > >
> > >
> > >> Thanks,
> > >> Jake Freeland
> > >>
> > >>
> > >>>
> > >>> I myself am bitten by several deadlocks on poudriere= full builds after
> > >>> upgrading base from stable/13 to stable/14, finally = finished with
> > >>> increasing kern.maxvnodes after powercycle on each d= eadlock and
> > >>> continue.
> > >>>
> > >>>
> > >>> Thanks in advance!
> > >>>
> > >>> [1]
> > >>>
> > >>> htt= ps://lists.freebsd.org/archives/freebsd-current/2023-August/004162.html=
> > >>>
> > >>> [2]
> > >>>
> > >>> https://cgit.freebsd.org/src/commit/?id=3D02f534b57f84d6f4f97c337b05= b383c8b3aaf18c
> > >>>
> > >>> [3]
> > >>>
> > >>> https://cgit.freebsd.org/src/commit/?id=3D5eab523053db79b4bd4f926c7d= 7ac04444d9c1da
> > >>>
> > >>> [4]
> > >>>
> > >>> https://cgit.freebsd.org/src/commit/?id=3Df4296cfb409a48de00bfa60e76= f686c2b031876f
> > >>>
> > >>> [5] https://reviews.freebsd.org/D41600<= br> > > >>>
> > >>> [6] https://reviews.freebsd.org/D41640<= br> > > >>>
> > >>> [7] https://reviews.freebsd.org/D41641<= br> > > >>>
> > >>> --
> > >>> Tomoaki AOKI=C2=A0 =C2=A0 <junchoon@dec.sakura.ne.jp><= br> >
> --
> Tomoaki AOKI=C2=A0 =C2=A0 <junchoon@dec.sakura.ne.jp>

Hi. Thanks for your hard work on it.

I could confirm commits to main as below, but they are not yet
MFC'ed/MFS'/ed.
It would be nice these commits to be incorporated at worst on first RC.
Any plans for MFC and following MFS?

commit=C2=A0 02f534b57f84d6f4f97c337b05b383c8b3aaf18c
=C2=A0 timerfd: fix up a memory leak and missing locking

commit=C2=A0 5eab523053db79b4bd4f926c7d7ac04444d9c1da
=C2=A0 timerfd: compute fflags before calling falloc

commit=C2=A0 f4296cfb409a48de00bfa60e76f686c2b031876f
=C2=A0 timerfd: convert timerfd_list_lock from sx to mtx

commit=C2=A0 a1f506156c4db885d3cc177c93e9c8a28d535d30
=C2=A0 timerfd: Define a locking regime

commit=C2=A0 918966a27479b4fb7c4c8999c4926d83c2c081e5
=C2=A0 timerfd: Relocate 32-bit compat code

commit=C2=A0 fb5daae920bae84e3eec8175bf9e46304c3b2ae6
=C2=A0 timerfd: Namespace pollution adjustments


Note that first 3 commits are authord/committed by mjg@.
These are all I could confirm landed with commit histories of
sys/kern/sys_timerfd.c and sys/sys/timerfd.c, excluding
already-in-stable/14 one.

I've found a related commit to sys/compat/linux/linux_event.c but
intentionally excluded it, as it's already MFC'ed and MFS'ed.
And I found another request on dev-commits-src-main ML archive [1],
without any reply.

Attached is the hand-merged patch to cherry pick them to stable/14,
created before last 3 commits landed on main. HTH.

[1]
https://lists.free= bsd.org/archives/dev-commits-src-main/2023-September/018407.html

Regards.

--
Tomoaki AOKI=C2=A0 =C2=A0 <junchoon@dec.sakura.ne.jp>
--0000000000003e854306058f6919--