From nobody Fri Dec 31 16:40:12 2021 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 57D4C19110BF for ; Fri, 31 Dec 2021 16:40:24 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-vk1-xa2a.google.com (mail-vk1-xa2a.google.com [IPv6:2607:f8b0:4864:20::a2a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JQW8m19H5z3DFJ for ; Fri, 31 Dec 2021 16:40:24 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-vk1-xa2a.google.com with SMTP id u1so15135595vkn.10 for ; Fri, 31 Dec 2021 08:40:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20210112.gappssmtp.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=LH07DO+K4QtrwU6appq56VxiE+cmjfpV2dyt+rsKdfY=; b=tZgeIm5GVM6REqw8t7RRUGkolbPkwG0woGBgNG4A/mMCm73weVrYUOVNsDqltrvpCM 6fkKL5ZoaI199oDKKkhEfg5uYcsh+75cr/jXe8Z2hnxCgzye7RI8vY+N0PGH4m2BsRI/ F/5kulexsoP3gxFZcOMemvUd62+t01PrOa51VmzrpH7uYBRtHvfLtrrfp1kAuKnoVlWj cDuFtt5zgMGb3bziK1PZkbHBYx+13b1zXxbSG5wc4BHe19O+mWDsCvuvoR5xxFcAJH5o 9TWEUYszZs5Ql7WaLOgQ9ZtJH0iAgc4w5skIq0NSDvT18o5i26vYLxR3gYUOTnqFlbVh S58Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=LH07DO+K4QtrwU6appq56VxiE+cmjfpV2dyt+rsKdfY=; b=V5/slO3jf6FFlAausl5+228yliUaevtIHJ584L7vFW3OUUBxI4NnXRbWptWagUTAdZ 2zpSAFcs7Czew5RJTZrowcQehIbtji5bm3XI09rfUXs9q+63rzGqL7tnmyr1SE/FjvNv RcxgW5fEvuWJ8V740QvF8pS9vAEnqyD/5OrgtYFulPLBJ8sKj7XdJFDixHw1RqDwUZLx eeWl0kU/9KkADYq5tgMYp3osFpiP0CFuTRKDUl0nXgfEeijyR4Yg8bZfGCnLcuTTENDm zHSAae78rFSULYrhFLT9dWn/EnR+1YwHiTWr01PvYVdnyXI4BwlV2iDw1c1dK6QV0AvA L+RA== X-Gm-Message-State: AOAM532B1Rownfe24SHU9nqiFy8cZuPI/YdHptbmMn4kGNdPG2PnJx8H oJzZRkGtt6NvUAo/PAmwXn2Rkmshz8ve6Zi+fg+I4g== X-Google-Smtp-Source: ABdhPJzpvPeh0XKX35CY96NufuAsxb1SGYypLtoYF1U2QEw7hTQ5Nsg/VQsGptWAkHL7oWNNKdlE8O1au2IZyMbOVzI= X-Received: by 2002:a1f:45d4:: with SMTP id s203mr11157538vka.5.1640968823487; Fri, 31 Dec 2021 08:40:23 -0800 (PST) List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 References: <202112301154.1BUBsR1q017491@gitrepo.freebsd.org> In-Reply-To: From: Warner Losh Date: Fri, 31 Dec 2021 09:40:12 -0700 Message-ID: Subject: Re: git: e2650af157bc - main - Make CPU_SET macros compliant with other implementations To: Kyle Evans Cc: Konstantin Belousov , Stefan Esser , Antoine Brodin , src-committers , "" , dev-commits-src-main@freebsd.org, FreeBSD Ports Management Team Content-Type: multipart/alternative; boundary="000000000000cfd2d605d473d36d" X-Rspamd-Queue-Id: 4JQW8m19H5z3DFJ X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N --000000000000cfd2d605d473d36d Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Fri, Dec 31, 2021 at 9:31 AM Kyle Evans wrote: > On Fri, Dec 31, 2021 at 10:19 AM Konstantin Belousov > wrote: > > > > On Fri, Dec 31, 2021 at 09:37:16AM -0600, Kyle Evans wrote: > > > On Fri, Dec 31, 2021 at 4:22 AM Stefan Esser wrote: > > > > > > > > Am 31.12.21 um 09:01 schrieb Antoine Brodin: > > > > > On Thu, Dec 30, 2021 at 11:54 AM Stefan E=C3=9Fer > wrote: > > > > >> > > > > >> The branch main has been updated by se: > > > > >> > > > > >> URL: > https://cgit.FreeBSD.org/src/commit/?id=3De2650af157bc7489deaf2c9054995f0= f88a6e5da > > > > >> > > > > >> commit e2650af157bc7489deaf2c9054995f0f88a6e5da > > > > >> Author: Stefan E=C3=9Fer > > > > >> AuthorDate: 2021-12-30 11:20:32 +0000 > > > > >> Commit: Stefan E=C3=9Fer > > > > >> CommitDate: 2021-12-30 11:20:32 +0000 > > > > >> > > > > [...] > > > > >> Ports that have added -D_WITH_CPU_SET_T to build on -CURRENT > do > > > > >> no longer require that option. > > > > >> > > > > >> The FreeBSD version has been bumped to 1400046 to reflect th= is > > > > >> incompatible change. > > > > >> > > > > >> Reviewed by: kib > > > > >> MFC after: 2 weeks > > > > >> Relnotes: yes > > > > >> Differential Revision: https://reviews.freebsd.org/D33451 > > > > > > > > > > Hi, > > > > > > > > > > This breaks a lot of ports, like lang/python38. > > > > > Could these kinds of changes on public headers be tested with an > > > > > exp-run, and reverted in the mean-time? > > > > > > > > I'm sorry for the breakage. The commit had the goal to lessen > > > > port build problems caused by the misled assumptions that the > > > > port was being built on a GLIBC based system. > > > > > > > > > > Given that we've now iterated on this a couple of times, this likely > > > should have all been backed out and exp-run'd *way* sooner. > > Exp-runs are great when there is a path forward from fixing the breakag= e. > > In the case where you have some random ports broken, and no ports > maintainers > > responses for explicit queries, it is basically a deadlock. > > > > I definitely cannot go over some random but large set of ports fixing > them, > > while maintainers are silent. > > > > We do not know in advance that maintainers are silent, because we do > not know in advance which ports are even affected. We can't outright > claim there's a problem without knowing the scope. Nevertheless, yes, > you're right- this is even a conversation we've had recently re: > toolchain breakage, and I don't recall what the outcome of that > conversation was. > > > In fact, with this set of changes, I initially provided some tools in > base > > that were intended to ease the ports life, but still required some > (minimal) > > involvement from the maintainers, like passing -DWITH_CPU_SET_T to C/C+= + > > compiler. I asked more than once if these tools are desirable helper o= r > > not, with no avail. > > > > And that was indeed helpful, thanks! > > > So my only route forward was to leave the state in the minimal damaging > > mode as I see it from bug reports, and wait for maintainers to do > > _something_. I am very grateful that Stefan took the torch and started > > massaging the CPU_XXX ugliness into more compatibility with glibc. This > > again happens in the same silence mode from maintainers, so if we want = a > > progress in this area, it have to go this way. > > > > None of this is really applicable to this specific commit, though. The > breakage identified this time was that a couple more definitions were > expected, which does lean towards iteration on the patch rather than > yet requiring the maintainer or ports committer aide. > > I suspect the answer for other scenarios is that we run the exp-run > and shoot out a solicitation for help to -ports@ to cast a broader > net. Ports committers already get stuck with the fallout from this > stuff when maintainers don't notice or step up to the plate or even > when the breakage is just bad enough. I imagine you'd have no problem > catching some folks willing to help be proactive rather than reactive. > > > > > > > > In the case of the Python language ports, one additional macro > > > > was required and has been added in commit cb65d4432aed11. > > > > > > > > Since the official package builders have not been upgraded to > > > > a -CURRENT with this change, they are not affected. But I'll > > > > watch the failed build logs on beefy18. > > Right. > > > > > > > > This is a mindset that we all take, but we really need to work toward= s > > > improving. Once we're watching fallout logs on the official builders, > > > we've already lost. This is the kind of thing that helps promote the > > > idea that -CURRENT isn't stable enough for production uses: we start > > > accepting that we can be a little more lenient on identifying > > > ports-breaking changes because it's -CURRENT and we lose a fraction o= f > > > the ports tree because we've only sniped off individual ports as they > > > come up. > > > > > > portmgr@ is able and willing to run exp-runs for changes like this, w= e > > > really need to take advantage of that to avoid this kind of follow-up= . > > There, you are blocking src changes by putting unreasonable requirement= s > > on src committers to fix ports breakage. I am willing to work together > > with ports maintainers, but I am not willing to handle things in silenc= e > > and neglect of other' (my) work. > > > > > > I have similar experience with ino64 FWIW, but I was too naive at that > time > > and indeed tried to fix all ports breakage, including digging into > rust/ghc > > builds. I learned since, I will not do that again. > > > > I don't want src committers to fix all of the breakage, I want them to > at least do the bare minimum to identify when a change is going to > break a lot of stuff and figure out the magnitude of that breakage. > Sometimes we have to work with ports people on it, sometimes we just > need to iterate on our own changes further. We can't punt on the > latter because we haven't perfected the former yet. > I have a bunch of endian.h changes. I've been iterating for months, though most of that is my fault. After the initial exp-run, I've been able to use poudriere to identify whether or not my fixes actually fix things, or if it's time to try to get things in upstream. I'm down to 4 ports I need to fix and upstream because they do stupid things that accidentally worked in my case. When I'm ready, I've found portmgr quite responsive to my requests and in general the ports maintainers I've contacted for this and other changes responsive enough for my needs. I have the changes in a branch for when I can get back to them, and a list of currently broken ports I can run poudriere on when I have time to pick it back up. Even though there's only a few that were broken by my last iteration, they are important enough that I'm holding off pushing those int= o the tree... I strongly agree with Kyle that we need to move away from the mindset of futility. If there's maintainers that are unresponsive, we have timeouts. If we need a lot of changes to ports, portmgr can grant blanket approval. There's no urgency to get these changes in, to be honest. Breaking python and who knows what else really isn't acceptable on the basis that people had a bad experience in the past. That's how things stay dysfunctional and how we never get any better. Especially for a change that's motivation is to make us more compatible and for there to be less work as a result... Warner --000000000000cfd2d605d473d36d Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Fri, Dec 31, 2021 at 9:31 AM Kyle = Evans <kevans@freebsd.org> = wrote:
On Fri, D= ec 31, 2021 at 10:19 AM Konstantin Belousov
<kostikbel@gmai= l.com> wrote:
>
> On Fri, Dec 31, 2021 at 09:37:16AM -0600, Kyle Evans wrote:
> > On Fri, Dec 31, 2021 at 4:22 AM Stefan Esser <se@freebsd.org> wrote:
> > >
> > > Am 31.12.21 um 09:01 schrieb Antoine Brodin:
> > > > On Thu, Dec 30, 2021 at 11:54 AM Stefan E=C3=9Fer <<= a href=3D"mailto:se@freebsd.org" target=3D"_blank">se@freebsd.org> w= rote:
> > > >>
> > > >> The branch main has been updated by se:
> > > >>
> > > >> URL: https://cgit.FreeBSD.org/src/commit/?id=3De2650af157bc7489deaf2= c9054995f0f88a6e5da
> > > >>
> > > >> commit e2650af157bc7489deaf2c9054995f0f88a6e5da
> > > >> Author:=C2=A0 =C2=A0 =C2=A0Stefan E=C3=9Fer <se@= FreeBSD.org>
> > > >> AuthorDate: 2021-12-30 11:20:32 +0000
> > > >> Commit:=C2=A0 =C2=A0 =C2=A0Stefan E=C3=9Fer <se@= FreeBSD.org>
> > > >> CommitDate: 2021-12-30 11:20:32 +0000
> > > >>
> > > [...]
> > > >>=C2=A0 =C2=A0 =C2=A0Ports that have added -D_WITH_CP= U_SET_T to build on -CURRENT do
> > > >>=C2=A0 =C2=A0 =C2=A0no longer require that option. > > > >>
> > > >>=C2=A0 =C2=A0 =C2=A0The FreeBSD version has been bum= ped to 1400046 to reflect this
> > > >>=C2=A0 =C2=A0 =C2=A0incompatible change.
> > > >>
> > > >>=C2=A0 =C2=A0 =C2=A0Reviewed by:=C2=A0 =C2=A0 kib > > > >>=C2=A0 =C2=A0 =C2=A0MFC after:=C2=A0 =C2=A0 =C2=A0 2= weeks
> > > >>=C2=A0 =C2=A0 =C2=A0Relnotes:=C2=A0 =C2=A0 =C2=A0 = =C2=A0yes
> > > >>=C2=A0 =C2=A0 =C2=A0Differential Revision:=C2=A0 https://reviews.freebsd.org/D33451
> > > >
> > > > Hi,
> > > >
> > > > This breaks a lot of ports,=C2=A0 like=C2=A0 lang/pytho= n38.
> > > > Could these kinds of changes on public headers be teste= d with an
> > > > exp-run,=C2=A0 and reverted in the mean-time?
> > >
> > > I'm sorry for the breakage. The commit had the goal to l= essen
> > > port build problems caused by the misled assumptions that th= e
> > > port was being built on a GLIBC based system.
> > >
> >
> > Given that we've now iterated on this a couple of times, this= likely
> > should have all been backed out and exp-run'd *way* sooner. > Exp-runs are great when there is a path forward from fixing the breaka= ge.
> In the case where you have some random ports broken, and no ports main= tainers
> responses for explicit queries, it is basically a deadlock.
>
> I definitely cannot go over some random but large set of ports fixing = them,
> while maintainers are silent.
>

We do not know in advance that maintainers are silent, because we do
not know in advance which ports are even affected. We can't outright claim there's a problem without knowing the scope. Nevertheless, yes, you're right- this is even a conversation we've had recently re: toolchain breakage, and I don't recall what the outcome of that
conversation was.

> In fact, with this set of changes, I initially provided some tools in = base
> that were intended to ease the ports life, but still required some (mi= nimal)
> involvement from the maintainers, like passing -DWITH_CPU_SET_T to C/C= ++
> compiler.=C2=A0 I asked more than once if these tools are desirable he= lper or
> not, with no avail.
>

And that was indeed helpful, thanks!

> So my only route forward was to leave the state in the minimal damagin= g
> mode as I see it from bug reports, and wait for maintainers to do
> _something_. I am very grateful that Stefan took the torch and started=
> massaging the CPU_XXX ugliness into more compatibility with glibc. Thi= s
> again happens in the same silence mode from maintainers, so if we want= a
> progress in this area, it have to go this way.
>

None of this is really applicable to this specific commit, though. The
breakage identified this time was that a couple more definitions were
expected, which does lean towards iteration on the patch rather than
yet requiring the maintainer or ports committer aide.

I suspect the answer for other scenarios is that we run the exp-run
and shoot out a solicitation for help to -ports@ to cast a broader
net. Ports committers already get stuck with the fallout from this
stuff when maintainers don't notice or step up to the plate or even
when the breakage is just bad enough. I imagine you'd have no problem catching some folks willing to help be proactive rather than reactive.

> >
> > > In the case of the Python language ports, one additional mac= ro
> > > was required and has been added in commit cb65d4432aed11. > > >
> > > Since the official package builders have not been upgraded t= o
> > > a -CURRENT with this change, they are not affected. But I= 9;ll
> > > watch the failed build logs on beefy18.
> Right.
>
> >
> > This is a mindset that we all take, but we really need to work to= wards
> > improving. Once we're watching fallout logs on the official b= uilders,
> > we've already lost. This is the kind of thing that helps prom= ote the
> > idea that -CURRENT isn't stable enough for production uses: w= e start
> > accepting that we can be a little more lenient on identifying
> > ports-breaking changes because it's -CURRENT and we lose a fr= action of
> > the ports tree because we've only sniped off individual ports= as they
> > come up.
> >
> > portmgr@ is able and willing to run exp-runs for changes like thi= s, we
> > really need to take advantage of that to avoid this kind of follo= w-up.
> There, you are blocking src changes by putting unreasonable requiremen= ts
> on src committers to fix ports breakage.=C2=A0 I am willing to work to= gether
> with ports maintainers, but I am not willing to handle things in silen= ce
> and neglect of other' (my) work.
>
>
> I have similar experience with ino64 FWIW, but I was too naive at that= time
> and indeed tried to fix all ports breakage, including digging into rus= t/ghc
> builds.=C2=A0 I learned since, I will not do that again.
>

I don't want src committers to fix all of the breakage, I want them to<= br> at least do the bare minimum to identify when a change is going to
break a lot of stuff and figure out the magnitude of that breakage.
Sometimes we have to work with ports people on it, sometimes we just
need to iterate on our own changes further. We can't punt on the
latter because we haven't perfected the former yet.

I have a bunch of endian.h changes. I've been iteratin= g for months, though
most of that is my fault. After the initial = exp-run, I've been able to use poudriere
to identify whether = or not my fixes actually fix things, or if it's time to try to
get things in upstream. I'm down to 4 ports I need to fix and upstrea= m because
they do stupid things that accidentally worked in my ca= se. When I'm ready,
I've found portmgr quite responsive t= o my requests and in general the ports
maintainers I've conta= cted for this and other changes responsive enough
for my needs. I= have the changes in a branch for when I can get back to them,
an= d a list of currently broken ports I can run poudriere on when I have time = to
pick it back up. Even though there's only a few that were = broken by my last
iteration, they are important enough that I'= ;m holding off pushing those into
the tree...

I strongly agree with Kyle that we need to move away from the mindset= of
futility. If there's maintainers that are unresponsive, w= e have timeouts. If we
need a lot of changes to ports, portmgr ca= n grant blanket approval. There's
no urgency to get these cha= nges in, to be honest. Breaking python and who
knows what else re= ally isn't acceptable on the basis that people had a bad
expe= rience in the past. That's how things stay dysfunctional and how we
never get any better. Especially for a change that's motivation = is to make us
more compatible and for there to be less work as a = result...

Warner

--000000000000cfd2d605d473d36d--