Re: git: fb5f03a87cf4 - main - Mk/bsd.lto.mk: add global LTO support for ports

On 21-10-05 22:48:14, Matthias Andree wrote:
> 
> Am 05.10.21 um 22:41 schrieb Piotr Kubaj:
> > On 21-10-05 22:04:32, Matthias Andree wrote:
> >>
> >> Am 05.10.21 um 18:49 schrieb Piotr Kubaj:
> >>> On 21-10-05 18:31:52, Mathieu Arnold wrote:
> >>>> On Tue, Oct 05, 2021 at 06:27:15PM +0200, Piotr Kubaj wrote:
> >>>>> On 21-10-04 15:30:56, Mathieu Arnold wrote:
> >>>>>> On Thu, Sep 30, 2021 at 06:34:20PM +0000, Piotr Kubaj wrote:
> >>>>>>> The branch main has been updated by pkubaj:
> >>>>>>>
> >>>>>>> URL: https://cgit.FreeBSD.org/ports/commit/?id=fb5f03a87cf432751fae1f0ae7f29c9d4fc65917
> >>>>>>>
> >>>>>>> commit fb5f03a87cf432751fae1f0ae7f29c9d4fc65917
> >>>>>>> Author:     Piotr Kubaj <pkubaj@FreeBSD.org>
> >>>>>>> AuthorDate: 2021-09-30 18:27:50 +0000
> >>>>>>> Commit:     Piotr Kubaj <pkubaj@FreeBSD.org>
> >>>>>>> CommitDate: 2021-09-30 18:27:50 +0000
> >>>>>>>
> >>>>>>>     Mk/bsd.lto.mk: add global LTO support for ports
> >>>>>>>     
> >>>>>>>     It's well known that LTO provides both performance and size benefits for
> >>>>>>>     binaries.
> >>>>>>>     
> >>>>>>>     Add preliminary, opt-in support for global LTO enforcement to ports. Ports that
> >>>>>>>     provide LTO option on their own and the ones that don't work with LTO will need
> >>>>>>>     to set LTO_UNSAFE in the future.
> >>>>>>>     
> >>>>>>>     PR:     258536
> >>>>>>
> >>>>>> Not to be picky about approval and all, but this was added to the
> >>>>>> framework, and the framework is maintained by portmgr.  When you want to
> >>>>>> add something to it, you must consult with portmgr before anything gets
> >>>>>> committed.
> >>>>>>
> >>>>>> In that case, we would have told you not to do it this way, but to make
> >>>>>> this a Mk/Uses/lto.mk.
> >>>>>>
> >>>>>> So please, turn this into a USES=lto.
> >>>>>
> >>>>> I did consult, but no one replied.
> >>>>
> >>>> There is absolutely no maintainer timeout for the framework, you cannot
> >>>> just add code there without explicit approval.
> >>>
> >>> And this is a port of a bigger issue, where portmgr ignores emails until numerously asked for (if one is lucky).
> >>> As one of users wrote in https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=251117, for which portmgr is assigned, "portmgr@ more and more feels to me like a black hole, or /dev/null: Anything sent there seems to disappear without effect."
> >>>
> >>> Since it was a change that doesn't change anything out-of-the-box, I decided to commit it.
> >>>
> >>>>
> >>>>> IMO adding it to USES is not a good idea, since USES are supposed to be used per port and my idea was to force LTO for all ports, same way that SSP already does.
> >>>>
> >>>> All I see in the patch is a USE_LTO knob, and a LTO_UNSAFE one, without
> >>>> any documentation of what it is for, what it does, what it might do,
> >>>> what it is about, or anything else.
> >>> Neither has SSP, I don't see any documentation for it (including commiter handbook which has just one line regarding USES=kmod at https://docs.freebsd.org/en/books/porters-handbook/book/#uses-kmod).
> >>
> >> Piotr,
> >>
> >> While I sympathize with your findings about portmgr@ 'responsiveness'
> >> having had my shares of ignores and brushes, I would tend to agree that
> >> we should not add undocumented knobs anywhere, so:
> >>
> >> 1. please add documentation including motivation
> > OK, just take note that there are several issues with it right now, so it's definitely not ready for use. E.g. libffi, perl and pkgconf don't build.
> > 
> > Other than that, what is the best place for documentation? Do you mean porter's handbook?
> 
> Assuming we're going the bsd.lto.mk path, the comment banner at the top
> of this new file would be the most obvious place.  Purpose, how-to,
> reference.  Anything else might be a proposal in whatever format.  Talk
> to the doc committers or doceng@ maybe?
> 
> > That's what we have package builders for. AFAIK they run Poudriere without ALLOW_MAKE_JOBS, so mostly single-threaded.
> > If LTO is ever enabled for everyone, we should still keep a knob to disable it globally.
> 
> Yes, but my personal poudriere builders are on disk-space constrained
> VMs so get maximum parallelity within one job, and run few jobs in
> parallel.  It's not ideal, but I can't build LLVM, Rust, and JDK or
> texmf in parallel with its truckloads-of-GBytes build directories.
> 
> Before committing intrusive changes, we normally do -exp runs.
> Personally, for OpenEXR which is one of the more "central" ports I have,
> I dare building all ports depending on it locally before committing, if
> I couldn't do that I'd have to go for an -exp run.
I myself have enough RAM for using LTO even with large software. If your hardware can't allow that, there were free Azure VMs that AFAIK were quite beefy.


> 
> HTH