FYI: what it takes for RAM+swap to build devel/llvm40 with 4 processors or cores and WITH__DEBUG= (powerpc64 example)

[Mark Millard]

On 2017-Mar-30, at 1:22 PM, Mark Millard <markmi at dsl-only.net> wrote:

> On 2017-Mar-29, at 8:53 AM, Brooks Davis <brooks at freebsd.org> wrote:
> 
>> On Mon, Mar 27, 2017 at 03:25:04AM -0700, Mark Millard wrote:
>>> On 2017-Mar-27, at 2:41 AM, Dimitry Andric <dim at FreeBSD.org> wrote:
>>> 
>>>> On 26 Mar 2017, at 23:36, Mark Millard <markmi at dsl-only.net> wrote:
>>>>> 
>>>>> I upgraded from llvm40 r4 to final. An interesting result was
>>>>> its creation of a backup package for llvm40-4.0.0.r4:
>>>>> 
>>>>> about 13 cpu-core-hours running pkg create
>>>>> 
>>>>> (Remember: I've been building with WITH_DEBUG= ) Its
>>>>> single-threaded status stands out via elapsed time
>>>>> approximately matching.
>>>>> 
>>>>> I'll note that it was somewhat under 6 elapsed hours for
>>>>> staging to have been populated (-j4 with 4 cores present
>>>>> helps for this part).
>>>>> 
>>>>> (Of course these elapsed-time figures are rather system
>>>>> dependent, although the ratio might be more stable.)
>>>>> 
>>>>> 
>>>>> 
>>>>> Also interesting was:
>>>>> 
>>>>> Installed packages to be REMOVED:
>>>>> 	llvm40-4.0.0.r4
>>>>> 
>>>>> Number of packages to be removed: 1
>>>>> 
>>>>> The operation will free 49 GiB.
>>>> 
>>>> Yes, this is big.  But there is no real need to build the llvm ports
>>>> with debug information, unless you want to hack on llvm itself.  And
>>>> in that case, you are better served by a Subversion checkout or Git
>>>> clone from upstream instead.
>>>> 
>>>> -Dimitry
>>> 
>>> FYI:
>>> 
>>> Historically unless something extreme like this ends up
>>> involved I build everything using WITH_DEBUG=  or explicit
>>> -g's in order to have better information on any failure.
>>> 
>>> This is extreme enough that next time I synchronize
>>> /usr/ports and it has a devel/llvm40 update I'll
>>> likely rebuild devel/llvm40 without using WITH_DEBUG= .
>>> I'm more concerned with the time it takes than with
>>> the file system space involved.
>> 
>> In the case of LLVM, enabling debug builds does a LOT more than adding
>> symbols.  It's much more like enabling WITNESS and INVARIANTS in your
>> kernel, except that the performance of the resulting binary is much
>> worse than a WITNESS kernel (more like 10x slowdown).
>> 
>> As Dimitry points out, these builds are of questionable value in ports
>> so garbage collecting the knob might be the sensable thing to do.
> 
> Sounds like the ALLOW_OPTIMIZATIONS_FOR_WITH_DEBUG technique
> would not change the "WITNESS and INVARIANTS"-like part of the
> issue. In fact if WITH_DEBUG= causes the cmake debug-style
> llvm40 build ALLOW_OPTIMIZATIONS_FOR_WITH_DEBUG might not
> make any difference: separate enforcing of lack of optimization.
> 
> But just to see what results I've done "pkg delete llvm40"
> and am doing another build with ALLOW_OPTIMIZATIONS_FOR_WITH_DEBUG=
> and its supporting code in place in addition to using WITH_DEBUG=
> as the type of build fro FreeBSD's viewpoint.
> 
> If you know that the test is a waste of machine cycles, you can
> let me know if you want.

The experiment showed that ALLOW_OPTIMIZATIONS_FOR_WITH_DEBUG
use made no difference for devel/llvm40 so devel/llvm40 itself
has to change such as what Dimitry Andric reported separately
as a working change to the Makefile .

(ALLOW_OPTIMIZATIONS_FOR_WITH_DEBUG would still have its uses
for various other ports.)

===
Mark Millard
markmi at dsl-only.net