FYI: what it takes for RAM+swap to build devel/llvm40 with 4 processors or cores and WITH__DEBUG= (powerpc64 example)

[Dimitry Andric]

On 30 Mar 2017, at 19:06, Alexey Dokuchaev <danfe at FreeBSD.org> wrote:
> 
> On Mon, Mar 27, 2017 at 11:41:40AM +0200, Dimitry Andric wrote:
>> On 26 Mar 2017, at 23:36, Mark Millard <markmi at dsl-only.net> wrote:
>>> ...
>>> Also interesting was:
>>> 
>>> Installed packages to be REMOVED:
>>> 	llvm40-4.0.0.r4
>>> 
>>> Number of packages to be removed: 1
>>> 
>>> The operation will free 49 GiB.
>> 
>> Yes, this is big.  But there is no real need to build the llvm ports
>> with debug information, unless you want to hack on llvm itself.
> 
> Cc'ing jmd@ and rezny at .
> 
> I've been watching increasing size of our LLVM packages with increasing
> worry.  This is from my tinderbox cache:
> 
>  $ % env LANG=C ls -lh llvm3*
>  -rw-r--r--  1 root  wheel    17M Jan 29  2016 llvm35-3.5.2_1.txz
>  -rw-r--r--  1 root  wheel    18M Mar  7  2016 llvm36-3.6.2_2.txz
>  -rw-r--r--  1 root  wheel    27M Feb 28 01:05 llvm37-3.7.1_4.txz
>  -rw-r--r--  1 root  wheel   207M Jan 19 18:20 llvm38-3.8.1_5.txz
>  -rw-r--r--  1 root  wheel   244M Mar 23 16:42 llvm39-3.9.1_2.txz
> 
> Dimitry, do you know what had causes such a huge bump in 37 -> 38?

Yes, up to llvm37, the ports were split in devel/llvmXY and lang/clangXY
parts, with separate ports for e.g. compiler-rt and other LLVM projects.

For llvm38 and later, the devel/llvmXY port contains almost *all*
upstream LLVM components, which are then selectable at port configure
time.  For instance, devel/llvm40 shows:

┌─────────────────────────────── llvm40-4.0.0 ─────────────────────────────────┐
│ ┌──────────────────────────────────────────────────────────────────────────┐ │
│ │ [x] CLANG        Build clang                                             │ │
│ │ [x] COMPILER_RT  Sanitizer libraries                                     │ │
│ │ [x] DOCS         Build and/or install documentation                      │ │
│ │ [x] EXTRAS       Extra clang tools                                       │ │
│ │ [x] GOLD         Build the LLVM Gold plugin for LTO                      │ │
│ │ [x] LIT          Install lit and FileCheck test tools                    │ │
│ │ [x] LLD          Install lld, the LLVM linker                            │ │
│ │ [x] LLDB         Install lldb, the LLVM debugger (ignored on 9.x)        │ │
│ │ [x] OPENMP       Install libomp, the LLVM OpenMP runtime library         │ │
│ └──────────────────────────────────────────────────────────────────────────┘ │
├──────────────────────────────────────────────────────────────────────────────┤
│                       <  OK  >            <Cancel>                           │
└──────────────────────────────────────────────────────────────────────────────┘

If you want to reduce the size of the package, only select the part(s)
you need.  I think you can get by with just the CLANG option, for most
dependent ports.



> They take lots of time to build and package.  And given that llvm
> is indirect dependency of any X11-related port, it pessimises their
> build times as well (devel/libclc now requires devel/llvm40 after
> r437268).

The previous split looks pretty hard to maintain, so that is most likely
the reason for combining all components in one port after 3.8.
Unfortunately the side effect is that it is way less granular.

If we ever get infrastructure for generating multiple packages out of
one port, the devel/llvm* ports are very good candidates. :)


> With 49 GiB llvm40, I guess I won't be able to build-test post as my
> hardware would just not be capable enough.

As said, this is because of WITH_DEBUG.  Don't use that for the llvm
ports, for now.  It will also allow you to build them with much less RAM
in the machine: especially linking can take multiple GB when debuginfo
is enabled, and optimization is off.

-Dimitry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 194 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.freebsd.org/pipermail/freebsd-toolchain/attachments/20170330/6960f8ec/attachment.sig>