ports-mgmt/poudriere-devel, lang/rust (for example), and USE_TMPFS that includes wrkdir (or yes)

[Bryan Drewery]

There is no solution at the moment and is a common complaint (about
concurrent large builds). I had TMPFS_LIMIT=15 and had rust fail on me
from that. Quite large...

Bryan

On 5/10/2021 10:19 PM, Mark Millard wrote:
> I've been using USE_TMPFS=yes (so "wrkdir data") on
> various systems, both ZFS (recently) and UFS
> (generally, even now). Only one system builds rust
> (in order for something else to be built), at least
> so far.
> 
> An example of the wrkdirs tmpfs use for rust is
> (UFS context):
> 
> # df -m | grep tmpfs
> Filesystem 1M-blocks   Used  Avail Capacity  Mounted on
> . . .
> tmpfs         301422  17859 283563     6%    /usr/local/poudriere/data/.m/FBSDFSSDjail-default/01/wrkdirs
> . . .
> 
> This was near the end but the maximum figure was probably
> somewhat higher than the 17 GiByte+ figure above. The
> context the example is from is for the only large capacity
> build machine that I have access to, an amd64 context. I
> have other build contexts as well, but, so far, none have
> had to deal with building rust.
> 
> Rust likely would fit the 8 GiByte RAM + 24 GiByte swap
> aarch64 build context with USE_TMPFS including wrkdir if
> it was the only builder running at the time. But the
> existing builds for the context allow 4 builders in
> parallel, one per core. [This deals just fine with
> llvm10, llvm11, llvm12, and, gcc10 (no bootstrap) being
> what happens to build in parallel, even with USE_TMPFS
> that includes wrkdir. Rust is just uses more space all
> by itself.]
> 
> If I end up with something that requires rust for the
> aarch64 builder context, is there a different technique
> to deal with the tradeoff other than giving up on
> USE_TMPFS spanning wrkdir for all other other
> ports/builder-instances as well, presuming the same
> media and partitioning (such as total swap space)?
> 
> Imaginary examples could be:
> 
> A) Tell poudriere that lang/rust is to be built by itself
>    despite the general 4-builder context.
> 
> B) Tell poudriere that USE_TMPFS excludes wrkdir for
>    lang/rust's specific builder.
> 
> C) . . . (good question) . . .
> 
> So far all I've come up with is explicitly building
> lang/rust by itself first, a form of (A):
> 
> # poudriere bulk -jNAME -w lang/rust
> # poudriere bulk -jNAME -w -f ~/origins/CA72-origins.txt
> 
> (Hopefully, reliably remembering to do so.)
> 
> Is there any better technique that I've not noticed?
> 
> To some extent here, lang/rust is being used an example
> of a more general issue: Other ports could have similar
> issues with attempted wrkdir-included USE_TMPFS use.
> 
> Note: If I build using WITH_DEBUG, the one system that
> I have access to that can build such a lang/rust with
> workdir included in USE_TMPFS shows over 130 GiBytes
> in the tmpfs earn the end of the builder's activity.
> (This is a amd64 context with 128 GiBytes of RAM and
> 192 GiBytes of swapping/paging space.)
> 
> ===
> Mark Millard
> marklmi at yahoo.com
> ( dsl-only.net went
> away in early 2018-Mar)
> 


-- 
Regards,
Bryan Drewery

-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 495 bytes
Desc: OpenPGP digital signature
URL: <http://lists.freebsd.org/pipermail/freebsd-toolchain/attachments/20210512/77c1aa64/attachment.sig>