[Proof of Concept] Stacked unionfs based 'tinderbox'

[David Naylor]

On 25 February 2010 15:53, Ulrich Spörlein <uqs at freebsd.org> wrote:
> On Thu, 25.02.2010 at 10:08:15 +0200, David Naylor wrote:
>> Hi,
>>
>> As some may have noticed on -current I have been working on using
>> stacked unionfs to implement a 'tinderbox' type build system.  I have
>> successfully used the scripts to build x11/xorg (and have compared the
>> results to using the traditional approach using pkg_add).  The build
>> system is stable except for one nasty corner case: deadlocks.
>
> When I did this a couple of years ago, the major problems were failing
> chdir(2) calls during ports build, etc.
>
>> To setup a compatible test environment requires:
>>  - recompile the kernel with `options KSTACK_PAGES=32`, otherwise the
>>  kernel will panic with a double fault.  WITNESS options results in
>>  substantial performance degradation.
>>  - patch mtree (see below) [optional]
>>  - create the appropriate chroot environment (and reboot) [see below
>>  for corner case]
>>
>> A performance bottleneck in mtree was identified.  This resulted in
>> mtree (as run by port install) consuming ~20% of the build time.  See
>> bin/143732 for a patch and further details.
>
> Good work!
>
>> The normal tinderbox approach takes ~80% more time to install compared to the
>> quick and dirty approach.  The stacked unionfs approach takes ~170% more time
>> (an increase of ~50% over the tinderbox approach).  Some performance gains can
>> be had if one uses memory backed storage (vs HDD in this case).
>
> Please explain: what is the quick and dirty approach and which one is
> faster now?

The quick and dirty is `make -C /usr/ports/x11/xorg install clean`.
The stacked unionfs
is still the slowest (even with a 20% improvement from patching mtree).

> As your scripts did not make it through, perhaps you can upload them to
> the wiki? What I did back then was using a clean base system as the
> underlying unionfs store to avoid re-generating the clean base over and
> over again. Nowadays, a ZFS clone would probably be the way to go.
>
> I'm not sure if a recursive approach is feasible here, as you can have
> only one underlying unionfs mount. But special casing, e.g., perl may
> still give a massive speedup. So for each port that has perl as
> dependancy, you would not pull in the clean base + pkg_add perl, but
> instead grab the clean-base+perl directory as an underlying unionfs.
>
> Cheers
> Uli
>