Short Term Focus

Greetings,

We had an impromptu discussion about pre-commit testing after the vendor
summit today.

We talked about different ways to split things up. There are a fair number
of 'free' services that can be used to run CI pipelines. However, in
general, they are best used for 'testing' or 'orchestration'. Large,
intensive compute jobs (like for building FreeBSD) can be done
occasionally, but will need to be offloaded from these services if there's
a lot of builds since the limitations of the free services would be
exhausted too quickly.

We had a side conversation about optimization at this point.  I presented
David Chisnel's idea of creating a snapshot of 'yesterday's. build, and all
pushes would use that to do a meta-mode build for the recent deltas and
proposed changes. This idea was presented again later when others joined
the call. While a good idea, there's a number of things that need to be
done before we can optimize.

The general consensus was the first steps would be to investigate building
freebsd (multiple architectures) and running kyua (and maybe other) tests.
This dovetails well with our Asia Timed meeting where Li-Wen, Philip and i
discussed taking the current Jenkins scripts and adapting them to run in
the cloud. There's a number of different means to do this. We have Azure
credits, Oracle credits, and a few free things to stand up runners. We also
can use the work Ed presented on terraform to interface to these different
cloud providers so we can use it for runners to do the heavy lifting. This
will also give us a number of choices for how to move jobs between
different services as prices change, free offers are there, etc down the
line.

There's a number of different ways to kick off builds when changes are
pushed. We have a little bit of that in github today, but it's used to kick
off the Cirrus CI builds. We do kernel builds on pull requests, but only on
Ubuntu and macOS. There's currently nothing done when a push happens to
github. So Ed Maste, myself and John Baldwin will be looking at different
aspects of expanding this integration (others are welcome to help). I'll be
looking at gitlab kicking off the standard cirrus-ci builds we can get from
github now (to compare and contrast). I'll also look at standing up other
jobs that use the testing features of gitlab using the artifacts built part
of the cirrus ci integration and maybe a few other checking things. There's
a number of other software projects that I can crib from. Ed and John will
be doing their experiments as well.

And to be clear: others are welcome to run their own experiments to learn
what's available, what we can use, how we can optimize. Mention was made of
using the general github push hook that others have used in other projects
that might be useful. We also talked about different ways to parameterize
the CI pipeline so different pushes can get different levels of testing
depending on things like gitlab CI variables. This is a very rich field,
and there's a number of other areas that  can be applied to FreeBSD's
specific needs that aren't being explored. We'd love to hear from others
that have done these things.

Once these initial experiments are done, we're planning on getting together
with anybody else that's done their own experiments and a few experts that
have joined the group who have time to advise but not to experiment. To
give people time, and to account for the US Thanksgiving holiday, we'll
meet next on December 1th, again on December 15th and again January 5th to
talk about the experiments, to plan next steps, etc. I'll send out meeting
details.

So I hope I've captured all we talked about, but it was a somewhat informal
conversation that ranged over a number of areas not related to CI. If
anybody else wants to add anything, please do.

Warner