From nobody Wed Sep 29 08:56:31 2021 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 5CCD4178ACBE for ; Wed, 29 Sep 2021 08:56:33 +0000 (UTC) (envelope-from manu@bidouilliste.com) Received: from mx.blih.net (mail.blih.net [212.83.155.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mx.blih.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HK9GT00gJz4pHB; Wed, 29 Sep 2021 08:56:32 +0000 (UTC) (envelope-from manu@bidouilliste.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bidouilliste.com; s=mx; t=1632905791; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=U+K8YGv6BGY8fI7A0Pc7Q50tMBg6dgAJiev0biRuJh8=; b=U0IMfkVJEO69KxGi4Cp4IMVEOQEuzyWSfJ0hW9xFqdWldyQXi9HyrH/VNTbYCsBvvqR7hy +cRKJ1qZ+6XYlIQPFL2eKYtCkyNHAv9743RwIIyhRmPstTlsrOqLRtWinwG/xctIADDPc0 U85AlTOiTC+hy+TVnms/YhkfWokoEVA= Received: from amy (lfbn-idf2-1-644-191.w86-247.abo.wanadoo.fr [86.247.100.191]) by mx.blih.net (OpenSMTPD) with ESMTPSA id ebec6fea (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Wed, 29 Sep 2021 08:56:31 +0000 (UTC) Date: Wed, 29 Sep 2021 10:56:31 +0200 From: Emmanuel Vadot To: David Chisnall Cc: freebsd-current@freebsd.org Subject: Re: FreeBSD base pkg (packaging) and critical ports build alongside Message-Id: <20210929105631.f9cba69265edcab0a05b25d6@bidouilliste.com> In-Reply-To: <74afb0a9-be7f-c797-bd15-a96022188d9a@FreeBSD.org> References: <20210929102805.1ac3a59d@freyja> <74afb0a9-be7f-c797-bd15-a96022188d9a@FreeBSD.org> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; amd64-portbld-freebsd14.0) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4HK9GT00gJz4pHB X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On Wed, 29 Sep 2021 09:42:46 +0100 David Chisnall wrote: > Hi, > > I think your best option would be to do the opposite of what you > suggest. Poudriere can build pkgbase sets from a source tree and > populate a jail from them. No it doesn't populate the jail from the pkgbase sets, the jail will still be populated using make installworld and friends. I still need to add the possibility to create a jail (and update it) just from a pkgbase but haven't got time yet for this. > The flow that I'd suggest is: > > - Poudriere jail to build a jail from an existing source tree. > - If there are kernel changes, install the packages on the package > builder and reboot. > - Poudriere bulk in the new jail to build the new package set. > > Note: You can *normally* skip the second step (drm ports, for example, > will be built against the new kernel sources in the jail, though they > might not be loadable on the host) but there's no guarantee that you can > run a newer userland on an older kernel so things may break. > > If you enable reproduceable builds in the src.conf that you use for > building the jail then you should be able to just diff the kernel binary > to see if anything has changed. > > If you have bhyve or are running on a cloud platform then you can > replace the second step with a poudriere image invocation to build a VM > image containing poudriere and your newly-built base system and deploy > this to build the packages. I'm planning on working on some tooling to > do this in Azure with GitHub Actions. > > Note that poudriere uses packages installed on the host system to build > a jail. If you have, for example, installed llvm12 then you can put a > line in your src-env.conf for the jail to tell it to use that as an > external toolchain and skip the toolchain-bootstrap phase of the build. > This means that the base-build is fairly fast even on quite modest > hardware (it still builds clang, but at least it does it only once). > > David > > > On 29/09/2021 09:28, FreeBSD User wrote: > > Hello, > > > > I use FreeBSD-base packages built on self hosted systems to update 13-STABLE > > and CURRENT hosts. I run into the problem, that the packages of the FreeBSD > > base, built via the FreeBSD framework and from most recent 13-STABLE sources, > > are often oit of synchronisation with our poudriere packaging builders, that is > > especially true for critical ports with kernel modules, like i915 drm, > > virtualbox and so on. The problem is, obviously, barehanded: 13-STABLE sources > > and probably the API changes more rapidly than those of the appropriate builder > > hosts for poudriere and since it takes a bunch of days to build a whole > > poudriere packages repository, there is often a gap between the revision of the > > kernel and the port containing kernel modules. > > > > So, the question is: how can I add ports to the building process of the FreeBSD > > sources tree in the way they get build every time I build the FreeBSD-base > > packages alongside the OS? > > > > Thanks in advance, > > > > oh > > > -- Emmanuel Vadot