From nobody Sat May 14 13:03:59 2022 X-Original-To: freebsd-jail@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 6C75B1ADC8DA for ; Sat, 14 May 2022 13:04:19 +0000 (UTC) (envelope-from dfr@rabson.org) Received: from mail-lj1-x22c.google.com (mail-lj1-x22c.google.com [IPv6:2a00:1450:4864:20::22c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4L0m1Z3V4gz4Zp3 for ; Sat, 14 May 2022 13:04:18 +0000 (UTC) (envelope-from dfr@rabson.org) Received: by mail-lj1-x22c.google.com with SMTP id q130so13198501ljb.5 for ; Sat, 14 May 2022 06:04:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rabson-org.20210112.gappssmtp.com; s=20210112; h=mime-version:from:date:message-id:subject:to; bh=LBpzV9BVO7VnSFee4xLOnR9vPwxYaENk5qRSkNUAq00=; b=LKzAq7nzFajakWGk8MZU5dF59AlHmNNSOIPBm5G85MukCoppypCoGALLLS4aSaEELs B6E9bokfvp7wLnFtkNLauY4ViqosEm/4ZCjOlj7rx0PbVcvNHIBwoHc5bsPLmnVbEgh+ qntRbNZSOP0TAo+Ve+0NRo+YkS1BheREFSb3mKQFmxyLd1tCo86bfjknKc6+zlh6IuMN NLqEwxxnnTwiUk3zH0GGCyHvAzpDTVNOMKzVJJsd8hUCxvIMWOnQ1gINwNo2g8gcQG5e 71y0u7TLldfRrMgxE3uZC8Uw4lIs1Q6ZF1q7b3y92o+cPGscT7v9Dyj2jjN4q9He6l1o o15Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=LBpzV9BVO7VnSFee4xLOnR9vPwxYaENk5qRSkNUAq00=; b=UnYcIPjfqZBR2B2BsOB7dcDd9DfedeHTu/5FR0PJc/1u876XYrkOCHNhjKv/jLpTWD 2ryaItW3mvTxoGojPuc7if/oVs870NOmq+0Lnj6qjm49y424tHXPQOllpgyC8FhQfBPW H+TroTss9lJHf+w7usqHJr7nI/68UgVqxdTKy3oYnZ7cMAUfq4s2eY3YsQAtEhoIj7/Y pmtcEjxyZmaxTQHggmWb2jc+W5Ws5ts7KZNKN6guaSaTUXQA+zhoGJF3+7v09uv4wHoQ BgxgFXh/D15dx+KjRwDICv4VvgNoA901trM/8p3U712u6CkrXw5IukettHi2lKjMR+Jp h17g== X-Gm-Message-State: AOAM532kTIj6EYLFz1yoehmSW9O8ncAWKSd1F3Xztm/DZZTs1jJUolVe lj/nGRZoe+S/NkkwR6Gm+BxFWav8NrcuQufxtsRlFgPFhesF2NjQ X-Google-Smtp-Source: ABdhPJxK3VHwlvU6ec69U++k5eFNRXbpW+cmXseDR4uVOj1QhJQi5SSa05sA/NmBJ80PC44T0XCSXV8mIQTFx8s/af4= X-Received: by 2002:a2e:aa27:0:b0:250:9109:2e7f with SMTP id bf39-20020a2eaa27000000b0025091092e7fmr5714816ljb.134.1652533450904; Sat, 14 May 2022 06:04:10 -0700 (PDT) List-Id: Discussion about FreeBSD jail(8) List-Archive: https://lists.freebsd.org/archives/freebsd-jail List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-jail@freebsd.org MIME-Version: 1.0 From: Doug Rabson Date: Sat, 14 May 2022 14:03:59 +0100 Message-ID: Subject: FreeBSD containers with podman and buildah To: freebsd-jail@freebsd.org Content-Type: multipart/alternative; boundary="00000000000052279d05def86d88" X-Rspamd-Queue-Id: 4L0m1Z3V4gz4Zp3 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=rabson-org.20210112.gappssmtp.com header.s=20210112 header.b=LKzAq7nz; dmarc=none; spf=pass (mx1.freebsd.org: domain of dfr@rabson.org designates 2a00:1450:4864:20::22c as permitted sender) smtp.mailfrom=dfr@rabson.org X-Spamd-Result: default: False [-3.50 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[rabson-org.20210112.gappssmtp.com:s=20210112]; FREEFALL_USER(0.00)[dfr]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-jail@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000]; DMARC_NA(0.00)[rabson.org]; DKIM_TRACE(0.00)[rabson-org.20210112.gappssmtp.com:+]; NEURAL_HAM_SHORT(-1.00)[-0.999]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::22c:from]; MLMMJ_DEST(0.00)[freebsd-jail]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: N --00000000000052279d05def86d88 Content-Type: text/plain; charset="UTF-8" Recently I've been working on porting the buildah and podman container tools to FreeBSD. Podman is a drop-in replacement for docker and buildah focuses on the narrower problem of building container images. At this point, there is enough functionality to show that these tools are viable on FreeBSD so I thought I would write a note here about how to install and try out my proof-of-concept. This will pull in source code for buildah and related modules, build everything and install to /usr/local. Be aware that if you have sysutils/runj installed, it will be overwritten with a modified version. This all happens in a directory named 'build' which can be deleted to clean up or to force a clean build: mkdir -p build fetch https://gist.github.com/dfr/ac4dc043ee3780b690c5887a61f53494/raw/11474779a16bdff1ca31c94437ddb25a8f1f364b/buildah-install.sh chmod +x buildah-install.sh (cd build && ../buildah-install.sh) Make a container and run things inside it: c=$(sudo buildah from docker.io/kwiat/freebsd:13.0-RELEASE) sudo buildah run $c freebsd-version sudo buildah run $c ifconfig sudo buildah rm $c Download and run images in podman: sudo podman run --rm docker.io/dougrabson/hello The containers will use the default 'podman' network which is defined in /usr/local/etc/cni/net.d/87-podman-bridge.conflist. This relies on NAT to allow the container traffic out to the internet and I use pf with the following simple pf.conf: nat on egress inet from to any -> (egress) nat on egress inet6 from to !ff00::/8 -> (egress) rdr-anchor "cni-rdr/*" table Note: I'm using the OpenBSD convention to identify the host's main interface by putting it into the 'egress' group using ifconfig, e.g.: sudo ifconfig vtnet0 group egress There is a lot of room for improvement in this area - NAT works fairly well for ipv4 but can get confused with ipv6 if the egress interface has non-routable addresses assigned to it. Port mapping is very limited and does not work for connections from localhost. Perhaps someone with better pf skills can help figure out how to get this working (probably needs to NAT from localhost back to the container network). --00000000000052279d05def86d88 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Recently I've been working on porting the buildah= and podman container tools to FreeBSD. Podman is a drop-in replacement for= docker and buildah=C2=A0focuses on the narrower problem of building contai= ner=C2=A0images. At this point, there is enough functionality=C2=A0to show = that these tools are viable on=C2=A0FreeBSD so I thought I would write a no= te here about=C2=A0how to install and try out my proof-of-concept.

This will pull in source code for buildah and related modules, = build everything and install to /usr/local. Be aware that if you have sysut= ils/runj installed, it will be overwritten with a modified version. This al= l happens in a directory named 'build' which can be deleted to clea= n up or to force a clean build:

mkdir -p build
fetc= h https://gis= t.github.com/dfr/ac4dc043ee3780b690c5887a61f53494/raw/11474779a16bdff1ca31c= 94437ddb25a8f1f364b/buildah-install.sh
chmod +x buildah-install.sh(cd build && ../buildah-install.sh)

Make a= container and run things inside it:

c=3D$(sudo builda= h from docker.io/kw= iat/freebsd:13.0-RELEASE)
sudo buildah run $c freebsd-version
sud= o buildah run $c ifconfig
sudo buildah rm $c

Down= load and run images in podman:

sudo podman run --rm docker.io/dougrabson/hello<= /font>

The containers will use the default 'podman'= network which is defined in /usr/local/etc/cni/ne= t.d/87-podman-bridge.conflist. This relies on NAT to allow the conta= iner traffic out to the internet and I use pf with the following simple pf.conf:

nat on egress in= et from <cni-nat> to any -> (egress)
nat on egress inet6 from &= lt;cni-nat> to !ff00::/8 -> (egress)
rdr-anchor "cni-rdr/*&qu= ot;
table <cni-nat>

Note: I'm using the= OpenBSD convention to identify the host's main interface by putting it= into the 'egress' group using ifconfig, e.g.:

sudo ifconfig vtnet0 group egress

There is a lot of= room for improvement in this area - NAT works fairly well for ipv4 but can= get confused with ipv6 if the egress interface has non-routable addresses = assigned to it. Port mapping is very limited and does not work for connecti= ons from localhost. Perhaps someone with better pf skills can help figure o= ut how to get this working (probably needs to NAT from localhost back to th= e container network). --00000000000052279d05def86d88--