From nobody Thu Jun 30 07:04:24 2022 X-Original-To: freebsd-jail@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 6496D8799F4 for ; Thu, 30 Jun 2022 07:04:37 +0000 (UTC) (envelope-from dfr@rabson.org) Received: from mail-lf1-x134.google.com (mail-lf1-x134.google.com [IPv6:2a00:1450:4864:20::134]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4LYTpr4wYbz3Pgq for ; Thu, 30 Jun 2022 07:04:36 +0000 (UTC) (envelope-from dfr@rabson.org) Received: by mail-lf1-x134.google.com with SMTP id y16so8203239lfb.9 for ; Thu, 30 Jun 2022 00:04:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rabson-org.20210112.gappssmtp.com; s=20210112; h=mime-version:from:date:message-id:subject:to:cc; bh=p4VvHA3BYktBTM3ydmQ2UGT9diFJQW4CucBjOT2Yogo=; b=VZgO5Ul21RGSoNsvmPx20ZSSHGpbG3BWFReiJc1G+b+GhI8J7JlX8wPkbMs+RB3jSL VGJZnN43+0V9E8BNAGP65QJGvA3QNQXoiupPc2iyLQOR65GkHdvUwSmoqCKEYU/3D7Ox DwVPjBztGQozs3aTAM7gFAjg8ulnS5oZNZgkXW1+mwQ22n3Z2v3+ky0RzKe/GU0f0eDK KJYwln8PwzXGOiqchyV6RxZ6cRGbcqIB0bg1Buui8MZTBTytnZpk5uNlrJQDTlqRqAJg +LGUyNsaTcKblXTib3kPaCNYZ1BJES3IJ3XTfxyjMSOhNuqPesGoe2BPmr8L2GBrDgGY etHQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=p4VvHA3BYktBTM3ydmQ2UGT9diFJQW4CucBjOT2Yogo=; b=6uKMFinXbuWQjEvR50OwxM8u36YZOsSlEezLNeUzTEZ4Kqe3fXqDz6cDdE/Pg21SIA guSw8VBcc2HGy/2+96xAURnRkHqoDyTdlOdeeqbjjU4vR7UddkY8WLlnXJc0Ynlq+4go 3VDd6DzLrLn1hiYBlG+DsHynXpQ68rw5q+oKbxK/4ZHeGroiu3GHqDNduqWtcsHN/CtF VxtNLrSmwdnX+j+LUIZ65X0QlsVtc6bCyHf8LbCtRiAHW17XtvoOMwtIuwQVytwFfkyt l4PkeO7CEJWdDV3UxNmYvV8baT4n9wkxmVJijZ+JJdcb00uEXdH0UX7kLoWrQrCTjgui GPJw== X-Gm-Message-State: AJIora8ktJi9GWCQFGUwfbo15a+KCNuk8e3szMU4Y9mzDZIgbQc6VC7X 4lTit8/oYV3nHvgSIEmfvpO8DNz0auBqZdOEUJglgBbDSzdT0AgK X-Google-Smtp-Source: AGRyM1seoLjlAbr7jxMFehu27NzvZGEFgZ0U0evnDjfyKOPfR8PmD/kYxHj0BOC8M7vr3WHLbmiOhYUd9YOMKekS1oY= X-Received: by 2002:a05:6512:25a3:b0:481:25b8:51b5 with SMTP id bf35-20020a05651225a300b0048125b851b5mr4785965lfb.472.1656572674896; Thu, 30 Jun 2022 00:04:34 -0700 (PDT) List-Id: Discussion about FreeBSD jail(8) List-Archive: https://lists.freebsd.org/archives/freebsd-jail List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-jail@freebsd.org MIME-Version: 1.0 From: Doug Rabson Date: Thu, 30 Jun 2022 08:04:24 +0100 Message-ID: Subject: Container Networking for jails To: freebsd-jail@freebsd.org, freebsd-net@freebsd.org Cc: Samuel Karp Content-Type: multipart/alternative; boundary="000000000000d509bf05e2a4e17e" X-Rspamd-Queue-Id: 4LYTpr4wYbz3Pgq X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=rabson-org.20210112.gappssmtp.com header.s=20210112 header.b=VZgO5Ul2; dmarc=none; spf=pass (mx1.freebsd.org: domain of dfr@rabson.org designates 2a00:1450:4864:20::134 as permitted sender) smtp.mailfrom=dfr@rabson.org X-Spamd-Result: default: False [-3.50 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[rabson-org.20210112.gappssmtp.com:s=20210112]; FREEFALL_USER(0.00)[dfr]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-jail@freebsd.org]; DMARC_NA(0.00)[rabson.org]; NEURAL_HAM_LONG(-1.00)[-1.000]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[rabson-org.20210112.gappssmtp.com:+]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::134:from]; MLMMJ_DEST(0.00)[freebsd-jail]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: N --000000000000d509bf05e2a4e17e Content-Type: text/plain; charset="UTF-8" I wanted to get a quick sanity check for my current approach to container networking with buildah and podman. These systems use CNI ( https://www.cni.dev) to set up the network. This uses a sequence of 'plugins' which are executables that perform successive steps in the process - a very common setup uses a 'bridge' plugin to add one half of an epair to a bridge and put the other half into the container's vnet. IP addresses are managed by an 'ipam' plugin and an optional 'portmap' plugin can be used to advertise container service ports on the host. All of these plugins run on the host with root privileges. In kubernetes and podman, it is possible for more than one container to share a network namespace in a 'pod'. Each container in the pod can communicate with its peers directly via localhost and they all share a single IP address. Mapping this over to jails, I am using one vnet jail to manage the network namespace and child jails of this to isolate the containers. The vnet jail uses '/' as its root path and the only things which run inside this jail are the CNI plugins. Using the host root means that a plugin can safely call host utilities such as ifconfig and route without having to trust the container's version of them. An important factor here is that the CNI plugins will only be run strictly before the container (to set up) or strictly after (to tear down) - at no point will CNI plugins be executed at the same time as container executables. The child jails use ip4/6=inherit to share the vnet and each will use a root path to the container's contents in the same way as a normal non-hierarchical jail. Can anyone see any potential security problems here, particularly around the use of nested jails? I believe that the only difference between this setup and a regular non-nested jail is that the vnet outlives the container briefly before it is torn down. --000000000000d509bf05e2a4e17e Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

<= div>In kubernetes and podman, it is possible for more than one container to= share a network namespace in a 'pod'. Each container in the pod ca= n communicate with its peers directly via localhost and they all share a si= ngle IP address.



Can anyone see any potential security problems here, particularly around = the use of nested jails? I believe that the only difference between this se= tup and a regular non-nested jail is that the vnet outlives the container b= riefly before it is torn down. --000000000000d509bf05e2a4e17e--