From nobody Tue Apr 11 07:11:59 2023 X-Original-To: freebsd-questions@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PwcTz72PLz44HHW for ; Tue, 11 Apr 2023 07:12:07 +0000 (UTC) (envelope-from tim@timpreston.net) Received: from wout4-smtp.messagingengine.com (wout4-smtp.messagingengine.com [64.147.123.20]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PwcTy4Vtmz4X4G for ; Tue, 11 Apr 2023 07:12:06 +0000 (UTC) (envelope-from tim@timpreston.net) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=timpreston.net header.s=fm2 header.b="j QtjAdk"; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=C1udRcdQ; spf=pass (mx1.freebsd.org: domain of tim@timpreston.net designates 64.147.123.20 as permitted sender) smtp.mailfrom=tim@timpreston.net; dmarc=none Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.west.internal (Postfix) with ESMTP id 9519232002D8; Tue, 11 Apr 2023 03:12:03 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Tue, 11 Apr 2023 03:12:03 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=timpreston.net; h=cc:cc:content-type:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to; s=fm2; t=1681197123; x= 1681283523; bh=OFByuV2jKb3+NzDuenhVOUXTcbnxZXV2FHnmcGD1bS8=; b=j QtjAdk5O4nJmHLhD8oWZRQAuvXxvZGZd6pl+bEnd3yBVRJ0bOsDBsnzT/y0m8xMo hdEpWoUCrV97JbzFVw580FJPHLZVUxJKeWclqH1x4T/7CNu7Z0Nu1b0EbIlyBZb4 zF7chKppH8UEV7/pP0JtKbWstEWawEamSp7cU5T57ZzEPiNdAF0qzmKauBMO5jkd CRFfIwglAK6VYx34OgUk3OuGJtJm1daanYgSko5Q5hl6/8BiigTybF/uv4dW+V0h UIYS2sBtw4+xZhUGeUHCpDLDUX4WTOVSUbID/07MqBkHOYE7SGtLsdDnjvJPchCH KKGLU8MjJBLF1PuYmL97w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; t=1681197123; x=1681283523; bh=OFByuV2jKb3+N zDuenhVOUXTcbnxZXV2FHnmcGD1bS8=; b=C1udRcdQkHB/O5vLRKoZkI3NdVmqK N+yOTmIkszHH+pzseQ7Ym+vhOuVgyaFN15/jYgELrBKwfts4JWaKuN5PNUEstXhi d4fNm54/ZwXlpTB5tp0PGlMzXaejGxuSJwsrobKW2S+4xj97lcJeyKuEVSfzAbvx vcQYkPsxKFGwx4CyLVHfbK6P9GClRen6fZ0iCxDqFBLZ2Pyl20szDwIs9GggCYjO /uN7AbFyZcAYvFUG2FWtvoibM5h5b56nGjHgS+hSlD2UDNP/QAQkJiNcqvXq8lZJ 0aQ1UpG6kWKMJ1vOBZZmnojFpQfF/sX6aOezgHVn/p+vYc8eTkqYgpfjw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrvdekfedguddulecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpegtkfffgggfuffvvehfhfgjsegrtderredtfeejnecuhfhrohhmpefvihhm ucfrrhgvshhtohhnuceothhimhesthhimhhprhgvshhtohhnrdhnvghtqeenucggtffrrg htthgvrhhnpeeftdfggedvffehudevtedvkeekhffgteetieefheffleeliefghfeltdfh feelkeenucffohhmrghinhepthhruhgvnhgrshdrtghomhdpphhrohguuhgtthhiohhnfi hithhhshgtihhsshhorhhsrdhruhhnpdhgihhthhhusgdrtghomhdphhhonhgvhihguhhi uggvrdhnvghtnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrh homhepthhimhesthhimhhprhgvshhtohhnrdhnvght X-ME-Proxy: Feedback-ID: i1d2040e0:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 11 Apr 2023 03:12:02 -0400 (EDT) Content-Type: multipart/alternative; boundary="------------HoX8A1ohXtJRxbaflXIMx3LT" Message-ID: <06be3a1e-9319-1a21-88b9-4f87328ee127@timpreston.net> Date: Tue, 11 Apr 2023 17:11:59 +1000 List-Id: User questions List-Archive: https://lists.freebsd.org/archives/freebsd-questions List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-questions@freebsd.org X-BeenThere: freebsd-questions@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: Docker Content-Language: en-AU To: Mario Marietto Cc: FreeBSD Mailing List References: <20230329053443.6ADA6B6AFED5@dhcp-8e64.meeting.ietf.org> <8E16D624-2655-4A10-844A-93E4F63E9859@gromit.dlib.vt.edu> <078a1cf8-7ae2-c593-615b-f5f37fa2b3eb@timpreston.net> From: Tim Preston In-Reply-To: X-Spamd-Result: default: False [-4.60 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[timpreston.net:s=fm2,messagingengine.com:s=fm3]; R_SPF_ALLOW(-0.20)[+ip4:64.147.123.20]; RCVD_IN_DNSWL_LOW(-0.10)[64.147.123.20:from]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; FROM_EQ_ENVFROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-questions@freebsd.org]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:29838, ipnet:64.147.123.0/24, country:US]; RCPT_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[64.147.123.20:from]; ARC_NA(0.00)[]; DKIM_TRACE(0.00)[timpreston.net:+,messagingengine.com:+]; FROM_HAS_DN(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; DMARC_NA(0.00)[timpreston.net]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[] X-Rspamd-Queue-Id: 4PwcTy4Vtmz4X4G X-Spamd-Bar: ---- X-ThisMailContainsUnwantedMimeParts: N This is a multi-part message in MIME format. --------------HoX8A1ohXtJRxbaflXIMx3LT Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit The port mentioned in the first article doesn't work any more, if I remember correctly. The second link describes how Linux containers might be run via Linux binary compatibility. It's not Docker exactly, but could be used to run Docker images. But the question I'd ask is "why?". If you need to run Docker images you should probably run them on Linux, to ensure 100% kernel compatibility. After all, Docker is a Linux-only technology. There doesn't seem much point trying to shoehorn it into FreeBSD. On 10/4/23 00:04, Mario Marietto wrote: > It seems that docker now can run on FreeBSD natively,not with the > collaboration of bhyve. What do you think ? > > He says : "Yes, OCI Containers on FreeBSD. What was proposed ages ago > as Docker done right" > > https://productionwithscissors.run/2022/09/04/containerd-linux-on-freebsd/ > > On Tue, Apr 4, 2023 at 4:23 AM Tim Preston wrote: > > It can be done, with a bit of manual tinkering. > > Here is a gist which explains how to run Docker in a CentOS 8 VM > (under bhyve). > > https://gist.github.com/tehpeh/7e5329d295eca9539e6462f36b6ce9c0 > > It's a bit out of date but the general idea would be the same for > CentOS stream, Alpine etc: install Docker, enable the service, > open firewall/networking, nfs mount a local directory. This is > pretty much what Docker for Mac does. > > If you're looking for the Docker hub image repository equivalent > for FreeBSD, take a look at Bastille templates or Potluck > (https://potluck.honeyguide.net/). > > However, and this is only my personal opinion, a pre-baked > container image repository is a bad idea. Apart from the security > issues and recent drama around Docker shutting down free accounts, > container images are often set up with default parameters not > useful in a production environment (or even your specific dev > environment) and are built against a particular kernel version, so > may not run as expected on a different kernel version. > > Again, only my opinion, but you're much better off building your > own, private, images targeting the particular OS/Kernel version > you use in dev/staging/production. In summary, prefer Dockerfiles > over pre-built images. > > I think the conversation we really need to have is not about > copying Docker, but instead how do we consistently create, run, > and scale jails across multiple FreeBSD hosts easily. > > Tim > > > On 2/4/23 02:54, Paul Mather wrote: >> On Mar 29, 2023, at 1:34 AM, John Levine wrote: >> >>> It appears that Tomek CEDRO said: >>>> if there are lots of images for linux docker, and docker is linux only >>>> solution, there is no reason to talk about it on bsd or even offer some >>>> sort of images of bsd for linux right? >>> Docker runs on MacOS with a linux emulation layer. FreeBSD already has >>> some linux emulation so in principle one could do the same thing, but >>> it'd be a lot of work for dubious benefit. >> I disagree it would be of dubious benefit. MacOS is a Tier 1 platform in the Docker ecosystem. Using Docker Desktop on macOS makes using Docker and Kubernetes for development work very easy on that platform, meaning you can stay in the environment you prefer. MacOS is not Linux, but the implementation on there is to use a shim Linux VM via the built-in macOS hypervisor (which, IIRC, is a derivative of bhyve). >> >> It would be great if the same thing could be done on FreeBSD. It would be beneficial if there was a supported docker machine driver for bhyve on FreeBSD. Right now, I believe the road to running Linux containers on FreeBSD is to use the VirtualBox docker machine driver, which is a bit heavyweight (in terms of added dependencies) for my liking. It would be nice if bhyve could be used to run the shim Linux VM. >> >> Other than that, much of the tooling to run Docker and Kubernetes is already in ports. But, those (e.g., in the case of Kubernetes) need to point to non-FreeBSD systems that are running the actual containers, pods, etc. It would be nice to be able to do it all on FreeBSD, at least for development and kicking-the-tyres purposes. >> >> Cheers, >> >> Paul. >> > > > > -- > Mario. --------------HoX8A1ohXtJRxbaflXIMx3LT Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit The port mentioned in the first article doesn't work any more, if I remember correctly.

The second link describes how Linux containers might be run via Linux binary compatibility. It's not Docker exactly, but could be used to run Docker images.

But the question I'd ask is "why?". If you need to run Docker images you should probably run them on Linux, to ensure 100% kernel compatibility.

After all, Docker is a Linux-only technology. There doesn't seem much point trying to shoehorn it into FreeBSD.


On 10/4/23 00:04, Mario Marietto wrote:
It seems that docker now can run on FreeBSD natively,not with the collaboration of bhyve. What do you think ?

He says : "Yes, OCI Containers on FreeBSD. What was proposed ages ago as Docker done right"


On Tue, Apr 4, 2023 at 4:23 AM Tim Preston <tim@timpreston.net> wrote:
It can be done, with a bit of manual tinkering.

Here is a gist which explains how to run Docker in a CentOS 8 VM (under bhyve).

https://gist.github.com/tehpeh/7e5329d295eca9539e6462f36b6ce9c0

It's a bit out of date but the general idea would be the same for CentOS stream, Alpine etc: install Docker, enable the service, open firewall/networking, nfs mount a local directory. This is pretty much what Docker for Mac does.

If you're looking for the Docker hub image repository equivalent for FreeBSD, take a look at Bastille templates or Potluck (https://potluck.honeyguide.net/).

However, and this is only my personal opinion, a pre-baked container image repository is a bad idea. Apart from the security issues and recent drama around Docker shutting down free accounts, container images are often set up with default parameters not useful in a production environment (or even your specific dev environment) and are built against a particular kernel version, so may not run as expected on a different kernel version.

Again, only my opinion, but you're much better off building your own, private, images targeting the particular OS/Kernel version you use in dev/staging/production. In summary, prefer Dockerfiles over pre-built images.

I think the conversation we really need to have is not about copying Docker, but instead how do we consistently create, run, and scale jails across multiple FreeBSD hosts easily.

Tim


On 2/4/23 02:54, Paul Mather wrote:
On Mar 29, 2023, at 1:34 AM, John Levine <johnl@iecc.com> wrote:


It appears that Tomek CEDRO <tomek@cedro.info> said:

if there are lots of images for linux docker, and docker is linux only
solution, there is no reason to talk about it on bsd or even offer some
sort of images of bsd for linux right?

Docker runs on MacOS with a linux emulation layer.  FreeBSD already has
some linux emulation so in principle one could do the same thing, but
it'd be a lot of work for dubious benefit.

I disagree it would be of dubious benefit.  MacOS is a Tier 1 platform in the Docker ecosystem.  Using Docker Desktop on macOS makes using Docker and Kubernetes for development work very easy on that platform, meaning you can stay in the environment you prefer.  MacOS is not Linux, but the implementation on there is to use a shim Linux VM via the built-in macOS hypervisor (which, IIRC, is a derivative of bhyve).

It would be great if the same thing could be done on FreeBSD.  It would be beneficial if there was a supported docker machine driver for bhyve on FreeBSD.  Right now, I believe the road to running Linux containers on FreeBSD is to use the VirtualBox docker machine driver, which is a bit heavyweight (in terms of added dependencies) for my liking.  It would be nice if bhyve could be used to run the shim Linux VM.

Other than that, much of the tooling to run Docker and Kubernetes is already in ports.  But, those (e.g., in the case of Kubernetes) need to point to non-FreeBSD systems that are running the actual containers, pods, etc.  It would be nice to be able to do it all on FreeBSD, at least for development and kicking-the-tyres purposes.

Cheers,

Paul.



--
Mario.

--------------HoX8A1ohXtJRxbaflXIMx3LT--