From nobody Sat Mar 12 20:00:13 2022 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 575B31A1C614 for ; Sat, 12 Mar 2022 20:00:32 +0000 (UTC) (envelope-from grembo@freebsd.org) Received: from mail.evolve.de (mail.evolve.de [213.239.217.29]) (using TLSv1.3 with cipher TLS_CHACHA20_POLY1305_SHA256 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA512 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mail.evolve.de", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4KGDDv1pZhz3LRT; Sat, 12 Mar 2022 20:00:31 +0000 (UTC) (envelope-from grembo@freebsd.org) Received: by mail.evolve.de (OpenSMTPD) with ESMTP id 849c85d1; Sat, 12 Mar 2022 20:00:22 +0000 (UTC) Received: by mail.evolve.de (OpenSMTPD) with ESMTPSA id fcb53d28 (TLSv1.3:AEAD-CHACHA20-POLY1305-SHA256:256:NO); Sat, 12 Mar 2022 20:00:18 +0000 (UTC) Date: Sat, 12 Mar 2022 21:00:13 +0100 From: Michael Gmelin To: Johan Hendriks Cc: Kristof Provost , Michael Gmelin , freebsd-net@freebsd.org, "Patrick M. Hausen" Subject: Re: epair and vnet jail loose connection. Message-ID: <20220312210013.5cc573e8.grembo@freebsd.org> In-Reply-To: References: <41ED1534-5E98-4D46-A562-811E80F82C5F@FreeBSD.org> <43AA6B37-6235-4787-A03F-B4C264C75A58@freebsd.org> X-Face: $wrgCtfdVw_H9WAY?S&9+/F"!41z'L$uo*WzT8miX?kZ~W~Lr5W7v?j0Sde\mwB&/ypo^}> +a'4xMc^^KroE~+v^&^#[B">soBo1y6(TW6#UZiC]o>C6`ej+i Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAJFBMVEWJBwe5BQDl LASZU0/LTEWEfHbyj0Txi32+sKrp1Mv944X8/fm1rS+cAAAACXBIWXMAAAsTAAAL EwEAmpwYAAAAB3RJTUUH3wESCxwC7OBhbgAAACFpVFh0Q29tbWVudAAAAAAAQ3Jl YXRlZCB3aXRoIFRoZSBHSU1QbbCXAAAAAghJREFUOMu11DFvEzEUAGCfEhBVFzuq AKkLd0O6VrIQsLXVSZXoWE5N1K3DobBBA9fQpRWc8OkWouaIjedWKiyREOKs+3PY fvalCNjgLVHeF7/3bMtBzV8C/VsQ8tecEgCcDgrzjekwKZ7TwsJZd/ywEKwwP+ZM 8P3drTsAwWn2mpWuDDuYiK1bFs6De0KUUFw0tWxm+D4AIhuuvZqtyWYeO7jQ4Aea 7jUqI+ixhQoHex4WshEvSXdood7stlv4oSuFOC4tqGcr0NjEqXgV4mMJO38nld4+ xKNxRDon7khyKVqY7YR4d+Cg0OMrkWXZOM7YDkEfKiilCn1qYv4mighZiynuHHOA Wq9QJq+BIES7lMFUtcikMnkDGHUoncA+uHgrP0ctIEqfwLHzeSo+eUA66AqzwN6n 2ZHJhw6Qh/PoyC/QENyEyC/AyNjq74Bs+3UH0xYwzDUC4B97HgLocg1QLYgDDO1v f3UX9Y307Ew4AHh67YAFFsxEpkXwpXY3eIgMhAAE3R19L919nNnuD2wlPcDE3UeT L2ytEICQib9BXgS2fU8PrD82ToYO1OEmMSnYTjSqSv9wdC0tPYC+rQRQD9ESnldF CyqfmiYW+tlALt8gH2xrMdC/youbjzPXEun+/ReXsMCDyve3dZc09fn2Oas8oXGc Jj6/fOeK5UmSMPmf/jL+GD8BEj0k/Fn6IO4AAAAASUVORK5CYII= List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4KGDDv1pZhz3LRT X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=softfail (mx1.freebsd.org: 213.239.217.29 is neither permitted nor denied by domain of grembo@freebsd.org) smtp.mailfrom=grembo@freebsd.org X-Spamd-Result: default: False [-0.59 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; FREEFALL_USER(0.00)[grembo]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999]; NEURAL_HAM_LONG(-1.00)[-1.000]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[freebsd.org]; R_SPF_SOFTFAIL(0.00)[~all]; RCPT_COUNT_FIVE(0.00)[5]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.99)[-0.995]; MID_CONTAINS_FROM(1.00)[]; MLMMJ_DEST(0.00)[freebsd-net]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:213.239.192.0/18, country:DE]; RCVD_TLS_ALL(0.00)[]; SUSPICIOUS_RECIPS(1.50)[] X-ThisMailContainsUnwantedMimeParts: N Hi Johan, I created a setup similar to yours - so it's based on your config files, but I had to tweak a few things to make it work. Basically this what it looks like: Host: Bare metal Runs bhyve, bridge has 10.1.1.1, does NAT to the world Bhyve VM runs 14-CURRENT (latest snapshot and has). vtnet0 - 10.1.1.16/24 - 10.1.1.17/32 bridge0: - ip: 10.233.185.1/24 - members: epair18a, epair20a jails: - haproxy.test.nl ip: 10.233.185.20 haproxy: listens on 443 (alpn h2,http/1.1 tls), backend to 10.233.185.18 port 80 binat: 10.1.1.16 - web01.test.nl ip: 10.233.185.18 nginx: listens on 80 and serves the static page from your example binat: 10.1.1.17 Runing hey on this setup from the bare metal host that hosts the bhyve vm works ok: # hey -h2 -n 10 -c 10 -z 300s https://10.1.1.16 Summary: Total: 300.0030 secs Slowest: 5.0101 secs Fastest: 0.0013 secs Average: 0.0039 secs Requests/sec: 2582.7142 Total data: 110024724 bytes On Sat, 12 Mar 2022 15:18:38 +0100 Johan Hendriks wrote: > .... > mount.devfs; > sysvshm="new"; > sysvsem="new"; > allow.raw_sockets; > allow.set_hostname = 0; > allow.sysvipc; > enforce_statfs = "2"; > devfs_ruleset = "11"; What is in devfs_ruleset 11? (it's not a standard one), I used "4" in my tests. > > path = "/storage/jails/${name}"; > host.hostname = "${name}.${domain}"; > > > web01 { > $ip = 18; > } I changed web01 to be the same setup as haproxy (that is, a full jail based in /storage/jails/${name}), as I didn't really know how it worked in your setup. > > haproxy { > $ip = 20; > mount.fstab = ""; > path = "/storage/jails/${name}"; > } Best Michael -- Michael Gmelin