From nobody Sat Sep 14 02:45:03 2024 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4X5Fs76DcBz5WnR3 for ; Sat, 14 Sep 2024 02:45:15 +0000 (UTC) (envelope-from zlei@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4X5Fs75mf8z47nm; Sat, 14 Sep 2024 02:45:15 +0000 (UTC) (envelope-from zlei@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1726281915; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7dR1TVlJo1EFEbPoJ/VGdnJDKmwogwK3eP/zhEOSDPY=; b=ar1OYxMJUBr2EKJtTLv8roa9Cr7k1V9fKsvzj+EbrjDkJYJ4FOxWtD+7K5+U7a8Z5j4Fwa QcucJd6PhzLVy9qhmJpOa3AdfYTYp01mUIABKBWPHpBiZaL20+xsYEgHz/hHMqrPuTQGgi x46tND/sqhBALXd3ru4PibwCXlff3T4CA7s2uEUwZS7GRg5DFl4ujDbgXaIy1qSbzn0X+c 6N3UzehmondtaiFZoUXWFYZF5xuGHrK/vD2Uwzcbf1o6A7l0gSyyLFWpBS0aan6uIr0WZ4 KZSK2LN+hnt+tRZrRaGd6tl/9C2teT9IK3Ln9wiPZQnewzoTbRsHbgDUGAApNg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1726281915; a=rsa-sha256; cv=none; b=ZoJpJVCWptsAz2RFVUOcWTdhgf6ba+P1hrZpaZRgYZHYbJ/2/aldUV9vMnDXyHJaDo6C+g 0VSlmLBzdOm23V0ZHQA4BPjApla++/SjR0Y9OEWtChWm0GPIwTqtvudhuUV20wuTgm264E zdsbLwKinbKCpTygM+hmVylFsFg8SfqlKiMwQbj0X+MbZASm0iY3FX+a0VUzbSBh/hIcKZ PXWF6Qewwf69Kt+5TkagDzv5LKxOddMxTFivhqXjKCVgPhtQZ5twSpRtS1RUaoHkX/qK/V 9VFLTFF2pJAc5S9kBvdsOsw6NJKNAfEuy0DlouuHLAfKb+/7CAsQOgot5t3rzw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1726281915; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7dR1TVlJo1EFEbPoJ/VGdnJDKmwogwK3eP/zhEOSDPY=; b=BYsizEG99bIkMIDoKWROcjbPRyHRnLTLqiv4TgTS5zolmh93iXk5jdIzqII0lBFSFcVNEW SdlBdPFO+Unhtd5p8GbbMX/zFpeBX0nrlkLNLDlcG0MEIyD1EpTV1xq2lCCWJssGEOin5T iLTBULKymE6Z3Ud+iSv29Uc2Ah1NuNh/CGAx0mCYSdh85waIV4cyKJA0A5UuNzpAuyL5wB KObwOBxq0dRAVb/dt0zrV+IuBC+zmkUXjRK8RA6CmUPqo1Y3eHpF/mxJV76H0tEhAqf3iU xFDgnnngjs6Jn1U+1f0NI59YgDshmcdKUnrEi4gUH9OwbiSEtr7A7Ife66DXrQ== Received: from smtpclient.apple (unknown [IPv6:2001:19f0:6001:9db:98f0:9fe0:3545:10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: zlei/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4X5Fs55gMzzFjS; Sat, 14 Sep 2024 02:45:12 +0000 (UTC) (envelope-from zlei@FreeBSD.org) Content-Type: text/plain; charset=us-ascii List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.10\)) Subject: Re: Performance issues with vnet jails + epair + bridge From: Zhenlei Huang In-Reply-To: <20240913155439.1e171a88bd01ce9b97558a90@gmail.com> Date: Sat, 14 Sep 2024 10:45:03 +0800 Cc: Mark Saad , FreeBSD Net Content-Transfer-Encoding: quoted-printable Message-Id: References: <20240913100938.3eac55c9fbd976fa72d58bb5@gmail.com> <39B2C95D-1E4F-4133-8923-AD305DFA9435@longcount.org> <20240913155439.1e171a88bd01ce9b97558a90@gmail.com> To: Sad Clouds X-Mailer: Apple Mail (2.3696.120.41.1.10) > On Sep 13, 2024, at 10:54 PM, Sad Clouds = wrote: >=20 > On Fri, 13 Sep 2024 08:08:02 -0400 > Mark Saad wrote: >=20 >> Sad >> Can you go back a bit you mentioned there is a RPi in the mix ? = Some of the raspberries have their nic usb attached under the covers . = Which will kill the total speed of things.=20 >>=20 >> Can you cobble together a diagram of what you have on either end ? >=20 > Hello, I'm not sending data across the network, only between the host > and the jails. I'm trying to evaluate how FreeBSD handles TCP data > locally within a single host. When you take vnet into account, the **locally** traffic should within on single vnet jail. If you want traffic across vnet jails, if_epair or = netgraph hooks should be employed, and it of course will introduce some overhead. >=20 > I understand that vnet jails will have more overhead, compared to a > shared TCP/IP stack via localhost. So I'm trying to measure it and see > where the bottlenecks are. The overhead of vnet jail should be neglectable, compared to legacy jail or no-jail. Bare in mind when VIMAGE option is enabled, there is a = default vnet 0. It is not visible via jls and can not be destroyed. So when you = see bottlenecks, for example this case, it is mostly caused by other = components such as if_epair, but not the vnet jail itself. >=20 > The Raspberry Pi 4 host has a single vnet jail, exchanging data with > the host via epair(4) and if_bridge(4) interfaces. I don't really know > what topology FreeBSD is using to represent all this so can't draw any > diagrams, but I think all data flows through the kernel internally and > never leaves the physical network interface. For vnet jails, when you try to describe the network topology, you can treat them as VM / physical boxes. I have one box with dozens of vnet jails. Each of them has very single responsibility, e.g. DHCP, LADP, pf firewall, OOB access. The topology = looks quite clear and it is easy to maintenance. The only overhead is too much hops between the vnet jail instances. For my use case the performance is not critical and it works great for years. >=20 Best regards, Zhenlei