From nobody Thu Jul 14 20:46:25 2022 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4LkRP43PRzz4T7wX for ; Thu, 14 Jul 2022 20:46:48 +0000 (UTC) (envelope-from nyan@myuji.xyz) Received: from wout4-smtp.messagingengine.com (wout4-smtp.messagingengine.com [64.147.123.20]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4LkRP346vbz3ZXW for ; Thu, 14 Jul 2022 20:46:47 +0000 (UTC) (envelope-from nyan@myuji.xyz) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.west.internal (Postfix) with ESMTP id 536A13200920 for ; Thu, 14 Jul 2022 16:46:46 -0400 (EDT) Received: from imap45 ([10.202.2.95]) by compute4.internal (MEProxy); Thu, 14 Jul 2022 16:46:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=myuji.xyz; h=cc :content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to; s=fm3; t=1657831605; x=1657918005; bh=XZjjUOfCH7 m3bikdYxST1JX3hX2XvPnlwOK30SsXY8k=; b=DMV5PcJmkqR1yzAe+58YFrPJOt jarPxPtvEIowJ1XwcggpsZhk9D9uJE3KTjwWRaDx9i11lPwMhjgZFzuHK5wq7vh+ T5Oe3qROh2abt15ZJWbxxgvr9/14De8GMfi+mc2yo4z6tDKnMWIGLqid/ufZD/uW pgGwyDYzUHqvNFrmkne4imZu4hLYKE/SFUTSBFmWLVNp0lZkqo7r6ZwPQjidXw3M J4JLmUNhAAL5bnGiD+PcrDAXKHFZKPt+fbHUvCG/fwjR85vQ76BjcAWcWsczzeym yMOxfS5MEaafBVnrplZiCD5x8DSbv2mpkv/1SEds6V53BWTH9R76R5JXnNhQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:date:feedback-id :feedback-id:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:sender:subject:subject:to:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; t=1657831605; x=1657918005; bh=XZjjUOfCH7m3bikdYxST1JX3hX2X vPnlwOK30SsXY8k=; b=OgvBTYLu37/8J6pfhF0q2ll04GkS1sZn7Sqz7W4oxCvG ZAIbfCCBCPt79I3AqhCrZ2URKHkUvshEEd7ZXB/0CXTW0l04iqt3Q7MXZigPe9Fr KPbXPU9OUcxNQL+wyw83AUxig9G5F4DwgjFqRsktgN/WDnxFxU//FcpTm+st1Yrh jZDy38M3Hd6e5BYmAyQ3/h84Uf7dYps77sM2jYumabbpJ9Z6tAAQR4yr1ZhAp5pI nQzGOBS8zbPZ8hRxGwBzhWQN4lupKRo/ub2nW77Zmy1FLcBL+pwdnDgsrOOFm6TZ bvyjmZA51J45oIdyUWDc2DPZfdcS26WgkyJxskvE3Q== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvfedrudejledgudehhecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecufghrlhcuvffnffculdefhedmnecujfgurhepof gfggfkjghffffhvffutgesrgdtreerreerjeenucfhrhhomhepfdfoihgthhgrvghlucgj rghnucfmrgcuvehhihhufdcuoehnhigrnhesmhihuhhjihdrgiihiieqnecuggftrfgrth htvghrnhepgefguedugfekveegffekudekjeeukedthedtvdffueetleegvdfgffeukedv ueehnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepnh ihrghnsehmhihujhhirdighiii X-ME-Proxy: Feedback-ID: i9dd946d0:Fastmail Received: by mailuser.nyi.internal (Postfix, from userid 501) id 80E3D2720078; Thu, 14 Jul 2022 16:46:45 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.7.0-alpha0-755-g3e1da8b93f-fm-20220708.002-g3e1da8b9 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 Message-Id: <01972a40-b8e3-494f-a3ca-a69c20f8fbee@www.fastmail.com> In-Reply-To: References: Date: Thu, 14 Jul 2022 22:46:25 +0200 From: "Michael Yan Ka Chiu" To: freebsd-hackers@freebsd.org Subject: Re: Dynamic VF mac address manipulation / Bhyve Content-Type: multipart/alternative; boundary=058093b9ed0d4b1dbddbc06093576862 X-Rspamd-Queue-Id: 4LkRP346vbz3ZXW X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=myuji.xyz header.s=fm3 header.b=DMV5PcJm; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=OgvBTYLu; dmarc=none; spf=pass (mx1.freebsd.org: domain of nyan@myuji.xyz designates 64.147.123.20 as permitted sender) smtp.mailfrom=nyan@myuji.xyz X-Spamd-Result: default: False [-3.59 / 15.00]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MID_RHS_WWW(0.50)[]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:64.147.123.20]; R_DKIM_ALLOW(-0.20)[myuji.xyz:s=fm3,messagingengine.com:s=fm3]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; RCVD_IN_DNSWL_LOW(-0.10)[64.147.123.20:from]; XM_UA_NO_VERSION(0.01)[]; RCVD_TLS_LAST(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; RCVD_COUNT_THREE(0.00)[4]; DMARC_NA(0.00)[myuji.xyz]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; ASN(0.00)[asn:29838, ipnet:64.147.123.0/24, country:US]; DKIM_TRACE(0.00)[myuji.xyz:+,messagingengine.com:+]; FROM_HAS_DN(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEFALL_USER(0.00)[nyan]; FROM_EQ_ENVFROM(0.00)[]; ARC_NA(0.00)[] X-ThisMailContainsUnwantedMimeParts: N --058093b9ed0d4b1dbddbc06093576862 Content-Type: text/plain;charset=utf-8 Content-Transfer-Encoding: quoted-printable On Thu, Jul 14, 2022, at 10:05 PM, John wrote: > Hi - >=20 > I am slowly converting a Linux based qemu/kvm solution to FreeBSD > Bhyve where VMs have mac addresses assigned to them. > I have tried the following steps. > # ifconfig ccv3 ether 02:22:33:44:55:66 AFAIK for SR-IOV, the PF driver is actually the one responsible to confi= gure the VF, so setting the mac address via the VF interface cannot crea= te a persistent mac address. > devctl set driver -f pci0:65:0:20 ppt >=20 > The VF device can now be passed into bhyve (65/0/20) but the new > value 02:22:33:44:55:66 is lost and the prior value 02:00:00:00:00:00 > is seen by the vm - but we want 02:22:33:44:55:66. So the new mac is > a property of the driver, not the VF. >=20 >=20 > At this point I'm wondering if adding a -U (update) option to iovctl > or a new standalone program to target an individual iface/vf pair > for update might be the answer. >=20 >=20 > Comments welcome. >=20 > Thanks, > John I=E2=80=99m just thinking out loud here. I have work on similar tools an= d have some thoughts on how to make VF more manageable. For example cur= rently there=E2=80=99s no way to really tell the pf of the vf actually b= elonging to, this is especially an issue with Sriov enabled nic as you c= an=E2=80=99t keep track of the physical port number. I think maybe a different ppt like driver (let=E2=80=99s just call it io= v for now) can be useful. Maybe with this iov device, we can have the P= F driver to register some hooks to reconfigure specific VF, show the top= ology between PFs and VFs and even allow dynamic creation of VF if the P= F driver can somehow support it? Best, Michael --058093b9ed0d4b1dbddbc06093576862 Content-Type: text/html;charset=utf-8 Content-Transfer-Encoding: quoted-printable
On Thu, Jul 14,= 2022, at 10:05 PM, John wrote:
Hi -

I am slowly conve= rting a Linux based qemu/kvm solution to FreeBSD
Bhyve whe= re VMs have mac addresses assigned to them.
<= br>
I have trie= d the following steps.
# ifconfig ccv3 ether 02:22:33:44:5= 5:66

AFAIK for SR-IOV, the PF = driver is actually the one responsible to configure the VF, so setting t= he mac address via the VF interface cannot create a persistent mac addre= ss.
devctl = set driver -f pci0:65:0:20 ppt

The VF devic= e can now be passed into bhyve (65/0/20) but the new
value= 02:22:33:44:55:66 is lost and the prior value 02:00:00:00:00:00
is seen by the vm - but we want 02:22:33:44:55:66. So the new mac= is
a property of the driver, not the VF.

At this point I'm wondering if adding a -U (= update) option to iovctl
or a new standalone program to ta= rget an individual iface/vf pair
for update might be the a= nswer.


Comments welcome.
=

Thanks,
John
I=E2=80=99m just thinking out loud here. I have work on similar = tools and have some thoughts on how to make  VF more manageable. Fo= r example currently there=E2=80=99s no way to really tell the pf of the = vf actually belonging to, this is especially an issue with Sriov enabled= nic as you can=E2=80=99t keep track of the physical port number.

I think maybe a different ppt like driver (let=E2= =80=99s just call it iov for now) can be useful. Maybe with this  i= ov device, we can have the PF driver to register some hooks to reconfigu= re specific VF, show the topology between PFs and VFs and even allow dyn= amic creation of VF if the PF driver can somehow support it?

Best,
Michael

--058093b9ed0d4b1dbddbc06093576862--