From nobody Sat Nov 27 09:22:24 2021 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id E0EE418A37C8; Sat, 27 Nov 2021 09:22:32 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4J1R3D4FPxz4tF2; Sat, 27 Nov 2021 09:22:32 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 1AR9MOjt024235 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Sat, 27 Nov 2021 11:22:27 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 1AR9MOjt024235 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 1AR9MOa5024234; Sat, 27 Nov 2021 11:22:24 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 27 Nov 2021 11:22:24 +0200 From: Konstantin Belousov To: Wei Hu Cc: src-committers@freebsd.org, dev-commits-src-all@freebsd.org, dev-commits-src-main@freebsd.org Subject: Re: git: 75412a521f60 - main - Hyper-V: vPCI: Prepopulate device bars Message-ID: References: <202111270655.1AR6tslh023868@gitrepo.freebsd.org> List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <202111270655.1AR6tslh023868@gitrepo.freebsd.org> X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.5 X-Spam-Checker-Version: SpamAssassin 3.4.5 (2021-03-20) on tom.home X-Rspamd-Queue-Id: 4J1R3D4FPxz4tF2 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On Sat, Nov 27, 2021 at 06:55:54AM +0000, Wei Hu wrote: > The branch main has been updated by whu: > > URL: https://cgit.FreeBSD.org/src/commit/?id=75412a521f60d4b0393c730ffb284e7c6ff9d2de > > commit 75412a521f60d4b0393c730ffb284e7c6ff9d2de > Author: Wei Hu > AuthorDate: 2021-11-27 06:42:34 +0000 > Commit: Wei Hu > CommitDate: 2021-11-27 06:42:34 +0000 > > Hyper-V: vPCI: Prepopulate device bars > > In recent Hyper-V releases on Windows Server 2022, vPCI code does not > initialize the last 4 bit of device bar registers. This behavior change > could result weird problems cuasing PCI code failure when configuring > bars. > > Just write all 1's to those bars whose probed values are not the same > as current read ones. This seems to make Hyper-V vPCI and > pci_write_bar() to cooperate correctly on these releases. Could you please elaborate both on the problem and on the solution more? It really makes not a lot of sense to me. Four low bits of BARs are space indicator, and for memory bar, 32/64 bitness and prefetch bit. It is some magic that, following your description, these values are wrong but writing 1's to them validate the value. Were other OSes affected by the bug? Do they use the same workaround? > > Reported by: khng@freebsd.org > Tested by: khng@freebsd.org > MFC after: 2 weeks > Sponsored by: Microsoft > --- > sys/dev/hyperv/pcib/vmbus_pcib.c | 43 ++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 43 insertions(+) > > diff --git a/sys/dev/hyperv/pcib/vmbus_pcib.c b/sys/dev/hyperv/pcib/vmbus_pcib.c > index 72e430c946db..c7df32044678 100644 > --- a/sys/dev/hyperv/pcib/vmbus_pcib.c > +++ b/sys/dev/hyperv/pcib/vmbus_pcib.c > @@ -1356,6 +1356,47 @@ _hv_pcifront_write_config(struct hv_pci_dev *hpdev, int where, int size, > } > } > > +/* > + * The vPCI in some Hyper-V releases do not initialize the last 4 > + * bit of BAR registers. This could result weird problems causing PCI > + * code fail to configure BAR correctly. > + * > + * Just write all 1's to those BARs whose probed values are not zero. > + * This seems to make the Hyper-V vPCI and pci_write_bar() to cooperate > + * correctly. > + */ > + > +static void > +vmbus_pcib_prepopulate_bars(struct hv_pcibus *hbus) > +{ > + struct hv_pci_dev *hpdev; > + int i; > + > + mtx_lock(&hbus->device_list_lock); > + TAILQ_FOREACH(hpdev, &hbus->children, link) { > + for (i = 0; i < 6; i++) { > + /* Ignore empty bar */ > + if (hpdev->probed_bar[i] == 0) > + continue; > + > + uint32_t bar_val = 0; > + > + _hv_pcifront_read_config(hpdev, PCIR_BAR(i), > + 4, &bar_val); > + > + if (hpdev->probed_bar[i] != bar_val) { > + if (bootverbose) > + printf("vmbus_pcib: initialize bar %d " > + "by writing all 1s\n", i); > + > + _hv_pcifront_write_config(hpdev, PCIR_BAR(i), > + 4, 0xffffffff); > + } > + } > + } > + mtx_unlock(&hbus->device_list_lock); > +} > + > static void > vmbus_pcib_set_detaching(void *arg, int pending __unused) > { > @@ -1479,6 +1520,8 @@ vmbus_pcib_attach(device_t dev) > if (ret) > goto vmbus_close; > > + vmbus_pcib_prepopulate_bars(hbus); > + > hbus->pci_bus = device_add_child(dev, "pci", -1); > if (!hbus->pci_bus) { > device_printf(dev, "failed to create pci bus\n");