From nobody Sat Jan 20 13:14:32 2024 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4THH564kSlz576FG for ; Sat, 20 Jan 2024 13:14:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4THH563ZJdz4gR5 for ; Sat, 20 Jan 2024 13:14:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1705756474; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zegybifrRzLgt0X+GiCTtWcE1Bf9d3ZXAb1QoN5QZxU=; b=J9V4ogP6f10j3pFcUvqf09vNnKokiTW+HbHMao8ScNDAJfYO7eTbbFPxQ0hNEtzpe8FYN+ W0P289kkWBFryIpO4RfQVtv7f/rfpG9RBXafjQRe2VFs1PQ91b0bLxN/BsgITzgitryySG XKHeF4yr7QERctYoggNGZnsHCTZqoqMdHCmM881NYhsJlJcbWl8YJ1ZMvc5F2RQYxhc4EN l2Medvva2bpW3kP6pvx9vXPXFyQ5LfZTrM4HD9A+BKWXPZfTdugfV6e7/lv0SG+vMsllxU JAPRLJAJnGLZ6ibXWIpnJ7FU992lxGnxaLrjXxM0GolSPGYspJA6Nki3hI30PQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1705756474; a=rsa-sha256; cv=none; b=u/JFOLG3MSBum7vdMhFuisXrn0up3je9hnBAfnoNNZatXBQky/629htlU/n9Pze8i3/Sy1 3YY+RRuQa0n62Q2XiqjFWrRa3LWpU/teZmpmKoz+UE1Jke8Nt23gy6aWzvJdFxUb0LmlmI eHUzsAFUpzCU4nT6RNyA6vm7GV2heeoDi98ExobqKl1WHlwaec2Fk8i4io9chQLfh8+JLI e3W0+zFgcm2cfzB/xQgks5Uz9AA/G17SfKO65H2pSCzI9aP6DPUSjHLwJjFGbQ/hu5jF+j STz2dAKS+Uco3VHiUUMm56LBpT9d5r0POb5C6pNBiNowxp2/3Q+VB/cG3ah34A== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4THH562f7Rzbvf for ; Sat, 20 Jan 2024 13:14:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 40KDEYX4062489 for ; Sat, 20 Jan 2024 13:14:34 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 40KDEYeI062488 for virtualization@FreeBSD.org; Sat, 20 Jan 2024 13:14:34 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 205549] bhyve pci passthru stops working after guest is restarted Date: Sat, 20 Jan 2024 13:14:32 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: ark@dudevich.dn.ua X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205549 --- Comment #20 from arkadyi --- Problem still persist on FreeBSD 14.0. FreeBSD 14.0-RELEASE-p4 FreeBSD 14.0-RELEASE-p4 with custop kernel(without xhci). Hardware: ThinkPad T530(2392-C3G). Before start VM: ppt0@pci0:0:20:0: class=3D0x0c0330 rev=3D0x04 hdr=3D0x00 vendor=3D0x8= 086 device=3D0x1e31 subvendor=3D0x17aa subdevice=3D0x21f6 vendor =3D 'Intel Corporation' device =3D '7 Series/C210 Series Chipset Family USB xHCI Host Controller' class =3D serial bus subclass =3D USB I'm using vm-bhyve as frontend(vm-bhyve-1.5.0_1). Vm is Linux Mint Conf file: loader=3D"uefi" uefi_vars=3D"yes" cpu=3D2 memory=3D4G network0_type=3D"virtio-net" network1_type=3D"virtio-net" network0_switch=3D"wan" network1_switch=3D"lan" #disk0_type=3D"ahci-hd" disk0_type=3D"nvme" disk0_name=3D"disk0.img" disk1_type=3D"virtio-9p" disk1_name=3D"home1=3D/home1"=20 disk1_dev=3D"custom" graphics=3D"yes" graphics_port=3D"5900" graphics_res=3D"1280x720" graphics_wait=3D"no" xhci_mouse=3D"yes" #debug=3D"yes" passthru0=3D"0/20/0" uuid=3D"3bf59fb9-2748-11ec-9d0d-3c970ea20e94" network0_mac=3D"58:9c:fc:0f:2f:fa" network1_mac=3D"58:9c:fc:0f:2f:fb" I have two devices installed in the USB 3.0 interface connector(Logitech ua= udio and flash disk). They are not visible on the host machine. After startup, both devices are visible in the virtual machine. [ 75.873606] usb 1-2: new high-speed USB device number 3 using xhci_hcd=20 [ 76.022651] usb 1-2: New USB device found, idVendor=3D058f, idProduct=3D= 6387, bcdDevice=3D 1.00=20 [ 76.022660] usb 1-2: New USB device strings: Mfr=3D1, Product=3D2, SerialNumber=3D3=20 [ 76.022663] usb 1-2:Product: Mass Storage Device [ 76.022665] usb 1-2: Manufacturer:JetFlash=20 [ 76.022667] usb 1-2: SerialNumber: I526KCO0 [ 76.050014] usb-storage 1-2:1.0: USB Mass Storage device detected [ 76.051116] scsi host0: usb-storage 1-2:1.0 [ 76.052927] usbcore: registered new interface driver usb-storage [ 76.060109] usbcore: registered new interface driver uas [ 77.062382] scsi 0:0:0:0: Direct-Access JetFlash TS8GJF168 8.07 PQ: 0 ANSI: 2=20 [ 77.063254] sd 0:0:0:0: Attached scsi generic sg0 type 0=20 [ 77.064328] sd 0:0:0:0: [sda] 15974400 512-byte logical blocks: (8.18 GB/7.62 GiB)=20 [ 77.064594] sd 0:0:0:0: [sda] Write Protect is off=20 [ 77.064600] sd 0:0:0:0: [sda] Mode Sense: 03 00 00 00 [ 77.064876] sd 0:0:0:0: [sda] No Caching mode page found [ 77.064885] sd 0:0:0:0: [sda] Assuming drive cache: write through [ 77.135379] sda: sda1 [ 77.137013] sd 0:0:0:0: [sda] Attached SCSI removable disk If I destroy Linux VM(I mean kill process bhyve), second start Linux VM work without problem. If the Linux VM is turned off correctly, then both devices i see in the host mashine. usbd_req_re_enumerate: addr=3D3, set address failed! (USB_ERR_STALLED, ignored) em0: link state changed to UP usbd_setup_device_desc: getting device descriptor at addr 3 failed, USB_ERR_STALLED ugen1.3: at usbus1 uaudio0 on uhub3 uaudio0: on usbus1 uaudio0: Play[0]: 48000 Hz, 2 ch, 16-bit S-LE PCM format, 2x2ms buffer. uaudio0: Record[0]: 48000 Hz, 1 ch, 16-bit S-LE PCM format, 2x2ms buffer. uaudio0: No MIDI sequencer. pcm2: on uaudio0 uaudio0: HID volume keys found. ugen1.3: at usbus1 (disconnected) uaudio0: at uhub3, port 1, addr 3 (disconnected) pcm2: unregister: channel pcm2:virtual:dsp2.vp0 busy (pid 1863) pcm2: Waiting for sound application to exit! pcm2: detached uaudio0: detached ugen1.3: at usbus1 uaudio0 on uhub3 uaudio0: on usbus1 uaudio0: Play[0]: 48000 Hz, 2 ch, 16-bit S-LE PCM format, 2x2ms buffer. uaudio0: Record[0]: 48000 Hz, 1 ch, 16-bit S-LE PCM format, 2x2ms buffer. uaudio0: No MIDI sequencer. pcm2: on uaudio0 uaudio0: HID volume keys found. ugen1.4: at usbus1 umass0 on uhub3 umass0: on usbus1 umass0: SCSI over Bulk-Only; quirks =3D 0x4101=20=20 umass0:5:0: Attached to scbus5 da0 at umass-sim0 bus 0 scbus5 target 0 lun 0 da0: Removable Direct Access SCSI-2 device da0: Serial Number I526KCO0 da0: 40.000MB/s transfers da0: 7800MB (15974400 512 byte sectors) da0: quirks=3D0x2 When restarting the Linux VM, these devices are not visible in the virtual machine, although pciconf -lv shows the status unchanged. ppt0@pci0:0:20:0: class=3D0x0c0330 rev=3D0x04 hdr=3D0x00 vendor=3D0x8= 086 device=3D0x1e31 subvendor=3D0x17aa subdevice=3D0x21f6 vendor =3D 'Intel Corporation' device =3D '7 Series/C210 Series Chipset Family USB xHCI Host Controller' class =3D serial bus subclass =3D USB This problem only occurs when running virtual machines with Linux. When starting Windows VM, this problem does not exist. --=20 You are receiving this mail because: You are the assignee for the bug.=