[Bug 205549] bhyve pci passthru stops working after guest is restarted

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=205549

--- Comment #20 from arkadyi <ark@dudevich.dn.ua> ---
Problem still persist on FreeBSD 14.0.
FreeBSD 14.0-RELEASE-p4 FreeBSD 14.0-RELEASE-p4 with custop kernel(without
xhci).
Hardware: ThinkPad T530(2392-C3G).

Before start VM:
ppt0@pci0:0:20:0:       class=0x0c0330 rev=0x04 hdr=0x00 vendor=0x8086
device=0x1e31 subvendor=0x17aa subdevice=0x21f6 vendor     = 'Intel
Corporation' device     = '7 Series/C210 Series Chipset Family USB xHCI
Host Controller' class      = serial bus
    subclass   = USB

I'm using vm-bhyve as frontend(vm-bhyve-1.5.0_1).
Vm is Linux Mint
Conf file:
loader="uefi"
uefi_vars="yes"
cpu=2
memory=4G
network0_type="virtio-net"
network1_type="virtio-net"
network0_switch="wan"
network1_switch="lan"
#disk0_type="ahci-hd"
disk0_type="nvme"
disk0_name="disk0.img"
disk1_type="virtio-9p"
disk1_name="home1=/home1" 
disk1_dev="custom"
graphics="yes"
graphics_port="5900"
graphics_res="1280x720"
graphics_wait="no"
xhci_mouse="yes"
#debug="yes"
passthru0="0/20/0"

uuid="3bf59fb9-2748-11ec-9d0d-3c970ea20e94"
network0_mac="58:9c:fc:0f:2f:fa"
network1_mac="58:9c:fc:0f:2f:fb"

I have two devices installed in the USB 3.0 interface connector(Logitech uaudio
and flash disk).

They are not visible on the host machine.

After startup, both devices are visible in the virtual machine.
[   75.873606] usb 1-2: new high-speed USB device number 3 using
xhci_hcd 
[   76.022651] usb 1-2: New USB device found, idVendor=058f, idProduct=6387,
bcdDevice= 1.00 
[   76.022660] usb 1-2: New USB device strings: Mfr=1, Product=2,
SerialNumber=3 
[   76.022663] usb 1-2:Product: Mass Storage Device [   76.022665] usb 1-2:
Manufacturer:JetFlash 
[   76.022667] usb 1-2: SerialNumber: I526KCO0
[   76.050014] usb-storage 1-2:1.0: USB Mass Storage device detected
[   76.051116] scsi host0: usb-storage 1-2:1.0
[   76.052927] usbcore: registered new interface driver usb-storage
[   76.060109] usbcore: registered new interface driver uas
[   77.062382] scsi 0:0:0:0: Direct-Access     JetFlash TS8GJF168
 8.07 PQ: 0 ANSI: 2 
[   77.063254] sd 0:0:0:0: Attached scsi generic
sg0 type 0 
[   77.064328] sd 0:0:0:0: [sda] 15974400 512-byte logical
blocks: (8.18 GB/7.62 GiB) 
[   77.064594] sd 0:0:0:0: [sda] Write
Protect is off 
[   77.064600] sd 0:0:0:0: [sda] Mode Sense: 03 00 00 00
[   77.064876] sd 0:0:0:0: [sda] No Caching mode page found
[   77.064885] sd 0:0:0:0: [sda] Assuming drive cache: write through
[   77.135379]  sda: sda1
[   77.137013] sd 0:0:0:0: [sda] Attached SCSI removable disk

If I destroy Linux VM(I mean kill process bhyve), second start Linux VM work
without problem.

If the Linux VM is turned off correctly, then both devices i see in the host
mashine.
usbd_req_re_enumerate: addr=3, set address failed! (USB_ERR_STALLED,
ignored) em0: link state changed to UP
usbd_setup_device_desc: getting device descriptor at addr 3 failed,
USB_ERR_STALLED ugen1.3: <Logitech product 0x0a29> at usbus1
uaudio0 on uhub3
uaudio0: <Logitech product 0x0a29, class 0/0, rev 2.00/76.57, addr 3>
on usbus1 uaudio0: Play[0]: 48000 Hz, 2 ch, 16-bit S-LE PCM format,
2x2ms buffer. uaudio0: Record[0]: 48000 Hz, 1 ch, 16-bit S-LE PCM
format, 2x2ms buffer. uaudio0: No MIDI sequencer.
pcm2: <USB audio> on uaudio0
uaudio0: HID volume keys found.
ugen1.3: <Logitech product 0x0a29> at usbus1 (disconnected)
uaudio0: at uhub3, port 1, addr 3 (disconnected)
pcm2: unregister: channel pcm2:virtual:dsp2.vp0 busy (pid 1863)
pcm2: Waiting for sound application to exit!
pcm2: detached
uaudio0: detached
ugen1.3: <Logitech Logitech Wireless Headset> at usbus1
uaudio0 on uhub3
uaudio0: <Logitech Logitech Wireless Headset, class 0/0, rev
2.00/76.57, addr 3> on usbus1 uaudio0: Play[0]: 48000 Hz, 2 ch, 16-bit
S-LE PCM format, 2x2ms buffer. uaudio0: Record[0]: 48000 Hz, 1 ch,
16-bit S-LE PCM format, 2x2ms buffer. uaudio0: No MIDI sequencer.
pcm2: <USB audio> on uaudio0
uaudio0: HID volume keys found.
ugen1.4: <JetFlash Mass Storage Device> at usbus1
umass0 on uhub3
umass0: <JetFlash Mass Storage Device, class 0/0, rev 2.00/1.00, addr
4> on usbus1 umass0:  SCSI over Bulk-Only; quirks = 0x4101  
umass0:5:0: Attached to scbus5
da0 at umass-sim0 bus 0 scbus5 target 0 lun 0
da0: <JetFlash TS8GJF168 8.07> Removable Direct Access SCSI-2 device
da0: Serial Number I526KCO0
da0: 40.000MB/s transfers
da0: 7800MB (15974400 512 byte sectors)
da0: quirks=0x2<NO_6_BYTE>

When restarting the Linux VM, these devices are not visible in the virtual
machine, although pciconf -lv shows the status unchanged.

ppt0@pci0:0:20:0:       class=0x0c0330 rev=0x04 hdr=0x00 vendor=0x8086
device=0x1e31 subvendor=0x17aa subdevice=0x21f6 vendor     = 'Intel
Corporation' device     = '7 Series/C210 Series Chipset Family USB xHCI
Host Controller' class      = serial bus
    subclass   = USB

This problem only occurs when running virtual machines with Linux. When
starting Windows VM, this problem does not exist.

-- 
You are receiving this mail because:
You are the assignee for the bug.