bhyve pci passthrough usb controller

Fri Mar 23 17:25:13 UTC 2018

On 23/3/18 11:40 pm, Wouter Snels wrote:

new wrapping is no better
It looks like the device you are passing though is not removed from 
freebsd first.
but it's hard to tell.

> |Hi, .. I'm trying to pci passhthrough a usb controller to a bhyve
> windows 7 virtualization. When I start the virtual machine it freezes
> the entire system and this shows up on the console. To be clear the
> system runs without any problems without the pci passthrough. |
> ||(ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 08 e0 ff 40 40 29 00
> 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was invalid
> (ada0:ahcich0:0:0:0): Error 22, Unretryable error
> (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00
> 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was invalid
> (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ada0 at ahcich0 bus
> 0 scbus0 target 0 lun 0 ada0: <ST500DM002-1SB10A CC43> s/n ZA4292G6
> detached (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40
> 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was
> invalid (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0:
> Poll timeout on slot 28 port 0 ahcich0: is 20000000 cs 10000000 ss
> 00000000 rs 10000000 tfd 1d0 serr 00000800 cmd 00005c17
> (aprobe0:ahcich0:0:0:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00
> 00 00 (aprobe0:ahcich0:0:0:0): CAM status: Command timeout
> (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted
> (ada0:ahcich0:0:0:0): SETFEATURES ENABLE RCACHE. ACB: ef aa 00 00 00 40
> 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was
> invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: Poll
> timeout on slot 30 port 0 ahcich0: is 20000000 cs 40000000 ss 00000000
> rs 40000000 tfd 1d0 serr 00000800 cmd 00005e17 (aprobe0:ahcich0:0:0:0):
> SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 00 00
> (aprobe0:ahcich0:0:0:0): CAM status: Command timeout
> (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted
> (ada0:ahcich0:0:0:0): SETFEATURES ENABLE WCACHE. ACB: ef 02 00 00 00 40
> 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was
> invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error
> (ada0:ahcich0:0:0:0): Periph destroyed em0: Watchdog timeout Queue[0]--
> resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw
> tdh = 428, hw tdt = 471 em0: Tx Queue Status = -2147483648 em0: TX
> descriptors avail = 981 em0: Tx Descriptors avail failure = 0 em0: RX
> Queue 0 ------ em0: hw rdh = 912, hw rdt = 908 em0: RX discarded packets
> = 0 em0: RX Next to Check = 909 em0: RX Next to Refresh = 908 xl0:
> watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface is
> RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh = 0, hw tdt = 58
> em0: Tx Queue Status = -2147483648 em0: TX descriptors avail = 966 em0:
> Tx Descriptors avail failure = 0 em0: RX Queue 0 ------ em0: hw rdh = 0,
> hw rdt = 1023 em0: RX discarded packets = 0 em0: RX Next to Check = 0
> em0: RX Next to Refresh = 1023 em0: Watchdog timeout Queue[0]--
> resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw
> tdh = 0, hw tdt = 58 em0: Tx Queue Status = -2147483648 em0: TX
> descriptors avail = 966 em0: Tx Descriptors avail failure = 0 em0: RX
> Queue 0 ------ em0: hw rdh = 0, hw rdt = 1023 em0: RX discarded packets
> = 0 em0: RX Next to Check = 0 em0: RX Next to Refresh = 1023 xl0:
> watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface is
> RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh = 0, hw tdt = 54
> em0: Tx Queue Status = -2147483648 em0: TX descriptors avail = 970 em0:
> Tx Descriptors avail failure = 0 em0: RX Queue 0 ------ em0: hw rdh = 0,
> hw rdt = 1023 em0: RX discarded packets = 0include GENERIC ident OFL
> options IPFIREWALL||||||
>
> |Hi, .. I'm trying to pci passhthrough a usb controller to a bhyve
> windows 7 virtualization. When I start the virtual machine it freezes
> the entire system and this shows up on the console. To be clear the
> system runs without any problems without the pci passthrough. |
> ||(ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 08 e0 ff 40 40 29 00
> 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was invalid
> (ada0:ahcich0:0:0:0): Error 22, Unretryable error
> (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00
> 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was invalid
> (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ada0 at ahcich0 bus
> 0 scbus0 target 0 lun 0 ada0: <ST500DM002-1SB10A CC43> s/n ZA4292G6
> detached (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40
> 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was
> invalid (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0:
> Poll timeout on slot 28 port 0 ahcich0: is 20000000 cs 10000000 ss
> 00000000 rs 10000000 tfd 1d0 serr 00000800 cmd 00005c17
> (aprobe0:ahcich0:0:0:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00
> 00 00 (aprobe0:ahcich0:0:0:0): CAM status: Command timeout
> (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted
> (ada0:ahcich0:0:0:0): SETFEATURES ENABLE RCACHE. ACB: ef aa 00 00 00 40
> 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was
> invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: Poll
> timeout on slot 30 port 0 ahcich0: is 20000000 cs 40000000 ss 00000000
> rs 40000000 tfd 1d0 serr 00000800 cmd 00005e17 (aprobe0:ahcich0:0:0:0):
> SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 00 00
> (aprobe0:ahcich0:0:0:0): CAM status: Command timeout
> (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted
> (ada0:ahcich0:0:0:0): SETFEATURES ENABLE WCACHE. ACB: ef 02 00 00 00 40
> 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was
> invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error
> (ada0:ahcich0:0:0:0): Periph destroyed em0: Watchdog timeout Queue[0]--
> resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw
> tdh = 428, hw tdt = 471 em0: Tx Queue Status = -2147483648 em0: TX
> descriptors avail = 981 em0: Tx Descriptors avail failure = 0 em0: RX
> Queue 0 ------ em0: hw rdh = 912, hw rdt = 908 em0: RX discarded packets
> = 0 em0: RX Next to Check = 909 em0: RX Next to Refresh = 908 xl0:
> watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface is
> RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh = 0, hw tdt = 58
> em0: Tx Queue Status = -2147483648 em0: TX descriptors avail = 966 em0:
> Tx Descriptors avail failure = 0 em0: RX Queue 0 ------ em0: hw rdh = 0,
> hw rdt = 1023 em0: RX discarded packets = 0 em0: RX Next to Check = 0
> em0: RX Next to Refresh = 1023 em0: Watchdog timeout Queue[0]--
> resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw
> tdh = 0, hw tdt = 58 em0: Tx Queue Status = -2147483648 em0: TX
> descriptors avail = 966 em0: Tx Descriptors avail failure = 0 em0: RX
> Queue 0 ------ em0: hw rdh = 0, hw rdt = 1023 em0: RX discarded packets
> = 0 em0: RX Next to Check = 0 em0: RX Next to Refresh = 1023 xl0:
> watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface is
> RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh = 0, hw tdt = 54
> em0: Tx Queue Status = -2147483648 em0: TX descriptors avail = 970 em0:
> Tx Descriptors avail failure = 0 em0: RX Queue 0 ------ em0: hw rdh = 0,
> hw rdt = 1023 em0: RX discarded packets = 0include GENERIC ident OFL
> options IPFIREWALL options IPFIREWALL_VERBOSE options
> IPFIREWALL_DEFAULT_TO_ACCEPT options DUMMYNET options IPDIVERT options
> CONSPEED=115200 em0: RX Next to Check = 0 em0: RX Next to Refresh = 1023 ||||
> |||I've recompiled the kernel without the xhci driver. I've tested both
> usb controllers. Both "|||||Sunrise Point-H USB 3.0 xHCI
> Controller|"||||| and "|||uPD720202 USB 3.0 Host Controller|", both
> showed the same error on the console screen.| # uname FreeBSD
> 11.1-STABLE #1 r331092M # custom kernel include GENERIC ident OFL
> options IPFIREWALL options IPFIREWALL_VERBOSE options
> IPFIREWALL_DEFAULT_TO_ACCEPT options DUMMYNET options IPDIVERT options
> CONSPEED=115200 * note commented "device xhci" in generic # svn info |||Path: . Working Copy Root Path: /usr/src URL:
> https://svn0.eu.freebsd.org/base/stable/11 Relative URL: ^/stable/11
> Repository Root: https://svn0.eu.freebsd.org/base Repository UUID:
> ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f Revision: 331092 Node Kind:
> directory Schedule: normal Last Changed Author: eadler Last Changed Rev:
> 331092 Last Changed Date: 2018-03-17 07:44:39 +0100 (Sat, 17 Mar 2018)
> |# vm passthru DEVICE BHYVE ID READY DESCRIPTION hostb0 0/0/0 No Xeon
> E3-1200 v5/E3-1500 v5/6th Gen Core Processor Host Bridge/DRAM Registers
> pcib1 0/1/0 No Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe
> Controller (x16) vgapci0 0/2/0 No HD Graphics 530 |||ppt0| 0/20/0 Yes Sunrise Point-H USB 3.0 xHCI Controller none0 0/20/2
> No Sunrise Point-H Thermal subsystem none1 0/22/0 No Sunrise Point-H
> CSME HECI ahci0 0/23/0 No Sunrise Point-H SATA controller [AHCI mode]
> pcib2 0/28/0 No Sunrise Point-H PCI Express Root Port pcib4 0/28/6 No
> Sunrise Point-H PCI Express Root Port isab0 0/31/0 No Sunrise Point-H
> LPC Controller none2 0/31/2 No Sunrise Point-H PMC hdac0 0/31/3 No
> Sunrise Point-H HD Audio none3 0/31/4 No Sunrise Point-H SMBus em0
> 0/31/6 No Ethernet Connection (2) I219-V pcib3 2/0/0 No XIO2001 PCI
> Express-to-PCI Bridge xl0 3/0/0 No 3c905B 100BaseTX [Cyclone] ppt1 4/0/0
> Yes uPD720202 USB 3.0 Host Controller ||ppt1 at pci0:4:0:0: class=0x0c0330 card=0x00151912 chip=0x00151912 rev=0x02
> hdr=0x00 vendor = 'Renesas Technology Corp.' device = 'uPD720202 USB 3.0
> Host Controller' class = serial bus subclass = USB |||ppt|0 at pci0:0:20:0: class=0x0c0330 card=0x06ba1028 chip=0xa12f8086
> rev=0x31 hdr=0x00 vendor = 'Intel Corporation' device = 'Sunrise Point-H
> USB 3.0 xHCI Controller' class = serial bus subclass = USB # grep -i
> pptdevfs loader.conf ||pptdevs="0/20/0 4/0/0" # vm configure ||passthru0="0/20/0"|
>
> |# vm configure ||passthru0="4/0/0"|
> ||Any suggestions on what I'm doing wrong?||
>   
> Regards, Ofloo
> ||||||
>
> ||options IPFIREWALL_VERBOSE options IPFIREWALL_DEFAULT_TO_ACCEPT options
> DUMMYNET options IPDIVERT options CONSPEED=115200 em0: RX Next to Check
> = 0 em0: RX Next to Refresh = 1023 ||||
> |||I've recompiled the kernel without the xhci driver. I've tested both
> usb controllers. Both "|||||Sunrise Point-H USB 3.0 xHCI
> Controller|"||||| and "|||uPD720202 USB 3.0 Host Controller|", both
> showed the same error on the console screen.| # uname FreeBSD
> 11.1-STABLE #1 r331092M # custom kernel include GENERIC ident OFL
> options IPFIREWALL options IPFIREWALL_VERBOSE options
> IPFIREWALL_DEFAULT_TO_ACCEPT options DUMMYNET options IPDIVERT options
> CONSPEED=115200 * note commented "device xhci" in generic # svn info |||Path: . Working Copy Root Path: /usr/src URL:
> https://svn0.eu.freebsd.org/base/stable/11 Relative URL: ^/stable/11
> Repository Root: https://svn0.eu.freebsd.org/base Repository UUID:
> ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f Revision: 331092 Node Kind:
> directory Schedule: normal Last Changed Author: eadler Last Changed Rev:
> 331092 Last Changed Date: 2018-03-17 07:44:39 +0100 (Sat, 17 Mar 2018)
> |# vm passthru DEVICE BHYVE ID READY DESCRIPTION hostb0 0/0/0 No Xeon
> E3-1200 v5/E3-1500 v5/6th Gen Core Processor Host Bridge/DRAM Registers
> pcib1 0/1/0 No Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe
> Controller (x16) vgapci0 0/2/0 No HD Graphics 530 |||ppt0| 0/20/0 Yes Sunrise Point-H USB 3.0 xHCI Controller none0 0/20/2
> No Sunrise Point-H Thermal subsystem none1 0/22/0 No Sunrise Point-H
> CSME HECI ahci0 0/23/0 No Sunrise Point-H SATA controller [AHCI mode]
> pcib2 0/28/0 No Sunrise Point-H PCI Express Root Port pcib4 0/28/6 No
> Sunrise Point-H PCI Express Root Port isab0 0/31/0 No Sunrise Point-H
> LPC Controller none2 0/31/2 No Sunrise Point-H PMC hdac0 0/31/3 No
> Sunrise Point-H HD Audio none3 0/31/4 No Sunrise Point-H SMBus em0
> 0/31/6 No Ethernet Connection (2) I219-V pcib3 2/0/0 No XIO2001 PCI
> Express-to-PCI Bridge xl0 3/0/0 No 3c905B 100BaseTX [Cyclone] ppt1 4/0/0
> Yes uPD720202 USB 3.0 Host Controller ||ppt1 at pci0:4:0:0: class=0x0c0330 card=0x00151912 chip=0x00151912 rev=0x02
> hdr=0x00 vendor = 'Renesas Technology Corp.' device = 'uPD720202 USB 3.0
> Host Controller' class = serial bus subclass = USB |||ppt|0 at pci0:0:20:0: class=0x0c0330 card=0x06ba1028 chip=0xa12f8086
> rev=0x31 hdr=0x00 vendor = 'Intel Corporation' device = 'Sunrise Point-H
> USB 3.0 xHCI Controller' class = serial bus subclass = USB # grep -i
> pptdevfs loader.conf ||pptdevs="0/20/0 4/0/0" # vm configure ||passthru0="0/20/0"|
>
> |# vm configure ||passthru0="4/0/0"|
> ||Any suggestions on what I'm doing wrong?||
>
> edit: previous wrapping was all wrong.
>    
> Regards, Ofloo
> ||||||
>
>