[Bug 244363] Assertion failed: error == 0, file pci_emul.c, line 517, function modify_bar_registration

[bugzilla-noreply at freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=244363

            Bug ID: 244363
           Summary: Assertion failed: error == 0, file pci_emul.c, line
                    517, function modify_bar_registration
           Product: Base System
           Version: 12.0-STABLE
          Hardware: amd64
                OS: Any
            Status: New
          Severity: Affects Some People
          Priority: ---
         Component: bhyve
          Assignee: virtualization at FreeBSD.org
          Reporter: sjorge+signup at blackdot.be

I'm running into an issue when using PCI passthrough with bhyve.

I am running into this on SmartOS but when googling the problem I found a few
FreeBSD users also running into the same problem. After chatting with Michael
Dexter, I as suggested to also fine a bug bere.


List of other people hitting the problem on FreeBSD:
-
https://forums.freebsd.org/threads/vm-bhyve-windows-2012-r2-and-passthru.60832/
-
https://www.ixsystems.com/community/threads/bhyve-pci-passthrough-errors.57284/
-
https://forums.freebsd.org/threads/bhyve-passthru-issue-with-lsi-logic-sas2008--
pci-express-fusion-mpt-sas-2.65269/
- bug #211062, comment #8

SmartOS bug report:
- https://github.com/joyent/smartos-live/issues/901

Bhyve is dumping core when the following assert is tripped:
Assertion failed: error == 0, file pci_emul.c, line 517, function
modify_bar_registration

This is only tripped when the guest OS is windows (10). I do not have issues
with a freebsd, illumos, or linux guest. So something windows is doing
triggeres it.

I also noticed I am only getting this on PCIe devices that have multiple BAR
entries.

I captured the output in a FreeBSD guest that got the PCIe devices via
passthrough, you can see they have 2 BARs. ( Found of no easy way to capture
this info on SmartOS, but this is probably easier for here anyway )

```
ixl0 at pci0:0:8:0:        class=0x020000 card=0x37d215d9 chip=0x37d28086 rev=0x09
hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Connection X722 for 10GBASE-T'
    class      = network
    subclass   = ethernet
    bar   [10] = type Prefetchable Memory, range 64, base 0xc1000000, size
16777216, enabled
    bar   [1c] = type Prefetchable Memory, range 64, base 0xc2000000, size
32768, enabled
    cap 01[40] = powerspec 3  supports D0 D3  current D0
    cap 05[50] = MSI supports 1 message, 64 bit, vector masks
    cap 11[70] = MSI-X supports 8 messages, enabled
                 Table in map 0x1c[0x0], PBA in map 0x1c[0x1000]
    cap 10[a0] = PCI-Express 2 endpoint max data 256(512) FLR RO
                 link x1(x1) speed 2.5(2.5) ASPM disabled(L0s/L1)
    cap 03[e0] = VPD
    VPD ident  = 'Example VPD'
ixl1 at pci0:0:8:1:        class=0x020000 card=0x37d215d9 chip=0x37d28086 rev=0x09
hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Connection X722 for 10GBASE-T'
    class      = network
    subclass   = ethernet
    bar   [10] = type Prefetchable Memory, range 64, base 0xc3000000, size
16777216, enabled
    bar   [1c] = type Prefetchable Memory, range 64, base 0xc4000000, size
32768, enabled
    cap 01[40] = powerspec 3  supports D0 D3  current D0
    cap 05[50] = MSI supports 1 message, 64 bit, vector masks
    cap 11[70] = MSI-X supports 8 messages, enabled
                 Table in map 0x1c[0x0], PBA in map 0x1c[0x1000]
    cap 10[a0] = PCI-Express 2 endpoint max data 256(512) FLR RO
                 link x1(x1) speed 2.5(2.5) ASPM disabled(L0s/L1)
    cap 03[e0] = VPD
    VPD ident  = 'Example VPD'
```

Michael Zeller who is also on the bhyve weekly called help me debug this a bit
(on SmartOS)

We traced it to `Also the call to unregister_mem() ends up seeing an ENOENT
from mmio_rb_lookup(&mmio_rb_root, memp->base, &entry);`

mmio_rb_lookup is macro that we (illumos) copied from FreeBSD and it wasn't
changed. So it would make sense that there is indeed a bug in there both
FreeBSd and SmartOS bhyve users would hit the same error.

-- 
You are receiving this mail because:
You are the assignee for the bug.