[Bug 222937] [byhve] Severe RAM corruption after PciPassThrough-guest shutdown

[bugzilla-noreply at freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=222937

            Bug ID: 222937
           Summary: [byhve] Severe RAM corruption after
                    PciPassThrough-guest shutdown
           Product: Base System
           Version: 11.1-STABLE
          Hardware: Any
                OS: Any
            Status: New
          Severity: Affects Some People
          Priority: ---
         Component: kern
          Assignee: freebsd-bugs at FreeBSD.org
          Reporter: bugzilla.freebsd at omnilan.de

Created attachment 187086
  --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=187086&action=edit
/etc/rc.d/pciptdetach: Work around RAM corruption at guest shutdown

Various different panics will happen after shutting down a bhyve guest with
PCIe passthrough NICs if some conditions are met.

This can lead to completely destroyed zpools, like I found out after some
dozend not so impacting crashes...

Quoting jhb@:
I suspect what is happening is that the PCI devices are still issuing DMAs
after the guest has been shutdown which end up trashing other parts of host
memory.  This may somewhat be my fault as I made a change which moves the
device back into the host domain after FLR during guest shutdown.  I should
perhaps leave the device disabled in the DMAR table instead if the FLR
doesn't succeed.  (We could also add some other forms of reset for devices
not supporting FLR.)
</quote>

Since I don't have the skills to help fixing the root cause, I wrote a little
workaround in form of a rc(8) script (to be copied to /etc/rc.d) which should
protect against accidental crashes and data losses, by bringing the
PciPassThrough devices down before shutting down, which prevents DMA writes
from the card after moving it back into host domain.

-harry

-- 
You are receiving this mail because:
You are the assignee for the bug.