[Bug 237321] vmx(4) iflib driver fails when number of CPU cores is not a power of two

[bugzilla-noreply at freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=237321

            Bug ID: 237321
           Summary: vmx(4) iflib driver fails when number of CPU cores is
                    not a power of two
           Product: Base System
           Version: 12.0-STABLE
          Hardware: Any
                OS: Any
            Status: New
          Severity: Affects Only Me
          Priority: ---
         Component: kern
          Assignee: bugs at FreeBSD.org
          Reporter: ncrogers at gmail.com

This is regarding the MFC of r343291 to 12-STABLE in r344027 (Convert vmx(4) to
being an iflib driver.)

If you configure a VMware ESXi guest with vmxnet3 interfaces and atypical
number of CPU cores (e.g 6), the iflib/vmx(4) driver throws an error "device
enable command failed" at bootup and periodically when trying to use the
interface. This is because , for reasons I do not understand, the driver is
unhappy when number of TX/RX queues is not a power of two. Because the driver
defaults to 8 queues, whenever a guest boots with 6 cores iflib automatically
reduces the number of queues to equal the number of CPU cores.

This is resolved by setting the number of tx/rx queues to 1,2, or 4, whichever
is less than or equal to the number of CPU cores.

For example, on a 6 core system the following works:
dev.vmx.0.iflib.override_ntxqs=4
dev.vmx.0.iflib.override_nrxqs=4

I believe either iflib or the vmx driver specifically needs to be corrected to
handle an irregular number of queues or limit it to powers of two when there is
fewer than 8 cores.

Note that this for whatever reason does NOT happen in VMware Fusion under
macOS, but seems to always happen in an ESXi environment.

In all cases, hw.pci.honor_msi_blacklist is set to 0.

-- 
You are receiving this mail because:
You are the assignee for the bug.