[Bug 292081] GCE /boot/loader.conf fixes to use vtnet(4) multiqueue and boot successfully on arm64

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=292081

            Bug ID: 292081
           Summary: GCE /boot/loader.conf fixes to use vtnet(4) multiqueue
                    and boot successfully on arm64
           Product: Base System
           Version: 16.0-CURRENT
          Hardware: Any
                OS: Any
            Status: New
          Severity: Affects Only Me
          Priority: ---
         Component: kern
          Assignee: bugs@FreeBSD.org
          Reporter: william.a@carrel.org

Created attachment 266708
  --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=266708&action=edit
gce.conf.patch

This patch provides three updates to the /boot/loader.conf for GCE image
builds:

 * Enable vtnet(4) Multiqueue: Removes hw.vtnet.mq_disable=1. This workaround
was originally introduced nearly a decade ago to address stability issues on
KVM that have long since been resolved in both the FreeBSD driver and the GCE
hypervisor. Removing this allows network interrupts to scale across multiple
vCPUs. Tested on n2-highcpu-16 VM with 15.0-RELEASE and confirmed multiple
queue pairs active and interrupts handling across cores.

 * Fix growfs path: The sed command was missing the ${DESTDIR} prefix, meaning
it was attempting to modify the build host's /etc/rc.d/growfs instead of the
target image's script. Tested in an arm64 builder that builds as non-root.

 * ARM64 Support: Add hw.pci.honor_msi_blacklist=0. This is required for MSI
support on GCE ARM64 instances which is prerequisite to gve(4) not panicking at
boot, and nvme(4) also has a real sad time without interrupts. Tested on a
variety of c4a VMs.

I'll admit the MSI blacklist feels a little icky, but the patches to do the
right thing without this will take a little longer and I'll send separately.
(Quickly summarized: add arm64 hypervisor detection by checking boot exception
level vs. what the CPU has support for; then add a PCI_QUIRK_ENABLE_MSI_VM for
the gve(4) device that reliably appears at 0:0:0 on GCE.)

-- 
You are receiving this mail because:
You are the assignee for the bug.