early boot netisr_init() panic on older AMD SMP machine with recent 11-STABLE

[Don Lewis]

My desktop machine has an older AMD SMP CPU and tracks 11-STABLE.  For
about six months or so it frequently panics early in boot.  If I retry a
sufficient number of times I can get a successful boot, but this is
rather annoying.

A normal boot looks like this:

Copyright (c) 1992-2018 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 11.2-STABLE #16 r339017M: Sat Sep 29 19:18:41 PDT 2018
    dl at mousie.catspoiler.org:/usr/obj/usr/src/sys/GENERICDDB amd64
FreeBSD clang version 6.0.1 (tags/RELEASE_601/final 335540) (based on LLVM 6.0.1
)
WARNING: WITNESS option enabled, expect reduced performance.
VT(vga): resolution 640x480
CPU: AMD Athlon(tm) II X3 450 Processor (3214.60-MHz K8-class CPU)
  Origin="AuthenticAMD"  Id=0x100f53  Family=0x10  Model=0x5  Stepping=3
  Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,C
MOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x802009<SSE3,MON,CX16,POPCNT>
  AMD Features=0xee500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM,3DNow!+,3DNow!
>
  AMD Features2=0x37ff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,IBS,
SKINIT,WDT>
  SVM: NP,NRIP,NAsids=64
  TSC: P-state invariant
real memory  = 34359738368 (32768 MB)
avail memory = 33275473920 (31733 MB)
Event timer "LAPIC" quality 100
ACPI APIC Table: <GBT    GBTUACPI>
FreeBSD/SMP: Multiprocessor System Detected: 3 CPUs
FreeBSD/SMP: 1 package(s) x 3 core(s)
ioapic0: Changing APIC ID to 2
ioapic0 <Version 2.1> irqs 0-23 on motherboard
SMP: AP CPU #1 Launched!
SMP: AP CPU #2 Launched!
Timecounter "TSC-low" frequency 1607298818 Hz quality 800
random: entropy device external interface
[SNIP]

An unsuccessful boot looks like this (hand transcribed):
[SNIP]
ACPI APIC Table: <GBT    GBTUACPI>
FreeBSD/SMP: Multiprocessor System Detected: 3 CPUs
FreeBSD/SMP: 1 package(s) x 3 core(s)
ioapic0: Changing APIC ID to 2
ioapic0 <Version 2.1> irqs 0-23 on motherboard
SMP: AP CPU #2 Launched!
SMP: AP CPU #1 Launched!
Timecounter "TSC-low" frequency 1607298818 Hz quality 800
panic: netisr_init: not on CPU 0
cpuid = 2
KDB: stack backtrace:
db_trace_selfwrapper() ...
vpanic() ...
doadump() ...
netisr_init() ...
mi_startup() ...
btext() ...

This problem may be silently occuring on many other machines.  This
machine is running a custom kernel with INVARIANTS and WITNESS.  The
panic is coming from a KASSERT(), which is only checked when the kernel
is built with INVARIANTS.

This KASSERT was removed from 12.0-CURRENT with this commit:
  https://svnweb.freebsd.org/base/head/sys/net/netisr.c?r1=301270&r2=302595

  Revision 302595 - (view) (download) (annotate) - [select for diffs]
  Modified Mon Jul 11 21:25:28 2016 UTC (2 years, 2 months ago) by nwhitehorn
  File length: 44729 byte(s)
  Diff to previous 301270

  Remove assumptions in MI code that the BSP is CPU 0.

Perhaps this should be MFC'ed, but it seems odd that the BSP is
non-deterministic.