Panic (gpf) in early boot after upgrading FreeBSD 10.4 -> 11.2 on Ganeti

Rob Austein sra at hactrn.net
Mon Nov 12 04:24:53 UTC 2018


Belated upgrade (don't ask) of a pair of FreeBSD 10.4 VMs to 11.2.
Each VM got as far as:

  freebsd-update -r 11.2-RELEASE update
  freebsd-update install
  reboot

Each VM got an immediate kernel panic after the reboot (log below).

The two VMs are basically identical at the system level, but run in
separate Ganeti clusters on opposite coasts, so no hardware in common.
A dozen or so other VMs run in each cluster without issues (including
at least one other FreeBSD 11.2 VM), and the VMs I'm trying to upgrade
have also been just fine until now, so the problem seems unlikely to
be hardware per se.

GENERIC, amd64, UFS2, no non-/boot/kernel modules, one CPU per VM.
The only things even slightly unusual about these VMs are:

* They're running in Ganeti clusters:
  * Ganeti version 2.15.2
  * KVM hypervisor

* They use virtio, so:
  * Disk is vtbd0
  * Net is vtnet0

Log of attempted boot with new kernel:

    /boot/kernel/kernel text=0x1547d48 data=0x144138+0x4e9818 syms=[0x8+0x16aef8+0x8+0x183f99]
    Booting...
    Copyright (c) 1992-2018 The FreeBSD Project.
    Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	    The Regents of the University of California. All rights reserved.
    FreeBSD is a registered trademark of The FreeBSD Foundation.
    FreeBSD 11.2-RELEASE-p4 #0: Thu Sep 27 08:16:24 UTC 2018
	root at amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64
    FreeBSD clang version 6.0.0 (tags/RELEASE_600/final 326565) (based on LLVM 6.0.0)
    VT(vga): text 80x25
    CPU: QEMU Virtual CPU version 2.1.2 (2666.81-MHz K8-class CPU)
      Origin="GenuineIntel"  Id=0x663  Family=0x6  Model=0x6  Stepping=3
      Features=0x783fbfd<FPU,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE,SSE2>
      Features2=0x80a02001<SSE3,CX16,x2APIC,POPCNT,HV>
      AMD Features=0x20100800<SYSCALL,NX,LM>
      AMD Features2=0x1<LAHF>
    Hypervisor: Origin = "KVMKVMKVM"
    real memory  = 4294967296 (4096 MB)
    avail memory = 4088406016 (3899 MB)
    Event timer "LAPIC" quality 100
    ACPI APIC Table: <BOCHS  BXPCAPIC>
    ioapic0 <Version 1.1> irqs 0-23 on motherboard
    kernel trap 9 with interrupts disabled


    Fatal trap 9: general protection fault while in kernel mode
    cpuid = 0; apic id = 00
    instruction pointer	= 0x20:0xffffffff810e9ae6
    stack pointer	        = 0x28:0xffffffff82272c20
    frame pointer	        = 0x28:0xffffffff82272c80
    code segment		= base 0x0, limit 0xfffff, type 0x1b
			    = DPL 0, pres 1, long 1, def32 0, gran 1
    processor eflags	= resume, IOPL = 0
    current process		= 0 (swapper)
    trap number		= 9
    panic: general protection fault
    cpuid = 0
    KDB: stack backtrace:
    #0 0xffffffff80b3d587 at kdb_backtrace+0x67
    #1 0xffffffff80af6b27 at vpanic+0x177
    #2 0xffffffff80af69a3 at panic+0x43
    #3 0xffffffff80f77fdf at trap_fatal+0x35f
    #4 0xffffffff80f7759e at trap+0x5e
    #5 0xffffffff80f57fbc at calltrap+0x8
    #6 0xffffffff810ec5f3 at apic_setup_io+0x53
    #7 0xffffffff80a92898 at mi_startup+0x118
    #8 0xffffffff8031002c at btext+0x2c
    Uptime: 1s
    Automatic reboot in 15 seconds - press a key on the console to abort
    --> Press a key on the console to reboot,
    --> or switch off the system now.

Goggling turned up a few theories about bad memory and incompatible
changes to video drivers, none of which seem likely to apply here.

Cluebat, please, somebody?

Thanks for reading.


More information about the freebsd-stable mailing list