Panic after update from r297313 -> r297348

[Konstantin Belousov]

On Mon, Mar 28, 2016 at 05:36:13AM -0700, David Wolfskill wrote:
> Unfortunately, the serial console only seemed to be echoing characters
> in quasi-random spurts, yielding such things as:
> 
> ...
> r() at lcpu_initcpu_initclocks_bsp+0x37f/frame 0xffffffff820dbb40
> initclocks() at initclocks+0x20/frame 0xffffffff820dbb50
> mi_start+0x2c
> 
> Tracing  td 0xffc_et_start+0x2d9/frame 0xffffffff820dbaa0
> loadtimer() at lrame 0xf configtsp() at clocks_bsp+0x37f/frame 0xffffffff820dbb40
> initclocks() at initclocks+0x20/
> Tracing pid 0 tid 100000 td 0xffffffff81d11200
> lapic_et_start()c_et_start+0x2d9/frame 0xffffffff820dbaa0
> +0x2cimeoadtimer+0x102/frame 0xf20dbac0
> db> 
> pid 0 tid 100000 td 0xffffffff81d11200
> lapic_et_start() at lapic_et_start+0x2d9xffffffff820dbaa0
> loadtimer() at loadtimer+0x102/frame 0xffffffff8imer+0x239/frame 0xffffffff820dbb10
> cpu_initsp() at f820dbb7pid 0 ti td 0xffffffff81_start()c_et_staf820dbaa0
> r() at loadtimer+0x102/ffffffff8
>                                 configtimer() atcpu_initxfffffffpid 0 tif820dbaa0| |___ _ __ ___  ___ | |_) | (___ | |  |   ___| '__/ _ \/ _ \|  _ < \___ \| |  | |dtimer() at loadtimer+0x102/frame 0xffffffff8 0xffffffff820dbb10
> c| |   | | |  __/  __/| |_) |____) | |__| |
> b| |   | |  \___|\|200
> configtimer() at configtimer+0x239/frame 0xffffffff820dbb10
> cpu_initclocks_bcpu_initsp+0x37f/frame 0f820dbb40
> initclocks() at initclocks+0x20/frame 0x820dbb50
> mi_startup() at mi_startup+0x118/frame 0
> Tracing d 100000lapic_et_start() at lapic_et_start+0x2d9sp+0x37f+0x2c
> db> 0
>  0xffffffff820dbsp+0x37ff820dbb4ks() at 
> mi_startup() at up+0x118/frame 0xffffffff820dbb70
> btext() at btext+0x2c
> db> bd 100000+0x102/frame 0xfcpu_initframe 0xffffffff+0x2c
> pic    [K]ernel (1 of 2)
> r|m 6.bConfigure Boot [O]ptions...
>  H| vpanic+0x182/frame 0xffffffff820db4a0+
> db_commaf0                                        .--         
> sp+0x37f0/frame 0xffffffff820db6kdb_trapf820d        .---.....----.______               ____   _____ _____   ____|             |  _ \ / ____|  __ \ 
> bte                                       
> Uptime: 1s                                                                     -c0+0x4db8+0x15eef8+0x8+0x]+-y+:.                  `:`
> 
> 
> at which point I had already (semi-blindly) tried "panic", so I
> tried "reset", then I was able to escape to the loader, unload the
> new kernel, boot the previous one, move the various kernel*
> difrectories around in /boot, then reboot again to old kernel:
> 
> FreeBSD freebeast.catwhisker.org 11.0-CURRENT FreeBSD 11.0-CURRENT #2028  r297313M/297313:1100104: Sun Mar 27 06:27:24 PDT 2016     root at freebeast.catwhisker.org:/common/S4/obj/usr/src/sys/GENERIC  amd64
> 
> I'm still building on my laptop; while I hope I don't have the character-
> dropping issues there (as I have no serial console on it), I also don't
> have much of a way to communicate what it displays if it (also) panics.
> 
> Apparently my attempt at "panic" didn't get a crash dump.. :-(
> 
> Any suggestions for diagnosing this?
> 

Show me verbose dmesg of the running kernel.
Also, does anything change with the faulting kernel if you set
hw.lapic_tsc_deadline to 0 in loader ?