kern/119890: debuggin pidgin with gdb causes kernel crash

[Bruce Evans]

On Sat, 26 Jan 2008, tadokoro wrote:

> kernel does not panic, so I cannot dump kernel.
> And I saw message "kernel trap 9 with interrupts disabled" on the console.
> So I built kernel with kdb and ddb,
> and I did single stepping execution with break point at printf.
>
> At doreti_iret_faut,

Applications can arrange to reach here by messing up their segment register
or stack.  Then interrupts are disabled, and it is a kernel bug to print
the "kernel trap 9 with interrupts disabled" message.  I think printing
the message doesn't cause any further problems.  Unprivileged applications
just have the privilege of spamming the console with the message.

> db> s
> Stopped at         doreti_iret_fault:      subq $0x98,%rsp
> db> s
> Stopped at         doreti_iret_fault+0x7:  testb $0x3,0xa0(%rsp)
> db> s
> Stopped at         doreti_iret_fault+0xf:  jz doreti_iret_fault+0x14
> db> s
> Stopped at         doreti_iret_fault+0x11: invlpg %ax
> db> s
>
> not return...
> Keyboard(num lock or caps lock) does not react.
> What should I do?

Don't do that :-).

This might be another kernel bug or two, now completely unrelated to
the original one:

- it's an amd64 kernel and the instruction at doreti_iret_fault+0x11 is
   actually "swapgs".  So it is a bug for the disassembler to print
   "invlpg %ax".

- The swapgs instruction is tricky and single stepping it is apparently
   broken.  I'm surprised the hang is after the swapgs instead of before.
   After the swapgs, we are in normal kernel mode, but before it we are
   half in user mode (haven't swapped %gs yet) and half in kernel mode
   (have the kernel %cs).  The trap handler for trace traps, like most
   trap handlers (all except NMI?), does a simplistic check of the %cs
   in the trap frame to decide whether to swap %gs.  This can never
   work right if we are in the in-between mode.  The in-between mode
   occurs for a few instructions (before %gs is swapped) on every entry
   to the kernel from user mode.  The NMI handler has complications to
   handle this problem, but the trace trap handler doesn't.  I think
   this bug isn't always harmful -- sometimes the mis-swapped %gs
   doesn't matter because it isn't used, and things work later on
   provided the total number of mis-swappings is even.

Bruce