kern/80166: Debugger SMP race panic

[Steve Sears]

>Number:         80166
>Category:       kern
>Synopsis:       Debugger SMP race panic
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Wed Apr 20 19:00:39 GMT 2005
>Closed-Date:
>Last-Modified:
>Originator:     Steve Sears
>Release:        5.3-RELEASE
>Organization:
>Environment:
FreeBSD sjs-linux 5.3-RELEASE FreeBSD 5.3-RELEASE #19: Mon Apr 18 11:28:21 EDT 2005     root at sjs-linux:/usr/src/sys-sjs/i386/compile/SJSKERN  i386

>Description:
      panic: mi_switch: did not reenter debugger

This panic happens often when setting a breakpoint and then hitting
it a few times.  I think I see the problem.

The bsd code in kern/subr_kdb.c does this:


#ifdef SMP
        if (did_stop_cpus)
                restart_cpus(stopped_cpus);
#endif

        kdb_active--;


The panic is this (mi_switch):

        /*
         * Don't perform context switches from the debugger.
         */
        if (kdb_active) {
                mtx_unlock_spin(&sched_lock);
                kdb_backtrace();
                kdb_reenter();
                panic("%s: did not reenter debugger", __func__);
        }


It's an mp-race.  It restarts the other CPU's before decrementing
kdb_active.   If the other CPU races into mi_switch first, a
panic is guaranteed.
>How-To-Repeat:
      Hit the same breakpoint several times on an SMP box.
>Fix:
      Decrement kdb_active before restarting cpus.
>Release-Note:
>Audit-Trail:
>Unformatted: