svn commit: r206878 - head/sys/kern

[Marius Strobl]

On Mon, Apr 19, 2010 at 11:27:54PM +0000, Attilio Rao wrote:
> Author: attilio
> Date: Mon Apr 19 23:27:54 2010
> New Revision: 206878
> URL: http://svn.freebsd.org/changeset/base/206878
> 
> Log:
>   Fix a deadlock in the shutdown code:
>   When performing a smp_rendezvous() or more likely, on amd64 and i386,
>   a smp_tlb_shootdown() the caller will end up with the smp_ipi_mtx
>   spinlock held, busy-waiting for other CPUs to acknowledge the operation.
>   As long as CPUs are suspended (via cpu_reset()) between the active mask
>   read and IPI sending there can be a deadlock where the caller will wait
>   forever for a dead CPU to acknowledge the operation.
>   Please note that on CPU0 that is going to be someway heavier because of
>   the spinlocks being disabled earlier than quitting the machine.
>   
>   Fix this bug by calling cpu_reset() with the smp_ipi_mtx held.
>   Note that it is very likely that a saner offline/online CPUs mechanism
>   will help heavilly in fixing similar cases as it is likely more bugs
>   of this type may arise in the future.
>   

This change causes a hang for me when running an SMP kernel on an
UP machine or an MP machine with SMP disabled as in these cases
smp_ipi_mtx isn't initialized (see mp_start()). The below patch
fixes this for me.

Marius

Index: kern_shutdown.c
===================================================================
--- kern_shutdown.c	(revision 207463)
+++ kern_shutdown.c	(working copy)
@@ -491,8 +491,8 @@ shutdown_reset(void *junk, int howto)
 	/*
 	 * Acquiring smp_ipi_mtx here has a double effect:
 	 * - it disables interrupts avoiding CPU0 preemption
-	 *   by fast handlers (thus deadlocking  against other CPUs)
-	 * - it avoids deadlocks against smp_rendezvous() or, more 
+	 *   by fast handlers (thus deadlocking against other CPUs)
+	 * - it avoids deadlocks against smp_rendezvous() or, more
 	 *   generally, threads busy-waiting, with this spinlock held,
 	 *   and waiting for responses by threads on other CPUs
 	 *   (ie. smp_tlb_shootdown()).
@@ -500,9 +500,11 @@ shutdown_reset(void *junk, int howto)
 	 * For the !SMP case it just needs to handle the former problem.
 	 */
 #ifdef SMP
-	mtx_lock_spin(&smp_ipi_mtx);
+	if (mtx_initialized(&smp_ipi_mtx) != 0)
+		mtx_lock_spin(&smp_ipi_mtx);
+	else
 #else
-	spinlock_enter();
+		spinlock_enter();
 #endif
 
 	/* cpu_boot(howto); */ /* doesn't do anything at the moment */