ath / 802.11n performance issues and timer code

[John Baldwin]

On Monday, September 26, 2011 11:36:26 pm Adrian Chadd wrote:
> .. and as a follow up (and cc'ing attillo and freebsd-mips, in case
> it's relevant to other platforms and there's a MIPS specific thing to
> fix):
> 
> * 2128: mi_switch to idle
> * 2129: kern_clocksource.c:762 - ie, cpu_idleclock() has been called
> * 2130: the ath interrupt comes in
> * 2134: it's skipped for now as the idle thread is in a critical section
> * 2136: kern_clocksource.c:266 - ie, getnextcpuevent(), inside 
cpu_idleclock().
> 
> What I bet is happening is this race between the critical section +
> cpu_idleclock() and the ath0 interrupt:
> 
> * idle gets scheduled
> * critical_enter() is called in the mips cpu_idle() routine
> * the ath interrupt comes in here and gets handled, but since we're in
> a critical section, it won't preempt things
> * the cpu_idleclock() code completes without releasing the preemption,
> and the only thing that wakes up from that wait is the next interrupt
> (clock, arge0, etc.)

I think this is a mips-specific bug, though it may be well to audit all the 
cpu_idle() implementations.  On x86 the idle hooks all check sched_runnable() 
with interrupts disabled and then atomically re-enable interrupts and sleep 
only if that is false, e.g.:

static void
cpu_idle_hlt(int busy)
{
	int *state;

	state = (int *)PCPU_PTR(monitorbuf);
	*state = STATE_SLEEPING;
	/*
	 * We must absolutely guarentee that hlt is the next instruction
	 * after sti or we introduce a timing window.
	 */
	disable_intr();
	if (sched_runnable())
		enable_intr();
	else
		__asm __volatile("sti; hlt");
	*state = STATE_RUNNING;
}

I don't know if it is possible to do the same thing with the mips "wait" 
instruction.

-- 
John Baldwin