Starting APs earlier during boot

[John Baldwin]

On Wednesday, February 17, 2016 11:42:41 AM Konstantin Belousov wrote:
> On Tue, Feb 16, 2016 at 12:50:22PM -0800, John Baldwin wrote:
> > However, we can run a slightly simpler version of our scheduler before
> > timers are working.  In fact, sleep/wakeup work just fine fairly early (we
> > allocate the necessary structures at SI_SUB_KMEM which is before the APs
> > are even started).  Once idle threads are created and ready we could in
> > theory let the APs startup and run other threads.  You just don't have working 
> > timeouts.  OTOH, you can sort of simulate timeouts if you modify the scheduler 
> > to yield the CPU instead of blocking the thread for a sleep with a timeout.  
> > The effect would be for threads that do sleeps with a timeout to fall back to 
> > polling before timers are working.  In practice, all of the early kernel 
> > threads use sleeps without timeouts when idle so this doesn't really matter.
> I understand that timeouts can be somewhat simulated this way.
> 
> But I do not quite understand how generic scheduling can work without
> (timer) interrupts. Suppose that we have two threads 1 and 2 of the same
> priority, both runnable, and due to some event thread 2 preempted thread
> 1. If thread 2 just runs without calling the preempt functions like
> msleep, what would guarentee that thread 1 eventually gets it CPU slice ?

Nothing, but the only thread we have that does that during early startup is
thread0 (which is what should be running as it is the one that makes
progress to getting timers setup).  Currently the sleep calls just always
return which means if any thread gets on the CPU it never yields and is
stuck forever.  My changes make it so only a CPU bound thread is stuck
forever, and we only have one such thread before timers are running: thread0.

> E.g. there might be no interrupts set up yet, and idle thread on UP
> gets on CPU, then the whole boot process could deadlock.

The idle threads are special as they yield explicitly if there are any
runnable threads on the run queues.

-- 
John Baldwin