svn commit: r312205 - in head/sys: kern net

[John Baldwin]

On Tuesday, January 17, 2017 12:02:49 PM Bruce Evans wrote:
> On Mon, 16 Jan 2017, John Baldwin wrote:
> 
> > On Sunday, January 15, 2017 12:50:10 AM Sean Bruno wrote:
> >> ...
> >> Log:
> >>   Fix hangs in a uniprocessor configuration (qemu, virtualbox, real hw).
> >>
> >>   sys/net/iflib.c:
> >>     Add ctx to filter_info and don't skpi interrupt early on unless we're on an
> >>     SMP system
> >
> > On an SMP system with EARLY_AP_STARTUP (default on x86) this code should
> > never see smp_started as false anyway, so these checks should likely just
> > be conditional on EARLY_AP_STARTUP (since that option will eventually go
> > away).
> 
> Ifdefs on EARLY_AP_STARTUP give strange results for the UP case (at least
> for SMP with 1 CPU).  First, the ifdef in sys/kernel.h moves SI_SUB_SMP
> from late to early, although the CPU initialization order has not changed
> when there is only 1 CPU.  Then everything that uses SI_SUB_SMP is
> reordered, and bugs in the new order show up even in the UP case.

Most of the things that used to use SI_SUB_SMP no longer do (there is now
a SI_SUB_LAST that many of those things use).

> sys/gtaskqueue.h uses a complicated combination of EARLY_AP_STARTUP and
> SI_SUB ifdefs to try to get the order right.  I think the last commit to
> it helps for the UP case but harms for the SMP case (both for the
> !EARLY_AP_STARTUP case).  Its code is simpler for the EARLY_AP_STARTUP
> case.  In the !EARLY_AP_STARTUP case, it has to split up the initialization
> to delay calculations depending on the number of CPUs until after SMP
> has started.  The last commit to it seems to have turned the split into
> nonsense by removing the delay.  But in the UP case, the split is not
> really necessary and removing the delay should help.
> 
> EARLY_AP_STARTUP works badly here.  It doesn't give a much faster startup
> for the early part of the boot.  Then it gives a much slower boot by
> hanging for almost a minute waiting for USB or something.  Starting all
> the CPUs early also unimproves debugging by interleaving the boot messages
> a bit.  I use colorized console messages (with the color indexed by the
> CPU).  This makes interleaved messages quite readable and gives interesting
> patterns when multiple CPUs are running.

The interleaving I am aware of is from USB which starts its own kthreads
during attach directly rather than using config_intrhook like other drivers
which do bus exploration later in boot (e.g. all the SCSI devices).  I tried
to describe this to Hans but will probably just come up with a patch to change
USB to use config_intrhook which might help.

-- 
John Baldwin