sio interrupt-level buffer overflows

[Bruce Evans]

On Tue, 29 Mar 2005, Oleg Tarasov wrote:

> Bruce Evans <bde at zeta.org.au> wrote:
>
>> I can't explain the panic.  The only relevant difference that I can think
>> of is that the change gives larger interrupt-level buffers in sio.  ppp
>> might be depending on them not being very large.  However, at the interrupt
>> level, ppp just copies data 1 character at a time into mbufs, so there is
>> not much to go wrong.  Does the panic occur in ppp (in the kernel) or in sio?
>
> This problem had unusual solution. It seems the key to this problem is
> that FreeBSD 5.3-STABLE-p5 (I have updated my system) has DEADLOCKS
> when intensively using sio device on high speed (115200) with SMP support.
> It seems that MPSAFE causes those deadlocks. Turning
>
> debug.mpsafenet="0"
>
> in /boot/loader.conf
> makes the system to work fine & stable instead of panicing every hour
> or more often

The upper layers of sio are not MPSAFE (and the lowest layer of interrupt
handling doesn't claim to be MPSAFE although it is, but that is not
the problem here -- it just a pessimization).  I think this prevents
ppp working with debug.mpsafe != 0.  I don't know if debug.mpsafe !=
0 works with !MPSAFE hardware network drivers, but ppp is mostly in
software so it is more likely to have problems.  ppp used to be locked
by splnet(), spltty() and splimp(), with spltty() stronger than splnet()
and splimp() stronger than splimp().  Now these functions are null,
and things only work if everything is Giant-locked.  Things are mostly
Giant-locked, but netisrs aren't when debug.mpsafenet != 0, so there
is a priority inversion: netisrs can interrupt lower level i/o code
that uses splimp() to attempt to lock out netisrs.

ppp is missing a NET_NEEDS_GIANT() declaration that would automatically
turn off debug.mpsafe (for the whole system :-() when ppp is configured,
even if ppp is never used (:-() or if it is used with a driver that
is MPSAFE.  However, no serial drivers are fully MPSAFE (since upper
tty layers aren't MPSAFE), so there may be no serial drivers that are
MPSAFE enough to support ppp, and ppp itself might not be MPSAFE.

This problem probably affects slip too.

Bruce