Intel 82574L interface wedging on em 7.1.9/7.2.3 when MSIX enabled

[Hooman Fazaeli]

On 11/8/2011 11:00 PM, Adrian Chadd wrote:
> On 8 November 2011 09:21, Hooman Fazaeli<hoomanfazaeli at gmail.com>  wrote:
>
>> With MSIX enabled, the link task (em_handle_link) does _not_ triggers
>> _start when the link changes state from inactive to active (which it
>> should).
>> If if_snd quickly fills up during a temporary link loss, transmission is
>> stopped forever and the driver never recovers from that state.
>>
>> The last patch should have reduced the frequency of the problem
>> but it assumes every IFQ_ENQUEUE is followed by a if_start which
>> is not a true assumption.
>
> FWIW, I saw something very similar with the if_arge code port from
> Linux. If the TX queue filled up and wasn't serviced before it hit
> completely full, it was never drained.
>
> It may be worthwhile auditing some of the other NIC drivers to ensure
> this kind of situation isn't occuring. Especially if they came from
> Linux. :-)
>
> That's a great catch, I hope it finally fixes the if_em issues with MSIX. :-)
>
>
> Adrian
Just for the record, I should inform you that igb, ixgb and ixbge have the
same issue. I have not checked other drivers.

And there is another subtle problem with all these drivers: if transmit (xxx_xmit)
fails for a temporary memory shortage (i.e., DMA failure for ENOMEM), the driver
may enter the OACTIVE state and _never_ recovers! The scenario is somehow as
before:

- if_start is executed.
- xxx_xmit fails with ENOMEM.
- xxx_start_locked sets OACTIVE. Note that this is different from a low TX descriptor
   condition which also sets OACTIVE.
- stack enqueues packets in if_snd but does not call if_start since driver is OACTIVE.
- stack enqueues more packets until if_snd fills up and packets start to drop.
- Since there is nowhere in the driver's code to re-try transmission when memory becomes
   available again (xxx_local_timer is a candidate), the driver remains OACTIVE forever
   until it is re-initialized.

I am working on patches for em/igb/ixgb/ixgbe to fix these issues and would be
happy to share them with anyone who is interested.

since these are really severe problems, I hope gurus apply official fixes ASAP.