ixgbe Jumbo race condition leading to Deadlock

[Kaushal Bhandankar]

The solution is:

I have a function which pre-calculates the buffers required for processing
the packet. This is to eliminate any lack-of-memory errors during
processing.
In this function I loop over descriptors from next_to_check onwards. If I
loop over some descriptors with DD set and do not see the EOP set till I
reach 5 descriptors, I will return a failure and not change next_to_check
etc.
So that on the next read, or later hopefully descriptor with EOP would have
been written back. This will ensure that race condition does not happen.

Let me know if it sounds good.

Regards,
Kaushal


On Sun, Jul 7, 2013 at 10:43 PM, Kaushal Bhandankar <kaushalgoa at gmail.com>wrote:

> Hi Jack,
> Thanks for the explanation. Do you suggest that I keep reading rx
> descriptor with DD bit and keep them pending till I get the descriptor with
> EOP set ? How much max delay can be expected for the EOP descriptor to be
> written back ?
>
> Regards,
> Kaushal
>
>
> On Sun, Jul 7, 2013 at 10:40 PM, Jack Vogel <jfvogel at gmail.com> wrote:
>
>> The "potential race condition" as the data sheet puts it, is only when
>> you are
>> trying to manage your RX ring by reading the RDH register, this is a bad
>> idea
>> anyway, none of our (Intel) drivers do this. Using the DD bit is what you
>> want
>> to do. The DD bit is set when the descriptor is written back, and that
>> happens
>> when the DMA is complete.
>>
>> The packet is incomplete until the descriptor with EOP set, in my code an
>> mbuf chain is created, and as each new descriptor is processed the pointer
>> to the head of the whole chain is kept in rxbuf->fmp, thus when you get to
>> the EOP descriptor you will be ready to send the whole chain to the stack.
>>
>> Its good that you are using ONEBUF since packet split has hardware issues
>> on 82599.
>>
>> Are you developing a new driver, or simply having issues using mine?
>>
>> Regards,
>>
>> Jack
>>
>>
>>
>> On Sun, Jul 7, 2013 at 2:24 AM, Kaushal Bhandankar <kaushalgoa at gmail.com>wrote:
>>
>>> In 82599, for a Jumbo packet of 9.5 K ( which consumes 5 descriptors of
>>> 2048 bytes each ), when does the Descriptor write back happen ? Does it
>>> happen per Descriptor or once per aggregated Descriptors ? Is it possible
>>> that all descriptors except last one to be written back and when you read
>>> RDH register, I get the last pending descriptor waiting inside 82599.
>>> We are using srrctl |= IXGBE_SRRCTL_DESCTYPE_ADV_ONEBUF;
>>>
>>> In my setup, I am seeing that, I don't see EOP set even when I read 5
>>> descriptors. Checking DD will return me an incomplete packet. What
>>> should I
>>> do in such a case ?
>>>
>>> References from Data sheet:
>>> -> Checking through DD bits eliminates a potential race condition: all
>>> descriptor data is posted internally prior to incrementing the head
>>> register and a read of the head register could potentially pass the
>>> descriptor waiting inside the 82599.
>>>
>>> Regards,
>>> Kaushal
>>> _______________________________________________
>>> freebsd-stable at freebsd.org mailing list
>>> http://lists.freebsd.org/mailman/listinfo/freebsd-stable
>>> To unsubscribe, send any mail to "freebsd-stable-unsubscribe at freebsd.org
>>> "
>>>
>>
>>
>