lagg/lacp poor traffic distribution

[Eugene Grosbein]

On 21.12.2010 19:11, Fabien Thomas wrote:

>>> I had this problem with igb driver, and I found, that lagg selects
>>> outgoing interface based on packet header flowid field if M_FLOWID field
>>> is set. And in the igb driver code flowid is set as 
>>>
>>> #if __FreeBSD_version >= 800000
>>> <------><------><------>rxr->fmp->m_pkthdr.flowid = que->msix;
>>> <------><------><------>rxr->fmp->m_flags |= M_FLOWID;
>>> #endif
>>>
>>> The same thing in em driver with MULTIQUEUE 
>>>
>>> That does not give enough number of flows to balance traffic well, so I
>>> commented check in if_lagg.c
>>>
>>> lagg_lb_start(struct lagg_softc *sc, struct mbuf *m)
>>> {
>>> <------>struct lagg_lb *lb = (struct lagg_lb *)sc->sc_psc;
>>> <------>struct lagg_port *lp = NULL;
>>> <------>uint32_t p = 0;
>>>
>>> //<---->if (m->m_flags & M_FLOWID)
>>> //<----><------>p = m->m_pkthdr.flowid;
>>> //<---->else
>>>
>>> and with this change I have much better load distribution across interfaces.
>>>
>>> Hope it helps.
>>
>> You are perfectly right. By disabling flow usage I've obtained load sharing
>> close to even (final patch follows). Two questions:
>>
>> 1. Is it a bug or design problem?
> 
> How many queues have you with igb? If it's one it will explain why the flowid is bad for load balancing with lagg.

How do I know? I've read igb(4) manual page and found no words
about queues within igb, nor I have any knowledge about them.

> The problem is that flowid is good if the number of queue is = or a multiple of the number of lagg ports.

Now I see, thanks.

Eugene Grosbein