SO_REUSEPORT: strange kernel balancer behaviour

[trafdev]

Yes, and if you kill this first thread - second thread will start to 
receive connections and so on.
That's why I've used one processes-shared acceptor socket which behaves 
better (load is balancing
between processes but equality of distribution is far from ideal).

Btw as per https://lwn.net/Articles/542629/ Linux 3.1 SOLVES all these 
problems via SO_REUSEPORT.


On Tue Jul 23 10:18:07 2013, Adrian Chadd wrote:
> Answering my own email:
>
> SO_REUSEPORT on FreeBSD doesn't load balance incoming connections.
>
> Test case:
>
> * 8 threads
> * each creates a TCP socket, listening on port 1667, with SO_REUSEPORT
> * only the first thread ever sees incoming requests.
>
> I think this load distribution feature is useful to implement, but it
> shouldn't be called SO_REUSEPORT.
>
> (Silly Linux, why would you do that too..)
>
>
>
> -adrian
>
> On 23 July 2013 07:39, Adrian Chadd <adrian at freebsd.org> wrote:
>> On 23 July 2013 00:09, trafdev <trafdev at mail.ru> wrote:
>>> It's like shared acceptor FD and N processes:
>>
>> [snip] looks like mine, but I use threads.
>>
>>> Accept conn callback is called in N processes on each connection, only one
>>> wins,
>>> others exit by errno == EAGAIN case. Overhead is almost zero.
>>> Problem is that "wins" distribution is far from equal.
>>
>> Right. I'm not at that stage yet, but I can totally see that happening.
>>
>> Ok. Time to hit up the TCP stack people to weigh in on the recent lkml
>> posts about this:
>>
>> http://lwn.net/Articles/542629/
>>
>> With SO_REUSEPORT, we should create one listen FD per thread, and let
>> the OS balance how that gets distributed. Rather than one listen
>> socket that is shared between all processes/threads. I'll try that
>> locally and see if that works right. Would you mind trying it locally
>> and see if it improves the distribution of work?
>>
>> Thanks,
>>
>>
>> -adrian
>
>