SMPable version of EM driver

[Vladimir Ivanov]

Hi,

Mike Tancsa wrote:
> On Wed, 01 Aug 2007 18:26:10 +0400, in sentex.lists.freebsd.net you
> wrote:
>
>   
>> Bill Marquette wrote:
>>     
>>> [skip]
>>> What type of performance differences are you seeing with these
>>> changes?  Is this with FreeBSD acting as a router/firewall, or purely
>>>   
>>>       
>> RX queue is being processed w/more than one thread.
>> TX queue thread isn't locked with RX anymore.
>>
>> Extra CPU time can be used by e.g. IPFW firewall or routing and so on.
>>
>>     
>
> Hi,
> 	I am interested in trying your version of the em driver.  On
> one of my routers, I am seeing
>
> kernel: em2: Missed Packets = 953
> kernel: em2: Receive No Buffers = 128
> kernel: em2: RX overruns = 7
> kernel: em2: Good Packets Rcvd = 62453961
> kernel: em2: Good Packets Xmtd = 31935910
>
> This is with the em driver currently in the RELENG_6 tree (version
> 6.6.6)..  Previous versions were the same.
> I notice that you have some different defaults as well
>
> dev.em.0.rx_int_delay: 0
> dev.em.0.tx_int_delay: 67108
> dev.em.0.rx_abs_int_delay: 1000
> dev.em.0.tx_abs_int_delay: 67108
>
> vs
>
> dev.em.1.rx_int_delay: 0
> dev.em.1.tx_int_delay: 66
> dev.em.1.rx_abs_int_delay: 66
> dev.em.1.tx_abs_int_delay: 66
> dev.em.1.rx_processing_limit: 100
>
> What are these tuned for ?  Hi pps ?  Low latency ?
>   
We've both problems and even more:
we need low latency, we've huge pps, we've to run firewall and so on

Tuning can not solve them.

Actually our rx/tx timeout defaults mostly are meaningless because:
1) we do not use TX interrupts et all
2) we use explicit SYSCTL (see dev.em.N.rx_kthread_priority) for tuning
RX threads' priority instead of rx_processing_limit.
3) we mask rx interrupts if aren't ready to catch that's why we do not
need interrupt pending/throttling.
> Thanks for any info,
>
> 	---Mike
>
>   
>> Also:
>> + RX and TX use different priority value. System seems to be more stable 
>> if RX scheduled w/less priority.
>> + RX/TX stay masked if there is no thread ready to catch interrupt.
>>
>>     
>>> as a server?  Any chance you are using the pf filtering engine (which
>>> I believe is still under giant in releng_6) with this?  Thanks
>>>   
>>>       
>> I have been talked that GIANT is a big problem for pf driver and they 
>> can not fix it easy.
>>
>> Regards,
>> -- 
>>     

[skip]

WBR,
PS: your personal e-mail doesn't work

-- 
Vladimir Ivanov
Network Operations Center
OOO "Yandex"
t: +7 495 739-7000
f: +7 495 739-7070
@: noc at yandex.net (corporate)
   wawa at yandex-team.ru (personal)
www: www.yandex.ru
--