Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp]

[Paul]

ngo Flaschberger wrote:
> Dear Paul,
>
>> Opteron 2222 UP mode, no polling
>>
>>           input          (em0)           output
>>  packets  errs      bytes    packets  errs      bytes colls
>>  1071020     0   66403248          2     0        404     0
>
> that looks good. (but seems to be near the limit).
>
Yes it is , any more and errors start.
>> Polling turned on provided better performance on 32 bit, but it gets 
>> strange errors on 64 bit..
>> Even at low pps I get small amounts of errors, and high pps same 
>> thing.. you would think that if
>> it got errors at low pps it would get more errors at high pps but 
>> that isn't the case..
>> Polling on:
>> packets  errs      bytes    packets  errs      bytes colls
>>   979736   963   60743636          1     0        226     0
>>   991838   496   61493960          1     0        178     0
>>   996125   460   61759754          1     0        178     0
>>
>>
>> what could cause this?
>
> *) kern.polling.idle_poll enabled?
> *) kern.polling.user_frac ?
> *) kern.polling.reg_frac ?
> *) kern.polling.burst_max ?
> *) kern.polling.each_burst ?

I tried tons of different values for these and nothing made any 
significant difference.
Idle polling makes a difference, allows more pps, but still errors.
Without idle polling it seems PPS is limited to HZ * descriptors, or 
1000 * 256 or 512
but 1000 * 1024 is the same as 512..  4000 * 256 or 2000 * 512 works but 
starts erroring 600kpps (SMP right now but it happens in UP too)

If anyone wants to log into the box and play with settings, recompile 
the kernel, etc. Let me know.


Paul