network performance
Stefan Lambrev
stefan.lambrev at moneybookers.com
Wed Feb 6 14:42:24 PST 2008
Greetings,
Kris Kennaway wrote:
> Stefan Lambrev wrote:
>
>>> I'll use again hwpmc and LOCK_PROFILING to see what's going on.
>>> And will try the same benchmark on quad core processor as now
>>> numbers of cores/cpus matter :)
>>>
>> Here are promised results - http://89.186.204.158/lock_profiling-8.txt
Finally I got pmcstat working - http://89.186.204.158/hwpmc-p4.txt
The stats are gathered during 600kpp incoming.
I think that syncache or what calls MD5Transform is not SMP able, and
that's why outgoing 250kpps is the limit that I can't beat.
>
> Thanks. There is further work needed on the route locking, and also
> you are hitting limitations of the em driver (or possibly hardware; if
> you only have a single transmit queue then outbound packets from
> multiple CPUs have to be serialized in the driver no matter what).
> Hopefully there will be further improvements in the coming months, and
> these changes will also migrate into CVS.
>
> If you want to start hacking things to see how much further progress
> is feasible, you can apply the attached hack that nulls out all route
> locking :) This should be OK as long as your routes are not changing,
> although you might get some spam on the console (if this is excessive,
> comment out the printfs also ;-). It may not help much though, all
> the contention will probably just fall through onto the ethernet driver.
I'll do tomorrow. I still did not tested how if_lagg will perform in
current situation.
I thing that I didn't reached the maximum of the network card today.
In this configuration best values was 1,122,674 incoming packets +
210kpps outgoing with both cpu cores 100% busy.
systat -ifstat showed:
em0 in 63.985 MB/s 63.985 MB/s 47.465 GB
out 11.674 MB/s 11.723 MB/s 12.975 GB
It is good, that now I hit a bottleneck that can be upgraded, if needed :)
>
>> Btw I got kernel panic first time when I run sysctl
>> debug.lock.prof.stats
>
> Yeah, it is a bit broken in 8.0 even in CVS. Also make sure not to
> reset it while the CPUs are loaded :)
>
>> I'm still trying to get hwpmc working with my cpu's and new kernel.
>> Do you have any patches Kris?
>> Is it supposed to work with your sources on my CPU?
>> I can fetch your latest src/lib/libpmc from from p4 if this will help :)
>
> It works on my systems...try with libpmc from my branch, make sure to
> install the new includes first and then rebuild and reinstall libpmc
> and pmcstat. I have attached a patch against the CVS libpmc which
> might be easier than checking it out from p4...it relies on kernel
> changes also though, which are in the kernel you already have but not
> in CVS.
>
> Kris
Yes there was a problem with the installed include file, I edit it by
hand, compiled new libpmc and it works again :)
More information about the freebsd-performance
mailing list