stefan.lambrev at moneybookers.com
Wed Feb 6 14:42:24 PST 2008
Kris Kennaway wrote:
> Stefan Lambrev wrote:
>>> I'll use again hwpmc and LOCK_PROFILING to see what's going on.
>>> And will try the same benchmark on quad core processor as now
>>> numbers of cores/cpus matter :)
>> Here are promised results - http://220.127.116.11/lock_profiling-8.txt
Finally I got pmcstat working - http://18.104.22.168/hwpmc-p4.txt
The stats are gathered during 600kpp incoming.
I think that syncache or what calls MD5Transform is not SMP able, and
that's why outgoing 250kpps is the limit that I can't beat.
> Thanks. There is further work needed on the route locking, and also
> you are hitting limitations of the em driver (or possibly hardware; if
> you only have a single transmit queue then outbound packets from
> multiple CPUs have to be serialized in the driver no matter what).
> Hopefully there will be further improvements in the coming months, and
> these changes will also migrate into CVS.
> If you want to start hacking things to see how much further progress
> is feasible, you can apply the attached hack that nulls out all route
> locking :) This should be OK as long as your routes are not changing,
> although you might get some spam on the console (if this is excessive,
> comment out the printfs also ;-). It may not help much though, all
> the contention will probably just fall through onto the ethernet driver.
I'll do tomorrow. I still did not tested how if_lagg will perform in
I thing that I didn't reached the maximum of the network card today.
In this configuration best values was 1,122,674 incoming packets +
210kpps outgoing with both cpu cores 100% busy.
systat -ifstat showed:
em0 in 63.985 MB/s 63.985 MB/s 47.465 GB
out 11.674 MB/s 11.723 MB/s 12.975 GB
It is good, that now I hit a bottleneck that can be upgraded, if needed :)
>> Btw I got kernel panic first time when I run sysctl
> Yeah, it is a bit broken in 8.0 even in CVS. Also make sure not to
> reset it while the CPUs are loaded :)
>> I'm still trying to get hwpmc working with my cpu's and new kernel.
>> Do you have any patches Kris?
>> Is it supposed to work with your sources on my CPU?
>> I can fetch your latest src/lib/libpmc from from p4 if this will help :)
> It works on my systems...try with libpmc from my branch, make sure to
> install the new includes first and then rebuild and reinstall libpmc
> and pmcstat. I have attached a patch against the CVS libpmc which
> might be easier than checking it out from p4...it relies on kernel
> changes also though, which are in the kernel you already have but not
> in CVS.
Yes there was a problem with the installed include file, I edit it by
hand, compiled new libpmc and it works again :)
More information about the freebsd-performance