network performance

Kris Kennaway kris at
Tue Feb 5 13:16:37 PST 2008

Stefan Lambrev wrote:

>>>> Thanks for investigating this. One thing to note is that ip flows from
>>>> the same connection always go down the same interface, this is because
>>>> Ethernet is not allowed to reorder frames. The hash uses
>>>> src-mac, dst-mac, src-ip and dst-ip (see lagg_hashmbuf), make sure when
>>>> performance testing that your traffic varies in these values. Adding
>>>> tcp/udp ports to the hashing may help.
>>> The traffic, that I generate is with random/spoofed src part, so it 
>>> is split between interfaces for sure :)
>>> Here you can find results when under load from hwpmc and lock_profiling:

OK, this shows the following major problems:

     39     22375065      1500649     5690741     3     0       119007 
      712359 /usr/src/sys/net/route.c:147 (sleep mutex:radix node head)
     21      3012732      1905704     1896914     1     1        14102 
      496427 /usr/src/sys/netinet/ip_output.c:594 (sleep mutex:rtentry)
     22          120      2073128          47     2 44109            0 
/usr/src/sys/modules/if_lagg/../../net/ieee8023ad_lacp.c:503 (rw:if_lagg 
     39     17857439      4262576     5690740     3     0        95072 
     1484738 /usr/src/sys/net/route.c:197 (sleep mutex:rtentry)

It looks like the if_lagg one has been fixed already in 8.0, it could 
probably be backported but requires some other infrastructure that might 
not be in 7.0.

The others are to do with concurrent transmission of packets (it is 
doing silly things with route lookups).  kmacy has a WIP that fixes 
this.  If you are interested in testing an 8.0 kernel with the fixes let 
me know.

>> I forget this file :)
> I found that MD5Transform aways uses ~14% (with rx/txcsum enabled or 
> disabled).

Yeah, these don't have anything to do with MD5.

> And when using without lagg MD5Transform pick up to 20% of the time.
> Is this normal?

It is probably from the syncache.  You could disable it 
(net.inet.tcp.syncookies_only) if you don't need strong protection 
against SYN flooding.


More information about the freebsd-performance mailing list