kern/100519: Suboptimal network polling

Gleb Smirnoff glebius at
Thu Sep 28 10:40:33 PDT 2006

The following reply was made to PR kern/100519; it has been noted by GNATS.

From: Gleb Smirnoff <glebius at>
To: freebsd-gnats-submit at
Subject: Re: kern/100519: Suboptimal network polling
Date: Thu, 28 Sep 2006 21:37:43 +0400

   attach this to PR, so that it won't be lost in
 my mailbox.
 ----- Forwarded message from Arthur Hartwig <Arthur.Hartwig at> -----
 >Hmm, interesting. Have you done any profiling?
 G'day Gleb,
    My work was then largely concerned with fast forwarding performance 
 between two em interfaces on FreeBSD 6.0 and in particular with trying 
 to get some performance improvement at high rates by using additional 
 CPUs. I did some timing using the tsc and found that the em driver in 
 polling mode was using roughly roughly half the time spent in 
 ether_input() to the point of putting a frame on the output queue. So I 
 thought if I ran the polling code on one CPU and ether_input() on 
 another CPU I would get an observable improvement. This turned out to be 
 the case.
    At some stage I noticed that vmstat showed lots of interrupts even 
 though I was using polling mode.  I noticed v_intr was incremented in 
 swi_sched() and intr_execute_handlers(). Another system not using 
 polling mode didn't show the same high interrupt rate. After making the 
 described change  I found the interrupt rate reported by vmstat dropped 
 significantly.  The change didn't result in a significant performance 
 improvement but clearly reduce the overhead in the polling thread, thus 
 making more CPU time available to the system.
    I found interrupt driven fast forwarding performed significantly 
 worse on a dual CPU system than on a single CPU system. Some profiling 
 showed a lot of contention for the routing table locks, sometimes with 
 significant delays between a thread attempting to acquire a lock and its 
 actual acquisition. Changing to polling mode seemed as if would help 
 reduce lock contention and so improve performance. A number of tweaks 
 combined to get noloss fast forwarding performance between two GigE 
 interfaces on a dual CPU system up to about what it was on a single CPU 
 interrupt driven system
 ----- End forwarded message -----

More information about the freebsd-bugs mailing list