network statistics in SMP

John Baldwin jhb at
Tue Dec 15 08:43:04 PST 2009

On Tuesday 15 December 2009 4:38:04 am Harti Brandt wrote:
> Hi all,
> I'm working on our network statistics (in the context of SNMP) and wonder, 
> to what extend we want them to be correct. I've re-read part of the past 
> discussions about 64-bit counters on 32-bit archs and got the impression, 
> that there are users that would like to have almost correct statistics 
> (for accounting, for example). If this is the case I wonder whether the 
> way we do the statistics today is correct.
> Basically all statistics are incremented or added to simply by a += b oder 
> a++. As I understand, this worked fine in the old days, where you had 
> spl*() calls at the right places. Nowadays when everything is SMP 
> shouldn't we use at least atomic operations for this? Also I read that on 
> architectures where cache coherency is not implemented in hardware even 
> this does not help (I found a mail from jhb why for the mutex 
> implementation this is not a problem, but I don't understand what to do 
> for the += and ++ operations). I failed to find a way, though, to 
> influence the caching policy (is there a function one can call to 
> change the policy?).

Atomic ops will always work for reliable statistics.  However, I believe 
Robert is working on using per-CPU statistics for TCP, UDP, etc. similar to 
what we do now for many of the 'cnt' stats (context switches, etc.).  For 
'cnt' each CPU has its own count of stats that are updated using non-atomic 
ops (since they are CPU local).  sysctl handlers then sum up the various per-
CPU counts to report global counts to userland.

John Baldwin

More information about the freebsd-arch mailing list