network statistics in SMP
Harti Brandt
hartmut.brandt at dlr.de
Tue Dec 15 09:45:17 PST 2009
On Tue, 15 Dec 2009, John Baldwin wrote:
JB>On Tuesday 15 December 2009 4:38:04 am Harti Brandt wrote:
JB>> Hi all,
JB>>
JB>> I'm working on our network statistics (in the context of SNMP) and wonder,
JB>> to what extend we want them to be correct. I've re-read part of the past
JB>> discussions about 64-bit counters on 32-bit archs and got the impression,
JB>> that there are users that would like to have almost correct statistics
JB>> (for accounting, for example). If this is the case I wonder whether the
JB>> way we do the statistics today is correct.
JB>>
JB>> Basically all statistics are incremented or added to simply by a += b oder
JB>> a++. As I understand, this worked fine in the old days, where you had
JB>> spl*() calls at the right places. Nowadays when everything is SMP
JB>> shouldn't we use at least atomic operations for this? Also I read that on
JB>> architectures where cache coherency is not implemented in hardware even
JB>> this does not help (I found a mail from jhb why for the mutex
JB>> implementation this is not a problem, but I don't understand what to do
JB>> for the += and ++ operations). I failed to find a way, though, to
JB>> influence the caching policy (is there a function one can call to
JB>> change the policy?).
JB>
JB>Atomic ops will always work for reliable statistics. However, I believe
JB>Robert is working on using per-CPU statistics for TCP, UDP, etc. similar to
JB>what we do now for many of the 'cnt' stats (context switches, etc.). For
JB>'cnt' each CPU has its own count of stats that are updated using non-atomic
JB>ops (since they are CPU local). sysctl handlers then sum up the various per-
JB>CPU counts to report global counts to userland.
I see. I was also thinking along these lines, but was not sure whether it
is worth the trouble. I suppose this does not help to implement 64-bit
counters on 32-bit architectures, though, because you cannot read them
reliably without locking to sum them up, right?
harti
More information about the freebsd-arch
mailing list