tcp_isn_tick() / dummynet() callout madness ?

[Robert Watson]

On Sun, 30 Jan 2005, Poul-Henning Kamp wrote:

> count    1061761 time   67.148 func 0xc05573e0 dummynet()
> count    1061762 time   56.688 func 0xc056c128 tcp_isn_tick()

I think the problem here isn't so much the level of work done in
tcp_isn_tick, it's that it runs every tick.  I do profiling of our
callouts about once every 10-12 months, but haven't re-run it since before
the HZ change from 100 to 1000.  I'm not surprised that we're seeing
increased cost for some of the previously less significant callouts given
that change.  Although I haven't looked through the dummynet code, it
wouldn't surprise me if the same conclusion can be drawn.  If you're
running a UP kernel, the mutex operations should be pretty cheap, so it's
more likely the general overhead of running this code ... a lot.

Note that the remainder run 1/100th as often, presumably because they do
hz/10 or hz/100 or the like.  I've talked to both Colin Percival and Mike
Silby about the problem -- since the callout_reset() is one of the more
expensive parts of this code, Colin has been looking at some locking
optimizations to lower the cost.  I attempted to convince Mike that if we
thought the ISN sequencing was "sufficiently secure" with HZ of 100, then
we should be able to still run it at hz/100 now instead of 1000 times per
second, but he seems resistant.  I'll CC him so he's forced to reconsider
:-).

Something to consider for dummynet is a similar change to the one made to
the NFS code: the NFS code used to fire a callout at a high rate to manage
retransmits and work queues.  It was modified to only do this when
actually active.  If dummynet isn't active on your box, or is not working
hard, it can probably afford to run the timer less frequently, noting that
the goal of increasing the timer rate is to smooth the bursting in rate
limiting (which can result in loss if queues fill unnecessarily as a
result of bursting), and to improve the accuracy of introduced latency.

Robert N M Watson


> count      10646 time    0.877 func 0xc04e193c 
> count      10609 time    0.778 func 0xc04e193c 
> count      10609 time    0.759 func 0xc04e193c 
> count      10650 time    0.751 func 0xc0455408 
> count      10608 time    0.751 func 0xc04e193c 
> count       5326 time    0.324 func 0xc04f4fe0 
> count      10651 time    0.323 func 0xc04d3f00 
> 
> We spend roughly 5% of the real time in those two callout functions,
> probably in locking ?
> 
> -- 
> Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
> phk at FreeBSD.ORG         | TCP/IP since RFC 956
> FreeBSD committer       | BSD since 4.3-tahoe
> Never attribute to malice what can adequately be explained by incompetence.
> _______________________________________________
> freebsd-current at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscribe at freebsd.org"
>