dummynet dropping too many packets
rihad
rihad at mail.ru
Mon Oct 5 08:53:23 UTC 2009
Luigi Rizzo wrote:
> On Mon, Oct 05, 2009 at 10:55:21AM +0800, Eugene Grosbein wrote:
>> On Sun, Oct 04, 2009 at 06:47:23PM +0500, rihad wrote:
>>
>>> sysctls:
>>> kern.ipc.nmbclusters=50000
>>> net.inet.ip.dummynet.io_fast=1
>> I guess you should also try to increase pipes length:
>>
>> net.inet.ip.dummynet.hash_size=65536
>> net.inet.ip.dummynet.pipe_slot_limit=1000
>
> in fact, i forgot to ask, we'd need to know the output of
> "ipfw pipe show" just to get the idea if there is any
> known reason for the drop (e.g. queues too short, etc.)
>
The pipes are fine, each normally having 100-120 concurrent consumers
(i.e. active users). (The max number of consumers allowed per pipe is
hash_size*max_chain_len, normally 64*16==1024.)
kern.ipc.nmbclusters=111111
And these are by default:
#net.inet.ip.dummynet.hash_size=64
#net.inet.ip.dummynet.max_chain_len=16
Yesterday I set up a cronjob logging the number of drops in the past 15
minutes:
*/15 * * * * (echo -n "$(date) "; netstat -z -s 2>/dev/null | fgrep
-w "output packets dropped") >> /tmp/bufs.log
And here's bufs.log:
Sun Oct 4 21:45:00 AZST 2009 418869 output packets dropped due to no
bufs, etc.
Sun Oct 4 22:00:00 AZST 2009 851693 output packets dropped due to no
bufs, etc.
Sun Oct 4 22:15:01 AZST 2009 932885 output packets dropped due to no
bufs, etc.
Sun Oct 4 22:30:00 AZST 2009 890522 output packets dropped due to no
bufs, etc.
Sun Oct 4 22:45:00 AZST 2009 1065931 output packets dropped due to no
bufs, etc.
Sun Oct 4 23:00:00 AZST 2009 937863 output packets dropped due to no
bufs, etc.
Sun Oct 4 23:15:01 AZST 2009 1018822 output packets dropped due to no
bufs, etc.
Sun Oct 4 23:30:00 AZST 2009 981922 output packets dropped due to no
bufs, etc.
Sun Oct 4 23:45:00 AZST 2009 1015124 output packets dropped due to no
bufs, etc.
Mon Oct 5 00:00:01 AZST 2009 1123926 output packets dropped due to no
bufs, etc.
Mon Oct 5 00:15:01 AZST 2009 948161 output packets dropped due to no
bufs, etc.
Mon Oct 5 00:30:00 AZST 2009 937277 output packets dropped due to no
bufs, etc.
Mon Oct 5 00:45:00 AZST 2009 875218 output packets dropped due to no
bufs, etc.
Mon Oct 5 01:00:00 AZST 2009 803527 output packets dropped due to no
bufs, etc.
Mon Oct 5 01:15:00 AZST 2009 728639 output packets dropped due to no
bufs, etc.
Mon Oct 5 01:30:00 AZST 2009 626154 output packets dropped due to no
bufs, etc.
Mon Oct 5 01:45:00 AZST 2009 519441 output packets dropped due to no
bufs, etc.
Mon Oct 5 02:00:00 AZST 2009 371098 output packets dropped due to no
bufs, etc.
Mon Oct 5 02:15:00 AZST 2009 681243 output packets dropped due to no
bufs, etc.
Mon Oct 5 02:30:00 AZST 2009 562909 output packets dropped due to no
bufs, etc.
Mon Oct 5 02:45:00 AZST 2009 426734 output packets dropped due to no
bufs, etc.
Mon Oct 5 03:00:00 AZST 2009 344619 output packets dropped due to no
bufs, etc.
Mon Oct 5 03:15:00 AZST 2009 90006 output packets dropped due to no
bufs, etc.
Mon Oct 5 03:30:00 AZST 2009 17064 output packets dropped due to no
bufs, etc.
Mon Oct 5 03:45:00 AZST 2009 3851 output packets dropped due to no
bufs, etc.
Mon Oct 5 04:00:00 AZST 2009 1323 output packets dropped due to no
bufs, etc.
Mon Oct 5 04:15:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 04:30:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 04:45:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 05:00:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 05:15:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 05:30:01 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 05:45:01 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 06:00:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 06:15:01 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 06:30:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 06:45:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 07:00:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 07:15:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 07:30:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 07:45:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 08:00:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 08:15:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 08:30:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 08:45:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 09:00:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 09:15:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 09:30:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 09:45:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 10:00:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 10:15:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 10:30:00 AZST 2009 177 output packets dropped due to no
bufs, etc.
Mon Oct 5 10:45:00 AZST 2009 1701 output packets dropped due to no
bufs, etc.
Mon Oct 5 11:00:01 AZST 2009 19933 output packets dropped due to no
bufs, etc.
Mon Oct 5 11:15:00 AZST 2009 30003 output packets dropped due to no
bufs, etc.
Mon Oct 5 11:30:00 AZST 2009 56712 output packets dropped due to no
bufs, etc.
Mon Oct 5 11:45:00 AZST 2009 78721 output packets dropped due to no
bufs, etc.
Mon Oct 5 12:00:01 AZST 2009 112518 output packets dropped due to no
bufs, etc.
Mon Oct 5 12:15:00 AZST 2009 7229 output packets dropped due to no
bufs, etc.
Mon Oct 5 12:30:01 AZST 2009 24965 output packets dropped due to no
bufs, etc.
Mon Oct 5 12:45:00 AZST 2009 75900 output packets dropped due to no
bufs, etc.
Mon Oct 5 13:00:00 AZST 2009 45002 output packets dropped due to no
bufs, etc.
Mon Oct 5 13:15:00 AZST 2009 67161 output packets dropped due to no
bufs, etc.
Mon Oct 5 13:30:00 AZST 2009 112591 output packets dropped due to no
bufs, etc.
As you can see the drops gradually went away completely at about 4:00
a.m., and started coming up at about 10:30 a.m., although at a lower
rate, probably thanks to me bumping "ipfw ... queue NNN" up to 5000 at
10a.m. this morning. The traffic flow between 4a.m. and 10:30a.m., the
"quiet" times, is about 200-330 mbit/s 5 minute average, without a
single drop. But after that, in come the drops, no matter how high I set
the queue. Should I try 10000 slots? 20000? I'm really sure there are
plenty of heavy downloaders between 4a.m. and 10a.m., but still without
a single drop before approx. 330 mbit/s! Strange, isn't it? This makes
me believe I'm hitting some other global memory limit, rather than
per-user limit, at around 340-350 mbit/s, causing drops. top and netstat
-m are OK right now, though:
Mem: 1870M Active, 1220M Inact, 481M Wired, 201M Cache, 214M Buf, 164M Free
15595/5385/20980/111112 mbuf clusters in use (current/cache/total/max)
More information about the freebsd-net
mailing list