Ping No buffer space with Dummynet

Mon Mar 24 05:17:43 UTC 2014

Dear all,
    I meet a problem that is ping another machine with Dummynet(bw 1Mbit/s
delay=10ms and plr=0.10). It says ping: sendto: No buffer space available.
I changed kern.ipc.nmbclusters and kern.ipc.nsfbufs from default to 32768.
The problem still happens.

PING 192.168.8.110 (192.168.8.110): 56 data bytes
64 bytes from 192.168.8.110: icmp_seq=0 ttl=64 time=20.898 ms
64 bytes from 192.168.8.110: icmp_seq=1 ttl=64 time=22.233 ms
64 bytes from 192.168.8.110: icmp_seq=2 ttl=64 time=22.163 ms
64 bytes from 192.168.8.110: icmp_seq=3 ttl=64 time=21.883 ms
64 bytes from 192.168.8.110: icmp_seq=4 ttl=64 time=21.917 ms
64 bytes from 192.168.8.110: icmp_seq=5 ttl=64 time=21.215 ms
64 bytes from 192.168.8.110: icmp_seq=6 ttl=64 time=21.926 ms
64 bytes from 192.168.8.110: icmp_seq=7 ttl=64 time=23.918 ms
64 bytes from 192.168.8.110: icmp_seq=8 ttl=64 time=21.855 ms
ping: sendto: No buffer space available
64 bytes from 192.168.8.110: icmp_seq=10 ttl=64 time=22.941 ms
ping: sendto: No buffer space available
64 bytes from 192.168.8.110: icmp_seq=12 ttl=64 time=21.938 ms
64 bytes from 192.168.8.110: icmp_seq=13 ttl=64 time=21.761 ms
64 bytes from 192.168.8.110: icmp_seq=14 ttl=64 time=22.911 ms
64 bytes from 192.168.8.110: icmp_seq=15 ttl=64 time=21.877 ms
64 bytes from 192.168.8.110: icmp_seq=16 ttl=64 time=21.919 ms
64 bytes from 192.168.8.110: icmp_seq=17 ttl=64 time=21.922 ms

The command I use
kldload dummynet
sysctl net.inet.sctp.cmt_on_off=1
ipfw add 400 pipe 1 ip from 192.168.6.0/24. to 192.168.6.0/24
ipfw add 400 pipe 2 ip from 192.168.8.0/24 to 192.168.8.0/24
ipfw pipe 1 config delay 10ms queue 50 bw 10Mbit/s plr 0
ipfw pipe 2 config delay 10ms queue 50 bw 1Mbit/s plr 0.10

ping 192.168.8.110 (pipe2 plr = 0.10)
will display ping: sendto: No buffer space available

ping 192.168.6.110(pipe1 plr=0)
is OK

Did packet loss leads that? is it normal?

system parameters are as following

uname -a
FreeBSD freetest0 10.0-RELEASE FreeBSD 10.0-RELEASE #0: Sun Mar 23 14:13:36
HKT 2014     root at freetest0:/mirror/usr/src/sys/i386/compile/HZ1000_NO_SCTP_DEBUG
 i386

cat /mirror/usr/src/sys/i386/conf/HZ1000_NO_SCTP_DEBUG
include GENERIC
ident MYKERNEL

#options SCTP_DEBUG
options HZ=1000

sysctl -a net.inet
net.inet.ip.portrange.lowfirst: 1023
net.inet.ip.portrange.lowlast: 600
net.inet.ip.portrange.first: 10000
net.inet.ip.portrange.last: 65535
net.inet.ip.portrange.hifirst: 49152
net.inet.ip.portrange.hilast: 65535
net.inet.ip.portrange.reservedhigh: 1023
net.inet.ip.portrange.reservedlow: 0
net.inet.ip.portrange.randomized: 1
net.inet.ip.portrange.randomcps: 10
net.inet.ip.portrange.randomtime: 45
net.inet.ip.forwarding: 0
net.inet.ip.redirect: 1
net.inet.ip.ttl: 64
net.inet.ip.rtexpire: 3600
net.inet.ip.rtminexpire: 10
net.inet.ip.rtmaxcache: 128
net.inet.ip.sourceroute: 0
net.inet.ip.intr_queue_maxlen: 256
net.inet.ip.intr_queue_drops: 0
net.inet.ip.accept_sourceroute: 0
net.inet.ip.keepfaith: 0
net.inet.ip.gifttl: 30
net.inet.ip.no_same_prefix: 0
net.inet.ip.random_id_period: 8192
net.inet.ip.random_id_collisions: 0
net.inet.ip.random_id_total: 0
net.inet.ip.mcast.maxgrpsrc: 512
net.inet.ip.mcast.maxsocksrc: 128
net.inet.ip.mcast.loop: 1
net.inet.ip.fastforwarding: 0
net.inet.ip.sendsourcequench: 0
net.inet.ip.random_id: 0
net.inet.ip.check_interface: 0
net.inet.ip.fragpackets: 0
net.inet.ip.maxfragsperpacket: 16
net.inet.ip.maxfragpackets: 1024
net.inet.ip.process_options: 1
net.inet.ip.fw.one_pass: 1
net.inet.ip.fw.autoinc_step: 100
net.inet.ip.fw.verbose: 0
net.inet.ip.fw.verbose_limit: 0
net.inet.ip.fw.default_rule: 65535
net.inet.ip.fw.tables_max: 128
net.inet.ip.fw.default_to_accept: 0
net.inet.ip.fw.static_count: 14
net.inet.ip.fw.enable: 1
net.inet.ip.fw.dyn_buckets: 256
net.inet.ip.fw.curr_dyn_buckets: 256
net.inet.ip.fw.dyn_count: 0
net.inet.ip.fw.dyn_max: 4096
net.inet.ip.fw.dyn_ack_lifetime: 300
net.inet.ip.fw.dyn_syn_lifetime: 20
net.inet.ip.fw.dyn_fin_lifetime: 1
net.inet.ip.fw.dyn_rst_lifetime: 1
net.inet.ip.fw.dyn_udp_lifetime: 10
net.inet.ip.fw.dyn_short_lifetime: 5
net.inet.ip.fw.dyn_keepalive: 1
net.inet.ip.dummynet.hash_size: 64
net.inet.ip.dummynet.pipe_slot_limit: 100
net.inet.ip.dummynet.pipe_byte_limit: 1048576
net.inet.ip.dummynet.io_fast: 0
net.inet.ip.dummynet.debug: 0
net.inet.ip.dummynet.red_lookup_depth: 256
net.inet.ip.dummynet.red_avg_pkt_size: 512
net.inet.ip.dummynet.red_max_pkt_size: 1500
net.inet.ip.dummynet.tick_delta: 4
net.inet.ip.dummynet.tick_delta_sum: -184
net.inet.ip.dummynet.tick_adjustment: 13988
net.inet.ip.dummynet.tick_diff: 16350
net.inet.ip.dummynet.tick_lost: 0
net.inet.ip.dummynet.expire: 1
net.inet.ip.dummynet.expire_cycle: 0
net.inet.ip.dummynet.schk_count: 4
net.inet.ip.dummynet.si_count: 0
net.inet.ip.dummynet.fsk_count: 2
net.inet.ip.dummynet.queue_count: 0
net.inet.ip.dummynet.io_pkt: 192
net.inet.ip.dummynet.io_pkt_fast: 0
net.inet.ip.dummynet.io_pkt_drop: 6
net.inet.icmp.maskrepl: 0
net.inet.icmp.icmplim: 200
net.inet.icmp.icmplim_output: 1
net.inet.icmp.maskfake: 0
net.inet.icmp.log_redirect: 0
net.inet.icmp.reply_src:
net.inet.icmp.reply_from_interface: 0
net.inet.icmp.quotelen: 8
net.inet.icmp.bmcastecho: 0
net.inet.icmp.drop_redirect: 0
net.inet.igmp.recvifkludge: 1
net.inet.igmp.sendra: 1
net.inet.igmp.sendlocal: 1
net.inet.igmp.v1enable: 1
net.inet.igmp.v2enable: 1
net.inet.igmp.legacysupp: 0
net.inet.igmp.default_version: 3
net.inet.igmp.gsrdelay: 10
net.inet.tcp.rfc1323: 1
net.inet.tcp.mssdflt: 536
net.inet.tcp.keepidle: 7200000
net.inet.tcp.keepintvl: 75000
net.inet.tcp.sendspace: 32768
net.inet.tcp.recvspace: 65536
net.inet.tcp.keepinit: 75000
net.inet.tcp.delacktime: 100
net.inet.tcp.v6mssdflt: 1220
net.inet.tcp.cc.algorithm: newreno
net.inet.tcp.cc.available: newreno
net.inet.tcp.hostcache.cachelimit: 15360
net.inet.tcp.hostcache.hashsize: 512
net.inet.tcp.hostcache.bucketlimit: 30
net.inet.tcp.hostcache.count: 0
net.inet.tcp.hostcache.expire: 3600
net.inet.tcp.hostcache.prune: 300
net.inet.tcp.hostcache.purge: 0
net.inet.tcp.log_in_vain: 0
net.inet.tcp.blackhole: 0
net.inet.tcp.delayed_ack: 1
net.inet.tcp.drop_synfin: 0
net.inet.tcp.rfc3042: 1
net.inet.tcp.rfc3390: 1
net.inet.tcp.experimental.initcwnd10: 1
net.inet.tcp.rfc3465: 1
net.inet.tcp.abc_l_var: 2
net.inet.tcp.ecn.enable: 0
net.inet.tcp.ecn.maxretries: 1
net.inet.tcp.insecure_rst: 0
net.inet.tcp.recvbuf_auto: 1
net.inet.tcp.recvbuf_inc: 16384
net.inet.tcp.recvbuf_max: 2097152
net.inet.tcp.path_mtu_discovery: 1
net.inet.tcp.tso: 1
net.inet.tcp.sendbuf_auto: 1
net.inet.tcp.sendbuf_inc: 8192
net.inet.tcp.sendbuf_max: 2097152
net.inet.tcp.reass.maxsegments: 2222
net.inet.tcp.reass.cursegments: 0
net.inet.tcp.reass.overflows: 0
net.inet.tcp.sack.enable: 1
net.inet.tcp.sack.maxholes: 128
net.inet.tcp.sack.globalmaxholes: 65536
net.inet.tcp.sack.globalholes: 0
net.inet.tcp.minmss: 216
net.inet.tcp.log_debug: 0
net.inet.tcp.tcbhashsize: 32768
net.inet.tcp.do_tcpdrain: 1
net.inet.tcp.pcbcount: 5
net.inet.tcp.icmp_may_rst: 1
net.inet.tcp.isn_reseed_interval: 0
net.inet.tcp.soreceive_stream: 0
net.inet.tcp.syncookies: 1
net.inet.tcp.syncookies_only: 0
net.inet.tcp.syncache.bucketlimit: 30
net.inet.tcp.syncache.cachelimit: 15360
net.inet.tcp.syncache.count: 0
net.inet.tcp.syncache.hashsize: 512
net.inet.tcp.syncache.rexmtlimit: 3
net.inet.tcp.syncache.rst_on_sock_fail: 1
net.inet.tcp.msl: 30000
net.inet.tcp.rexmit_min: 30
net.inet.tcp.rexmit_slop: 200
net.inet.tcp.always_keepalive: 1
net.inet.tcp.fast_finwait2_recycle: 0
net.inet.tcp.finwait2_timeout: 60000
net.inet.tcp.keepcnt: 8
net.inet.tcp.rexmit_drop_options: 0
net.inet.tcp.per_cpu_timers: 0
net.inet.tcp.timer_race: 0
net.inet.tcp.maxtcptw: 20349
net.inet.tcp.nolocaltimewait: 0
net.inet.udp.checksum: 1
net.inet.udp.maxdgram: 9216
net.inet.udp.recvspace: 42080
net.inet.udp.log_in_vain: 0
net.inet.udp.blackhole: 0
net.inet.sctp.sendspace: 1864135
net.inet.sctp.recvspace: 1864135
net.inet.sctp.auto_asconf: 1
net.inet.sctp.ecn_enable: 1
net.inet.sctp.strict_sacks: 1
net.inet.sctp.peer_chkoh: 256
net.inet.sctp.maxburst: 4
net.inet.sctp.fr_maxburst: 4
net.inet.sctp.maxchunks: 4096
net.inet.sctp.tcbhashsize: 1024
net.inet.sctp.pcbhashsize: 256
net.inet.sctp.min_split_point: 2904
net.inet.sctp.chunkscale: 10
net.inet.sctp.delayed_sack_time: 200
net.inet.sctp.sack_freq: 2
net.inet.sctp.sys_resource: 1000
net.inet.sctp.asoc_resource: 10
net.inet.sctp.heartbeat_interval: 30000
net.inet.sctp.pmtu_raise_time: 600
net.inet.sctp.shutdown_guard_time: 180
net.inet.sctp.secret_lifetime: 3600
net.inet.sctp.rto_max: 60000
net.inet.sctp.rto_min: 1000
net.inet.sctp.rto_initial: 3000
net.inet.sctp.init_rto_max: 60000
net.inet.sctp.valid_cookie_life: 60000
net.inet.sctp.init_rtx_max: 8
net.inet.sctp.assoc_rtx_max: 10
net.inet.sctp.path_rtx_max: 5
net.inet.sctp.path_pf_threshold: 65535
net.inet.sctp.add_more_on_output: 1452
net.inet.sctp.incoming_streams: 2048
net.inet.sctp.outgoing_streams: 10
net.inet.sctp.cmt_on_off: 1
net.inet.sctp.nr_sack_on_off: 0
net.inet.sctp.cmt_use_dac: 0
net.inet.sctp.cwnd_maxburst: 1
net.inet.sctp.asconf_auth_nochk: 0
net.inet.sctp.auth_disable: 0
net.inet.sctp.nat_friendly: 1
net.inet.sctp.abc_l_var: 2
net.inet.sctp.max_chained_mbufs: 5
net.inet.sctp.do_sctp_drain: 1
net.inet.sctp.hb_max_burst: 4
net.inet.sctp.abort_at_limit: 0
net.inet.sctp.strict_data_order: 0
net.inet.sctp.min_residual: 1452
net.inet.sctp.max_retran_chunk: 30
net.inet.sctp.log_level: 0
net.inet.sctp.default_cc_module: 0
net.inet.sctp.default_ss_module: 0
net.inet.sctp.default_frag_interleave: 1
net.inet.sctp.mobility_base: 0
net.inet.sctp.mobility_fasthandoff: 0
net.inet.sctp.udp_tunneling_port: 0
net.inet.sctp.enable_sack_immediately: 0
net.inet.sctp.nat_friendly_init: 0
net.inet.sctp.vtag_time_wait: 60
net.inet.sctp.buffer_splitting: 0
net.inet.sctp.initial_cwnd: 3
net.inet.sctp.rttvar_bw: 4
net.inet.sctp.rttvar_rtt: 5
net.inet.sctp.rttvar_eqret: 0
net.inet.sctp.rttvar_steady_step: 20
net.inet.sctp.use_dcccecn: 1
net.inet.sctp.blackhole: 0
net.inet.raw.maxdgram: 9216
net.inet.raw.recvspace: 9216
net.inet.accf.unloadable: 0

Regards,
Niu Zhixiong
－－－－－－－－－－－－－－－
 kaiaixi at gmail.com