Packet drops and queue length upon bandwidth limiting in PF

[Bill Moran]

"Ashish Awasthi" <awasthi.ashish at gmail.com> wrote:

> On 3/30/06, Bill Moran <wmoran at collaborativefusion.com> wrote:
> >
> > "Ashish Awasthi" <awasthi.ashish at gmail.com> wrote:
> >
> > > I am a relative newbie, so please don't flame me if my question doesn't
> > make
> > > sense.
> > >
> > > In a network experiment to determine appropriate length of router
> > buffers, I
> > > am using pfctl on FreeBSD 5.3 to limit the bandwidth to 100 Mbps on a 1
> > Gig
> > > link and limit the queue to 240 packets, and I use iperf for sending out
> > > data. Connection is maintained between two routers running FreeBSD 5.3,
> > > connected by a 1 Gig link. I monitor on sender the pfctl and iperf
> > > statisitcs.
> > >
> > > As I see the iperf throughput go down from 94 Mbps to 50 Mbps and then
> > rise
> > > again in accordance with the classic sawtooth curve of TCP, it is clear
> > that
> > > there must have been a packet drop, but "pfctl -s -queue -v -v" at the
> > > sender shows 0 losses and 0 drops. Moreover, the queue length as
> > reported
> > > never overflows. Even netstat shows 0 retransmissions!
> > >
> > > I tried this with queue lengths of 50, 100, 240, 10 and 5. Only when
> > queue
> > > length is on the order of 5 or 10 do I see packet drops in pfctl report
> > (and
> > > also retransmissions in the netstat report); however, since I have
> > limited
> > > the bandwidth and the outgoing traffic is shaped by this limitation, it
> > is
> > > clear that there must be some packet losses in other cases as well.
> > >
> > > So, I tend to think that some other queueing is occuring apart from the
> > > ALTQ, and drops are occuring there. If so, how can I obtain those
> > > statistics?
> >
> > You're making a lot of assumptions about how things work, so I'll follow
> > in kind.
> >
> > I would assume that pf is sending ICMP source quench messages to the
> > sending machine to avoid overflowing its queues.  If it's proactive
> > in doing this, it would never overflow, except in the case where the
> > queue is so short that it can't reply with a source quench fast enough.
> > To me, this would be expected behaviour.  A little packet sniffing should
> > show whether this is what is actually happening or not.
> >
> > As a side note, this is why arbitrarily blocking all ICMP messages is a
> > bad
> > idea.
> 
> Hi,
> 
> Thanks for your response. However, the problem still remains. I did check
> for ICMP packets at both the source and the router, but there are NO source
> quench packets at all in the tcpdump traces.
> 
> Where should I be looking? Any suggestions?

Look at the network traffic.  If you're getting dropped packets, those will
be obvious from the retransmits.  If not, you'll be able to see what is
actually controlling the speed.  I suppose the kernel could be limiting
how it sends ACKs.

I suggest Ethereal for this kind of thing.  It has a lot of nifty features
that make it easy (i.e. it automagically flags retransmitted packets).

-- 
Bill Moran
Potential Technologies
http://www.potentialtech.com