ALTQ weirdness

[Morgan Wesström]

Hi.

For five years I've used ALTQ to shape the upload of my DSL connection 
and it has worked very well. All details can be found further down in 
this mail but the basic setup is a default CBQ queue with 10% of the 
bandwidth and another queue for the remaining 90% with three child 
queues where I put traffic I want to prioritize. All queues can borrow 
bandwidth from each other. When the prio queues are fully utilized the 
default queue falls back to 10% just as expected. The strange thing 
happens if I saturate my DOWNLOAD at this point. The default queue is 
then suddenly able to "steal" much more than its 10% share even though 
the demand for bandwidth in the prio queues are unchanged at max. The 
bahaviour can be seen in the lower of the pfstat graphs here at -1.5 hours:

http://pp.dyndns.biz/altq-weirdness.png

The red spike is the TCP ACKs from the download which are correctly put 
in the highest priority child queue. Another download can be seen at -6 
hours but it doesn't saturate the download bandwidth and the default 
queue remains at 10% there. Obviously there is something I'm doing wrong 
and don't understand and I'm just curious of what it is so I can make 
the queues behave exactly the way I intended.

Queue definitions from /etc/pf.conf:

altq on $ext_if cbq bandwidth $up_limit queue {q_def, q_pri}
queue q_def bandwidth 10% qlimit 400 cbq( borrow default )
queue q_pri bandwidth 90% cbq( borrow ) {q_hv, q_p2p, q_p1, q_p2}
queue  q_hv  bandwidth 10% priority 2 cbq( borrow )
queue  q_p2p bandwidth 10% priority 3 cbq( borrow )
queue  q_p1  bandwidth 20% priority 5 cbq( borrow )
queue  q_p2  bandwidth 60% priority 7 cbq( borrow )

# uname -a
FreeBSD gatekeeper.pp.dyndns.biz 7.0-RELEASE-p3 FreeBSD 7.0-RELEASE-p3 
#1: Fri Aug 15 14:12:27 CEST 2008 
pp at gatekeeper.pp.dyndns.biz:/usr/obj/usr/src/sys/MYKERNEL  amd64

This is on amd64 but I experienced the exact same behaviour on my old 
router running FreeBSD 6.3/i386

Kind regards
Morgan Wesström