Questioning altq (cbq) performance above 4Mb on gif or above 40 Mb on e1000

[Ricky Charlet]

Howdy,

(FreeBsd 8.0REL plus a modified e1000 driver borrowing the 8.1 altq capabilities, amd64, 8 core)

        I have been measuring altq performance on my system. I set a cbq bandwith limit for a queue, match traffic to the queue with an appropriate rule and pump data.

        Without altq enabled, my traffic gen tool (nuttcp) can pump 800Mb across my gig links.

        With altq(cbq) setting bandwith limits (no borrow), up to about 40Mb on em0 altq keeps up. But from 40Mb bandwith limit and up, altq seems to slow down my traffic much slower than the bandwidth limits.

        Same effect is observed on gif interfaces (where my real interest lies) but much worse. With altq(cbq) setting bandwith limits (no borrow), up to about 4Mb on gif0 altq keeps up. But from 4Mb bandwidth limit and up, altq seems to slow down my traffic much lower than the bandwidth limits.

        I have a large file of sample data in excel formant which I'd like to share. Is there an appropriate way to share it with this list?

        I know I'm not cpu bound (97% idle while nuttcp pushes 800Mb traffic and altq configured with a bandwidth limit of 50Mb actually passes 30Mb.)

        I sort of suspect I'm mutex bound but have no idea how to test that.



 (1) this config allows about 4Mb tcp traffic to pass:
----------cut------------------
altq on em0 cbq bandwidth 100Mb queue { queue1, queue2 }
queue queue1  bandwidth 1Mb priority 7 cbq  (default)
queue queue2 bandwidth 50Mb priority 1 cbq
pass out quick on gif10 inet proto tcp no state queue queue2
pass out quick on gif10 proto icmp queue queue2
----------paste----------------

(2) this config allows about 40 Mb tcp traffic to pass
----------cut------------------
altq on em0 cbq bandwidth 100Mb queue { queue1, queue2 }
queue queue1  bandwidth 1Mb priority 7 cbq  (default)
queue queue2 bandwidth 50Mb priority 1 cbq
pass out quick on em0 inet proto tcp no state queue queue2
pass out quick on em0 proto icmp queue queue2
----------paste----------------

        Yes, I send appropriate traffic in each case to be 'caught' by the tcp pass out rule wether it be found on a gif or an em.


        My goal is to get 'expected' behavior (a bandwidth limit of 40 Mb allows 40Mb to pass) at T3 link speed (45Mb). Is this a reasonable expectation? Any ideas about config jigs or tests to run?



---
Ricky Charlet
Adara Networks
USA 408-433-4942