Questioning altq (cbq) performance above 4Mb on gif or above 40 Mb on e1000

[Ricky Charlet]

Yes.. this works very well.

the first thing I tried:
I had a kern.hz of 1000 and changed to 10000 (through loader.conf + reboot)
I had a queue depth of 50 and changed to 900 (through `qlimit` parameter in pf.conf)


And now I am hitting my configured rate limits beyond my goal of 45Mb (T3 speed) even into 100 Mb range. I'm going to continue to experiment around with hz and q-depth and pick some eventual values for my implementation.

But thanks, Emeral. This seems to be exactly the parameters to play around with to increase altq performance into the 50~100 Mb range.

---
Ricky Charlet
Adara Networks
USA 408-433-4942







> -----Original Message-----
> From: ermal.luci at gmail.com [mailto:ermal.luci at gmail.com] On Behalf Of
> Ermal Luçi
> Sent: Friday, November 05, 2010 10:18 AM
> To: Ricky Charlet
> Cc: freebsd-pf at freebsd.org; altq at csl.sony.co.jp
> Subject: Re: Questioning altq (cbq) performance above 4Mb on gif or
> above 40 Mb on e1000
>
> On Fri, Nov 5, 2010 at 1:33 AM, Ricky  Charlet <RCharlet at adaranet.com>
> wrote:
> > Has anyone out there run altq with cbq with bandwidth limits set
> around 40 ~ 50 Mb and seen it work well (actual through put allowed to
> come near that speed)?
> >
> > Thanks
> > ---
> > Ricky Charlet
> > Adara Networks
> > USA 408-433-4942
> >
>
> I can suggest to try with higher hz and increase queue lengths in altq.
> By default they are 50 which is quite low in that regard.
>
> >
> >> -----Original Message-----
> >> From: owner-freebsd-pf at freebsd.org [mailto:owner-freebsd-
> >> pf at freebsd.org] On Behalf Of Ricky Charlet
> >> Sent: Wednesday, November 03, 2010 9:47 PM
> >> To: 'freebsd-pf at freebsd.org'; 'altq at csl.sony.co.jp'
> >> Subject: Questioning altq (cbq) performance above 4Mb on gif or
> above
> >> 40 Mb on e1000
> >>
> >> Howdy,
> >>
> >> (FreeBsd 8.0REL plus a modified e1000 driver borrowing the 8.1 altq
> >> capabilities, amd64, 8 core)
> >>
> >>         I have been measuring altq performance on my system. I set a
> >> cbq bandwith limit for a queue, match traffic to the queue with an
> >> appropriate rule and pump data.
> >>
> >>         Without altq enabled, my traffic gen tool (nuttcp) can pump
> >> 800Mb across my gig links.
> >>
> >>         With altq(cbq) setting bandwith limits (no borrow), up to
> about
> >> 40Mb on em0 altq keeps up. But from 40Mb bandwith limit and up, altq
> >> seems to slow down my traffic much slower than the bandwidth limits.
> >>
> >>         Same effect is observed on gif interfaces (where my real
> >> interest lies) but much worse. With altq(cbq) setting bandwith
> limits
> >> (no borrow), up to about 4Mb on gif0 altq keeps up. But from 4Mb
> >> bandwidth limit and up, altq seems to slow down my traffic much
> lower
> >> than the bandwidth limits.
> >>
> >>         I have a large file of sample data in excel formant which
> I'd
> >> like to share. Is there an appropriate way to share it with this
> list?
> >>
> >>         I know I'm not cpu bound (97% idle while nuttcp pushes 800Mb
> >> traffic and altq configured with a bandwidth limit of 50Mb actually
> >> passes 30Mb.)
> >>
> >>         I sort of suspect I'm mutex bound but have no idea how to
> test
> >> that.
> >>
> >>
> >>
> >>  (1) this config allows about 4Mb tcp traffic to pass:
> >> ----------cut------------------
> >> altq on em0 cbq bandwidth 100Mb queue { queue1, queue2 }
> >> queue queue1  bandwidth 1Mb priority 7 cbq  (default)
> >> queue queue2 bandwidth 50Mb priority 1 cbq
> >> pass out quick on gif10 inet proto tcp no state queue queue2
> >> pass out quick on gif10 proto icmp queue queue2
> >> ----------paste----------------
> >>
> >> (2) this config allows about 40 Mb tcp traffic to pass
> >> ----------cut------------------
> >> altq on em0 cbq bandwidth 100Mb queue { queue1, queue2 }
> >> queue queue1  bandwidth 1Mb priority 7 cbq  (default)
> >> queue queue2 bandwidth 50Mb priority 1 cbq
> >> pass out quick on em0 inet proto tcp no state queue queue2
> >> pass out quick on em0 proto icmp queue queue2
> >> ----------paste----------------
> >>
> >>         Yes, I send appropriate traffic in each case to be 'caught'
> by
> >> the tcp pass out rule wether it be found on a gif or an em.
> >>
> >>
> >>         My goal is to get 'expected' behavior (a bandwidth limit of
> 40
> >> Mb allows 40Mb to pass) at T3 link speed (45Mb). Is this a
> reasonable
> >> expectation? Any ideas about config jigs or tests to run?
> >>
> >>
> >>
> >> ---
> >> Ricky Charlet
> >> Adara Networks
> >> USA 408-433-4942
> >>
> >>
> >> _______________________________________________
> >> freebsd-pf at freebsd.org mailing list
> >> http://lists.freebsd.org/mailman/listinfo/freebsd-pf
> >> To unsubscribe, send any mail to "freebsd-pf-
> unsubscribe at freebsd.org"
> > _______________________________________________
> > freebsd-pf at freebsd.org mailing list
> > http://lists.freebsd.org/mailman/listinfo/freebsd-pf
> > To unsubscribe, send any mail to "freebsd-pf-unsubscribe at freebsd.org"
> >
>
>
>
> --
> Ermal