Questioning altq (cbq) performance above 4Mb on gif or above 40
Mb on e1000
Ricky Charlet
RCharlet at adaranet.com
Fri Nov 5 23:55:45 UTC 2010
OK, so what is the theory of operation behind this (empirically effective) solution? I'll post my guess, please correct me.
<guess>
The slowness in the system is waiting on the timer to pop. The timer did not need to be set any quicker, just the granularity of how quickly the timers are run needed to be updated.
</guess>
> -----Original Message-----
> From: owner-freebsd-pf at freebsd.org [mailto:owner-freebsd-
> pf at freebsd.org] On Behalf Of Ricky Charlet
> Sent: Friday, November 05, 2010 1:43 PM
> To: 'Ermal Luçi'
> Cc: altq at csl.sony.co.jp; freebsd-pf at freebsd.org
> Subject: RE: Questioning altq (cbq) performance above 4Mb on gif or
> above 40 Mb on e1000
>
> Yes.. this works very well.
>
> the first thing I tried:
> I had a kern.hz of 1000 and changed to 10000 (through loader.conf +
> reboot)
> I had a queue depth of 50 and changed to 900 (through `qlimit`
> parameter in pf.conf)
>
>
> And now I am hitting my configured rate limits beyond my goal of 45Mb
> (T3 speed) even into 100 Mb range. I'm going to continue to experiment
> around with hz and q-depth and pick some eventual values for my
> implementation.
>
> But thanks, Emeral. This seems to be exactly the parameters to play
> around with to increase altq performance into the 50~100 Mb range.
>
> ---
> Ricky Charlet
> Adara Networks
> USA 408-433-4942
>
>
>
>
>
>
>
> > -----Original Message-----
> > From: ermal.luci at gmail.com [mailto:ermal.luci at gmail.com] On Behalf Of
> > Ermal Luçi
> > Sent: Friday, November 05, 2010 10:18 AM
> > To: Ricky Charlet
> > Cc: freebsd-pf at freebsd.org; altq at csl.sony.co.jp
> > Subject: Re: Questioning altq (cbq) performance above 4Mb on gif or
> > above 40 Mb on e1000
> >
> > On Fri, Nov 5, 2010 at 1:33 AM, Ricky Charlet
> <RCharlet at adaranet.com>
> > wrote:
> > > Has anyone out there run altq with cbq with bandwidth limits set
> > around 40 ~ 50 Mb and seen it work well (actual through put allowed
> to
> > come near that speed)?
> > >
> > > Thanks
> > > ---
> > > Ricky Charlet
> > > Adara Networks
> > > USA 408-433-4942
> > >
> >
> > I can suggest to try with higher hz and increase queue lengths in
> altq.
> > By default they are 50 which is quite low in that regard.
> >
> > >
> > >> -----Original Message-----
> > >> From: owner-freebsd-pf at freebsd.org [mailto:owner-freebsd-
> > >> pf at freebsd.org] On Behalf Of Ricky Charlet
> > >> Sent: Wednesday, November 03, 2010 9:47 PM
> > >> To: 'freebsd-pf at freebsd.org'; 'altq at csl.sony.co.jp'
> > >> Subject: Questioning altq (cbq) performance above 4Mb on gif or
> > above
> > >> 40 Mb on e1000
> > >>
> > >> Howdy,
> > >>
> > >> (FreeBsd 8.0REL plus a modified e1000 driver borrowing the 8.1
> altq
> > >> capabilities, amd64, 8 core)
> > >>
> > >> I have been measuring altq performance on my system. I set
> a
> > >> cbq bandwith limit for a queue, match traffic to the queue with an
> > >> appropriate rule and pump data.
> > >>
> > >> Without altq enabled, my traffic gen tool (nuttcp) can
> pump
> > >> 800Mb across my gig links.
> > >>
> > >> With altq(cbq) setting bandwith limits (no borrow), up to
> > about
> > >> 40Mb on em0 altq keeps up. But from 40Mb bandwith limit and up,
> altq
> > >> seems to slow down my traffic much slower than the bandwidth
> limits.
> > >>
> > >> Same effect is observed on gif interfaces (where my real
> > >> interest lies) but much worse. With altq(cbq) setting bandwith
> > limits
> > >> (no borrow), up to about 4Mb on gif0 altq keeps up. But from 4Mb
> > >> bandwidth limit and up, altq seems to slow down my traffic much
> > lower
> > >> than the bandwidth limits.
> > >>
> > >> I have a large file of sample data in excel formant which
> > I'd
> > >> like to share. Is there an appropriate way to share it with this
> > list?
> > >>
> > >> I know I'm not cpu bound (97% idle while nuttcp pushes
> 800Mb
> > >> traffic and altq configured with a bandwidth limit of 50Mb
> actually
> > >> passes 30Mb.)
> > >>
> > >> I sort of suspect I'm mutex bound but have no idea how to
> > test
> > >> that.
> > >>
> > >>
> > >>
> > >> (1) this config allows about 4Mb tcp traffic to pass:
> > >> ----------cut------------------
> > >> altq on em0 cbq bandwidth 100Mb queue { queue1, queue2 }
> > >> queue queue1 bandwidth 1Mb priority 7 cbq (default)
> > >> queue queue2 bandwidth 50Mb priority 1 cbq
> > >> pass out quick on gif10 inet proto tcp no state queue queue2
> > >> pass out quick on gif10 proto icmp queue queue2
> > >> ----------paste----------------
> > >>
> > >> (2) this config allows about 40 Mb tcp traffic to pass
> > >> ----------cut------------------
> > >> altq on em0 cbq bandwidth 100Mb queue { queue1, queue2 }
> > >> queue queue1 bandwidth 1Mb priority 7 cbq (default)
> > >> queue queue2 bandwidth 50Mb priority 1 cbq
> > >> pass out quick on em0 inet proto tcp no state queue queue2
> > >> pass out quick on em0 proto icmp queue queue2
> > >> ----------paste----------------
> > >>
> > >> Yes, I send appropriate traffic in each case to be
> 'caught'
> > by
> > >> the tcp pass out rule wether it be found on a gif or an em.
> > >>
> > >>
> > >> My goal is to get 'expected' behavior (a bandwidth limit
> of
> > 40
> > >> Mb allows 40Mb to pass) at T3 link speed (45Mb). Is this a
> > reasonable
> > >> expectation? Any ideas about config jigs or tests to run?
> > >>
> > >>
> > >>
> > >> ---
> > >> Ricky Charlet
> > >> Adara Networks
> > >> USA 408-433-4942
> > >>
> > >>
> > >> _______________________________________________
> > >> freebsd-pf at freebsd.org mailing list
> > >> http://lists.freebsd.org/mailman/listinfo/freebsd-pf
> > >> To unsubscribe, send any mail to "freebsd-pf-
> > unsubscribe at freebsd.org"
> > > _______________________________________________
> > > freebsd-pf at freebsd.org mailing list
> > > http://lists.freebsd.org/mailman/listinfo/freebsd-pf
> > > To unsubscribe, send any mail to "freebsd-pf-
> unsubscribe at freebsd.org"
> > >
> >
> >
> >
> > --
> > Ermal
> _______________________________________________
> freebsd-pf at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-pf
> To unsubscribe, send any mail to "freebsd-pf-unsubscribe at freebsd.org"
More information about the freebsd-pf
mailing list