igb(4) Raising IGB_MAX_TXD ??
seanbru at yahoo-inc.com
Wed Apr 18 16:49:53 UTC 2012
ok, good. that at least confirms that I correctly translated between
the driver code and documented specification.
I will try 8k as a test for now and see how that runs.
On Wed, 2012-04-18 at 09:46 -0700, Jack Vogel wrote:
> The MAX value is something I set, not a hardware thing, it was based
> on reports
> I had from the various driver engineers in our org. If you increase
> the ring size
> you might run into other performance issues, however there's nothing
> you from trying. Just be aware that its not something that's been
> Let me know how it goes please :)
> On Wed, Apr 18, 2012 at 9:27 AM, Sean Bruno <seanbru at yahoo-inc.com>
> On Wed, 2012-04-18 at 00:28 -0700, Luigi Rizzo wrote:
> > On Tue, Apr 17, 2012 at 04:24:24PM -0700, Sean Bruno wrote:
> > > We're running a service with a 82576 configured with 4
> queues and a
> > > maxed rxd/txd configuration:
> > >
> > > http://people.freebsd.org/~sbruno/igb_stats.txt
> > these stats show that over half of your incoming traffic is
> > made of small packets (65..127 bytes) but especially, that
> > the "missed packets" count is very small (18k out of 40G
> > none of them is reported as "no_desc_avail", and only 76 are
> > "recv_no_buffer".
> > Are you dropping packets in the ip interrupt handler by
> chance ?
> > what are your settings there ?
> nope, doesn't look like it.
> > BTW it seems that there is only one global setting for the
> > policy, but for instance there are two netisr_dispatch()
> > in the incoming path, one for layer2 and one for layer3.
> > The former has relatively little work to do and so it might
> > make sense to have direct dispatch, the other can be
> > so i wonder if it wouldn't be better to use deferred
> > If not, perhaps you might try to reduce the
> > to bring down the load on the intr thread.
> I don't really see any issue with horsepower on this host at
> the moment
> with 4 queues. I mean it looks a little something like this
> under high
> I guess my question still stands though, since the ethernet
> is reporting that it doesn't have any more descriptors
> available is the
> hardcoded 4k max descriptors a limit that an be raised?
> > With your numbers i doubt that raising the queue size helps.
> Indeed, you're probably right and this is more than likely an
> application problem that will have to be resolved. However,
> I'm still
> curious if the MAX_RXD/TXD is really 4k or if the
> documentation is
> correct and we can raise it to 32k for testing?
More information about the freebsd-net