Is there any way to limit the amount of data in an mbuf chain submitted to a driver?

Sat May 18 16:16:25 UTC 2013

Version numbers result from my work internally, and sometimes they
increment due to a build issue, or a bug found that needs to be corrected,
so unfortunately its not always a neat progression. I have thought at times
about having a separate revision sequence but that would make for other
issues, also not ever change made into the community source changes
the version either. Its all because of the driver living in two different
worlds you might say :)

I don't have the list at hand right now, what were you planning to deploy
instead?

Jack

On Sat, May 18, 2013 at 5:08 AM, Laurie Jennings <
laurie_jennings_1977 at yahoo.com> wrote:

> Can you outline the changes? Anything with the general processing? I have
> to make
> a case to hold off a deployment.
>
> and what happened to 9, 10, and 11?
>
> Laurie
>
>
> --- On *Fri, 5/17/13, Jack Vogel <jfvogel at gmail.com>* wrote:
>
>
> From: Jack Vogel <jfvogel at gmail.com>
> Subject: Re: Is there any way to limit the amount of data in an mbuf chain
> submitted to a driver?
> To: "Laurie Jennings" <laurie_jennings_1977 at yahoo.com>
> Cc: "FreeBSD Net" <freebsd-net at freebsd.org>, "Adrian Chadd" <
> adrian at freebsd.org>, "Richard Sharpe" <realrichardsharpe at gmail.com>
> Date: Friday, May 17, 2013, 3:32 PM
>
> No Laurie, I ran into an issue that I needed to resolve, and then my
> validation engineer
> went out of town a couple days. Should be early next week.
>
> Jack
>
>
>
> On Fri, May 17, 2013 at 3:25 PM, Laurie Jennings <
> laurie_jennings_1977 at yahoo.com<http://mc/compose?to=laurie_jennings_1977@yahoo.com>>
> wrote:
>
> > Did you get to commit that new code yet?
> >
> > Laurie
> >
> > --- On *Sat, 5/4/13, Jack Vogel <jfvogel at gmail.com<http://mc/compose?to=jfvogel@gmail.com>>*
> wrote:
> >
> >
> > From: Jack Vogel <jfvogel at gmail.com<http://mc/compose?to=jfvogel@gmail.com>
> >
> > Subject: Re: Is there any way to limit the amount of data in an mbuf
> chain
> > submitted to a driver?
> > To: "Richard Sharpe" <realrichardsharpe at gmail.com<http://mc/compose?to=realrichardsharpe@gmail.com>
> >
> > Cc: "FreeBSD Net" <freebsd-net at freebsd.org<http://mc/compose?to=freebsd-net@freebsd.org>>,
> "Adrian Chadd" <
> > adrian at freebsd.org <http://mc/compose?to=adrian@freebsd.org>>
> > Date: Saturday, May 4, 2013, 2:18 PM
> >
> >
> > Ahh, Twinville, new hardware :)  The version at the tip is 2.5.8 and I am
> > working on version 2.5.12 internally that I hope to commit next week...
> > so your version is "a bit old" :) I would do some testing on newer code.
> >
> > Jack
> >
> >
> >
> > On Sat, May 4, 2013 at 1:54 PM, Richard Sharpe
> > <realrichardsharpe at gmail.com<http://mc/compose?to=realrichardsharpe@gmail.com>
> <http://mc/compose?to=realrichardsharpe@gmail.com>
> > >wrote:
> >
> > > On Sat, May 4, 2013 at 1:41 PM, Jack Vogel <jfvogel at gmail.com<http://mc/compose?to=jfvogel@gmail.com>
> <http://mc/compose?to=jfvogel@gmail.com>>
>
> > wrote:
> > > > If you don't use TSO you will hurt your TX performance significantly
> > from
> > > > the tests that I've run. What exactly is the device you are using, I
> > > don't
> > > > have the source in front of me now, but I'm almost sure that the
> limit
> > is
> > > > not 64K but 256K, or are you using some ancient version of the
> driver?
> > >
> > >             ix0 pnpinfo vendor=0x8086 device=0x1528 subvendor=0x8086
> > > subdevice=0x0001 class=0x020000 at slot=0 function=0
> > >             ix1 pnpinfo vendor=0x8086 device=0x1528 subvendor=0x8086
> > > subdevice=0x0001 class=0x020000 at slot=0 function=1
> > >
> > > The version calls itself ixgbe-2.4.4 ...
> > >
> > > Hmmm, copyright is 2001-2010 ... so perhaps a bit old.
> > >
> > > > Jack
> > > >
> > > >
> > > >
> > > > On Sat, May 4, 2013 at 1:30 PM, Richard Sharpe <
> > > realrichardsharpe at gmail.com<http://mc/compose?to=realrichardsharpe@gmail.com>
> <http://mc/compose?to=realrichardsharpe@gmail.com>
> > >
> > > > wrote:
> > > >>
> > > >> On Sat, May 4, 2013 at 10:39 AM, Adrian Chadd <adrian at freebsd.org<http://mc/compose?to=adrian@freebsd.org>
> <http://mc/compose?to=adrian@freebsd.org>
> > >
> > > wrote:
> > > >> > On 4 May 2013 06:52, Richard Sharpe <realrichardsharpe at gmail.com<http://mc/compose?to=realrichardsharpe@gmail.com>
> <http://mc/compose?to=realrichardsharpe@gmail.com>
>
> > >
> > > wrote:
> > > >> >> Hi folks,
> > > >> >>
> > > >> >> I understand better why I am seeing EINVAL intermittently when
> > > sending
> > > >> >> data from Samba via SMB2.
> > > >> >>
> > > >> >> The ixgbe driver, for TSO reasons, limits the amount of data that
> > can
> > > >> >> be DMA'd to 65535 bytes. It returns EINVAL for any mbuf chain
> > larger
> > > >> >> than that.
> > > >> >>
> > > >> >> The SO_SNDBUF for that socket is set to 131972. Mostly there is
> > less
> > > >> >> than 64kiB of space available, so that is all TCP etc can put
> into
> > > the
> > > >> >> socket in one chain of mbufs. However, every now and then there
> is
> > > >> >> more than 65535 bytes available in the socket buffers, and we
> have
> > an
> > > >> >> SMB packet that is larger than 65535 bytes, and we get hit.
> > > >> >>
> > > >> >> To confirm this I am going to set SO_SNDBUF back to the default
> of
> > > >> >> 65536 and test again. My repros are very reliable.
> > > >> >>
> > > >> >> However, I wondered if my only way around this if I want to
> > continue
> > > >> >> to use SO_SNDBUF sizes larger than 65536 is to fragment large
> mbuf
> > > >> >> chains in the driver?
> > > >> >
> > > >> > Hm, is this is a problem without TSO?
> > > >>
> > > >> We are using the card without TSO, so I am thinking of changing that
> > > >> limit to 131072 and retesting.
> > > >>
> > > >> I am currently testing with SO_SNDBUF=32768 and have not hit the
> > > problem.
> > > >>
> > > >> > Is the problem that the NIC can't handle a frame that big, or a
> > buffer
> > > >> > that big?
> > > >> > Ie - if you handed the hardware two descriptors of 64k each, for
> the
> > > >> > same IP datagram, will it complain?
> > > >>
> > > >> I can't find any documentation, but it seems that with TSO it cannot
> > > >> handle a frame that big. Actually, since we are not using TSO, there
> > > >> really should not be a problem with larger frames.
> > > >>
> > > >> > Or do you need to break it up into two separate IP datagrams,
> facing
> > > >> > the driver, with a maximum size of 64k each?
> > > >>
> > > >> Not sure, but it looks like we need to do that.
> > > >>
> > > >>
> > > >> --
> > > >> Regards,
> > > >> Richard Sharpe
> > > >> (何以解憂？唯有杜康。--曹操)
> > > >> _______________________________________________
> > > >> freebsd-net at freebsd.org<http://mc/compose?to=freebsd-net@freebsd.org>
> <http://mc/compose?to=freebsd-net@freebsd.org>mailing list
> > > >> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> > > >> To unsubscribe, send any mail to "
> freebsd-net-unsubscribe at freebsd.org<http://mc/compose?to=freebsd-net-unsubscribe@freebsd.org>
> <http://mc/compose?to=freebsd-net-unsubscribe@freebsd.org>
>
> > "
> > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Regards,
> > > Richard Sharpe
> > > (何以解憂？唯有杜康。--曹操)
> > >
> > _______________________________________________
> > freebsd-net at freebsd.org <http://mc/compose?to=freebsd-net@freebsd.org> <
> http://mc/compose?to=freebsd-net@freebsd.org>mailing list
> > http://lists.freebsd.org/mailman/listinfo/freebsd-net
> > To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org<http://mc/compose?to=freebsd-net-unsubscribe@freebsd.org>
> <http://mc/compose?to=freebsd-net-unsubscribe@freebsd.org>
>
> > "
> >
> >
> _______________________________________________
> freebsd-net at freebsd.org <http://mc/compose?to=freebsd-net@freebsd.org>mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org<http://mc/compose?to=freebsd-net-unsubscribe@freebsd.org>
> "
>
>