Is there any way to limit the amount of data in an mbuf chain submitted to a driver?

[Jack Vogel]

Ahh, Twinville, new hardware :)  The version at the tip is 2.5.8 and I am
working on version 2.5.12 internally that I hope to commit next week...
so your version is "a bit old" :) I would do some testing on newer code.

Jack



On Sat, May 4, 2013 at 1:54 PM, Richard Sharpe
<realrichardsharpe at gmail.com>wrote:

> On Sat, May 4, 2013 at 1:41 PM, Jack Vogel <jfvogel at gmail.com> wrote:
> > If you don't use TSO you will hurt your TX performance significantly from
> > the tests that I've run. What exactly is the device you are using, I
> don't
> > have the source in front of me now, but I'm almost sure that the limit is
> > not 64K but 256K, or are you using some ancient version of the driver?
>
>             ix0 pnpinfo vendor=0x8086 device=0x1528 subvendor=0x8086
> subdevice=0x0001 class=0x020000 at slot=0 function=0
>             ix1 pnpinfo vendor=0x8086 device=0x1528 subvendor=0x8086
> subdevice=0x0001 class=0x020000 at slot=0 function=1
>
> The version calls itself ixgbe-2.4.4 ...
>
> Hmmm, copyright is 2001-2010 ... so perhaps a bit old.
>
> > Jack
> >
> >
> >
> > On Sat, May 4, 2013 at 1:30 PM, Richard Sharpe <
> realrichardsharpe at gmail.com>
> > wrote:
> >>
> >> On Sat, May 4, 2013 at 10:39 AM, Adrian Chadd <adrian at freebsd.org>
> wrote:
> >> > On 4 May 2013 06:52, Richard Sharpe <realrichardsharpe at gmail.com>
> wrote:
> >> >> Hi folks,
> >> >>
> >> >> I understand better why I am seeing EINVAL intermittently when
> sending
> >> >> data from Samba via SMB2.
> >> >>
> >> >> The ixgbe driver, for TSO reasons, limits the amount of data that can
> >> >> be DMA'd to 65535 bytes. It returns EINVAL for any mbuf chain larger
> >> >> than that.
> >> >>
> >> >> The SO_SNDBUF for that socket is set to 131972. Mostly there is less
> >> >> than 64kiB of space available, so that is all TCP etc can put into
> the
> >> >> socket in one chain of mbufs. However, every now and then there is
> >> >> more than 65535 bytes available in the socket buffers, and we have an
> >> >> SMB packet that is larger than 65535 bytes, and we get hit.
> >> >>
> >> >> To confirm this I am going to set SO_SNDBUF back to the default of
> >> >> 65536 and test again. My repros are very reliable.
> >> >>
> >> >> However, I wondered if my only way around this if I want to continue
> >> >> to use SO_SNDBUF sizes larger than 65536 is to fragment large mbuf
> >> >> chains in the driver?
> >> >
> >> > Hm, is this is a problem without TSO?
> >>
> >> We are using the card without TSO, so I am thinking of changing that
> >> limit to 131072 and retesting.
> >>
> >> I am currently testing with SO_SNDBUF=32768 and have not hit the
> problem.
> >>
> >> > Is the problem that the NIC can't handle a frame that big, or a buffer
> >> > that big?
> >> > Ie - if you handed the hardware two descriptors of 64k each, for the
> >> > same IP datagram, will it complain?
> >>
> >> I can't find any documentation, but it seems that with TSO it cannot
> >> handle a frame that big. Actually, since we are not using TSO, there
> >> really should not be a problem with larger frames.
> >>
> >> > Or do you need to break it up into two separate IP datagrams, facing
> >> > the driver, with a maximum size of 64k each?
> >>
> >> Not sure, but it looks like we need to do that.
> >>
> >>
> >> --
> >> Regards,
> >> Richard Sharpe
> >> (何以解憂？唯有杜康。--曹操)
> >> _______________________________________________
> >> freebsd-net at freebsd.org mailing list
> >> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> >> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
> >
> >
>
>
>
> --
> Regards,
> Richard Sharpe
> (何以解憂？唯有杜康。--曹操)
>