9.2 ixgbe tx queue hang
Rick Macklem
rmacklem at uoguelph.ca
Thu Mar 27 00:35:50 UTC 2014
Christopher Forgeron wrote:
> That's interesting. I see here in the r251296 commit Andre says :
>
> Drivers can set ifp->if_hw_tsomax before calling ether_ifattach()
> to
> change the limit.
>
> I wonder if we add your same TSO patch to if_lagg.c before line
> 356's
> ether_ifattach() will fix it.
>
I think the value(s) for underlying hardware drivers have to somehow
be propagated up through lagg. I haven't looked at the code, so I
don't know what that would be.
Putting the patch for ixgbe.c in lagg wouldn't make sense, since it
doesn't know if the underlying devices have the 32 limit.
I've suggested in the other thread what you suggested in a recent
post...ie. to change the default, at least until the propagation
of driver set values is resolved.
rick
> Ultimately, it will need to load the if_hw_tsomax from the if below
> it -
> but then again, if the calculation for ixgbe is good enough for that
> driver, why wouldn't it be good enough for lagg?
>
> Unless people think I'm crazy, I'll compile that in at line 356 in
> if_lagg.c and give it a test run tomorrow.
>
> This may need to go into vlan and carp as well, I'm not sure yet.
>
>
> On Tue, Mar 25, 2014 at 8:16 PM, Rick Macklem <rmacklem at uoguelph.ca>
> wrote:
>
> > Christopher Forgeron wrote:
> > > Update:
> > >
> > > I'm changing my mind, and I believe Rick's TSO patch is fixing
> > > things
> > > (sorry). In looking at my notes, it's possible I had lagg on for
> > > those
> > > tests. lagg does seem to negate the TSO patch in my case.
> > >
> > Ok, that's useful information. It implies that r251296 doesn't
> > quite
> > work and needs to be fixed for "stacked" network interface drivers
> > before it can be used. I've cc'd Andre who is the author of that
> > patch, in case he knows how to fix it.
> >
> > Thanks for checking this, rick
> >
> > > kernel.10stable_basicTSO_65535/
> > >
> > > - IP_MAXPACKET = 65535;
> > > - manually forced (no if statement) ifp->if_hw_tsomax =
> > > IP_MAXPACKET
> > > -
> > > (ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN);
> > > - Verified on boot via printf that ifp->if_hw_tsomax = 65517
> > > - Boot in a NON LAGG environment. ix0 only.
> > >
> > > ixgbe's printf is showing packets up to 65530. Haven't run long
> > > enough yet
> > > to see if anything will go over 65535
> > >
> > > I have this tcpdump running to check packet size.
> > > tcpdump -ennvvXS -i ix0 greater 65518
> > >
> > > I do expect to get packets over 65518, but I was just curious to
> > > see
> > > if any
> > > of them would go over 65535. Time will tell.
> > >
> > > In a separate test, If I enable lagg, we have LOTS of oversized
> > > packet
> > > problems. It looks like tsomax is definitely not making it
> > > through in
> > > if_lagg.c - Any recommendations there? I will eventually need
> > > lagg,
> > > as I'm
> > > sure will others.
> > >
> > > With dtrace, it's showing t_tsomax >= 65518. Shouldn't that not
> > > be
> > > happening?
> > >
> > >
> > > dtrace -n 'fbt::tcp_output:entry / args[0]->t_tsomax != 0 &&
> > > args[0]->t_tsomax >= 65518 / { printf("unexpected tp->t_tsomax:
> > > %i\n",
> > > args[0]->t_tsomax); stack(); }'
> > >
> > >
> > > 6 31403 tcp_output:entry unexpected
> > > tp->t_tsomax:
> > > 65535
> > >
> > > kernel`tcp_do_segment+0x2c99
> > > kernel`tcp_input+0x11a2
> > > kernel`ip_input+0xa2
> > > kernel`netisr_dispatch_src+0x5e
> > > kernel`ether_demux+0x12a
> > > kernel`ether_nh_input+0x35f
> > > kernel`netisr_dispatch_src+0x5e
> > > kernel`bce_intr+0x765
> > > kernel`intr_event_execute_handlers+0xab
> > > kernel`ithread_loop+0x96
> > > kernel`fork_exit+0x9a
> > > kernel`0xffffffff80c75b2e
> > >
> > > 3 31403 tcp_output:entry unexpected
> > > tp->t_tsomax:
> > > 65535
> > >
> > > kernel`tcp_do_segment+0x2c99
> > > kernel`tcp_input+0x11a2
> > > kernel`ip_input+0xa2
> > > kernel`netisr_dispatch_src+0x5e
> > > kernel`ether_demux+0x12a
> > > kernel`ether_nh_input+0x35f
> > > kernel`netisr_dispatch_src+0x5e
> > > kernel`bce_intr+0x765
> > > kernel`intr_event_execute_handlers+0xab
> > > kernel`ithread_loop+0x96
> > > kernel`fork_exit+0x9a
> > > kernel`0xffffffff80c75b2e
> > >
> > > 6 31403 tcp_output:entry unexpected
> > > tp->t_tsomax:
> > > 65535
> > >
> > > kernel`tcp_do_segment+0x2c99
> > > kernel`tcp_input+0x11a2
> > > kernel`ip_input+0xa2
> > > kernel`netisr_dispatch_src+0x5e
> > > kernel`ether_demux+0x12a
> > > kernel`ether_nh_input+0x35f
> > > kernel`netisr_dispatch_src+0x5e
> > > kernel`bce_intr+0x765
> > > kernel`intr_event_execute_handlers+0xab
> > > kernel`ithread_loop+0x96
> > > kernel`fork_exit+0x9a
> > > kernel`0xffffffff80c75b2e
> > >
> > > 1 31403 tcp_output:entry unexpected
> > > tp->t_tsomax:
> > > 65535
> > >
> > > kernel`tcp_do_segment+0x2c99
> > > kernel`tcp_input+0x11a2
> > > kernel`ip_input+0xa2
> > > kernel`netisr_dispatch_src+0x5e
> > > kernel`ether_demux+0x12a
> > > kernel`ether_nh_input+0x35f
> > > kernel`netisr_dispatch_src+0x5e
> > > kernel`bce_intr+0x765
> > > kernel`intr_event_execute_handlers+0xab
> > > kernel`ithread_loop+0x96
> > > kernel`fork_exit+0x9a
> > > kernel`0xffffffff80c75b2e
> > > _______________________________________________
> > > freebsd-net at freebsd.org mailing list
> > > http://lists.freebsd.org/mailman/listinfo/freebsd-net
> > > To unsubscribe, send any mail to
> > > "freebsd-net-unsubscribe at freebsd.org"
> > >
> >
> _______________________________________________
> freebsd-net at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to
> "freebsd-net-unsubscribe at freebsd.org"
>
More information about the freebsd-net
mailing list