tcp_output starving -- is due to mbuf get delay?

[Mike Silbersack]

On Thu, 10 Apr 2003, Borje Josefsson wrote:

> What we did in NetBSD (-current) was to increase IFQ_MAXLEN in (their)
> sys/net/if.h, apart from that it's only "traditional" TCP tuning.
>
> My hosts are connected directly to core routers in a 10Gbps nationwide
> network, so if anybody is interested in some testing I am more than
> willing to participate. If anybody produces a patch, I have a third system
> that I can use for piloting of that too.
>
> --Börje

This brings up something I've been wondering about, which you might want
to investigate:

From tcp_output:

		if (error == ENOBUFS) {
	                if (!callout_active(tp->tt_rexmt) &&
                            !callout_active(tp->tt_persist))
	                        callout_reset(tp->tt_rexmt, tp->t_rxtcur,
                                      tcp_timer_rexmt, tp);
			tcp_quench(tp->t_inpcb, 0);
			return (0);
		}

That tcp_quench knocks the window size back to one packet, if I'm not
mistaken.  You might want to put a counter there and see if that's
happening frequently to you; if so, it might explain some loss of
performance.

Have you tried running kernel profiling yet?  It would be interesting to
see which functions are using up the largest amount of time.

Mike "Silby" Silbersack