svn commit: r334804 - in head/sys: kern modules/tcp modules/tcp/rack netinet netinet/tcp_stacks sys

[hiren panchasara]

On 06/07/18 at 08:58P, Randall Stewart wrote:
> 
> 
> > On Jun 7, 2018, at 6:01 PM, hiren panchasara <hiren at strugglingcoder.info> wrote:
> > 
> > On 06/07/18 at 06:18P, Randall Stewart wrote:
> >> Author: rrs
> >> Date: Thu Jun  7 18:18:13 2018
> >> New Revision: 334804
> >> URL: https://svnweb.freebsd.org/changeset/base/334804
> >> 
> >> Log:
> >>  This commit brings in a new refactored TCP stack called Rack.
> >>  Rack includes the following features:
> >>   - A different SACK processing scheme (the old sack structures are not used).
> >>   - RACK (Recent acknowledgment) where counting dup-acks is no longer done
> >>          instead time is used to knwo when to retransmit. (see the I-D)
> >>   - TLP (Tail Loss Probe) where we will probe for tail-losses to attempt
> >>          to try not to take a retransmit time-out. (see the I-D)
> >>   - Burst mitigation using TCPHTPS
> >>   - PRR (partial rate reduction) see the RFC.
> >> 
> >>  Once built into your kernel, you can select this stack by either
> >>  socket option with the name of the stack is "rack" or by setting
> >>  the global sysctl so the default is rack.
> >> 
> >>  Note that any connection that does not support SACK will be kicked
> >>  back to the "default" base  FreeBSD stack (currently known as "default").
> >> 
> >>  To build this into your kernel you will need to enable in your
> >>  kernel:
> >>     makeoptions WITH_EXTRA_TCP_STACKS=1
> >>     options TCPHPTS
> >> 
> >>  Sponsored by:	Netflix Inc.
> >>  Differential Revision:		https://reviews.freebsd.org/D15525
> >> 
> >> Added:
> >>  head/sys/modules/tcp/rack/
> >>  head/sys/modules/tcp/rack/Makefile   (contents, props changed)
> >>  head/sys/netinet/tcp_stacks/rack.c   (contents, props changed)
> >>  head/sys/netinet/tcp_stacks/rack_bbr_common.h   (contents, props changed)
> >>  head/sys/netinet/tcp_stacks/sack_filter.c   (contents, props changed)
> >>  head/sys/netinet/tcp_stacks/sack_filter.h   (contents, props changed)
> >>  head/sys/netinet/tcp_stacks/tcp_rack.h   (contents, props changed)
> >> Modified:
> >>  head/sys/kern/uipc_sockbuf.c
> >>  head/sys/modules/tcp/Makefile
> >>  head/sys/netinet/tcp.h
> >>  head/sys/netinet/tcp_log_buf.h
> >>  head/sys/netinet/tcp_output.c
> >>  head/sys/netinet/tcp_stacks/fastpath.c
> >>  head/sys/netinet/tcp_timer.c
> >>  head/sys/netinet/tcp_timer.h
> >>  head/sys/netinet/tcp_var.h
> >>  head/sys/sys/mbuf.h
> >>  head/sys/sys/queue.h
> >>  head/sys/sys/sockbuf.h
> >>  head/sys/sys/time.h
> > 
> > I thought we'd have more time to review/test this. Looks like BSDCan
> > commit-spree in effect. :-)
> 
> The Phabricator review has been up since May 22nd. Thats over 2.5 weeks,
> this was also discussed on the Thursday conference calls.

Fair enough. (I am out of touch a little so shouldn't really complain ;-))
> > 
> > A few questions:
> > 1) Does RACK work reliably without HPTS? If yes, has that config been
> > tested?
> > 
> No it requires the pacer.
> 
> > 2) It looks like PRR is tied to RACK. Why did we go that route?
> > Shouldn't it be easily used with the 'default' stack also?
> > 
> 
> It is what I developed.. and I had no desire to work with the default stack. That
> is a fifth rail that no one wants touched.
> 
> > 3) Can new SACK be used with the traditional stack?
> 
> Well if you want to rework the base stack you might be able to do that :)
> 
> It would be quite some effort.. I think Robert wants eventually the old
> stack to be de-composed and then slowly work at getting more common
> code between them until eventually you can have a diff and somehow
> figure out how to integrate the two.

Okay. I believe there might be situations where we may want to still
keep the 'default' stack alive. I know Windows doesn't yet use RACK when
rtt is lesser than 10ms (or something like that), as an example.

Such optimizations (PRR, better SACK) should be made available to that
also if we see non-RACK having a viable future.

> 
> > 
> > 4) Where should manpage like info for RACK go? a new man-page or
> > extending tcp(4)? Info like how to enable system-wide or per socket
> > should go here.
> > 
> 
> The enable/disable or per-socket I think is in with the pluggable stack
> stuff. We might want a Rack man page.. have to think about it.
> 
> > 5) Any perf numbers to go along with this commit? Synthetic or
> > production numbers showing improvements in transfer speed or any other
> > impact on CPU usage (specially with HPTS) that you can share?
> > 
> 
> CPU will be more but we see close to a drop in rebuffers by about 12% I am told.

Great. Thanks.
> 
> > 6) In your testing, have you found cases where RACK does poorly compared
> > to the 'default' stack? Any recommendations on when should RACK be
> > enabled? (Something like this could go in the manpage.)
> 
> Nope. 

Okay. I assuming the 2 places this got tested at (NFLX and LLNW) would have
provided enough variety in terms of traffic types.

Thanks again,
Hiren
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 603 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/svn-src-head/attachments/20180607/72c9e415/attachment.sig>