svn commit: r296689 - head/sys/dev/cxgbe

[Slawa Olhovchenkov]

On Sun, Mar 13, 2016 at 12:15:18AM -0800, Navdeep Parhar wrote:

> On Sun, Mar 13, 2016 at 04:14:34AM +0300, Slawa Olhovchenkov wrote:
> > On Fri, Mar 11, 2016 at 11:24:05PM +0000, Navdeep Parhar wrote:
> > 
> > > Author: np
> > > Date: Fri Mar 11 23:24:04 2016
> > > New Revision: 296689
> > > URL: https://svnweb.freebsd.org/changeset/base/296689
> > > 
> > > Log:
> > >   cxgbe(4): sysctls to display the TOE's TCP timers.
> > >   
> > >   cask:~# sysctl -d dev.t5nex.0.toe
> > >   dev.t5nex.0.toe.finwait2_timer: FINWAIT2 timer (us)
> > >   dev.t5nex.0.toe.initial_srtt: Initial SRTT (us)
> > >   dev.t5nex.0.toe.keepalive_intvl: Keepidle interval (us)
> > >   dev.t5nex.0.toe.keepalive_idle: Keepidle idle timer (us)
> > >   dev.t5nex.0.toe.persist_max: Persist timer max (us)
> > >   dev.t5nex.0.toe.persist_min: Persist timer min (us)
> > >   dev.t5nex.0.toe.rexmt_max: Retransmit max (us)
> > >   dev.t5nex.0.toe.rexmt_min: Retransmit min (us)
> > >   dev.t5nex.0.toe.dack_timer: DACK timer (us)
> > >   dev.t5nex.0.toe.dack_tick: DACK tick (us)
> > >   dev.t5nex.0.toe.timestamp_tick: TCP timestamp tick (us)
> > >   dev.t5nex.0.toe.timer_tick: TP timer tick (us)
> > >   ...
> > >   
> > >   cask:~# sysctl dev.t5nex.0.toe
> > >   dev.t5nex.0.toe.finwait2_timer: 9765440
> > >   dev.t5nex.0.toe.initial_srtt: 244128
> > >   dev.t5nex.0.toe.keepalive_intvl: 73240800
> > >   dev.t5nex.0.toe.keepalive_idle: 7031116800
> > >   dev.t5nex.0.toe.persist_max: 9765440
> > >   dev.t5nex.0.toe.persist_min: 976544
> > >   dev.t5nex.0.toe.rexmt_max: 9765440
> > >   dev.t5nex.0.toe.rexmt_min: 244128
> > >   dev.t5nex.0.toe.dack_timer: 19520
> > >   dev.t5nex.0.toe.dack_tick: 32.768
> > >   dev.t5nex.0.toe.timestamp_tick: 1048.576
> > >   dev.t5nex.0.toe.timer_tick: 32.768
> > >   ...
> > 
> > Can you more describe it in man?
> 
> There are updates and additions to cxgbe documentation in progress,
> including new man pages: t4_tom(4) for the TCP offload module (aka TOE
> driver), cxgbei(4) for the iSCSI offload driver, and iw_cxgbe(4) for the
> iWARP driver.  I'll get these sysctls documented in t4_tom(4).  But
> writing decent documentation is harder than writing code and the effort
> is taking longer than I'd anticipated, so please be patient.  
> 
> > I am see this timers different from system, is this problem? What
> > correlations between?
> 
> The kernel's TCP stack is different from the TCP implementation in the
> chip's silicon and they happen to use different default timers.  Note
> that the driver requires you to explicitly load a module (t4_tom) and
> then enable IFCAP_TOE on the interfaces in order to use the hardware TCP
> offload.  This opt-in model is by design and makes sure you aren't using
> a stack that you didn't explicitly ask for.  Offloaded connections show
> up as "toe4/6" instead of "tcp4/6" in the output of netstat/sockstat/etc.
> so it's easy to tell which socket is using which stack (and which timers).

Please also describe how TOE connection handled (and limits, how many
connections can handled in silicon) and some recomendation (sndbuf?).
What happen when timer expired? Connection pass to handled to software
(and in this case silicon timers desirable have less then software?)
or connection totaly handled in silicon (and timers tuned specially?)