bce(4) - com_no_buffers (Again)

[Pyun YongHyeon]

On Mon, Sep 13, 2010 at 09:11:25PM +0200, Andre Oppermann wrote:
> On 13.09.2010 20:48, Pyun YongHyeon wrote:
> >On Mon, Sep 13, 2010 at 10:04:25AM -0500, Tom Judge wrote:
> >>Without BCE_JUMBO_HDRSPLIT then we see no errors.  With it we see number
> >>of errors, however the rate seems to be reduced compaired to the
> >>previous version of the driver.
> >>
> >
> >It seems there are issues in header splitting and it was disabled
> >by default. Header splitting reduces packet processing overhead in
> >upper layer so it's normal to see better performance with header
> >splitting.
> 
> I'm not sure that header splitting really helps much at least for TCP.
> The only place where it could make a difference is at socket buffer
> append time.  There the header get 'thrown away'.  With header splitting
> the first mbuf in the chain containing the header can be returned to the
> free pool.  Without header splitting it's just a offset change in the
> mbuf.
> 
> IIRC header splitting was introduced with the Tigeon cards which were
> the first programmable network cards and the first to support putting
> the header in a different mbuf.  Header splitting, in theory, could
> make a difference with zero copy sockets where the data portion in a
> separate mbuf is flipped by VM magic into userspace.  The trouble is
> that no driver fully supports the semantics required for page flipping
> and the zero copy code, if compiled in, is less much less optimized for
> the non-flipping case than the standard code path.  With the many dozen
> gigabyte per second memory copy bandwidth of current CPU's it remains
> questionable whether the page-flipping VM magic is actually faster than
> a plain kernel/userspace copy as in the standard code path.  I generally
> recommend not to use ZERO_COPY_SOCKETS.
> 
> I suspect in the case of the bce(4) driver the change in header splitting
> is probably not the cause of the performance difference.
> 

I'm under the impression the header splitting in bce(4) is for
LRO(opposite of TSO), not for VM magic to enable page flipping
tricks.

> -- 
> Andre