FreeBSD problems and preliminary ways to solve

[Slawa Olhovchenkov]

On Fri, Aug 19, 2011 at 09:56:02AM +0100, Robert Watson wrote:

> 
> On Fri, 19 Aug 2011, Lev Serebryakov wrote:
> 
> >> Our network stack is actually pretty parallel as such things go, and there are
> >> a number of changes in FreeBSD 9.x that extend this work.  Most of the
> >> performance work is being done on edge nodes rather than middle nodes -- i.e.,
> >> maxing out multiple 10gbps links serving content, etc, rather than in routing
> >> configurations, though.  We also have a strong and growing collection of
> >> 10gbps drivers.  You'll find our drivers lifted for many other systems,
> >> including Solaris :-).
> >
> >  I need to bribe our admins (OPs) on my paid work to try FreeBSD instead of 
> > Solaris on new servers :) We are processing huge amount of multicast streams 
> > (up to 2.5-3Gbit/s with 500-1000 bytes packets) and have difficulties not to 
> > lose any packets on Solaris :) They tried Linux without success, but FreeBSD 
> > is unknown to them.
> >
> > One problem for FreeBSD is that our applications are Java-based... Problems 
> > are not in Java, but in Intel drivers (igb / ixgb in FreeBSD terms), which 
> > sometimes lose packets with "buffer is not available" diagnostics when 
> > consumer is heavily-multithreaded.
> 
> Is the issue here that FreeBSD is dropping more packes, or just that FreeBSD 
> is reporting that it drops packets?  Historically, we've returned ENOBUFS from 
> datagram sockets when the interface queue is overflowed, but some other 
> systems (most noticeably Linux) simply return success when they drop a packet 
> on an outgoing interface queue.  You can debate which is the better model, but 
> one impact is that sometimes people report errors on FreeBSD that they don't 
> see on Linux -- when actually, the same failure is present, we just allow the 
> application to learn about it.

Historically, Linux on datagram (UDP) socket allow use select, FreeBSD
-- don't allow. FreeBSD always report 'UDP socket ready to transmit'.
And after try to send packet -- 'oops, ENOBUFS'.




-- 
Slawa Olhovchenkov