bce(4) - com_no_buffers (Again)

[David Christensen]

> >> Under testing I have yet to see a memory fragmentation issue with
> this
> >> driver.  I follow up if/when I find a problem with this again.
> >>
> >>
> So here we are again.  The system is locking up again because of 9k
> mbuf
> allocation failures.

Failure to allocate a new buffer should cause the driver to
drop the received frame and reuse the buffer, not lock up the
system.  Are you seeing the lockup come from bce(4) or does
it come from somewhere else due to the dropped data?

> >> Is there a way to fix the RX buffer shortage issues (when header
> >> splitting is turned on) so that they are guarded by flow control.
> Maybe
> >> change the low watermark for flow control when its enabled?
> >>
> >>
> > I'm not sure how much it would help but try changing RX low
> > watermark. Default value is 32 which seems to be reasonable value.
> > But it's only for 5709/5716 controllers and Linux seems to use
> > different default value.
> >
> These are: NetXtreme II BCM5709 Gigabit Ethernet
> 
> So my next task is to turn the watermark related defines into sysctls
> and turn on header splitting so that I can try to tune them without
> having to reboot.
>

Do you have flow control enabled?  There are arguments both for
and against flow control.  For bce(4), I haven't tested flow control
for quite a while and it's behavior may have changed since it is
controlled by firmware.   Keep an eye on the hardware statistics
to see that's it's actively generating pause frames.
> 
> My next question is, is it possible to increase the size of the RX ring
> without switching to RSS?
>

I have a change I've been working on to allow RX/TX ring size
to be adjusted through a sysctl.  Let me pretty it up a bit and
send it to you for test.  You should be able to adjust the ring
size without enabling RSS.

Dave