bce packet loss

[Kevin Oberman]

On Jul 5, 2011 4:49 PM, "Chuck Swiger" <cswiger at mac.com> wrote:
>
> On Jul 4, 2011, at 6:32 PM, Charles Sprickman wrote:
> > We're running a few 8.1-R servers with Broadcom bce interfaces (Dell
R510) and I'm seeing occasional packet loss on them (enough that it trips
nagios now and then).  Cabling seems fine as neither the switch nor the
sysctl info for the device show any errors/collisions/etc, however there is
one odd one, which is "dev.bce.1.stat_IfHCInBadOctets: 539369".  See [1]
below for full sysctl output.  The switch shows no errors but for "Dropped
packets 683868".
>
> A quick check of the #'s is showing an error rate of 8.6e-6, or about one
byte out of 100,000.  That's much better than what TCP/IP and ethernet were
designed for.  At a guess, your box or the switch occasionally gets busy
enough that it drops a few packets now and again.
>
> Regards,
> --
> -Chuck
>

1 in 10**6? That is totally excessive. The Ethernet spec requires no worse
than 10**13 and that is far worse than should ever be seen in the real
world. At one in a million, any remotely high volume transfer will crawl,
especially over a long path.

If dropped packets ate being reported, the most common cause is fan-in. If
two input ports are both trying to talk a line rate to a single output port,
the buffer will fill an packets will be dropped. Most switches do tail drop,
so queue management is terrible, compounding the effects.

R. Kevin Oberman, Network Engineer
Retired
kob6558 at gmail.com