Increasing number of "requests for jumbo clusters denied" in netstat -m

[Tom Judge]

Tim Chen wrote:
> My machine is an IBM HS21 blade server and the NIC is bce.
> 
> bce1: <Broadcom NetXtreme II BCM5708 1000Base-SX (B2)> mem
> 0xd8000000-0xd9ffffff  irq 19 at device 0.0 on pci6
> miibus1: <MII bus> on bce1
> bce1: Ethernet address: 00:1a:64:34:f2:fa
> bce1: [ITHREAD]
> bce1: ASIC (0x57081021); Rev (B2); Bus (PCI-X, 64-bit, 133MHz); B/C
> (0x03040405) ; Flags( MSI )
> 
> That machine is serving as a mail and web server. Recently I found that the
> number of "requests for jumbo clusters denied" in netstat -m increases all
> the time.
> 
> 1031/3469/4500 mbufs in use (current/cache/total)
> 510/3326/3836/32768 mbuf clusters in use (current/cache/total/max)
> 510/2278 mbuf+clusters out of packet secondary zone in use (current/cache)
> 1/1453/1454/8704 4k (page size) jumbo clusters in use
> (current/cache/total/max)
> 510/1086/1596/4352 9k jumbo clusters in use (current/cache/total/max)
> 0/0/0/2176 16k jumbo clusters in use (current/cache/total/max)
> 5871K/23105K/28977K bytes allocated to network (current/cache/total)
> 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
> 0/4337166/0 requests for jumbo clusters denied (4k/9k/16k)
> 0/0/0 sfbufs in use (current/peak/max)
> 0 requests for sfbufs denied
> 0 requests for sfbufs delayed
> 0 requests for I/O initiated by sendfile
> 0 calls to protocol drain routines
> 
> ifconfig shows:
> bce1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000
> 
> options=1bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,
> TSO4>
>         ether 00:1a:64:34:f2:fa
>         inet 192.168.152.152 netmask 0xffffff00 broadcast 192.168.152.255
>         media: Ethernet autoselect (1000baseSX <full-duplex>)
>         status: active
> 
> And in rc.conf:
> ifconfig_bce1="inet 192.168.152.152  netmask 255.255.255.0 mtu 9000"
> 
> Is the problem of increasing "requests for jumbo clusters denied" resulted
> from my "mtu 9000" jumbo frame setting?
> Will it be the problem of "bce" nic's driver or hardware?
> Most of all, how will my system be efftected by that ""requests for jumbo
> clusters denied" problem? Will it degrade
> the performance or it is harmless?
> 
> 
Second thread on this bug today :)

There is a bug in bce(4) that causes memory fragmentation, which results 
in denied mbuf requests as you are seeing.

You can correct this issue by applying the patch created by this command:

svn diff -r 198319:198320 http://svn.freebsd.org/base/head


Once you have applied the patch you need to add:

options        BCE_JUMBO_HDRSPLIT

To you kernel config and recompile and install the new kernel.


Hope this works for you.

Tom