vmstat buckets

[Mark van der Meulen]

Hi All,

I have question about buckets in vmstat -z output which I haven’t been able to find an answer for online…

I have 6 FreeBSD 9 boxes running as routers and they are seeing varying results in vmstat -z. I’m interested in understanding what the Buckets represent/mean and how I can influence usage of them.

Some of the routers show 0 free buckets under 64 or 128 Bucket and so I did some research and haven’t been able to find out what it means - most people who have asked online have received an answer that it is nothing to worry about, which is meaningless to me.

It seems that the higher the usage of the router, especially when it is also running userspace applications the less Bucket availability there is. Also I found someone speculating it was related to free kernel memory so that was updated on some routers however I saw little difference(although not sure if adding it in /etc/sysctl.conf actually does anything as some haven’t had the vm.kmem_size value change at all, despite reboots).

Can anyone explain what the buckets (16,32,64,128) do and what they represent? Also if you know anything about troubleshooting bucket failures? There doesn’t appear to be anything useful online despite many searches.

I am aware that it may not have anything to do with the problems I am seeing, but I would still like to understand.

Thanks,

Mark


-------------------------------------------------------------------------------

Background:

I have a number of FreeBSD 9 boxes running as routers in a testing environment where I am seeing odd latency issues. Most of them have different hardware and configurations now just to rule out configuration or hardware issues.

What I am seeing is occasional latency increases when routing traffic so that on the “client” end the experience is like the link has hung for a split second - this happens for both L2TP/PPPoE clients and plain ethernet connected clients. I haven’t been able to identify anything obvious like fragmentation, mtu issues, etc.

The setup is:

3 x Dual Quad Core, 8GB RAM running as LAC & LNS w/ Quagga(BGP, OSPF), MPD5, Netflow, IPFW Stateless(recently introduced and my problem existed before filtering was introduced).
3 x Dual Quad Core, 8GB RAM running as Routers w/ Quagga(BGP, OSPF)

Router 1
========

ITEM                   SIZE  LIMIT     USED     FREE      REQ FAIL SLEEP
16 Bucket:              152,      0,     198,       2,     198,   0,   0
32 Bucket:              280,      0,     291,       3,     291,  16,   0
64 Bucket:              536,      0,     221,       3,     221,  58,   0
128 Bucket:            1048,      0,    7993,       2,    7993, 505,   0
vm.kmem_size: 4127952896

Router 2(LNS)
========
ITEM                   SIZE  LIMIT     USED     FREE      REQ FAIL SLEEP
16 Bucket:              152,      0,     199,       1,     199,   0,   0
32 Bucket:              280,      0,     329,       7,     333,   0,   0
64 Bucket:              536,      0,     241,       4,     248,  57,   0
128 Bucket:            1048,      0,    7960,       2,    9479,1002,   0
vm.kmem_size: 2147483648

Router 3(LNS)
========
ITEM                   SIZE  LIMIT     USED     FREE      REQ FAIL SLEEP
16 Bucket:              152,      0,     194,       6,     194,   0,   0
32 Bucket:              280,      0,     325,      11,     325,  20,   0
64 Bucket:              536,      0,     278,       2,     278,  57,   0
128 Bucket:            1048,      0,    8031,       0,    8031, 955,   0
vm.kmem_size: 4127952896

Router 4(LNS)
========
ITEM                   SIZE  LIMIT     USED     FREE      REQ FAIL SLEEP
16 Bucket:              152,      0,     197,       3,     197,   0,   0
32 Bucket:              280,      0,     265,       1,     265,   0,   0
64 Bucket:              536,      0,     227,       4,     227,  57,   0
128 Bucket:            1048,      0,    7824,       0,    7824, 721,   0
vm.kmem_size: 2147483648

Router 5
========
ITEM                   SIZE  LIMIT     USED     FREE      REQ FAIL SLEEP
16 Bucket:              152,      0,     204,      21,     204,   0,   0
32 Bucket:              280,      0,     313,       9,     313,   1,   0
64 Bucket:              536,      0,     268,       5,     268,  57,   0
128 Bucket:            1048,      0,    9204,       0,    9204,5723,   0
vm.kmem_size: 4127952896

Router 6
========
ITEM                   SIZE  LIMIT     USED     FREE      REQ FAIL SLEEP
16 Bucket:              152,      0,      80,      20,      80,   0,   0
32 Bucket:              280,      0,      86,      12,      86,   0,   0
64 Bucket:              536,      0,      49,       0,      49,  57,   0
128 Bucket:            1048,      0,    8697,       0,    8697,5604,   0
vm.kmem_size: 3108118528