netstat fails with memory allocation error and error in kvm_read
J. Martin Petersen
techlists at motrix.dk
Thu Dec 16 05:11:27 PST 2004
> You might also log 'sysctl vm.kvm_free' and 'sysctl vm.zone'.
I just got another "netstat: kvm_read: Bad address". vm.kvm_free was
675278848 two seconds earlier and a couple of seconds later (and also
currently).
The output from vm.zone a couple of seconds after:
vm.zone:
ITEM SIZE LIMIT USED FREE REQUESTS
pfosfp: 28, 0, 155, 226, 1085
pfospfen: 108, 0, 293, 31, 2051
pfiaddrpl: 92, 0, 5, 79, 28
pfstatescrub: 8, 0, 26676, 28540, 37487180
pffrcent: 12, 50141, 0, 0, 0
pffrcache: 48, 10062, 0, 0, 0
pffrag: 48, 0, 0, 234, 378044
pffrent: 16, 5075, 0, 203, 934804
pfrkentry: 152, 0, 4, 48, 8
pfrktable: 248, 0, 4, 28, 12
pfpooladdrpl: 68, 0, 21, 91, 126
pfaltqpl: 128, 0, 19, 71, 133
pfstatepl: 256, 65010, 14396, 23014, 29835551
pfrulepl: 560, 0, 248, 221, 1693
pfsrctrpl: 80, 0, 0, 0, 0
FFS2 dinode: 256, 0, 32828, 427, 1154768
FFS1 dinode: 128, 0, 0, 0, 0
FFS inode: 140, 0, 32828, 352, 1154768
SWAPMETA: 276, 63672, 9, 47, 40
rtentry: 132, 0, 219, 187, 6666
unpcb: 140, 17220, 9, 47, 5217
ripcb: 180, 8140, 0, 44, 15
sackhole: 16, 0, 0, 203, 3107
tcpreass: 20, 1183, 0, 169, 134
hostcache: 88, 15400, 3, 85, 39
syncache: 108, 15372, 0, 72, 374
tcptw: 56, 1675, 2, 132, 48
tcpcb: 448, 8136, 4, 23, 489
inpcb: 180, 8140, 6, 38, 489
udpcb: 180, 8140, 11, 33, 6297398
socket: 324, 8136, 24, 36, 6303121
KNOTE: 68, 0, 0, 112, 12538429
PIPE: 384, 0, 2, 78, 251051
DIRHASH: 1024, 0, 1621, 11, 4318
NAMEI: 1024, 0, 3, 17, 131064562
L VFS Cache: 291, 0, 38, 456, 8482
S VFS Cache: 68, 0, 29954, 5550, 1214821
VNODEPOLL: 64, 0, 0, 118, 2
VNODE: 264, 0, 33205, 20, 33205
ata_request: 200, 0, 0, 38, 2322599
g_bio: 132, 0, 0, 2204, 4301499
MbufClust: 2048, 17216, 906, 124, 12121216
Mbuf: 256, 0, 907, 188, 2520398988
Packet: 256, 0, 740, 355, 2728311990
VMSPACE: 300, 0, 23, 81, 726569
UPCALL: 44, 0, 0, 0, 0
KSEGRP: 104, 0, 144, 31, 144
TID: 140, 0, 1, 53, 1
THREAD: 388, 0, 144, 6, 144
PROC: 452, 0, 73, 71, 726627
Files: 68, 0, 71, 153, 122894112
4096: 4096, 0, 129, 60, 746477
2048: 2048, 0, 130, 30, 24039
1024: 1024, 0, 23, 61, 14341894
512: 512, 0, 244, 92, 656520
256: 256, 0, 256, 1094, 7153814
128: 128, 0, 1842, 318, 15428344
64: 64, 0, 4481, 1183, 8923514
32: 32, 0, 2595, 682, 4033338850
16: 16, 0, 1930, 303, 22115093
DP fakepg: 72, 0, 0, 0, 0
PV ENTRY: 24, 944530, 5540, 17080, 315728548
MAP ENTRY: 68, 0, 439, 1241, 32683625
KMAP ENTRY: 68, 32200, 40, 464, 5444457
MAP: 192, 0, 7, 33, 5
VM OBJECT: 132, 0, 32560, 1428, 14688659
128 Bucket: 524, 0, 966, 0, 0
64 Bucket: 268, 0, 30, 40, 0
32 Bucket: 140, 0, 30, 26, 0
16 Bucket: 76, 0, 15, 35, 0
UMA Hash: 128, 0, 2, 28, 0
UMA RCntSlab: 104, 0, 515, 3, 0
UMA Slabs: 64, 0, 793, 33, 0
UMA Zones: 88, 0, 73, 7, 0
UMA Kegs: 136, 0, 73, 23, 0
vm.zone a couple of seconds before:
ITEM SIZE LIMIT USED FREE REQUESTS
pfosfp: 28, 0, 155, 226, 1085
pfospfen: 108, 0, 293, 31, 2051
pfiaddrpl: 92, 0, 5, 79, 28
pfstatescrub: 8, 0, 27146, 28070, 37485813
pffrcent: 12, 50141, 0, 0, 0
pffrcache: 48, 10062, 0, 0, 0
pffrag: 48, 0, 0, 234, 378044
pffrent: 16, 5075, 0, 203, 934804
pfrkentry: 152, 0, 4, 48, 8
pfrktable: 248, 0, 4, 28, 12
pfpooladdrpl: 68, 0, 21, 91, 126
pfaltqpl: 128, 0, 19, 71, 133
pfstatepl: 256, 65010, 14858, 22552, 29834555
pfrulepl: 560, 0, 248, 221, 1693
pfsrctrpl: 80, 0, 0, 0, 0
FFS2 dinode: 256, 0, 32820, 435, 1154765
FFS1 dinode: 128, 0, 0, 0, 0
FFS inode: 140, 0, 32820, 360, 1154765
SWAPMETA: 276, 63672, 9, 47, 40
rtentry: 132, 0, 222, 184, 6666
unpcb: 140, 17220, 9, 47, 5217
ripcb: 180, 8140, 0, 44, 15
sackhole: 16, 0, 0, 203, 3107
tcpreass: 20, 1183, 0, 169, 134
hostcache: 88, 15400, 1, 87, 39
syncache: 108, 15372, 0, 72, 374
tcptw: 56, 1675, 0, 134, 48
tcpcb: 448, 8136, 4, 23, 489
inpcb: 180, 8140, 4, 40, 489
udpcb: 180, 8140, 11, 33, 6296980
socket: 324, 8136, 24, 36, 6302703
KNOTE: 68, 0, 0, 112, 12537593
PIPE: 384, 0, 2, 78, 251017
DIRHASH: 1024, 0, 1621, 11, 4318
NAMEI: 1024, 0, 3, 17, 131046202
L VFS Cache: 291, 0, 38, 456, 8482
S VFS Cache: 68, 0, 29953, 5551, 1214811
VNODEPOLL: 64, 0, 0, 118, 2
VNODE: 264, 0, 33205, 20, 33205
ata_request: 200, 0, 0, 38, 2322417
g_bio: 132, 0, 0, 2204, 4301228
MbufClust: 2048, 17216, 906, 124, 12121216
Mbuf: 256, 0, 907, 188, 2520288299
Packet: 256, 0, 674, 421, 2728222296
VMSPACE: 300, 0, 23, 81, 726488
UPCALL: 44, 0, 0, 0, 0
KSEGRP: 104, 0, 144, 31, 144
TID: 140, 0, 1, 53, 1
THREAD: 388, 0, 144, 6, 144
PROC: 452, 0, 73, 71, 726546
Files: 68, 0, 71, 153, 122876814
4096: 4096, 0, 129, 60, 746392
2048: 2048, 0, 130, 30, 24033
1024: 1024, 0, 23, 61, 14340393
512: 512, 0, 244, 92, 656448
256: 256, 0, 239, 1111, 7152982
128: 128, 0, 1785, 375, 15426920
64: 64, 0, 4243, 1421, 8922809
32: 32, 0, 2519, 758, 4033195752
16: 16, 0, 1928, 305, 22113261
DP fakepg: 72, 0, 0, 0, 0
PV ENTRY: 24, 944530, 5540, 17080, 315699586
MAP ENTRY: 68, 0, 439, 1241, 32680232
KMAP ENTRY: 68, 32200, 40, 464, 5443857
MAP: 192, 0, 7, 33, 5
VM OBJECT: 132, 0, 32552, 1436, 14687110
128 Bucket: 524, 0, 966, 0, 0
64 Bucket: 268, 0, 30, 40, 0
32 Bucket: 140, 0, 30, 26, 0
16 Bucket: 76, 0, 15, 35, 0
UMA Hash: 128, 0, 2, 28, 0
UMA RCntSlab: 104, 0, 515, 3, 0
UMA Slabs: 64, 0, 793, 33, 0
UMA Zones: 88, 0, 73, 7, 0
UMA Kegs: 136, 0, 73, 23, 0
There doesn't seem to be anything really changed between then and now (i.e.
in one hour), but netstat works again. Can anyone recommend a suitable
course of actions? I'm somewhat at a loss as to what I should look for now.
It seems that when the machine has been up for a few days this starts
happening once or twice a day, until the machine finally deadlocks (it
usually does this after running for a week).
I tried to enable WITNESS, but as fas as I can gather from the dmesg, it
doesn't work as there is not enough buffer space or something (the error is
"witness_get: witness exhausted").
-Martin
More information about the freebsd-stable
mailing list