deadlock in "zoneli" state on 6.2-PRERELEASE
Nikolay Pavlov
quetzal at zone3000.net
Mon Dec 11 10:30:48 PST 2006
On Monday, 11 December 2006 at 15:59:03 +0800, LI Xin wrote:
> Hi,
>
> Would you please give the following patch a try?
>
> http://people.freebsd.org/~delphij/misc/patch-zonelim-drain
>
> Note: Please revert my previous patch against sys/kern/kern_mbuf.c.
>
> This patch should be applied against sys/vm/ [RELENG_6 or RELENG_6_2],
> it schedules a drain of uma zones when they are low on memory.
This time things worked out a bit better, there was no Kernel panic and
my server managed to overcome the "magic" number 65550 mbufs. But very
soon the server reached another limit - 131072 mbuf clusters
(This is my limit for kern.ipc.nmbclusters).
And server started to drop the packets. After I've
removed the overload I found my server responding but when I actually
accessed it I found out that although the number of connections has
reduces considerably, the memory allocated for the net did not become
free. So I believe that there is still a mbufs leak somewhere.
root at accel1:~# sockstat -4 | wc -l
17
root at accel1:~# netstat -m
1082/131578/132660 mbufs in use (current/cache/total)
1080/129992/131072/131072 mbuf clusters in use (current/cache/total/max)
^^^^^^^^^^^^^^^^^^^^^^^^^^
1080/128712 mbuf+clusters out of packet secondary zone in use (current/cache)
0/0/0/0 4k (page size) jumbo clusters in use (current/cache/total/max)
0/0/0/0 9k jumbo clusters in use (current/cache/total/max)
0/0/0/0 16k jumbo clusters in use (current/cache/total/max)
2430K/292878K/295309K bytes allocated to network (current/cache/total)
^^^^^^^^^^^^^^^^^^^^^
0/1058420/529208 requests for mbufs denied
(mbufs/clusters/mbuf+clusters)
0/0/0 requests for jumbo clusters denied (4k/9k/16k)
0/4/6656 sfbufs in use (current/peak/max)
0 requests for sfbufs denied
0 requests for sfbufs delayed
0 requests for I/O initiated by sendfile
156 calls to protocol drain routines
root at accel1:~# vmstat -z
ITEM SIZE LIMIT USED FREE REQUESTS
UMA Kegs: 140, 0, 84, 12, 84
UMA Zones: 120, 0, 84, 6, 84
UMA Slabs: 64, 0, 904, 40, 31603
UMA RCntSlabs: 104, 0, 65536, 28, 65536
UMA Hash: 128, 0, 4, 26, 6
16 Bucket: 76, 0, 23, 27, 32
32 Bucket: 140, 0, 20, 8, 29
64 Bucket: 268, 0, 23, 33, 46
128 Bucket: 524, 0, 1423, 5, 1590808
VM OBJECT: 132, 0, 18094, 147, 43780
MAP: 192, 0, 7, 33, 7
KMAP ENTRY: 68, 57456, 17, 151, 151954
MAP ENTRY: 68, 0, 718, 290, 182847
PV ENTRY: 24, 2228360, 124232, 4528, 861505
DP fakepg: 72, 0, 0, 0, 0
mt_zone: 64, 0, 182, 54, 182
16: 16, 0, 2828, 1435, 273492
32: 32, 0, 1769, 491, 116114
64: 64, 0, 3199, 2701, 181154
128: 128, 0, 1813, 1517, 273347
256: 256, 0, 365, 415, 13230
512: 512, 0, 1006, 10, 35578
1024: 1024, 0, 49, 83, 72385
2048: 2048, 0, 170, 38, 40942
4096: 4096, 0, 129, 22, 89367
Files: 72, 0, 100, 5624, 227230
MAC labels: 20, 0, 20726, 15778, 368518
PROC: 536, 0, 71, 27, 1441
THREAD: 376, 0, 98, 22, 98
KSEGRP: 88, 0, 98, 62, 98
UPCALL: 44, 0, 0, 0, 0
VMSPACE: 296, 0, 28, 24, 1398
mbuf_packet: 256, 0, 1080, 128712, 6714669
mbuf: 256, 0, 2, 2866, 7912768
mbuf_cluster: 2048, 131072, 129792, 1280, 166272
mbuf_jumbo_pagesize: 4096, 0, 0, 0, 0
mbuf_jumbo_9k: 9216, 0, 0, 0, 0
mbuf_jumbo_16k: 16384, 0, 0, 0, 0
ACL UMA zone: 388, 0, 0, 0, 0
g_bio: 132, 0, 0, 783, 194850
ata_request: 204, 0, 0, 0, 0
ata_composite: 196, 0, 0, 0, 0
VNODE: 348, 0, 20116, 3, 22965
VNODEPOLL: 76, 0, 0, 0, 0
S VFS Cache: 68, 0, 16387, 77, 18383
L VFS Cache: 291, 0, 0, 0, 0
NAMEI: 1024, 0, 0, 12, 166126
DIRHASH: 1024, 0, 1688, 128, 3307
NFSMOUNT: 480, 0, 0, 0, 0
NFSNODE: 536, 0, 0, 0, 0
PIPE: 408, 0, 6, 21, 595
KNOTE: 68, 0, 0, 112, 64
socket: 356, 131076, 31, 5766, 111627
unpcb: 140, 131096, 12, 44, 171
ipq: 32, 4181, 0, 0, 0
udpcb: 180, 131076, 6, 38, 164
inpcb: 180, 131076, 13, 6081, 111291
tcpcb: 464, 131072, 13, 5555, 111291
tcptw: 48, 8268, 0, 858, 68707
syncache: 100, 15366, 0, 312, 92731
hostcache: 76, 15400, 4989, 61, 4989
tcpreass: 20, 8281, 0, 338, 3819
sackhole: 20, 0, 0, 338, 191981
ripcb: 180, 131076, 0, 0, 0
rtentry: 132, 0, 19, 39, 29
pfsrctrpl: 100, 15015, 0, 0, 0
pfrulepl: 604, 0, 9, 9, 9
pfstatepl: 260, 15000, 7, 10568, 92462
pfaltqpl: 128, 0, 0, 0, 0
pfpooladdrpl: 68, 0, 2, 110, 2
pfrktable: 1240, 0, 4, 5, 8
pfrkentry: 156, 0, 5, 45, 5
pfrkentry2: 156, 0, 0, 0, 0
pffrent: 16, 203, 0, 0, 0
pffrag: 48, 0, 0, 0, 0
pffrcache: 48, 10062, 0, 0, 0
pffrcent: 12, 50141, 0, 0, 0
pfstatescrub: 28, 0, 0, 0, 0
pfiaddrpl: 92, 0, 0, 0, 0
pfospfen: 108, 0, 345, 51, 345
pfosfp: 28, 0, 188, 193, 188
SWAPMETA: 276, 121576, 0, 0, 0
Mountpoints: 740, 0, 6, 4, 6
FFS inode: 132, 0, 20074, 52, 22922
FFS1 dinode: 128, 0, 0, 0, 0
FFS2 dinode: 256, 0, 20074, 41, 22922
Should i add this to my PR?
http://www.freebsd.org/cgi/query-pr.cgi?pr=106317
>
> Cheers,
> --
> Xin LI <delphij at delphij.net> http://www.delphij.net/
> FreeBSD - The Power to Serve!
>
--
======================================================================
- Best regards, Nikolay Pavlov. <<<-----------------------------------
======================================================================
More information about the freebsd-stable
mailing list