Today's -current panics
othermark
atkin901 at yahoo.com
Fri Jun 11 16:27:16 GMT 2004
Bosko Milekic wrote:
>
> othermark wrote:
>>I get a very similar stack track traversing through sossend(), under heavy
>>NFS load on a 1GB machine. Note the panic message here, and the
>>peculiarity that previous incarnations of -current did not panic under
>>similar load. It is highly reproduceable via a 'make installworld' via
>>NFS with /usr/src and /usr/obj mounted. The NFS serving machine will
>>always panic using vanilla GENERIC:
>>
>>[root at pippin root]$ panic: kmem_malloc(4096): kmem_map too small: 40894464
>>total
>> allocated
>
> Do you have the kern.ipc.nmbclusters boot-time tunable set to 0?
>
> I just noticed that if this is set to zero then kmem_map will not
> be scaled larger to accomodate clusters and mbufs. In this scenario,
> what I recommend that you do is increase the vm.kmem_size boot-time
> tunable to ~300,000,000 or ~400,000,000. Currently, your kmem_map
> is way too small (looks like only ~40M). Be careful not to overdo
> it, though, because you might also then have to increase the
> available KVA (KVA_PAGES).
>
> -Bosko
It's all a default installation (I try to keep the machines as untweaked as
possible). The only thing in loader.conf is to turn on DMA for the hdd.
I rebooted with kern.ipc.nmbclusters="0" and vm.kmem_size="300000000" in
/boot/loader.conf.
Another NFS installworld completed successfully (but appeared to going
slower). I'm assuming that GENERIC w/ no adjustments should complete this
type of task w/o panicing anyway? Thanks for your help Bosko!
Here are the vmstat, netstat numbers that Robert requested earlier in this
thread after the successful installworld. Hopefully the high/peak numbers
will be useful in analysis (long):
[root at pippin root]$ vmstat -m
Type InUse MemUse HighUse Requests Size(s)
acpidev 36 1K 1K 36 16
acpisem 15 1K 1K 15 64
acpitask 0 0K 1K 2 16,32
acpica 945 56K 56K 15810 16,32,64,128,256,512,1024,2048
atkbddev 2 1K 1K 2 32
pfs_fileno 1 20K 20K 1
pfs_nodes 20 3K 3K 20 128
nexusdev 3 1K 1K 3 16
memdesc 1 4K 4K 1 4096
MSDOSFS mount 1 16K 16K 1
UMAHash 2 3K 3K 5 256,512,1024,2048
VM pgdata 2 9K 9K 2 64
DEVFS 110 17K 17K 110 16,32,128,4096
UFS mount 3 7K 7K 3 256,2048,4096
UFS ihash 1 32K 32K 1
UFS dirhash 87 16K 26K 258 16,32,64,128,256,512
dirrem 0 0K 1K 32 32
mkdir 0 0K 1K 6 32
diradd 0 0K 1K 36 32
freefile 0 0K 1K 22 32
freeblks 0 0K 3K 19 256
freefrag 0 0K 1K 8 32
allocdirect 0 0K 2K 70 128
bmsafemap 0 0K 1K 50 32
newblk 1 1K 1K 71 64,256
inodedep 2 33K 39K 119 128
pagedep 1 4K 5K 11 64,4096
p1003.1b 1 1K 1K 1 16
agp 1 1K 1K 1 16
NFS daemon 69 7K 7K 69 64,256,512
NFSV3 srvdesc 0 0K 1K 321570 16,256
NFS srvsock 2 1K 1K 2 128
NFS hash 1 32K 32K 1
ip6_moptions 1 1K 1K 1 16
in6_multi 11 1K 1K 11 16,64
syncache 1 8K 8K 1
hostcache 1 24K 24K 1
in_multi 2 1K 1K 2 32
routetbl 55 8K 8K 88 16,32,64,128,256
USBdev 1 1K 2K 4 128,512
USB 15 2K 2K 15 16,32,64,128,256
lo 1 1K 1K 1 512
clone 4 16K 16K 4 4096
ether_multi 37 2K 2K 37 16,32,64
ifaddr 65 14K 14K 65 16,32,64,256,512,2048
BPF 6 1K 1K 6 64
mount 10 3K 3K 14 16,32,128,512,1024
vnodes 28 7K 7K 159 16,32,64,128,256
cluster_save buffer 0 0K 1K 1 32
vfscache 1 64K 64K 1
BIO buffer 288 576K 740K 3080 2048
pcb 26 5K 5K 100 16,32,64,2048
soname 2 1K 1K 162678 16,32,128
tag 0 0K 1K 5 32
ttys 1013 134K 134K 2378 128,512
shm 1 12K 12K 1
sem 4 7K 7K 4 512,1024,4096
msg 4 25K 25K 4 512,4096
ioctlops 0 0K 1K 14 512,1024
entropy 1024 64K 64K 1024 64
ppbusdev 3 1K 1K 3 128
turnstiles 111 7K 7K 111 64
taskqueue 5 1K 1K 5 64
sleep queues 111 4K 4K 111 32
sbuf 0 0K 13K 190
16,32,64,128,256,512,1024,2048,4096
rman 126 8K 8K 486 16,64
kobj 219 438K 438K 264 2048
eventhandler 33 2K 2K 33 32,128
devstat 4 9K 9K 4 16,4096
bus-sc 45 27K 111K 1323
16,32,64,128,256,512,1024,2048,4096
bus 841 34K 112K 3159 16,32,64,128,1024
SWAP 2 209K 209K 2 64
sysctltmp 0 0K 1K 724 16,32,64,128
sysctloid 1847 56K 56K 1847 16,32,64
sysctl 0 0K 1K 310 16,32,64
uidinfo 3 1K 1K 8 32,128
plimit 16 4K 5K 2974 256
cred 13 2K 3K 4272 128
subproc 78 312K 328K 879 4096
proc 2 1K 1K 2 512
session 24 3K 4K 282 128
pgrp 26 2K 2K 303 64
mtx_pool 1 8K 8K 1
module 329 21K 21K 329 64,128
ip6ndp 8 1K 1K 9 64,128
temp 35 78K 87K 37496
16,32,64,128,256,512,1024,2048,4096
devbuf 166 868K 868K 174
16,32,64,128,256,512,1024,2048,4096
lockf 3 1K 1K 11 64
linker 49 2K 2K 82 16,32,64,256
KTRACE 100 13K 13K 100 128
ithread 51 5K 5K 52 64,128
zombie 0 0K 1K 801 128
proc-args 30 2K 2K 1123 32,64,128
kqueue 0 0K 1K 22 256
kenv 105 6K 6K 106 16,32,64,2048
sigio 1 1K 1K 1 32
file desc 81 24K 25K 900 32,256,2048
dev_t 35 9K 9K 35 256
ACD driver 1 2K 2K 1 2048
AR driver 1 1K 3K 3 64,256,2048
AD driver 1 1K 1K 1 128
ATA DMA 2 1K 1K 2 128
ATA generic 4 2K 2K 4 16,512
ISOFS mount 1 32K 32K 1
isadev 19 2K 2K 19 64
GEOM 49 6K 15K 234
16,32,64,128,256,512,1024,2048,4096
[root at pippin root]$ vmstat -z
ITEM SIZE LIMIT USED FREE REQUESTS
FFS2 dinode: 256, 0, 0, 0, 0
FFS1 dinode: 128, 0, 5851, 39, 17801
FFS inode: 140, 0, 5851, 29, 17801
SWAPMETA: 276, 14980, 0, 0, 0
unpcb: 140, 0, 9, 47, 315
ripcb: 180, 1936, 0, 44, 2
tcpreass: 20, 0, 0, 0, 0
hostcache: 88, 15390, 1, 89, 1
syncache: 104, 15390, 0, 76, 261
tcptw: 56, 426, 0, 0, 0
tcpcb: 372, 1930, 17, 13, 282
inpcb: 180, 1936, 17, 27, 282
udpcb: 180, 1936, 18, 26, 141
socket: 244, 1936, 44, 20, 745
KNOTE: 64, 0, 0, 0, 0
PIPE: 376, 0, 3, 17, 250
NFSNODE: 452, 0, 0, 0, 0
NFSMOUNT: 424, 0, 0, 0, 0
DIRHASH: 1024, 0, 85, 11, 255
NAMEI: 1024, 0, 0, 12, 88425
L VFS Cache: 291, 0, 3, 36, 23
S VFS Cache: 68, 0, 6175, 89, 40269
VNODEPOLL: 60, 0, 0, 0, 0
VNODE: 260, 0, 5870, 10, 5870
ata_request: 180, 0, 0, 44, 18511
g_bio: 136, 0, 0, 116, 72421
MbufClust: 2048, 0, 384, 124, 124710
Mbuf: 256, 0, 160051, 104, 697905
Packet: 256, 0, 159987, 168, 167937
VMSPACE: 236, 0, 30, 21, 853
UPCALL: 44, 0, 0, 0, 0
KSE: 84, 0, 99, 65, 99
KSEGRP: 128, 0, 99, 56, 99
TID: 140, 0, 0, 0, 0
THREAD: 340, 0, 99, 11, 99
PROC: 440, 0, 77, 22, 900
Files: 68, 0, 94, 80, 6712
4096: 4096, 0, 117, 17, 7001
2048: 2048, 0, 525, 63, 3476
1024: 1024, 0, 13, 39, 759
512: 512, 0, 90, 30, 922
256: 256, 0, 211, 29, 165379
128: 128, 0, 1479, 71, 11187
64: 64, 0, 3130, 32, 20854
32: 32, 0, 1043, 187, 5661
16: 16, 0, 1889, 262, 348635
DP fakepg: 72, 0, 0, 0, 0
PV ENTRY: 28, 225680, 7576, 3064, 397167
MAP ENTRY: 60, 0, 568, 224, 33578
KMAP ENTRY: 60, 7656, 23, 109, 20105
MAP: 160, 0, 7, 18, 0
VM OBJECT: 132, 0, 5020, 110, 30065
128 Bucket: 524, 0, 31, 4, 0
64 Bucket: 268, 0, 16, 14, 0
32 Bucket: 140, 0, 23, 5, 0
16 Bucket: 76, 0, 18, 34, 0
UMA Hash: 128, 0, 2, 29, 0
UMA RCntSlab: 82, 0, 254, 28, 0
UMA Slabs: 38, 0, 507, 87, 0
UMA Zones: 328, 0, 59, 7, 0
UMA Kegs: 136, 0, 59, 16, 0
[root at pippin root]$ netstat -mb
159963 mbufs in use
256/0 mbuf clusters in use (current/max)
0/2/1456 sfbufs in use (current/peak/max)
40502 KBytes allocated to network
0 requests for sfbufs denied
0 requests for sfbufs delayed
0 requests for I/O initiated by sendfile
34 calls to protocol drain routines
--
othermark
atkin901 at nospam dot yahoo dot com
(!wired)?(coffee++):(wired);
More information about the freebsd-current
mailing list