misc/75510: panic: kmem_malloc(4096): kmem_map too small
Rob Swindell
rob at synchro.net
Sun Dec 26 03:20:18 PST 2004
>Number: 75510
>Category: misc
>Synopsis: panic: kmem_malloc(4096): kmem_map too small
>Confidential: no
>Severity: serious
>Priority: high
>Responsible: freebsd-bugs
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: sw-bug
>Submitter-Id: current-users
>Arrival-Date: Sun Dec 26 11:20:17 GMT 2004
>Closed-Date:
>Last-Modified:
>Originator: Rob Swindell
>Release: 5.3-RELEASE
>Organization:
Synchronet
>Environment:
FreeBSD devil.synchro.net 5.3-RELEASE FreeBSD 5.3-RELEASE #0: Sat Dec 25 02:42:54 PST 2004 root at devil.synchro.net:/usr/src/sys/i386/compile/MYKERNEL i386
>Description:
When running a multi-threaded server application (http://www.synchro.net), accessing configuration and data files on an smbfs-mounted volume, after 4-8 hours, I get the following panic (routinely):
panic: kmem_malloc(4096): kmem_map too small: 40898560 total allocated
I've seen another PR related to a panic like this one, but it was in relation to a system with 4GB+ of RAM. This particular system only has 128MB of RAM. Here is a current "top" output at the time of one of the panics:
last pid: 6429; load averages: 0.09, 0.07, 0.21 up 0+07:27:31 05:52:23
84 processes: 2 running, 57 sleeping, 25 waiting
CPU states: 0.4% user, 0.0% nice, 3.5% system, 2.3% interrupt, 93.8% idle
Mem: 31M Active, 2652K Inact, 47M Wired, 1556K Cache, 22M Buf, 35M Free
Swap: 231M Total, 88K Used, 231M Free
PID USERNAME PRI NICE SIZE RES STATE TIME WCPU CPU COMMAND
3158 root 20 0 19928K 17596K kserel 22:14 0.00% 0.00% sbbs
583 root 96 0 6092K 2172K select 0:01 0.00% 0.00% sshd
1298 root 96 0 6092K 2168K select 0:01 0.00% 0.00% sshd
529 root 96 0 6092K 2152K select 0:01 0.00% 0.00% sshd
476 root 96 0 6092K 1856K select 0:14 0.00% 0.00% sshd
379 root 96 0 3360K 1560K select 0:00 0.00% 0.00% sshd
528 root 96 0 2296K 1308K RUN 1:23 0.00% 0.00% top
1314 root 8 0 2208K 1616K wait 0:00 0.00% 0.00% bash
586 root 5 0 2204K 1620K ttyin 0:00 0.00% 0.00% bash
532 root 8 0 2204K 1580K wait 0:00 0.00% 0.00% bash
479 root 8 0 2204K 1524K wait 0:00 0.00% 0.00% bash
218 root 96 0 1784K 1048K select 0:01 0.00% 0.00% dhclient
392 root 8 0 1356K 860K nanslp 0:00 0.00% 0.00% cron
268 root 96 0 1312K 760K select 0:08 0.00% 0.00% syslogd
465 root 5 0 1280K 772K ttyin 0:00 0.00% 0.00% getty
471 root 5 0 1280K 772K ttyin 0:00 0.00% 0.00% getty
470 root 5 0 1280K 772K ttyin 0:00 0.00% 0.00% getty
468 root 5 0 1280K 772K ttyin 0:00 0.00% 0.00% getty
467 root 5 0 1280K 772K ttyin 0:00 0.00% 0.00% getty
469 root 5 0 1280K 772K ttyin 0:00 0.00% 0.00% getty
466 root 5 0 1280K 772K ttyin 0:00 0.00% 0.00% getty
1611 root 5 0 1280K 764K ttyin 0:00 0.00% 0.00% getty
343 root 96 0 1236K 656K select 0:00 0.00% 0.00% usbd
428 root 96 0 1232K 656K select 0:00 0.00% 0.00% moused
3500 root 4 0 1216K 580K kqread 0:00 0.00% 0.00% tail
167 root 20 0 1184K 572K pause 0:00 0.00% 0.00% adjkerntz
1 root 8 0 740K 220K wait 0:00 0.00% 0.00% init
248 root 111 0 516K 224K select 0:00 0.00% 0.00% devd
11 root 112 0 0K 12K RUN 340:17 89.36% 89.36% idle
407 root 8 0 0K 12K 90idle 4:01 0.93% 0.93% smbiod0
27 root -28 -147 0K 12K WAIT 14:22 0.15% 0.15% swi5: clock s
29 root -44 -163 0K 12K WAIT 2:41 0.00% 0.00% swi1: net
21 root -68 -187 0K 12K WAIT 2:40 0.00% 0.00% irq10: de0 uh
30 root 76 0 0K 12K - 0:38 0.00% 0.00% yarrow
36 root -24 -143 0K 12K WAIT 0:14 0.00% 0.00% swi6:+
40 root 171 52 0K 12K pgzero 0:09 0.00% 0.00% pagezero
42 root 20 0 0K 12K syncer 0:07 0.00% 0.00% syncer
4 root -8 0 0K 12K - 0:07 0.00% 0.00% g_down
Here's the top output at the time of another such panic:
last pid: 2904; load averages: 0.13, 0.11, 0.08
up 0+01:35:27 14:39:41
75 processes: 2 running, 48 sleeping, 25 waiting
CPU states: 0.4% user, 0.0% nice, 5.4% system, 2.3% interrupt, 91.8% idle
Mem: 22M Active, 7184K Inact, 47M Wired, 1188K Cache, 22M Buf, 40M Free
Swap: 231M Total, 231M Free
PID USERNAME PRI NICE SIZE RES STATE TIME WCPU CPU COMMAND
474 root 20 0 11908K 9956K kserel 4:10 0.00% 0.00% sbbs
482 root 96 0 6092K 2240K select 0:03 0.00% 0.00% sshd
470 root 96 0 6092K 2236K select 0:00 0.00% 0.00% sshd
380 root 96 0 3360K 1936K select 0:00 0.00% 0.00% sshd
487 root 8 0 2208K 1576K wait 0:00 0.00% 0.00% bash
473 root 8 0 2208K 1568K wait 0:00 0.00% 0.00% bash
488 root 96 0 2280K 1316K RUN 0:17 0.00% 0.00% top
216 root 96 0 1784K 1064K select 0:00 0.00% 0.00% dhclient
393 root 8 0 1356K 916K nanslp 0:00 0.00% 0.00% cron
462 root 5 0 1280K 808K ttyin 0:00 0.00% 0.00% getty
463 root 5 0 1280K 808K ttyin 0:00 0.00% 0.00% getty
469 root 5 0 1280K 808K ttyin 0:00 0.00% 0.00% getty
466 root 5 0 1280K 808K ttyin 0:00 0.00% 0.00% getty
465 root 5 0 1280K 808K ttyin 0:00 0.00% 0.00% getty
468 root 5 0 1280K 808K ttyin 0:00 0.00% 0.00% getty
467 root 5 0 1280K 808K ttyin 0:00 0.00% 0.00% getty
464 root 5 0 1280K 808K ttyin 0:00 0.00% 0.00% getty
266 root 96 0 1312K 800K select 0:02 0.00% 0.00% syslogd
342 root 96 0 1236K 688K select 0:00 0.00% 0.00% usbd
429 root 118 0 1232K 664K select 0:00 0.00% 0.00% moused
165 root 20 0 1184K 580K pause 0:00 0.00% 0.00% adjkerntz
1 root 8 0 740K 244K wait 0:00 0.00% 0.00% init
246 root 111 0 516K 224K select 0:00 0.00% 0.00% devd
11 root 104 0 0K 12K RUN 59:07 90.23% 90.23% idle
408 root 8 0 0K 12K 90idle 1:11 0.29% 0.29% smbiod0
27 root -44 -163 0K 12K WAIT 0:42 0.10% 0.10% swi1: net
28 root -28 -147 0K 12K WAIT 1:45 0.00% 0.00% swi5: clock sio
21 root -68 -187 0K 12K WAIT 1:19 0.00% 0.00% irq10: de0 uhci0
30 root 76 0 0K 12K - 0:09 0.00% 0.00% yarrow
40 root 171 52 0K 12K pgzero 0:04 0.00% 0.00% pagezero
32 root -24 -143 0K 12K WAIT 0:04 0.00% 0.00% swi6:+
43 root 20 0 0K 12K syncer 0:01 0.00% 0.00% syncer
3 root -8 0 0K 12K - 0:01 0.00% 0.00% g_up
4 root -8 0 0K 12K - 0:01 0.00% 0.00% g_down
45 root -8 0 0K 12K - 0:01 0.00% 0.00% schedcpu
2 root -8 0 0K 12K - 0:01 0.00% 0.00% g_event
25 root -64 -183 0K 12K WAIT 0:01 0.00% 0.00% irq14: ata0
410 root 8 0 0K 12K 90idle 0:00 0.00% 0.00% smbiod1
41 root -16 0 0K 12K psleep 0:00 0.00% 0.00% bufdaemon
42 root -4 0 0K 12K vlruwt 0:00 0.00% 0.00% vnlru
7 root -8 0 0K 12K - 0:00 0.00% 0.00% fdc0
Here is the mbuf allocation status not too long before a panic:
[devil:~]# netstat -m
108 mbufs in use
34/4800 mbuf clusters in use (current/max)
0/3/1456 sfbufs in use (current/peak/max)
95 KBytes allocated to network
0 requests for sfbufs denied
0 requests for sfbufs delayed
2 requests for I/O initiated by sendfile
57 calls to protocol drain routines
Here's a /var/crash/info file for a recent panic:
Architecture: i386
Architecture version: 1
Dump length: 134205440B (127 MB)
Blocksize: 512
Dumptime: Sat Dec 25 14:39:42 2004
Hostname: devil.synchro.net
Versionstring: FreeBSD 5.3-RELEASE #0: Sat Dec 25 02:42:54 PST 2004
root at devil.synchro.net:/usr/src/sys/i386/compile/MYKERNEL
Panicstring: kmem_malloc(4096): kmem_map too small: 40898560 total allocated
Bounds: 0
Here's a kgdb backtrace from that same panic:
(kgdb) bt
#0 doadump () at pcpu.h:159
#1 0xc05fdc49 in boot (howto=260) at ../../../kern/kern_shutdown.c:397
#2 0xc05fdf05 in panic (
fmt=0xc07c38d3 "kmem_malloc(%ld): kmem_map too small: %ld total allocated")
at ../../../kern/kern_shutdown.c:553
#3 0xc06fc8d9 in kmem_malloc (map=0xc103a0c0, size=4096, flags=258)
at ../../../vm/vm_kern.c:300
#4 0xc070b7c2 in page_alloc (zone=0xc1044d80, bytes=4096, pflag=0x0, wait=258)
at ../../../vm/uma_core.c:935
#5 0xc070b301 in slab_zalloc (zone=0xc1044d80, wait=258)
at ../../../vm/uma_core.c:805
#6 0xc070ca54 in uma_zone_slab (zone=0xc1044d80, flags=2)
at ../../../vm/uma_core.c:1962
#7 0xc070cc74 in uma_zalloc_bucket (zone=0xc1044d80, flags=2)
at ../../../vm/uma_core.c:2071
#8 0xc070c900 in uma_zalloc_arg (zone=0xc1044d80, udata=0x0, flags=2)
at ../../../vm/uma_core.c:1889
#9 0xc06455ba in cache_enter (dvp=0xc17afb58, vp=0xc17dc318, cnp=0xcc83fa48)
at uma.h:274
#10 0xc1575310 in ?? ()
#11 0xc17afb58 in ?? ()
#12 0xc17dc318 in ?? ()
#13 0xcc83fa48 in ?? ()
I don't know for sure that this is an smbfs-related problem, but I have my suspicions. I don't know if it's any clue, but for every file close(), I get the following error: "smb_maperror: Unmapped error 1:158".
Merry Christmas!
>How-To-Repeat:
Run Synchronet (http://www.synchro.net) (or probably any other multi-threaded server) with the configuration and data files stored in an smbfs-mounted directory.
>Fix:
>Release-Note:
>Audit-Trail:
>Unformatted:
More information about the freebsd-bugs
mailing list