(panic) Lots of network memory needed
Karim Fodil-Lemelin
kfl at xiphos.ca
Thu Jun 16 20:55:00 GMT 2005
I use FreeBSD 4.9. Actually I had to _reduce_ VM_KMEM_SIZE in order to
be able to create that much mnbclusters ...
Somehow it seems that vm_kmem_size cannot go over 1GB without crashing
the system so I had to reduce memory map from other subsystems in order
to get what I needed for network memory. From:
vm_kmem_size = VM_KMEM_SIZE + nmbclusters * PAGE_SIZE;
Assuming vm_kmem_size has some sort of hard limit, if I reduce VM_KMEM_SIZE I can get more nmbclusters into the system.
dima wrote:
>>I'm fairly sure that you are out of kernel virtual memory.
>>Look at kern/kern_malloc.c kmeminit (); you can play w/ VM_KMEM_SIZE_MAX
>>or TUNABLE_INT_FETCH("kern.vm.kmem.size", &vm_kmem_size);
>>
>> /*
>> * Try to auto-tune the kernel memory size, so that it is
>> * more applicable for a wider range of machine sizes.
>> * On an X86, a VM_KMEM_SIZE_SCALE value of 4 is good, while
>> * a VM_KMEM_SIZE of 12MB is a fair compromise. The
>> * VM_KMEM_SIZE_MAX is dependent on the maximum KVA space
>> * available, and on an X86 with a total KVA space of 256MB,
>> * try to keep VM_KMEM_SIZE_MAX at 80MB or below.
>> *
>> * Note that the kmem_map is also used by the zone allocator,
>> * so make sure that there is enough space.
>> */
>> vm_kmem_size = VM_KMEM_SIZE;
>> mem_size = cnt.v_page_count * PAGE_SIZE;
>>
>>
>Which version do you use? I see the following in my /sys/kern/kern_malloc.c:
> vm_kmem_size = VM_KMEM_SIZE + nmbclusters * PAGE_SIZE;
>....
> /*
> * Limit kmem virtual size to twice the physical memory.
> * This allows for kmem map sparseness, but limits the size
> * to something sane. Be careful to not overflow the 32bit
> * ints while doing the check.
> */
> if (((vm_kmem_size / 2) / PAGE_SIZE) > cnt.v_page_count)
> vm_kmem_size = 2 * cnt.v_page_count * PAGE_SIZE;
>It's 5.4-p2
>
>
>
>
>
>>#if defined(VM_KMEM_SIZE_SCALE)
>> if ((mem_size / VM_KMEM_SIZE_SCALE) > vm_kmem_size)
>> vm_kmem_size = mem_size / VM_KMEM_SIZE_SCALE;
>>#endif
>>
>>#if defined(VM_KMEM_SIZE_MAX)
>> if (vm_kmem_size >= VM_KMEM_SIZE_MAX)
>> vm_kmem_size = VM_KMEM_SIZE_MAX;
>>#endif
>>
>> /* Allow final override from the kernel environment */
>> TUNABLE_INT_FETCH("kern.vm.kmem.size", &vm_kmem_size);
>>
>>
>>Karim Fodil-Lemelin wrote:
>>
>>
>>>Thanks but the system still crashes (FreeBSD 4.9) with 131072. Here is a
>>>backtrace showing just that:
>>>
>>>Debugger (msg=0xc02b6cdb "panic") at ../../i386/i386/db_interface.c:321
>>>321 }
>>>(kgdb) bt
>>>#0 Debugger (msg=0xc02b6cdb "panic") at ../../i386/i386/db_interface.c:321
>>>#1 0xc016a230 in panic (
>>> fmt=0xc02ea380 "pmap_enter: invalid page directory pdir=%#llx,
>>>va=%#x\n")
>>> at ../../kern/kern_shutdown.c:593
>>>#2 0xc0283594 in pmap_enter (pmap=0xc0340460, va=4292141056, m=0xc24b2848,
>>> prot=7 '\a', wired=1) at ../../i386/i386/pmap.c:1943
>>>#3 0xc023ddd0 in vm_fault (map=0xc033322c, vaddr=4292141056,
>>> fault_type=7 '\a', fault_flags=1) at ../../vm/vm_fault.c:841
>>>#4 0xc023df0a in vm_fault_wire (map=0xc033322c, start=4292141056,
>>> end=4292149248) at ../../vm/vm_fault.c:915
>>>#5 0xc0240831 in vm_map_pageable (map=0xc033322c, start=4292141056,
>>> real_end=4292149248, new_pageable=0) at ../../vm/vm_map.c:1817
>>>#6 0xc023ea25 in kmem_alloc (map=0xc033322c, size=8192)
>>> at ../../vm/vm_kern.c:213
>>>#7 0xc024a46f in _zget (z=0xdb5c6e80) at ../../vm/vm_zone.c:425
>>>#8 0xc024a269 in zalloc (z=0xdb5c6e80) at ../../vm/vm_zone.c:60
>>>#9 0xc0196106 in namei (ndp=0xfa489ef4) at ../../kern/vfs_lookup.c:104
>>>#10 0xc01614ca in execve (p=0xfa482e00, uap=0xfa489f90)
>>> at ../../kern/kern_exec.c:165
>>>#11 0xc01590a1 in start_init (dummy=0x0) at ../../kern/init_main.c:543
>>>(kgdb) p nmbclusters
>>>$1 = 131072
>>>(kgdb)
>>>
>>>Goran Spirovski - MorEl On.net wrote:
>>>
>>>
>>>
>>>>AFAIK the number of mbufs (and consequently nmbclusters) has to be a
>>>>power
>>>>of 2, so you should set it to 131072
>>>>
>>>>MorEl
>>>>
>>>>----- Original Message ----- From: "Karim Fodil-Lemelin" <kfl at xiphos.ca>
>>>>To: <freebsd-net at freebsd.org>
>>>>Sent: Wednesday, June 15, 2005 6:08 PM
>>>>Subject: (panic) Lots of network memory needed
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>>Hello,
>>>>>
>>>>>From kernel tuning page
>>>>>
>>>>>
>>>>>
>>>>(http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/configtuning-kern
>>>>
>>>>el-limits.html)
>>>>
>>>>
>>>>
>>>>
>>>>>Some sentence about nmbclusters says:
>>>>>
>>>>>"Under no circumstances should you specify an arbitrarily high value for
>>>>>
>>>>>
>>>>>
>>>>this parameter as it could lead to a boot time crash."
>>>>
>>>>
>>>>
>>>>
>>>>>Now I want to push the limits where I need 4KB buffer for each of the
>>>>>
>>>>>
>>>>>
>>>>32000 connections I want this server to handle. If I do the math:
>>>>
>>>>
>>>>
>>>>
>>>>>(32000 (conns) * 4 (KB/buffer) * 2 (buffer/conn)) / 2048 (KB/cluster)
>>>>>
>>>>>= 128000 clusters
>>>>>
>>>>>So I set this arbitrary high value in loader.conf under
>>>>>
>>>>>
>>>>>
>>>>(kern.ipc.nmbclusters) and no surprises I get panic: pmap_enter
>>>>invalid page
>>>>directory pdir=0x3cb063, va=0xfff800
>>>>
>>>>
>>>>
>>>>
>>>>>(va has a weird address here)
>>>>>
>>>>>I know I am pushing the limits here but I have plenty of memory (2GB) on
>>>>>
>>>>>
>>>>>
>>>>this system (after all its just 250MB for network memory ;) and this is
>>>>mainly just experimentation.
>>>>
>>>>
>>>>
>>>>
>>>>>I would like some pointers toward fixing this. Is there another variable
>>>>>
>>>>>
>>>>>
>>>>tied into this (I guess so)? Could anybody points me to a technical
>>>>document
>>>>that would explain the relationship with that (those) other(s) presumed
>>>>variable(s)?
>>>>
>>>>
>>>>
>>>>
>>>>>Thank you,
>>>>>
>>>>>
>>>>>Karim
>>>>>
>>>>>
>>>>>_______________________________________________
>>>>>freebsd-net at freebsd.org mailing list
>>>>>http://lists.freebsd.org/mailman/listinfo/freebsd-net
>>>>>To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>--
>>Dave Baukus
>>dbaukus at chiaro.com
>> Chiaro Networks Ltd.
>> Richardson, Texas
>> USA
>>
>>_______________________________________________
>>freebsd-net at freebsd.org mailing list
>>http://lists.freebsd.org/mailman/listinfo/freebsd-net
>>To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>>
>>
>>
>
>
>
>
--
Karim Fodil-Lemelin
Lead Programmer
Xiphos Technologies Inc.
(514) 848-9640 x223
(514) 848-9644 fax
www.xiplink.com
--------------------------------------------------------------
The information transmitted is intended only for the
person or entity to which it is addressed and may contain
confidential and/or privileged material. If you have
received this in error, please contact the sender and delete
this communication and any copy immediately. Thank you.
More information about the freebsd-net
mailing list