(panic) Lots of network memory needed
dima
_pppp at mail.ru
Fri Jun 17 08:15:47 GMT 2005
> I use FreeBSD 4.9. Actually I had to _reduce_ VM_KMEM_SIZE in order to
> be able to create that much mnbclusters ...
>
> Somehow it seems that vm_kmem_size cannot go over 1GB without crashing
> the system so I had to reduce memory map from other subsystems in order
> to get what I needed for network memory. From:
>
> vm_kmem_size = VM_KMEM_SIZE + nmbclusters * PAGE_SIZE;
>
> Assuming vm_kmem_size has some sort of hard limit, if I reduce VM_KMEM_SIZE I can get more nmbclusters into the system.
Sorry. I only have 5.x systems handy.
According to /sys/i386/include/pmap.h
/*
* Size of Kernel address space. This is the number of page table pages
* (4MB each) to use for the kernel. 256 pages == 1 Gigabyte.
* This **MUST** be a multiple of 4 (eg: 252, 256, 260, etc).
*/
#ifndef KVA_PAGES
#ifdef PAE
#define KVA_PAGES 512
#else
#define KVA_PAGES 256
#endif
#endif
Well, be very carefull if you're going to change that. Check also the kernel base address in /sys/i386/include/vmparam.h
>
>
> dima wrote:
>
> >>I'm fairly sure that you are out of kernel virtual memory.
> >>Look at kern/kern_malloc.c kmeminit (); you can play w/ VM_KMEM_SIZE_MAX
> >>or TUNABLE_INT_FETCH("kern.vm.kmem.size", &vm_kmem_size);
> >>
> >> /*
> >> * Try to auto-tune the kernel memory size, so that it is
> >> * more applicable for a wider range of machine sizes.
> >> * On an X86, a VM_KMEM_SIZE_SCALE value of 4 is good, while
> >> * a VM_KMEM_SIZE of 12MB is a fair compromise. The
> >> * VM_KMEM_SIZE_MAX is dependent on the maximum KVA space
> >> * available, and on an X86 with a total KVA space of 256MB,
> >> * try to keep VM_KMEM_SIZE_MAX at 80MB or below.
> >> *
> >> * Note that the kmem_map is also used by the zone allocator,
> >> * so make sure that there is enough space.
> >> */
> >> vm_kmem_size = VM_KMEM_SIZE;
> >> mem_size = cnt.v_page_count * PAGE_SIZE;
> >>
> >>
> >Which version do you use? I see the following in my /sys/kern/kern_malloc.c:
> > vm_kmem_size = VM_KMEM_SIZE + nmbclusters * PAGE_SIZE;
> >....
> > /*
> > * Limit kmem virtual size to twice the physical memory.
> > * This allows for kmem map sparseness, but limits the size
> > * to something sane. Be careful to not overflow the 32bit
> > * ints while doing the check.
> > */
> > if (((vm_kmem_size / 2) / PAGE_SIZE) > cnt.v_page_count)
> > vm_kmem_size = 2 * cnt.v_page_count * PAGE_SIZE;
> >It's 5.4-p2
> >
> >
> >
> >
> >
> >>#if defined(VM_KMEM_SIZE_SCALE)
> >> if ((mem_size / VM_KMEM_SIZE_SCALE) > vm_kmem_size)
> >> vm_kmem_size = mem_size / VM_KMEM_SIZE_SCALE;
> >>#endif
> >>
> >>#if defined(VM_KMEM_SIZE_MAX)
> >> if (vm_kmem_size >= VM_KMEM_SIZE_MAX)
> >> vm_kmem_size = VM_KMEM_SIZE_MAX;
> >>#endif
> >>
> >> /* Allow final override from the kernel environment */
> >> TUNABLE_INT_FETCH("kern.vm.kmem.size", &vm_kmem_size);
> >>
> >>
> >>Karim Fodil-Lemelin wrote:
> >>
> >>
> >>>Thanks but the system still crashes (FreeBSD 4.9) with 131072. Here is a
> >>>backtrace showing just that:
> >>>
> >>>Debugger (msg=0xc02b6cdb "panic") at ../../i386/i386/db_interface.c:321
> >>>321 }
> >>>(kgdb) bt
> >>>#0 Debugger (msg=0xc02b6cdb "panic") at ../../i386/i386/db_interface.c:321
> >>>#1 0xc016a230 in panic (
> >>> fmt=0xc02ea380 "pmap_enter: invalid page directory pdir=%#llx,
> >>>va=%#x\n")
> >>> at ../../kern/kern_shutdown.c:593
> >>>#2 0xc0283594 in pmap_enter (pmap=0xc0340460, va=4292141056, m=0xc24b2848,
> >>> prot=7 '\a', wired=1) at ../../i386/i386/pmap.c:1943
> >>>#3 0xc023ddd0 in vm_fault (map=0xc033322c, vaddr=4292141056,
> >>> fault_type=7 '\a', fault_flags=1) at ../../vm/vm_fault.c:841
> >>>#4 0xc023df0a in vm_fault_wire (map=0xc033322c, start=4292141056,
> >>> end=4292149248) at ../../vm/vm_fault.c:915
> >>>#5 0xc0240831 in vm_map_pageable (map=0xc033322c, start=4292141056,
> >>> real_end=4292149248, new_pageable=0) at ../../vm/vm_map.c:1817
> >>>#6 0xc023ea25 in kmem_alloc (map=0xc033322c, size=8192)
> >>> at ../../vm/vm_kern.c:213
> >>>#7 0xc024a46f in _zget (z=0xdb5c6e80) at ../../vm/vm_zone.c:425
> >>>#8 0xc024a269 in zalloc (z=0xdb5c6e80) at ../../vm/vm_zone.c:60
> >>>#9 0xc0196106 in namei (ndp=0xfa489ef4) at ../../kern/vfs_lookup.c:104
> >>>#10 0xc01614ca in execve (p=0xfa482e00, uap=0xfa489f90)
> >>> at ../../kern/kern_exec.c:165
> >>>#11 0xc01590a1 in start_init (dummy=0x0) at ../../kern/init_main.c:543
> >>>(kgdb) p nmbclusters
> >>>$1 = 131072
> >>>(kgdb)
> >>>
> >>>Goran Spirovski - MorEl On.net wrote:
> >>>
> >>>
> >>>
> >>>>AFAIK the number of mbufs (and consequently nmbclusters) has to be a
> >>>>power
> >>>>of 2, so you should set it to 131072
> >>>>
> >>>>MorEl
> >>>>
> >>>>----- Original Message ----- From: "Karim Fodil-Lemelin" <kfl at xiphos.ca>
> >>>>To: <freebsd-net at freebsd.org>
> >>>>Sent: Wednesday, June 15, 2005 6:08 PM
> >>>>Subject: (panic) Lots of network memory needed
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>>Hello,
> >>>>>
> >>>>>From kernel tuning page
> >>>>>
> >>>>>
> >>>>>
> >>>>(http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/configtuning-kern
> >>>>
> >>>>el-limits.html)
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>>Some sentence about nmbclusters says:
> >>>>>
> >>>>>"Under no circumstances should you specify an arbitrarily high value for
> >>>>>
> >>>>>
> >>>>>
> >>>>this parameter as it could lead to a boot time crash."
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>>Now I want to push the limits where I need 4KB buffer for each of the
> >>>>>
> >>>>>
> >>>>>
> >>>>32000 connections I want this server to handle. If I do the math:
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>>(32000 (conns) * 4 (KB/buffer) * 2 (buffer/conn)) / 2048 (KB/cluster)
> >>>>>
> >>>>>= 128000 clusters
> >>>>>
> >>>>>So I set this arbitrary high value in loader.conf under
> >>>>>
> >>>>>
> >>>>>
> >>>>(kern.ipc.nmbclusters) and no surprises I get panic: pmap_enter
> >>>>invalid page
> >>>>directory pdir=0x3cb063, va=0xfff800
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>>(va has a weird address here)
> >>>>>
> >>>>>I know I am pushing the limits here but I have plenty of memory (2GB) on
> >>>>>
> >>>>>
> >>>>>
> >>>>this system (after all its just 250MB for network memory ;) and this is
> >>>>mainly just experimentation.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>>I would like some pointers toward fixing this. Is there another variable
> >>>>>
> >>>>>
> >>>>>
> >>>>tied into this (I guess so)? Could anybody points me to a technical
> >>>>document
> >>>>that would explain the relationship with that (those) other(s) presumed
> >>>>variable(s)?
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>>Thank you,
> >>>>>
> >>>>>
> >>>>>Karim
> >>>>>
> >>>>>
> >>>>>_______________________________________________
> >>>>>freebsd-net at freebsd.org mailing list
> >>>>>http://lists.freebsd.org/mailman/listinfo/freebsd-net
> >>>>>To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
> >>>>>
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>--
> >>Dave Baukus
> >>dbaukus at chiaro.com
> >> Chiaro Networks Ltd.
> >> Richardson, Texas
> >> USA
> >>
> >>_______________________________________________
> >>freebsd-net at freebsd.org mailing list
> >>http://lists.freebsd.org/mailman/listinfo/freebsd-net
> >>To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
> >>
> >>
> >>
> >
> >
> >
> >
>
> --
> Karim Fodil-Lemelin
> Lead Programmer
>
> Xiphos Technologies Inc.
> (514) 848-9640 x223
> (514) 848-9644 fax
> www.xiplink.com
>
> --------------------------------------------------------------
> The information transmitted is intended only for the
> person or entity to which it is addressed and may contain
> confidential and/or privileged material. If you have
> received this in error, please contact the sender and delete
> this communication and any copy immediately. Thank you.
>
>
>
More information about the freebsd-net
mailing list