svn commit: r216016 - head/sys/sparc64/include

[mdf at FreeBSD.org]

On Tue, Dec 7, 2010 at 5:41 AM, Marius Strobl <marius at alchemy.franken.de> wrote:
> On Mon, Dec 06, 2010 at 02:30:01PM -0800, mdf at FreeBSD.org wrote:
>> On Mon, Dec 6, 2010 at 2:07 PM, Marius Strobl <marius at alchemy.franken.de> wrote:
>> [lots of snip]
>>
>> > With that one the kernel now survies memguard_init() but then panics
>> > right afterwards when kmeminit() calls kmem_suballoc():
>> > KDB: debugger backends: ddb
>> > KDB: current backend: ddb
>> > Copyright (c) 1992-2010 The FreeBSD Project.
>> > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
>> > ? ? ? ?The Regents of the University of California. All rights reserved.
>> > FreeBSD is a registered trademark of The FreeBSD Foundation.
>> > FreeBSD 9.0-CURRENT #18 r215249:216120M: Mon Dec ?6 13:27:57 CET 2010
>> > ? ?marius at v20z.zeist.de:/home/marius/co/build/head2/sparc64.sparc64/usr/home/m4
>> > WARNING: WITNESS option enabled, expect reduced performance.
>> > panic: kmem_suballoc: bad status return of 3
>>
>> [more snip]
>>
>> Shooting in the dark a little...
>>
>> The bad status of 3 is presumably KERN_NO_SPACE because we attempted
>> to allocate too much space from the kernel_map.  What are the actual
>> values of vm_kmem_size, kernel_map->min_offset, kernel_map->max_offset
>> at panic time?
>
> vm_kmem_size=5610405888 min_offset=3221225472 max_offset=13958643712
> This is on a US3i machine with 16GB RAM.

So kernel_map is from 0xC000_0000 to 0x3_4000_0000, or 0x2_8000_0000
bytes large.  Double the vm_kmem_size is 0x2_9CD0_0000, which is why
this is failing.

This to me says that, for the moment, you need the
VM_MAX_KERNEL_ADDRESS define so that memguard does not take up too
much virtual space; at the moment this hardware is somewhat
constrained on virtual space.

Thanks,
matthew


>
>>  How much virtual space does sparc64 support (since
>> earlier it was reported it's computed based on hardware capability,
>> for this specific machine?)
>>
>
> Currently, the limiting factor is that the kernel TSB is addressed
> virtually, which means that the TTEs for kernel TSB itself have to be
> put into locked dTLB slots taking up 1 4MB dTLB slot per 1GB. US3
> family CPUs have 16 lockable 4MB dTLB and iTLB slots, which need to
> be shared between the TSB and the kernel itself though, i.e. a 9MB
> kernel also takes up 3 slots (US1/2 machines have 64 dTLB and iTLB
> slots but there these are also used for unlocked TTEs so we don't use
> more than 32 for kernel plus TSB on these). VM_MAX_KERNEL_ADDRESS is
> limited according to how many slots are available for the TSB, that's
> what I was referring to previously.
> Actually, US3 and later as well as SPARC64 V and later CPUs have
> a feature allowing the TSB to be addressed physically, circumventing
> the need to lock the TSB into the TLB, thus allowing the full 64-bit
> virtual address space to be used. Implementing that is on my TODO
> list but unfortunately that's not exactly straight forward and also
> requires some instructions to be patched at runtime so the kernel
> still works on US1/2 machines.
>
> Marius
>
>