only 3 of 4GB memory available on amd64 8.1-RC1

[Andriy Gapon]

on 19/06/2010 20:16 Andreas Tobler said the following:
> Hi all,
> 
> I got my hands on a t60 with 4GB of RAM (BIOS displays it)
> 
> And I installed 8.1-RC1 on it:
> 
> FreeBSD 8.1-RC1 #0: Mon Jun 14 13:40:28 UTC 2010
>     root at mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64
> Timecounter "i8254" frequency 1193182 Hz quality 0
> CPU: Intel(R) Core(TM)2 CPU         T5600  @ 1.83GHz (1828.76-MHz
> K8-class CPU)
>   Origin = "GenuineIntel"  Id = 0x6f6  Family = 6  Model = f  Stepping = 6
> 
> Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
> 
>   Features2=0xe3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM>
>   AMD Features=0x20100800<SYSCALL,NX,LM>
>   AMD Features2=0x1<LAHF>
>   TSC: P-state invariant
> real memory  = 4294967296 (4096 MB)
> avail memory = 3092344832 (2949 MB)
> ACPI APIC Table: <LENOVO TP-79   >
> FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
> FreeBSD/SMP: 1 package(s) x 2 core(s)
>  cpu0 (BSP): APIC ID:  0
>  cpu1 (AP): APIC ID:  1
> 
> 
> 
> Do I need to configure something special to get the full 4GB of memory?
> 
> I only found the PAE hint, but this is for x86 machines, right?

Yes, PAE is for what we call in FreeBSD land "i386".
"x86" we use for both i386 and amd64.

Now, to the rest.
I recently investigated this topic myself, so I can share what I learned.
The first thing you'd want to have is SMAP information.
You can get it at the loader prompt with 'smap' command.
It is also reported by kernel during a verbose boot, but it is only printed to
console; it is not saved to dmesg, because it is printed before msgbuf is created.

Example of how SMAP information looks:
SMAP type=01 base=0000000000000000 end=000000000009f800
SMAP type=02 base=00000000000f0000 end=0000000000100000
SMAP type=02 base=00000000fec00000 end=0000000100000000
SMAP type=02 base=00000000e0000000 end=00000000f0000000
SMAP type=02 base=000000000009f800 end=00000000000a0000
SMAP type=02 base=00000000bfdf0000 end=00000000bfe00000
SMAP type=01 base=0000000000100000 end=00000000bfde0000
SMAP type=03 base=00000000bfde3000 end=00000000bfdf0000
SMAP type=04 base=00000000bfde0000 end=00000000bfde3000
SMAP type=01 base=0000000100000000 end=0000000130000000

Type 1 ('01') is memory ranges free for OS use.
Note that the ranges could be unsorted.

SMAP would give you an idea what is free for OS use and what is taken away for
BIOS and hardware needs.  Also note that what's reported as "avail memory" is
smaller than sum of sizes of all unreserved regions.  Some space is taken away
by FreeBSD virtual memory code to store some core page/memory management
information (~3%).  Memory used by kernel and preloaded modules is also not
included into "avail memory".

If you discover that your system reserves, in your opinion, too much memory,
then consider the following things.
1. Some memory might be allocated as video adapter's aperture and/or video
adapter's memory in case of some integrated graphics solutions.
2. Some memory addresses below 4GB are used for MMIO (memory mapped
input/output) - accessing those addresses is actually communicating with some
hardware rather than accessing DRAM.  Some hardware+firmware combinations can
hoist (or remap) DRAM that corresponds to such an address range to a different
address range above 4G.  If you examine last line of my sample SMAP output, then
you'll see that there is 756MB of "type 01" RAM above 4GB and that machine has
exactly 4GB of DRAM installed - this is MMIO range 0xd0000000-0x100000000
hoisted to 0x100000000-0x130000000.
If your hardware doesn't support that option, then too bad - the memory is
"overshadowed" by MMIO and is effectively lost.
Sometimes hoisting option (under whatever name) has to be explicitly set in BIOS
configuration.

That's about all.
I am curious as to what you would discover about your system - please share with us.

-- 
Andriy Gapon