kernel panic with greater that 8 GB of memory

[David O'Brien]

On Tue, Nov 30, 2004 at 02:35:30PM -0500, Ketrien I. Saihr-Kenchedra wrote:
> On Tue, 30 Nov 2004, Steve Kargl wrote:
> 
> <slice and dice>
> >The system starts to boot and dmesg reports the 16 GB of memory, but
> >I rapidly get this panic.
> <snippity!>
> >So, it looks like ACPI on this Tyan motherboard is restricting
> >the memory to 8G.
> 
> I presume you're using CM74SD2048RLP-2700/S DIMMs? (Corsair/Samsung 2GB
> ECC Reg'd.) 
> More importantly, which BIOS version? You have to run the 2.03 BIOS on 
> the K8SPro if you're running 16GB. Make sure that ccNUMA support is off
> in the BIOS as well - this was a never-ending source of fun on both the
> S2882 and S4882's.

There is no 'ccNUMA' setting in the BIOS.  A multi-processor Opteron is a
NUMA architecture machine regardless of any BIOS settings.  I.E. there is
no way to disable that a MP Opteron is a NUMA machine.

The setting of interest is should the BIOS round-robin interleave
physical addresses across the NUMA nodes[*].  The reference AMI BIOS
refers to this as "Node Interleaving".  It should be "DISABLED".  Or if
the BIOS speaks of the SRAT table, it should be "ENABLED".  While FreeBSD
doesn't use the SRAT table (and cannot until ACPI 3.0 BIOS's); turning on
the SRAT turns off node interleaving.

There is a CPU errata for older CPU's that can cause problems if you have
both memory interleaving turned on and ECC turned enabled in the BIOS.

-- 
-- David  (obrien at FreeBSD.org)