kernel panic with greater that 8 GB of memory

[Steve Kargl]

kmem_suballoc: bad status return of 3
panic: kmem_suballoc
cpuid = 0
KDB: stack backtrace:
kdb_backtrace() at kdb_backtrace+0x37
panic() at painc+0x1d1
kmem_suballoc() at kmem_suballoc+0x93
kmeminit() at kmeminit+0x16d
mi_startuup() at mi_startup+0xb6
btext90 at btext+0x2c
KDB: enter: panic
[thread pid 0 tid 0]

This is a Tyan K8S Pro motherboard with 16 GB of PC2700
Cosair ECC memory.  The BIOS was flashed to the latest
available from Tyan.  The scheduler is 4BSD.  There are
no ATA devices in the system and no ATA subsystem in
the kernel.  This a SCSI only box.   World and kernel
were rebuilt today with CFLAGS=-O -pipe and
COPTFLAGS=-O -pipe.

If I set hw.physmem="#G" n /boot/loader.conf where # is
less than or equal to 8 the box boots up (see dmesg below).
If # > 8, then we have the above panic.

Any ideas where to look for the a fix?

-- 
Steve


Copyright (c) 1992-2004 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD 6.0-CURRENT #0: Mon Nov 29 12:18:32 PST 2004
    root at dhcp-78-96.apl.washington.edu:/home/obj/usr/src/sys/SPEW
WARNING: WITNESS option enabled, expect reduced performance.
ACPI APIC Table: <A M I  OEMAPIC >
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: AMD Opteron(tm) Processor 248 (2193.63-MHz K8-class CPU)
  Origin = "AuthenticAMD"  Id = 0xf58  Stepping = 8
  Features=0x78bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2>
  AMD Features=0xe0500800<SYSCALL,NX,MMX+,LM,3DNow+,3DNow>
real memory  = 8589934592 (8192 MB)
avail memory = 8252538880 (7870 MB)
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  1
MADT: Forcing active-low polarity and level trigger for SCI
ioapic0 <Version 1.1> irqs 0-23 on motherboard
ioapic1 <Version 1.1> irqs 24-27 on motherboard
ioapic2 <Version 1.1> irqs 28-31 on motherboard
acpi0: <A M I OEMRSDT> on motherboard
acpi0: Power Button (fixed)
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x5008-0x500b on acpi0
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 6.0 on pci0
pci3: <ACPI PCI bus> on pcib1
ohci0: <OHCI (generic) USB controller> mem 0xfeafc000-0xfeafcfff irq 19 at device 0.0 on pci3
ohci0: [GIANT-LOCKED]
usb0: OHCI version 1.0, legacy support
usb0: <OHCI (generic) USB controller> on ohci0
usb0: USB revision 1.0
uhub0: AMD OHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 3 ports with 3 removable, self powered
ohci1: <OHCI (generic) USB controller> mem 0xfeafd000-0xfeafdfff irq 19 at device 0.1 on pci3
ohci1: [GIANT-LOCKED]
usb1: OHCI version 1.0, legacy support
usb1: <OHCI (generic) USB controller> on ohci1
usb1: USB revision 1.0
uhub1: AMD OHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub1: 3 ports with 3 removable, self powered
pci3: <mass storage> at device 5.0 (no driver attached)
pci3: <display, VGA> at device 6.0 (no driver attached)
fxp0: <Intel 82551 Pro/100 Ethernet> port 0xa400-0xa43f mem 0xfeaa0000-0xfeabffff,0xfeafb000-0xfeafbfff irq 18 at device 8.0 on pci3
miibus0: <MII bus> on fxp0
inphy0: <i82555 10/100 media interface> on miibus0
inphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
fxp0: Ethernet address: 00:e0:81:2a:59:e7
isab0: <PCI-ISA bridge> at device 7.0 on pci0
isa0: <ISA bus> on isab0
pci0: <mass storage, ATA> at device 7.1 (no driver attached)
amdpm0: <AMD 756/766/768/8111 Power Management Controller> port 0xe0-0xff,0xcc00-0xcc1f irq 19 at device 7.2 on pci0
amdpm0: could not map i/o space
device_attach: amdpm0 attach returned 6
pci0: <bridge, PCI-unknown> at device 7.3 (no driver attached)
pcib2: <ACPI PCI-PCI bridge> at device 10.0 on pci0
pci2: <ACPI PCI bus> on pcib2
ahd0: <Adaptec AIC7902 Ultra320 SCSI adapter> port 0x9c00-0x9cff,0x9000-0x90ff mem 0xfc8fc000-0xfc8fdfff irq 24 at device 6.0 on pci2
ahd0: [GIANT-LOCKED]
aic7902: Ultra320 Wide Channel A, SCSI Id=7, PCI 33 or 66Mhz, 512 SCBs
ahd1: <Adaptec AIC7902 Ultra320 SCSI adapter> port 0x9400-0x94ff,0x9800-0x98ff mem 0xfc8fe000-0xfc8fffff irq 25 at device 6.1 on pci2
ahd1: [GIANT-LOCKED]
aic7902: Ultra320 Wide Channel B, SCSI Id=7, PCI 33 or 66Mhz, 512 SCBs
bge0: <Broadcom BCM5704C Dual Gigabit Ethernet, ASIC rev. 0x2003> mem 0xfc8a0000-0xfc8affff,0xfc8b0000-0xfc8bffff irq 24 at device 9.0 on pci2
miibus1: <MII bus> on bge0
brgphy0: <BCM5704 10/100/1000baseTX PHY> on miibus1
brgphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX, 1000baseTX-FDX, auto
bge0: Ethernet address: 00:e0:81:2a:da:46
bge1: <Broadcom BCM5704C Dual Gigabit Ethernet, ASIC rev. 0x2003> mem 0xfc8d0000-0xfc8dffff,0xfc8e0000-0xfc8effff irq 25 at device 9.1 on pci2
miibus2: <MII bus> on bge1
brgphy1: <BCM5704 10/100/1000baseTX PHY> on miibus2
brgphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX, 1000baseTX-FDX, auto
bge1: Ethernet address: 00:e0:81:2a:da:47
pci0: <base peripheral, interrupt controller> at device 10.1 (no driver attached)
pcib3: <ACPI PCI-PCI bridge> at device 11.0 on pci0
pci1: <ACPI PCI bus> on pcib3
pci0: <base peripheral, interrupt controller> at device 11.1 (no driver attached)
acpi_button0: <Power Button> on acpi0
pci_link0: <ACPI PCI Link LNKA> irq 15 on acpi0
pci_link1: <ACPI PCI Link LNKB> irq 9 on acpi0
pci_link2: <ACPI PCI Link LNKC> irq 11 on acpi0
pci_link3: <ACPI PCI Link LNKD> irq 10 on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x64,0x60 irq 1 on acpi0
atkbd0: <AT Keyboard> flags 0x1 irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
psm0: model Generic PS/2 mouse, device ID 0
sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
sio0: type 16550A
sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0
sio1: type 16550A
fdc0: <floppy drive controller (FDE)> port 0x3f7,0x3f0-0x3f5 irq 6 drq 2 on acpi0
fdc0: [FAST]
fd0: <1440-KB 3.5" drive> on fdc0 drive 0
orm0: <ISA Option ROMs> at iomem 0xc8000-0xcc7ff,0xc0000-0xc7fff on isa0
ppc0: cannot reserve I/O port range
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <8 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
Timecounters tick every 0.976 msec
Waiting 3 seconds for SCSI devices to settle
acpi_cpu: throttling enabled, 8 steps (100% to 12.5%), currently 100.0%
SMP: AP CPU #1 Launched!
cd0 at ahd0 bus 0 target 4 lun 0
cd0: <PLEXTOR CD-R   PX-W4012S 1.01> Removable CD-ROM SCSI-2 device 
cd0: 20.000MB/s transfers (20.000MHz, offset 15)
cd0: Attempt to query device size failed: NOT READY, Medium not present - tray closed
da1 at ahd0 bus 0 target 1 lun 0
da1: <SEAGATE ST373307LW 0007> Fixed Direct Access SCSI-3 device 
da1: 40.000MB/s transfers (20.000MHz, offset 63, 16bit), Tagged Queueing Enabled
da1: 70007MB (143374744 512 byte sectors: 255H 63S/T 8924C)
da0 at ahd0 bus 0 target 0 lun 0
da0: <SEAGATE ST373307LW 0007> Fixed Direct Access SCSI-3 device 
da0: 40.000MB/s transfers (20.000MHz, offset 63, 16bit), Tagged Queueing Enabled
da0: 70007MB (143374744 512 byte sectors: 255H 63S/T 8924C)
Trying to mount root from ufs:/dev/da0s1a
WARNING: / was not properly dismounted
WARNING: /home was not properly dismounted
WARNING: /usr was not properly dismounted
WARNING: /var was not properly dismounted
/var: mount pending error: blocks 16 files 8