Problems with AMD64 and 8 GB RAM?

[Alan Jay]

> Date: Thu, 31 Mar 2005 16:12:35 +0900
> From: Ganbold <ganbold at micom.mng.net>
> Subject: Re: Problems with AMD64 and 8 GB RAM?
> 
> Hi,
> 
> Since we are discussing AMD64 with 8GB RAM, I also would like to point my
> problem.
> 
> I'm still looking for possibility to run FreeBSD 5.3-STABLE with more than
> 4GB RAM
> on Dual amd64 2.2GHz machine (IBM @server 325) with ServeRAID 6M (ips
> driver)).
> Right now I'm using only 4GB RAM and this server is in production.
> 
> #uname -an
> FreeBSD publica.ub.mng.net 5.3-STABLE FreeBSD 5.3-STABLE #12: Mon Nov 22
> 12:04:57 ULAT 2004     tsgan at publicc.ub.mng.net:/usr/obj/usr/src/sys/AMD
> amd64
> 
> As Scott said a few months ago, problem is below:
> 
> "The ips driver looks like it will fail under heavy load when more than 4GB
> of RAM is present.  It tries to force busdma to not defer requests when the
> bounce page reserve is low, but that looks to be broken and
> will result in corrupted commands."

[Alan Jay] Since we are talking about FreeBSD on AMD64 on the AMD64 list I
have reported issues on that list.

I have a TyanThunder K8S pro S2882 twin Operteron with 8Gb of RAM and although
I can get the machine to run reasonably stably with 8Gb of RAM with limited
loading when pushed it falls over unpredictably.

We did some tests with the latest 5.3-STABLE / 5.4-PRERELEASE and still found
the same issues when using a mySQL database heavily hit over the Ethernet
controller.  Our final tests limited the memory on boot-up to 4Gb and the bug
is still there so we think it may well be some interaction with the Ethernet
controller.  The motherboard we have has a BroadcomBCM5704C 10/100/1000 based
card on board.  

Again this works fine initially but then we get a very dramatic failure with
no warning messages and the system falls over.  

There are still a few issues to be ironed out with the FreeBSD 5.x on AMD64
the latest STABLE/PRE-RELEASE is much improved but be aware there may be
issues.  We will be waiting a few more weeks before re-trying these tests to
see if the latest fixes that have been discussed have solved our problems.