System advice requested

Willem Jan Withagen wjw at
Sun Feb 22 05:26:36 PST 2004

----- Original Message ----- 
From: "Willem Jan Withagen" <wjw at>
> If I were to get a dual opteron board/system what things would be
> to watch out for, or would be required. Hardware wise that is.
> In the beginning I would like to use is as:
>     1) FreeBSD/windows/linux desktop
> and lateron it would turn into my 
>     2) Home NFS/Samba-server running FBSD
> In case 2) I'd remove any fancy user-interface stuff...
> All suggestions welcomed,

To follow-up on myself...

The attachement contains all(??) comments about hardware on this
list in the last 4 months. Currently totally completely unorganised.
(Other than the most recents comments are first in the file)

Given some more time, I'll try to organise it more sensible.
And of course: comments welcome.

-------------- next part --------------
Gobrien at
you want to populate memory in pairs per CPU.
This is so you can use both channels of the DDR memory controller.  If
you only put one DIMM per CPU, you're missing 1/2 your potential

james-freebsd-amd64 at 
> Note 5.2.1 also adds the Silicon Image 3114 SATA support needed to
> install on a SATA drive in many Opteron systems.

Is anyone else having trouble with this?

The 3114 is found no problem but the drive is not seen.
The culprit may be this code in ata_reset().

billsf at
> Is anyone using an Adptec 29320?  Mine is not working.
> I suspect is may be a BIOS problem on the Tyan K8W S2885.
> I also have a Highpoint 1820 that does not work right.

Not anymore. It worked for awhile and after a few days it just stopped.
Next i was given a PERC 320/DC Adaptec FSA RAID (aac) This was much better
but still an Adaptec. I'm using a 29160 now and it is going to go! These
cards just don't seem to be up to a busy Unix environment. In all cases
the firmware was the latest. (Surprisingly FreeDOS runs quite well on 
the amd64, and needed to run the flash.exe stuff.) I'm getting a real
Cirrus Logic card. These cost a third what Adaptec costs and my experience
has only been good with these cheap (smb) cards. I work with servers and
most BOFH's swear by them and deplore Adaptec. (Therefore all those 'free'
cards to try. ;-) Gladly stump up a hundred Euros or so for piece of mind.  

Presently I'm using an Asus K8 motherboard for an Opteron-100 marketed as
"Athlon-FX" or something. Not bad for Taiwan, but apparently Taiwan thought
amd64 was going to be Windows and therefore some really goofy features. 

All AMD products seem to work better slightly overclocked. 2250MHz for a 
2200MHz chip is perfectly inline with the 3 - 5% rule. When increasing the 
bus speed, Adaptec cards give out first, then SATA and then the video and 
so on. Sorry, this is a bit offtopic.

bob at
Well, my early Tyan Athlon MBs (Tiger S2460) didn't like Crucial
DIMMs much, and my later Tyan boards (Tiger s2466's) won't work with the
Crucial DIMMs at all (I have about 8 of them on a shelf that I can't
use at all now). I never have seen any ECC errors (there registered ECC
DIMMs) with these, just varying degrees of system unstability (random
memory erors/panics). A couple of years ago I spent quite a bit of time
on the phone with Crucial's tech support folks (shortly after I bought
the memory) with no real resolution (the tech support guy tried to be
helpful but couldn't really help).

Following this experience, I have come to worry much more than I
previously did about memory compatibility issues; and I began to wonder
just how much good ECC memory really does in these systems when you
wind up with all kinds of system instability. I now worry more about
compatibility than do ECC vs. non-ECC memory.

Note, that my newer Tyan dual MB systems (those with the S2466 MBs
that can run with unbufferred/non-ECC DIMMs) have been running with
unbufferred/non-ECC DIMMs for some time (over a year) and I have had
much better stability with these systems than when I tried the Crucial
reg/ECC DIMMs (generally, they either wouldn't boot at all or only
ran for a short time, depending on how many DIMMs I installed--more
meant shorter time to crash). Note that I never did try other brands of
reg/ECC memory on these systems since they were stable with the cheaper
stuff and I was growing tired of spending money on them.

obrien at
Tyan GX28 support?

> The onboard sATA on the S2885 isn't supported.  The GX28
> uses a Promise controller chip that is more likely to work.
> I have no idea if the LSI SCSI controller is supported.
> It's a BCM5704c ethernet: mine is a 5705.

Both the LSI SCSI and BCM5704c are supported.

obrien at
> what motherboard do you guys recommend for use on a dual cpu ssytem. 

Server or workstation??
Workstation get the Tyan Thunder K8W or Iwill DK8X.
Server get Arima RioWorks, or Tyan Thunder K8S PRO.

> would prefer ddr  non registered memory capability and sata drive support.

ALL Opterons require registered ECC memory.

james-freebsd-amd64 at
My Tyan K8W S2885 works fine with its onboard Broadcom.  Will's
problems appear caused by ACPI problems, which is a property of the
particular BIOS the motherboard interacting with whatever bugs exist
in FreeBSD's ACPI interpreter.  Your Tyan probably works as-is.

obrien at
> I downloaded the 5.2-RC2 AMD64 miniinst ISO to try on a MSI K8T800
> based dual opteron system today.

What is the board model number?  I am not aware of any K8T800 based SMP
boards.  The only MSI 2P board I know if is the K8D Master MS-9131 which
is AMD 8100.

chat95 at
Hm, I know it, but unfortunately, there are no data for
Tyan thunder k8w (S2885) and Rioworks HDAMB-WO... anyway,
for Rioworks HDAMB-WO,

  I found a PDF (though it was written in Japanese).

o AMD Opteron Processor Compatible DDR Memory Modules
  AMD Tested Memory
  Validated DIMMs - AMD Opteron Processor,,30_118_8796_8819%5E9394,00.html

obrien at
At work we're pretty sloppy about what random no-name DIMM's we use.
Only the Shuttle box has been picky.  That said, Micron (MT) / Crucial
DIMM's are probably the best quality.  Samsung DIMM's also.  These are
the two brands AMD uses in the development systems it sends out.

I've got a case load of Kingston 1GB ValueRAM in which 1 out of 4 DIMM's
is bad.

Markus at
> The motherboard has the same Sil3114 SATA controller as the Tyan Thunder K8W 
> (S2885).  I have seen other posts to this list that indicate success in 
> installing FreeBSD on the Thunder K8W, so I'm wondering why I can't see the 
> drives on my system.

I made a simple modification to the driver which essentially maps the 3114 as
a 3112, which works, sort of (not with the recent changes though, I'm getting
those "TIMEOUT - ATAPI_IDENTIFY retrying messages" that others had mentioned
as well). The drives do not go into SATA DMA mode (150Mbps), but I have the 
slight suspicion the code doesn't really support those yet with the SiL chipsets.

On very high load on the drives, I sometimes get 
  ad4: WARNING - WRITE_DMA UDMA ICRC error (retrying request)
style errors, but all in all, it's working very nicely.

obrien at
> You might be able to save some money by using an Athlon64 CPU and
> motherboard instead of Opteron, but registered DIMM might not work.

Registered(buffered) DIMM's will not work in Athlon64 motherboards.

obrien at
>     Adaptec 3950U2B hosting a pair of LVD drives
>     Adaptec 2400A IDE RAID hosting a quartet of WD 120GB Sp. Editions
>     ATI Radeon 7200 (aka Radeon VIVO)
>     Netgear FA310TX serving the outside world
>     Netgear FA302T serving the inside LAN

All of this will work.

WillS at
If you used an Opteron 240 on an MSI K8T Master2-FAR
board you would spend ~$600. That board is ATX-sized so it doesn't require
any special case.

Your RAM is too slow, and I don't know if you can run the memory bus 
asynchronously, so if it were me I would figure on buying more RAM. I think
you can get 1GB of Corsair Registered ECC DDR400 in a 'matched pair' of 
512MB modules for less than $400, so that would put you under your spending

obrien at
> Your RAM is too slow, and I don't know if you can run the memory bus 
> asynchronously, so if it were me I would figure on buying more RAM.

Huh?  I don't follow, why is running PC2100 in any AMD64 machine running
the memory bus asynchronously??

PC1600 memory will work fine any any AMD64 mobo, so will PC2100 and
PC2700.  You'll need the latest BIOS's and a rev.C CPU to run DDR400.

obrien at
Well tested are Matrox G4[05]0 and nVidia GeForce{3,4}.  I have no ATI
experience on FreeBSD/amd64.

tilman at
I use an

none0 at pci1:0:0: class=0x030000 card=0x596412ab chip=0x59641002 rev=0x01 hdr=0x00
    vendor   = 'ATI Technologies'
    device   = 'Radeon 9200 Series'
    class    = display
    subclass = VGA

obrien at
I don't quite follow you.  You are using the 2015S ZCR in 32-bit or
64-bit FreeBSD?  It should work with FreeBSD/i386 on Opteron HW.  asr(4)
isn't 64-bit clean and thus asr(4) devices aren't supported in the
FreeBSD/amd64 OS.

tomh at
All right, so I gave up on my Adaptec 2100S RAID controller, and managed to
locate a used HP NetRAID 3Si (aka MegaRAID 438).  The NetRAID is only U2W
(80 MB/s, as I recall?), but FreeBSD/amd64 seems to be much happier with it.

peter at
The only major chunk of hardware that I know of that has big issues is the 
Promise SX6000.  The driver tries to use a 32 bit field in a hardware data 
structure to store a 64 bit pointer.

james-freebsd-amd64 at
I'm using a Tyan S2885, which appears somewhat similar to this.
FreeBSD has run flawlessly for me - no reboots, crashes or
other headaches.  I'm using FreeBSD 5.2 RC2.

The onboard sATA on the S2885 isn't supported.  The GX28
uses a Promise controller chip that is more likely to work.
I have no idea if the LSI SCSI controller is supported.
It's a BCM5704c ethernet: mine is a 5705.

I'm using 8 GB of memory (8x Crucial part #CT12872Y335)
and a HighPoint 1542 sATA controller.

obrien at
> On the Tyan 2880 motherboard.

I know you've already bought it... but to everyone else I strongly
recommend not buying this motherboard.  The 4+2 DIMM configuration[*] is
sub-optimal and you can get 4+4 DIMM configuration from Tyan (K8S Pro)
and other makers.

[*] I fully know the bad history of Opteron 4+2 DIMM configurations...
but can't speak publically about it. :-(

peter at
The ones I have personal experience with are:

940 pin:
Asus SK8N: works (nForce3 chipset, no driver for onboard nVidia nic, onboard
                  promise 378 raid+SATA)
Rioworks HDAMA: works (AMD chipset, 2 cpu, onboard if_bge gigabit, in use on
                  the cluster as and at work)

754 pin:
Asus K8V: works (VIA K8T800 chipset, onboard ide not recognized without
                 patch which has integrated SATA support, decent onboard
                 3com/syskonnect gigabit nic (if_sk), onboard promise 378
Gigabyte K8NPro: works (nForce3, lousy tiny bios interface, realtek 8110
                 gigabit nic (if_re) - supposedly not bad.  onboard nVidia
                 ethernet isn't connected (but we dont have a driver anyway)).
AMD Solo4, Solo5 and Solo7 (developer reference boards)

I really like both of the ASUS boards for personal use and the Rioworks for
servers.  I have not seen any of the MSI boards so I can't vouch for them.
The MSI 2xCPU board only has 2 dimm slots on one of the cpus so I'm not
that interested in their server board.  Their consumer board appears to
have similar specs to the asus K8V but I haven't tried it.

All of these boards work with ACPI, minus some quirks with ordering the
serial ports.  Some of them list the COM2 port first in the tables which
means we assign 'sio0' to it.  Thats only a minor annoyance though.

The onboard nVidia ethernet is annoying because they only give out a binary
driver blob that you write an OS shim to use.  This works for i386, but
obviously isn't going to work too well for an amd64 kernel.  The only board
affected by this is the Asus SK8N, but ethernet cards are not a big deal.

I really really dislike the bios on the Gigabyte, and really like the ones
on the ASUS boards.  The Gigabyte bios interface is almost worthless and
gives you very little control over anything.  If it had to be named, I'd
call it a 'BIOS for dummies'.  On the other hand, the ASUS bios' give you
more than enough rope.  For example, Gigabyte have 'PNP OS: Yes' by default
and dont give you the ability to turn it off.  ASUS do let you turn it off.

obrien at
I've been asked a few times what case one can use in building a dual-proc
Opteron workstation.  So I thought I'd answer it publically for all.

AMD uses the InWin Q500 with the internal 3"
drive cage removed.  This case has tons of room and even with the 3"
drive cage removed there are still 5 drive bays left.  This case is
US$69.32 from

If you want a shorter case, I have also installed both Tyan K8W (s2885)
and MSI-9131 mobos into the Antec SX835II Performance Series II mid-Tower
Again with the internal 3" drive cage removed.  

eric at
The "Antec" cases are also available under a variety of other names, they are actually manufactured by:

WillS at
At least with my board (an 'awful' 4+0 MSI K8T Master2-FAR board) the onboard broadcom 
NIC does not work correctly with bge. It does work with ndis though. Also for whatever 
reason -current as of last weekend would lock the machine up hard after ~ 30 minutes. 
Haven't had time to work on why.


On Sat, Feb 14, 2004 at 11:26:06PM +0100, Adriaan de Groot wrote:
> Take a look at the archives of this list (via
> <> ), there's lot of
> hardware mentioned. I think a lot depends on how much memory you need to
> support your users: Athlon64 boards (single CPU) tend to top out at 4G
> theoretical and 3G practical (like my Asus k8v, with 3 slots so I could
> stick 3G of DDR266 in it) ; Opteron boards for single CPU with 4 slots top
> out at 8G theoretical (4x2G Registered), but I hear tales of an sk8n
> supporting only 3.5G. Dual boards run up to 8 slots, for 16G of memory.

Not just SK8N's. Just about every AMD64 mobo will "loose" physical
memory from 3.5-4GB. So one would get 17.5GB in your example of 8x2GB
DIMM's. The reasons have been explained in this list before.

nVidia nForce3 [for AMD64] chipsets are very problematic for
Unix(BSD)/Linux. I would avoid them if you want to run a non-MS-Windows
operating system.

-- David (obrien at <mailto:obrien at>)

From: "Haapanen, Tom" <tomh at>
Andrew Gallatin:
> Thanks. But are you saying the VIA K8T800 chipset is flakey too, and I 
> should only consider AMD-8000 based boards? Just from scanning the 
> last month or so of archives, it looks like the bad hardware is pretty 
> evenly distributed across chipsets, but perhaps that's not enough 
> perspective. I'm new here, after all..

Absolutely not saying that. I don't have any experience with K8T800-based
boards at all.

But many people have been highly successful with the Tyan boards (I have
been running a K8S Pro-based server in production for more than two weeks
now, and the mysqld process is approaching 100 hours of CPU time.) 

So I am only suggesting them as an option ...

Tyan K8W here with dual 242's. Love it.

Dylan Carlson [absinthe at]

More information about the freebsd-amd64 mailing list