Hardware RAID support? Which controller best to use?

[ray at redshift.com]

At 02:00 PM 11/18/2005 -0500, Nathan Vidican wrote:
| ray at redshift.com wrote:
| > | I'm trying to get him towards ordering a 3Ware-Escalade 9500S-4 RAID 
| > | controller (or possibly the -8 or -12 one).
| > 
| > The 9500S line is a good choice.  You can also sometimes find them used/NIB on
| > ebay at a fair price ($300 to $400).  BTW, in my testing, you don't get much
| > additional performance from having 8 vs 4 drives (if you want to save costs).
| > Although you do get more space :)  
| >  
| > | One thing which is of importance is that their budget may not allow for 
| > | a really high end server, so they may have to decide to buy a semi 
| > | high-end server. This then may result in them taking Athlons instead of 
| > | Opterons, and it may result in a MoBo with 32-bits PCI slots instead of 
| > | 64-bits PCI slots. Judging from the pictures of the 9500S it has a 
| > | 64-bits PCI slot, but it looks like it might be compatible with the 
| > | 32-bits slots as well (at a performance penalty, of course).
| > | Would you happen to know if that's indeed possible, or whether they'd 
| > | better choose a different RAID controller?
| > 
| > You'd have to check the 3ware website to be sure, but a lot of times those
cards
| > are backwards compatible with the 32 bit slots.  I just checked the manuals I
| > have for the 9000 series and it says 3Ware cards will run properly in any
PCI or
| > PCI-X slot, but you'll get better performance in a 64 bit slot - here is the
| > text from the manual:
| > 
| > --------------------------------------------------------
| > Selecting the Slot in Which to Install the Controller
| > Consider these factors when deciding on the slot in which to insert the
controller:
| > .. While the 3ware RAID Controller runs properly in any PCI or
| > PCI-X slot, not all slots give equal performance due to the
| > architecture of the PCI bus.
| > These slots typically give the best performance:
| > .. Slots closest to the Accelerated Graphics Port (AGP), if
| > included.
| > .. 64-bit PCI slots
| > Although the controller fits in both 32-bit and 64-bit PCI or
| > PCI-X slots with 5V as well as with 3.3V, install it in a 64-
| > bit slot to take full advantage of the controller’s
| > performance.
| > --------------------------------------------------------
| >  
| > | Regarding the MySQL versions and their settings: tnx for giving me 
| > | enough comfort to indeed give heavy preference for the FreeBSD amd64 
| > | version. Will (source) installing the version from the ports do, or do 
| > | you mean something else when you say that you compiled MySQL yourself?
| > 
| > from the tar.gz off the mysql site.  E-mail me privately for a full run down on
| > the install I use, but here is the basic idea:
| > 
| > as root do this:
| > ------------------------------------------------------
| > tar -xzf mysql-4.1.15.tar.gz
| > cd mysql-4.1.15
| > pw groupadd mysql
| > adduser {user you are creating is mysql}
| > 
| > CFLAGS="-O3" CXX=gcc CXXFLAGS="-O3 -felide-constructors -fno-exceptions
| > -fno-rtti" ./configure --prefix=/usr/local/mysql --enable-assembler
| > --with-mysqld-ldflags=-all-static --without-debug
| > 
| > make
| > make install
| > make clean
| > 
| > ln -s /usr/local/mysql/share/mysql/mysql.server /usr/local/bin/mysql.server
| > ln -s /usr/local/mysql/bin/mysql /usr/local/bin/mysql
| > ln -s /usr/local/mysql/bin/mysqld_safe /usr/local/bin/mysqld_safe
| > ------------------------------------------------------
| > 
| > There is more too it, but that's the basic idea.  Check the readme/install
files
| > with the tar.gz - I will also send you my entire run down privately in a
second.
| >  
| > | Regarding the benchmark results: I'd love to receive them. Can you 
| > | perhaps send them off-list to me?
| > 
| > Actually, I forgot that I put them on-line a while back.  Here you go:
| > 
| > http://www.redshift.com/~ray/mysql_bench/
| >  
| > | Regarding W*nd*ws vs. FreeBSD: I love your remark; I wonder if the 
| > | person I forwarded it to can laugh as loudly about it as I did. :D
| > 
| > :)
| >  
| > | Finally regarding SCSI vs. SATA:
| > | >I've had far better luck using SATA over SCSI in the recent couple of years.
| > | > We have several machines setup using FreeBSD and 3Ware RAID 0+1 that 
| > | > routinely run with no problems and uptimes of 200 to 300 days at a time.
| > | 
| > | Very interesting to know. At present, I myself have a 754 socket AMD 
| > | Athlon 64 3.2 GHz (IIRC), running FreeBSD 5.4 release AMD-64, with an 
| > | Adaptec 2200S U320 SCSI RAID controller with 4 Maxtor Atlas 10KIV 36GB 
| > | drives attached to it in RAID-10 mode. So far it works a charm (though I 
| > | too had to effectively downgrade it to U160 due to the lack of 64-bits 
| > | PCI slots, grrr). I hope it'll keep performing well (so far uptimes in 
| > | the order magnitude you mention have been working fine for me as well on 
| > | SCSI - Adaptec 2100S RAID set-ups in my (now) fall-back server, and ever 
| > | after installing the AMD-64 one 34 days ago I haven't had to restart it 
| > | so far)...
| > 
| > The file will send you with the benchmarks includes the PHP files to run the
| > benchmarks, so if you feel like adding your #'s to the mix, just let me know.
| > Or feel free to run the code and send me the results.  I'll e-mail you
privately
| > in a second here.
| >  
| > | Yet, it'll be interesting to keep an eye on the SATA RAID performance 
| > | and costs. With such uptimes SATA will surely become (if it hasn't 
| > | already become so, that is) a very good alternative for SCSI.
| > 
| > SATA is built on top of SCSI I think (not sure exactly how it all sorts out,
but
| > I think some of the SATA stuff is tied into some of the SCSI stuff in the
kernel
| > or something).  If I recall correctly, you have to leave some SCSI drivers in
| > the kernel when you setup SATA on FreeBSD.  I may be off, but I think that's
| > what we had to do.  Maybe someone more familiar with the kernel stuff can
| > comment further on that count.
| > 
| > I will e-mail you the install run down in a sec.
| > 
| > Ray
| > 
| > 
| > 
| Actually, that's quite wrong. S-ATA is ATA, not SCSI-like... S-ATA in fact is 
| almost the same as ATA (EIDE) from a code/kernel standpoint. The physical 
| hardware interface differs slightly and offers greater stability and performance 
| than standard ATA (EIDE). However, you are correct if not confusing when you 
| state that you must 'leave some SCSI drivers in the kernel', this is not for ATA 
| so much as it is for the use of the RAID card. If you're using plain old S-ATA, 
| no SCSI disk devices are required, but with the 3Ware Escalades, you will need:
| 
| device scbus
| device da
| device twa
| 
| 
| Anyhow, long story short - the only 'real' difference from S-ATA to P-ATA 
| (Parallel ATA aka 'ATA' or 'EIDE'), is the interface - the drive internals and 
| functionality are the same - but both differ from SCSI.

Hi Nathan,

  Thanks for the run down.  I was refering mainly to the drivers (specifically
the ones you mention above, scbus and da, etc), not so much the hardware per se.

Ray