am2 MBs - 4g + SCSI wipes out root partition

Scott Long scottl at
Sun Oct 12 09:10:44 PDT 2008

Jeremy Chadwick wrote:
> On Sat, Oct 11, 2008 at 12:26:29PM -0400, Adam McDougall wrote:
>> Jeremy Chadwick wrote:
>>> On Sat, Oct 11, 2008 at 04:45:29PM +0200, Gary Jennejohn wrote:
>>>> On Sat, 11 Oct 2008 03:13:16 -0700
>>>> Jeremy Chadwick <koitsu at> wrote:
>>>>> On Sat, Oct 11, 2008 at 11:30:57AM +0200, Gary Jennejohn wrote:
>>>>>> On Fri, 10 Oct 2008 14:29:37 -0300
>>>>>> JoaoBR <joao at> wrote:
>>>>>>> I tried MBs as Asus, Abit and Gigabyte all same result
>>>>>>> Same hardware with SATA works perfect
>>>>>>> Same hardware with scsi up to 3.5Gigs installed works perfect
>>>>>>> what calls my attention that all this MBs do not have the 
>>>>>>> memroy hole remapping feature so the complete 4gigs are 
>>>>>>> available what normally was not the case with amd64 Mbs for the 
>>>>>>> Athlon 64 CPUs
>>>>>>> some has an opinion if this is a freebsd issue or MB falure or 
>>>>>>> scsi drv problem?
>>>>>> It's a driver problem.  If you want to use SCSI then you'll have to limit
>>>>>> memory to 3.5 GB.
>>>>> What you're saying is that Adaptec and LSI Logic SCSI controllers behave
>>>>> badly (and can cause data loss) on amd64 systems which contain more than
>>>>> 3.5GB of RAM.  This is a very big claim.
>>>>> Have you talked to Scott Long about this?
>>>>> Please expand on this, and provide evidence or references.  I need to
>>>>> document this in my Wiki if it is indeed true.
>>>> See the freebsd-scsi thread with Subject "data corruption with ahc driver
>>>> and 4GB of memory using a FBSD-8 64-bit installation?" from Wed, 30 Jan
>>>> 2008.
>>>> This was for ahc, but the bit-rot which Scott mentions in his reply might
>>>> also apply to the LSI Logic controllers.
>>>> Basically the driver doesn't correctly handle DMA above 4GB.  Since the PCI
>>>> hole gets mapped above 4GB it causes problems.  the (S)ATA drivers don't seem
>>>> to have this problem.
>>> Thank you -- this is the exact information I was looking for.
>>> I will update my Wiki page to reflect this quite major problem.
>> I am using some LSI (mpt driver) ultra4 (U320 scsi) and LSI SAS  
>> controllers in FreeBSD 7.x amd64 with 20G of ram, and Adaptec (aac  
>> driver) with a 5th generation RAID card with 8G of ram, both have no  
>> such corruption problems.  Providing this as a counter-example just to  
>> document some evidence of which products seem to work fine.
> Is your LSI SAS controller driven by mpt(4) or mfi(4)?

I can personal vouch for MPT and MFI drivers working just fine with >4GB.

> Let's break down what we know for sure at this point:
> aac(4) - not affected

Works fine

> aha(4) - unknown
> ahb(4) - unknown

These two will likely be using bounce buffers and should work, albeit

> ahc(4) - affected
> ahd(4) - unknown; no one answered the OP's question in the thread

Both ahc and ahd were designed _AND_TESTED_ to work with >4GB.  If they 
don't work anymore, it's due to unintended bitrot.

> asr(4) - unknown

Danger!  Achtung!  Beware of Dog!

> ips(4) - unknown

I'm pretty sure this works just fine.

> mpt(4) - not affected
> mfi(4) - unknown

Both work just fine

> sym(4) - unknown

This has had problems in the past, but I think that it might have been 
fixed recently

You forgot to mention isp(4), which also works just fine.

> Could the problem be specific to certain firmware revisions on the
> cards?

ahc/ahd use custom "firmware" that is part of the driver.  Their BIOS
can be flashed, but that does little to affect OS operation of the card.
So, "firmware revisions" has nothing to do with whatever this problem

Please do keep in mind that 32bit vs 64bit support, and by correlary, 
 >4GB support, is something that is completely isolated on a per-driver 
basis.  Trying to draw patterns between drivers to say, "FreeBSD SCSI
support is broken," is not valid.  In fact, traditionally, SCSI drivers
in general have had the best support because they are so much more 
common in the high-end systems that need the support.

Out of your whole list, the only card to explicitly stay away from is
the asr(4) family, but that's been known for years.  If ahc and/or ahd
has problems, we need someone willing to dig into code and trace through
the DMA path.


More information about the freebsd-stable mailing list