Update on data corruption with Tyan/3Ware

[Erik Trulsson]

On Mon, Sep 24, 2007 at 10:58:20PM -0700, Ted Mittelstaedt wrote:
> 
> 
> > -----Original Message-----
> > From: owner-freebsd-questions at freebsd.org
> > [mailto:owner-freebsd-questions at freebsd.org]On Behalf Of Bart
> > Silverstrim
> > Sent: Monday, September 24, 2007 7:05 PM
> > To: Ted Mittelstaedt
> > Cc: Chris Boyd; freebsd-questions at freebsd.org
> > Subject: Re: Update on data corruption with Tyan/3Ware
> >
> >
> >
> >
> > Ted Mittelstaedt wrote:
> > > 3ware is supported by the manufacturer - what do they say?
> > >
> > > Ted
> > >
> > >> -----Original Message-----
> > >> From: owner-freebsd-questions at freebsd.org
> > >> [mailto:owner-freebsd-questions at freebsd.org]On Behalf Of Chris Boyd
> > >> Sent: Monday, September 24, 2007 2:54 PM
> > >> To: freebsd-questions at freebsd.org
> > >> Subject: Update on data corruption with Tyan/3Ware
> > >>
> > >>
> > >> Here's an update on my odd problem.  Thanks to Don B for some hints
> > >> that helped us start looking in a better directions.
> > >>
> > >> System is a Tyan Thunder K8SE motherboard with dual Opteron 250
> > >> 2.4GHz CPUs and a 3Ware 9550SX-4LP PCI Express four port RAID
> > >> controller running in RAID 5.  Disks are 4x Seagate 500GB SATA.  4GB
> > >> Memory.
> > >>
> > >> Latest BIOS and firmware on mobo and RAID controller.
> > >>
> > >> FreeBSD 6.2 AMD64 with all patches as of 9-21-2007
> > >>
> > >> We've narrowed the problem down to files that are > 4GB.  Anytime we
> > >> have a file that's > 4GB, we get inconsistent checksums, can't
> > >> uncompress it, etc.  Files < 4GB are fine.
> > >>
> > >> So is this a RAID controller issue?  A filesystem problem?  All hints
> > >> appreciated.
> >
> > How are you getting the files on the system?  Network transfer? Direct
> > copy from a disc?
> >
> > What filesystem is it you're using?
> >
> > 3ware is well supported under Linux, from what I can tell and what I've
> > experienced, and can't imagine that a manufacturer with a good track
> > record of driver support for Linux for so long would not support FreeBSD
> > as well.
> 
> Bart and Chris,
> 
>   The problem might be that the 3ware driver uses a 32 bit "int" to
> represent
> a file size.

The 3ware driver handles blocks on the device. It knows nothing
about files or their sizes.  That is the job of the filesystem which resides
at a higher level in the OS.

It is true that the 3ware twe(4) driver uses 32-bit ints to represent the
*disk* size in number of disk-blocks.  With the standard 512-byte block size
this gives a maximum disk size of (512*4G==) 2TB, which is a known
limitation of the twe(4) driver as well as many other disk controllers.
Anyways the twa(4) driver is supposed to handle volumes larger than 2TB.

(twe(4) handles 3ware 7000 and 8000-series controllers, while the twa(4)
driver is for the 9000-series.)

>  In FreeBSD, stat() ftruncate() lseek() and
> friends which are based on "strut stat" had this limitation under FreeBSD
> 4.xx.
> 
>   Note line# 821 of twe_freebsd.c the driver:
> 
>     sc->twed_disk->d_maxsize = (TWE_MAX_SGL_LENGTH - 1) * PAGE_SIZE;
>     sc->twed_disk->d_sectorsize = TWE_BLOCK_SIZE;
>     sc->twed_disk->d_mediasize = TWE_BLOCK_SIZE *
> (off_t)sc->twed_drive->td_size;
>     sc->twed_disk->d_fwsectors = sc->twed_drive->td_sectors;
> 
> that off_t also appears elsewhere.

Since off_t is a 64-bit type, and since none of that code fragment has
anything to do with *file* sizes, I am not sure what your point is.

> 
>   I'm not a driver programmer but I'd bet the driver hasn't been updated
> for 64 bit FreeBSD.




-- 
<Insert your favourite quote here.>
Erik Trulsson
ertr1013 at student.uu.se