Problem with stripe across mirrors

[Pawel Jakub Dawidek]

On Mon, Jun 26, 2006 at 10:55:16PM -0500, Jason Hitt wrote:
> I'm running into a big problem with a new 4 disk array im attempting to configure.
> 
> I'm using an SIIG 3114 PCI SATA controller with four WD3200SD 320 gig disks attached to it.  This is only going to be bulk storage, not a system disk.  The disks appear in 
> the system as ad{4,6,8,10}.  My test is to configure my geom provider and then dump/restore a 100 gig mount from a separate provider.
> 
> Setup:
> 
> apply "dd if=/dev/zero of=/dev/%1 bs=512 count=79" ad4 ad6 ad8 ad10
> apply "fdisk -v -B -I /dev/%1" ad4 ad6 ad8 ad10
> I then go back in with "fdisk -v -f- -i /dev/ad?" and reduce the size of the slices by 1 so the consumers will be ad?s1, not ad?.
> 
> From this point i can successully use gmirror to mirror 2 disks and run my test, or i can use gstripe to stripe 2 disks and run my test (presumably a stripe across all 4 
> would work as well, but this is untested).  The failure comes when i do this:
> gmirror label storage0 ad4s1 ad6s1
> gmirror label storage1 ad8s1 ad10s1
> gstripe label storage mirror/storage0 mirror/storage1
> newfs /dev/stripe/storage
> 
> At this point i mount /dev/stripe/storage and attempt to copy data to it using dump/restore (a large cp suffices as well).  After a random amount of time (typically < 60 
> seconds), the system will completely hang.  There is no panic, there is no crash.  The system is completely hung.  With no crash dump or panic, it's quite difficult to 
> pinpoint the problem.  I also cannot find any solid resources to use as a "how-to" for a geom raid10 setup, so i'm not sure if im approaching it correctlly or not.
> 
> Anyone familiar with this that can lend a hand?

I was trying to reproduce it, but no luck so far.
I've a very simlar environment:

# gmirror status
           Name    Status  Components
mirror/storage0  COMPLETE  ad5s1
                           ad6s1
mirror/storage1  COMPLETE  ad8s1
                           ad9s1

# gstripe status
          Name  Status  Components
stripe/storage      UP  mirror/storage0
                        mirror/storage1

# egrep '^ad[5689]' /var/run/dmesg.boot
ad5: 305245MB <Seagate ST3320620AS 3.AAC> at ata5-master SATA150
ad6: 305245MB <Seagate ST3320620AS 3.AAC> at ata6-master SATA150
ad8: 305245MB <Seagate ST3320620AS 3.AAC> at ata8-master SATA150
ad9: 305245MB <Seagate ST3320620AS 3.AAC> at ata9-master SATA150

# egrep '^ata[5689]' /var/run/dmesg.boot
ata5: <ATA channel 1> on atapci1
ata6: <ATA channel 2> on atapci1
ata8: <ATA channel 0> on atapci3
ata9: <ATA channel 1> on atapci3

# egrep '^atapci[13]' /var/run/dmesg.boot
atapci1: <SiI 3114 SATA150 controller> port 0xa400-0xa407,0xa000-0xa003,0x9c00-0x9c07,0x9800-0x9803,0x9400-0x940f mem 0xff8dec00-0xff8defff irq 10 at device 4.0 on pci2
atapci3: <Intel ICH5 SATA150 controller> port 0xe800-0xe807,0xe400-0xe403,0xe000-0xe007,0xdc00-0xdc03,0xd800-0xd80f irq 10 at device 31.2 on pci0

As you can see, not all the disks used are connected to Silicon Image
controler. From what I know those are the worst controllers ever (3112
beeing the really worst one). Will you be able to try to reproduce it
with a different controller?

-- 
Pawel Jakub Dawidek                       http://www.wheel.pl
pjd at FreeBSD.org                           http://www.FreeBSD.org
FreeBSD committer                         Am I Evil? Yes, I Am!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-geom/attachments/20060627/fdafe9b0/attachment.pgp