Current status of support for high end SAN hardware

[Wilko Bulte]

Quoting Andy Kosela, who wrote on Mon, Jun 09, 2008 at 02:41:24PM +0200 ..
> On Mon, Jun 9, 2008 at 12:36 PM, Russell Vincent <rv at openusenet.org> wrote:
> >
> > The FreeBSD support for multipath/SAN is fairly poor. It's fiddly
> > to get to work and boot times are a little variable (into the
> > minutes) as it tries to discover the devices.  Once it is configured
> > and booted, it just works as long as things don't go wrong.  SAN
> > outages cause the machine to hang up until the issue is resolved
> > (in which case it just seems to continue) or it doesn't recover at
> > all and requires a reboot.  Note that I don't spend a significant
> > amount of time on this, so it may be that I could do things a little
> > better.  I have also not tested the failover stuff very well (I
> > only upgraded this machine to 7-STABLE fairly recently).  Disk
> > access seems to be restricted to a single path at a time.  Problem
> > solving is very tricky as there is very little information to trace
> > which path/disk refers to which fabric/storage device/LUN.
> >
> 
> Russell,
> Thank you for your insights. It's good to see you have no problems
> with isp(4) and Qlogic HBAs. Though I'm concerned about
> multipathing. We run 6.x-RELEASE releases so it seems we have
> to upgrade to 7.0-RELEASE to achieve that goal. gmultipath(8)
> code is fairly new so I suppose it's not that mature yet as in
> Linux. Unfortunately it is only an active/passive approach with
> no load balancing (the active path is active until a BIO request is
> failed with EIO or ENXIO)

Well, it is worse than that: HP EVA arrays have 2 different multipath
behaviours.  VCS 3.x firmware is active/passive, whereas VCS 4.x is 
active/active.  Active/active as well are all XCS firmware versions.
All newer EVAs run XCS.

To properly do multipathing on active/active EVAs your multipathing 
software should be aware of the ALUA stuff.  On active/active EVAs
you should fire your read commands to the right HSV controller, if you
take the other HSV things will work, but the controller you
sent the read to will have to do a proxy read to the HSV controller that
"owns" the LUN.  This will be slower that taking the right HSV in one go.
It also consumes bandwith on the HSV controller's mirror port(s).  Writes do
not matter, they are mirrored to the write cache in each HSV anyway.

Linux devicemapper is sort-of getting there now.  For the longest time
HP did not support devicemapper, and for good reasons too.

To make things interesting I think I remember Matt (the isp(4) author)
telling me that some ispfw(4) versions not getting it right.  If the adapter
firmware does not properly inform the isp(4) driver of SAN disruptions you are
obviously doomed as a multipathing driver that depends on the underlying 
HBA drivers to feed SAN problems/status upstream.  In short: if you use 
isp(4), always load ispfw(4) too.

Getting all the components in a FC SAN to DTRT is unfortunately still a major 
pain in the backside.

hth
Wilko

-- 
Wilko Bulte				wilko at FreeBSD.org