ISCSI on zvols with two maschines. What needs to be done?

Gary Palmer gpalmer at freebsd.org
Sat Jul 21 02:35:46 UTC 2012


On Fri, Jul 20, 2012 at 03:02:08PM +0200, dennis berger wrote:
> Hi John,
> see inline.
> 
> 
> Am 20.07.2012 um 14:32 schrieb John:
> 
> > Hi Dennis,
> > 
> > ----- dennis berger's Original Message -----
> >> Hi folks,
> >> 
> >> The following question often comes to me, when I speak to my customers.
> >> 
> >> Is it possible to attach two systems to the same storage array using shared SAS and use istgt + zfs to provide 
> >> a reliable ISCSI target. This could be active/passive or later active/active. In case it's not possible, what needs to be done?
> >> 
> >> The questions that came into my mind.
> >> 
> >> Can I connect two freebsd to the storage and boot them up? Just that both systems would see the physical devices. 
> > 
> > Yes, please see:
> > 
> > http://people.freebsd.org/~jwd/zfsnfsserver.jpg
> 
> This is exactly how I would like to use it.
> 
> > 
> > for a possible diagram. Dual path all the shelves and use gmultipath to
> > access them. Use 9-stable.
> OK. And with new multipath code. I remember I read something in quarterly reports?
> 
> > 
> > Make sure each host is dual homed and a client (initiator) can access
> > the target with dual pathing, one connection over each network path.
> > 
> OK but it's still accessing one target right.
> 
> >> Can I start ZFS on both systems without writing on it? I would like to use just block devices for ISCSI. Is everything still consistent?
> > 
> > You can start ZFS on both systems and import different pools. The same pool
> > cannot be imported on two different systems (you can, you won't be happy).
> > 
> Ahh you mean to create a separate zfs pool with the different devices names according to multipath. Which in in essence are the same physical devices?!
> 
> PoolA with da0 and da2 
> PoolB with da1 and da3
>  
> da0 and da1 are the same device but different paths?
> But where is the difference?! Maybe I don't understand exactly why 2 pools


You cannot have both systems accessing the same device, whether in one pool
or in two pools, at the same time.  You will end up with trouble.

The two pools in John's reply indicated that each system would have its
own pool of disks.


> >> Does starting ZFS up already writing some sort of meta data to the physical devices?!
> > 
> > Yes, others would know better than I.
> 
> OK
> 
> > 
> >> Can I start istgt on both systems without actually using the istgt on one system?! Then CARP would be enough, wouldn't it.
> > 
> > You start istgt on both systems handing out luns from the two different
> > pools you have mounted on each head. Hint, put your istgt configs for
> > each pool in each pool.
> > 
> > Using CARP/DEVD and some shell code, if head A dies, you import the
> > failing heads pool on the other system and bring istgt up. CARP handles
> > the IP address switch over.
> 
> Do I need to zfs import?! 

This is the key bit.  What John suggests is that on system A you have
poolA and on system B you have pool B.  Both pools have their own,
separate, disks.  

If system A goes down, system B notices this and imports poolA and shares
out its iSCSI devices.  

You have to be careful when system A comes back up that it doesn't
auto-import the pool and negotiates with system B to get it back

It may be easier to think of a situation where there is only 1 pool.
In this situation, the two servers will be in active/passive mode,
and the passive system will only ever take load if the active node
goes down.  

The important thing to remember is NO filesystem which is included with
FreeBSD by default is cluster aware.  This means that two servers
CANNOT both mount filesystems on direct-attached storage without
corrupting the data on the storage.  The best you can get active/passive
right now, where only 1 server mounts the filesystem at a time and
the second server takes over in event of failure of the primary server.

In this instance consider ZFS, even at the zpool layer, to be a
filesystem.  Mounting the same zpool from shared storage on two servers
at the same time will almost certainly lead to corruption and probably a
restore from backup tapes.

Gary


More information about the freebsd-fs mailing list