Qlogic 2432, Multipath, Dynamic LUNs, and NTFS

[Rainer Duffner]

Greg H. wrote:
> Hello All,
>
> I've just installed a new HP BladeSystem with BL460c blades, connected via 
> dual on-board Qlogic 2432 Fibre Channel controllers, to an EVA4000 SAN.  After 
> getting the isp driver from 6.2-Stable, everything runs very smoothly.
>
> Now, I've been asked to use our very stable FreeBSD blades to backup the often 
> flakey Windows 2003 Server drives.  Both the FreeBSD blades and the Windows 
> blades have their own LUNs on the EVA4000 SAN.
>
> To do this, I've created a snapshot of one of the non-compressed NTFS drives 
> on the EVA, and presented it to the FreeBSD system, dynamically.  After 
> doing a camcontrol rescan all, both a camcontrol devlist, or an ls /dev/ntfs 
> crash the system.  Because of the silly HP blade environment, I don't have 
> access to the console during the crashes.  I also haven't found any signs 
> in the logs.  
>   

I've only played with BL20 G2 (and FreeBSD) briefly.
I also witnessed some crashes in such cases.

Mr. Jacobs is currently writing a complete replacement of the driver 
that is supposed to be able to also properly function in MPIO environments.

> Here is an additional quirk.  When the system comes back up, the snapshot LUN 
> show up, and seems to mount properly with mount -t ntfs.  I can't get ntfs-3g 
> to work properly when loaded as a package, and the port won't compile.
>
> The strange thing is that an ls of the NTFS snapshot shows a file that 
> is 7,653,312,512 bytes, but cat | bzip2 (and dd) only reads 3,358,345,216 bytes.
> Reading a larger file of 50,135,415,808 bytes crashes the system.
>
>   


When you create a LUN, you have to choose which OS you want to place on 
it. EVA4000 has provisions for Windoze, Linux, Solaris and VMware (IIRC).
This may alter the way, the LUNs are presented to the hosts - I'm not 
sure it is a reason for your problems.


> These are the files from the SAN snapshot of the NTFS drive:
> -rwxr-xr-x  1 root  wheel   7653312512 Jun 26 20:31 MV-C.bkf
> -rwxr-xr-x  1 root  wheel  50135415808 Jun 26 21:28 MV-E.bkf
> -rwxr-xr-x  1 root  wheel    575225856 Jun 26 20:12 MV-SysState.bkf
>
> Here's what happens when I try to read them:
> Copying MV-SysState.bkf 
> Wed Jun 27 17:58:00 PDT 2007
> (stdin):  2.792:1,  2.866 bits/byte, 64.18% saved, 575225856 in, 206056048 out
>  
> Copying MV-C.bkf 
> Wed Jun 27 18:01:08 PDT 2007
> (stdin):  4.047:1,  1.977 bits/byte, 75.29% saved, 3358345216 in, 829840937 out
>  
> Copying MV-E.bkf 
> Wed Jun 27 18:19:34 PDT 2007
> (stdin): <insert crash here>
>
> If this is the wrong list, I'll happily send it to the proper list if someone 
> tells me what it is.
>
>   


You could try the HP ITRC forums.


> Other interesting facts are that each LUN presented by the SAN appears 4 times, 
> once on each FC port (as was expected), and GEOM seems to handle it just fine.
> Mounts reference the GEOM label such as /dev/ufs/mail2root or /dev/ntfs/MV-Backup.
>
>   


That's the multi-pathing. See my above comment about a forthcomming driver.
I would recommend disabling multi-pathing for systems that are not 
MPIO-capable.


> Any suggestions of how to dynamically mount/unmount FC LUNs, and faithfully 
> read files from an NTFS file system?
>   


I'm sorry - I can't really relate any experience to your primary problem.


BTW: Despite the name, freebsd-isp is not about the isp(4)-driver ;-)
It's about FreeBSD @ work at Internet Service Providers.


cheers,
Rainer