FC questions, specifically isp(4), sa(4) and mt(1)

[Kenneth D. Merry]

On Thu, Nov 17, 2016 at 19:44:32 +0100, Harry Schmalzbauer wrote:
>  Dear scsi gurus,
> 
> I'm asking for help regarding FibreChannel support in FreeBSD. I thought
> questions@ wasn't really adequate, please tell me if scsi@ isn't either.
> 
> First I'd like to ask if there's alreday any translation (documentation
> newer than isp(4)) between dev.isp.N.topo|loopstate|etc. and
> hint.isp.N.topology etc.
> The only matching component I found in man 4 isp was "role". Especially
> hint.isp.0.fullduplex seems weird to me...
> I will have a look into the driver to see if I can answer some questions
> myself, but any links to more info are welcome.

Well, the role you want is initiator, which is 2, if you're setting
hint.isp.N.role.  It should be configured as an initiator by default.

You should also make sure you're loading the ispfw module (ispfw_load=true
in /boot/loader.conf).  That insures that you're using the firmware that
the driver knows how to talk to, instead of whatever happens to be flashed
on the card.

As for topology, you can set that to one of four values:

lport, nport, lport-only, nport-only

Usually the default negotiation should be fine, but you might try
lport-only to force loop mode if you're having problems.  nport-only will
force fabric/point to point mode, and you're not connected to a switch from
what it sounds like.  That may work fine, but usually the drives will
negotiate loop mode when they're directly connected.

You might also try setting fctape=1 and see if that makes any difference.
I have changes to the isp(4) driver to essentially force turning on FC-Tape
by default if a drive claims support (instead of just paying attention to
what is in the NVRAM), but I haven't put them into FreeBSD/head yet.  So
the driver may or may not enable FC-Tape.

> My specific problem is that mt(1) seem to crash my LTO3 drive.
> I'm quiet new to FC, but I think using a QLE2532 in N_port and initiator
> role togehter with the FC drive shouldn't need any more
> understanding/tuning than doing the same with LSI2008 and an SAS drive,
> besides the different wiring ;-)

You're right, it should just work.  Obviously something is wrong.  I
haven't tested with any HP Fibre Channel drives, but they should work in
theory.

> At first, it looks like I was right with that assumption, since
> camcontrol(8) behaves as usual:
> 
>     'camcontrol devl'
> <HP Ultrium 3-SCSI L63S>           at scbus8 target 0 lun 0 (sa0,pass3)
>     'camcontrol inq sa0'
> pass3: <HP Ultrium 3-SCSI L63S> Removable Sequential Access SCSI-3 device
> pass3: Serial Number HUE07491PW
> pass3: 200.000MB/s transfers, Command Queueing Enabled
>     'camcontrol load sa0 && camcontrol tur sa0'
> Unit started successfully, Media loaded
> Unit is ready
> ...
> 
> But as soon as I issue a 'mt status', the drive seems to crash and I get
> the following kernel messages:
> isp0: Chan 0 LIP destroyed 1 active commands
> isp0: Chan 0 LIP destroyed 1 active commands
> (sa0:isp0:0:0:0): unable to rewind after test read
> mt: /dev/nsa0: Input/output error
> or if port is forced to N_ instead L_:
> sa0: <HP Ultrium 3-SCSI L63S> s/n HUE07491PW detached
> (sa0:isp1:0:0:0): unable to rewind after test read
> (sa0:isp1:0:0:0): Periph destroyed
> mt: /dev/nsa0: Input/output error
> 
> Like mentioned, the drive seems to crash/reboot according to the LEDs.
> 
> Am I wrong that mt(1) shouldn't make a difference if it's a P-SCSI, SAS
> or FC drive?
> 
> Is there a document where some cmd/smpcmd are collected which I could
> use to start narrowing down the source of the error?

Well, in addition to the above (make sure you're loading the firmware, try
forcing l-port or n-port, try turning on FC-tape), you can turn on some
Qlogic debugging in /boot/loader.conf:

hint.isp.0.debug=0x10f
hint.isp.1.debug=0x10f

The other thing to do is make sure your kernel is compiled with CAMDEBUG,
and try doing the following before you run 'mt status':

camcontrol debug -Ic 8:0:0

You can run 'camcontrol debug off' to turn off debugging afterwards.  The
goal here is to get a dump of the SCSI CDBs that get sent to the drive, and
see what in particular is causing the problem.

It may be that one of the commands that the sa(4) driver is sending is
causing the tape drive to crash.  If we can figure out what it is, we may
be able to come up with a work-around or see if there is a bug.

Just in case, you may want to see if you can locate any firmware updates
for that particular drive.  If there is a bug, they may have fixed it in a
later firmware revision.

Hopefully one of those things will either fix it or help us figure out what
the problem is.

Ken
-- 
Kenneth Merry
ken at FreeBSD.ORG