misc/113419: geom fox multipathing not failing back

[Dave Costakos]

>Number:         113419
>Category:       misc
>Synopsis:       geom fox multipathing not failing back
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Wed Jun 06 15:10:06 GMT 2007
>Closed-Date:
>Last-Modified:
>Originator:     Dave Costakos
>Release:        6.2-PRERELEASE
>Organization:
QUALCOMM Incorporated
>Environment:
dcostako at galena ~> uname -a
FreeBSD galena.qualcomm.com 6.2-PRERELEASE FreeBSD 6.2-PRERELEASE #1: Mon Oct  9 17:09:41 PDT 2006     root at galena.qualcomm.com:/usr/obj/usr/src/sys/GALENA  amd64

>Description:
We are having an issue with geom fox when used for multipath.  We have dual QLogic FC HBAs connected to this AMD64 machine and are booting from FC SAN.  Whenever there is SAN maintenance, we lose a single FC path, then it returns and we lose the other FC path and it returns.

However, in our geom fox configuration, when we lose a path, it never returns.  So whenever there is maintenance, we have to either shutdown the machine, or lose the machine completely.

Not sure if this is a configuration error on my part or a bug, but I could use some help.

Here are some configuration details:


dcostako at galena ~> uname -a
FreeBSD galena.qualcomm.com 6.2-PRERELEASE FreeBSD 6.2-PRERELEASE #1: Mon Oct  9 17:09:41 PDT 2006     root at galena.qualcomm.com:/usr/obj/usr/src/sys/GALENA  amd64
dcostako at galena ~> sudo geom fox list
Password:
Geom name: da0.fox
Providers:
1. Name: da0.fox
   Mediasize: 117625060864 (110G)
   Sectorsize: 512
   Mode: r1w1e2
Consumers:
1. Name: da0
   Mediasize: 117625061376 (110G)
   Sectorsize: 512
   Mode: r1w1e3
2. Name: da7
   Mediasize: 117625061376 (110G)
   Sectorsize: 512
   Mode: r0w0e1

Geom name: da1.fox
Providers:
1. Name: da1.fox
   Mediasize: 117625060864 (110G)
   Sectorsize: 512
   Mode: r1w1e2
Consumers:
1. Name: da1
   Mediasize: 117625061376 (110G)
   Sectorsize: 512
   Mode: r1w1e3
2. Name: da8
   Mediasize: 117625061376 (110G)
   Sectorsize: 512
   Mode: r0w0e1

Geom name: da2.fox
Providers:
1. Name: da2.fox
   Mediasize: 117625060864 (110G)
   Sectorsize: 512
   Mode: r1w1e2
Consumers:
1. Name: da2
   Mediasize: 117625061376 (110G)
   Sectorsize: 512
   Mode: r1w1e3
2. Name: da9
   Mediasize: 117625061376 (110G)
   Sectorsize: 512
   Mode: r0w0e1

Geom name: da3.fox
Providers:
1. Name: da3.fox
   Mediasize: 117625060864 (110G)
   Sectorsize: 512
   Mode: r1w1e2
Consumers:
1. Name: da3
   Mediasize: 117625061376 (110G)
   Sectorsize: 512
   Mode: r1w1e3
2. Name: da10
   Mediasize: 117625061376 (110G)
   Sectorsize: 512
   Mode: r0w0e1

Geom name: da4.fox
Providers:
1. Name: da4.fox
   Mediasize: 117625060864 (110G)
   Sectorsize: 512
   Mode: r1w1e2
Consumers:
1. Name: da4
   Mediasize: 117625061376 (110G)
   Sectorsize: 512
   Mode: r1w1e3
2. Name: da11
   Mediasize: 117625061376 (110G)
   Sectorsize: 512
   Mode: r0w0e1

Geom name: da5.fox
Providers:
1. Name: da5.fox
   Mediasize: 117625060864 (110G)
   Sectorsize: 512
   Mode: r1w1e2
Consumers:
1. Name: da5
   Mediasize: 117625061376 (110G)
   Sectorsize: 512
   Mode: r1w1e3
2. Name: da12
   Mediasize: 117625061376 (110G)
   Sectorsize: 512
   Mode: r0w0e1

Geom name: da6.fox
Providers:
1. Name: da6.fox
   Mediasize: 117625060864 (110G)
   Sectorsize: 512
   Mode: r1w1e2
Consumers:
1. Name: da6
   Mediasize: 117625061376 (110G)
   Sectorsize: 512
   Mode: r1w1e3
2. Name: da13
   Mediasize: 117625061376 (110G)
   Sectorsize: 512
   Mode: r0w0e1

dcostako at galena ~> sudo geom fox status
   Name  Status  Components
da0.fox     N/A  da0
                 da7
da1.fox     N/A  da1
                 da8
da2.fox     N/A  da2
                 da9
da3.fox     N/A  da3
                 da10
da4.fox     N/A  da4
                 da11
da5.fox     N/A  da5
                 da12
da6.fox     N/A  da6
                 da13
dcostako at galena ~> mount
/dev/ad0s1a on / (ufs, local)
devfs on /dev (devfs, local)
/dev/ad0s1e on /usr (ufs, NFS exported, local, soft-updates)
/dev/ad0s1d on /var (ufs, local, soft-updates)
/dev/stripe/opt on /opt (ufs, local)
procfs on /proc (procfs, local)
pid763 at galena:/host on /host (nfs)
pid763 at galena:/usr2 on /usr2 (nfs)
pid763 at galena:/net on /net (nfs)
barrel:/vol/vol2/usr2/dcostako on /.amd_mnt/barrel/vol/vol2/usr2/dcostako (nfs, nosuid)
dcostako at galena ~> df -h
Filesystem                        Size    Used   Avail Capacity  Mounted on
/dev/ad0s1a                       1.9G     73M    1.7G     4%    /
devfs                             1.0K    1.0K      0B   100%    /dev
/dev/ad0s1e                        43G    9.5G     30G    24%    /usr
/dev/ad0s1d                       1.9G    273M    1.5G    15%    /var
/dev/stripe/opt                   743G    352G    331G    52%    /opt
procfs                            4.0K    4.0K      0B   100%    /proc
barrel:/vol/vol2/usr2/dcostako    577G    553G     24G    96%    /.amd_mnt/barrel/vol/vol2/usr2/dcostako

>How-To-Repeat:
Take down SAN paths 1-by-1 and they don't fail back.  Causes downtime whenever there is FC SAN maintenance requiring switch reboots.
>Fix:


>Release-Note:
>Audit-Trail:
>Unformatted: