ZFS attaching direct access drives as Scsibus#target#drive#

Tue Oct 14 21:08:27 UTC 2008

Hello,

I'm in the process of building several zfs pools for a backup-server,
the machine has 2 24 port sata 3 controllers and 48 1TB drives for
storage.  I've done a little testing and found that building  a pool by
using the da# device names can be inconsistent.  For example if a device
is lost from the first controller and the box is re-booted all the da#s
shuffle up,  which creates problems with adjacent pools, and the
potential for catastrophic failure. In the Sun documentation zpool
allows a pool to be built by using S#T#D# (Scsibus Target Drive) syntax,
when I attempt it in FBSD It returns:" cannot use '/dev/c0t0d0': must be
a GEOM provider" does anyone know a way to statically map a
scsibus/target/drive to a specific da# or a fix to allow specific drives
by target numbers to be added to a zpool?

Daniel

Relevant portions of dmesg:

twa0: <3ware 9000 series Storage Controller> port 0x2000-0x20ff mem
0xd8000000-0xd9ffffff,0xdc100000-0xdc100fff irq 48 at device 0.0 on pci1

twa1: <3ware 9000 series Storage Controller> port 0x3000-0x30ff mem
0xda000000-0xdbffffff,0xdc400000-0xdc400fff irq 54 at device 0.0 on pci6

da0 at twa0 bus 0 target 0 lun 0

da0: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da0: 100.000MB/s transfers

da0: 953664MB (1953103872 512 byte sectors: 255H 63S/T 121575C)

da1 at twa0 bus 0 target 1 lun 0

da1: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da1: 100.000MB/s transfers

da1: 953664MB (1953103872 512 byte sectors: 255H 63S/T 121575C)

da2 at twa0 bus 0 target 2 lun 0

da2: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da2: 100.000MB/s transfers

da2: 953664MB (1953103872 512 byte sectors: 255H 63S/T 121575C)

da3 at twa0 bus 0 target 3 lun 0

da3: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da3: 100.000MB/s transfers

da3: 953664MB (1953103872 512 byte sectors: 255H 63S/T 121575C)

da4 at twa0 bus 0 target 4 lun 0

da4: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da4: 100.000MB/s transfers

da4: 953664MB (1953103872 512 byte sectors: 255H 63S/T 121575C)

da5 at twa0 bus 0 target 5 lun 0

da5: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da5: 100.000MB/s transfers

da5: 953664MB (1953103872 512 byte sectors: 255H 63S/T 121575C)

da6 at twa0 bus 0 target 6 lun 0

da6: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da6: 100.000MB/s transfers

da6: 953664MB (1953103872 512 byte sectors: 255H 63S/T 121575C)

da7 at twa0 bus 0 target 7 lun 0

da7: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da7: 100.000MB/s transfers

da7: 953664MB (1953103872 512 byte sectors: 255H 63S/T 121575C)

da8 at twa0 bus 0 target 8 lun 0

da8: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da8: 100.000MB/s transfers

da8: 953664MB (1953103872 512 byte sectors: 255H 63S/T 121575C)

da9 at twa0 bus 0 target 9 lun 0

da9: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da9: 100.000MB/s transfers

da9: 953664MB (1953103872 512 byte sectors: 255H 63S/T 121575C)

da10 at twa0 bus 0 target 10 lun 0

da10: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da10: 100.000MB/s transfers

da10: 953664MB (1953103872 512 byte sectors: 255H 63S/T 121575C)

da11 at twa0 bus 0 target 11 lun 0

da11: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da11: 100.000MB/s transfers

da11: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)

da12 at twa0 bus 0 target 12 lun 0

da12: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da12: 100.000MB/s transfers

da12: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)

da13 at twa0 bus 0 target 13 lun 0

da13: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da13: 100.000MB/s transfers

da13: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)

da14 at twa0 bus 0 target 14 lun 0

da14: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da14: 100.000MB/s transfers

da14: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)

da15 at twa0 bus 0 target 15 lun 0

da15: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da15: 100.000MB/s transfers

da15: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)

da16 at twa0 bus 0 target 16 lun 0

da16: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da16: 100.000MB/s transfers

da16: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)

da17 at twa0 bus 0 target 17 lun 0

da17: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da17: 100.000MB/s transfers

da17: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)

da18 at twa0 bus 0 target 18 lun 0

da18: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da18: 100.000MB/s transfers

da18: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)

da19 at twa0 bus 0 target 19 lun 0

da19: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da19: 100.000MB/s transfers

da19: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)

da20 at twa0 bus 0 target 20 lun 0

da20: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da20: 100.000MB/s transfers

da20: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)

da21 at twa0 bus 0 target 21 lun 0

da21: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da21: 100.000MB/s transfers

da21: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)

da22 at twa0 bus 0 target 22 lun 0

da22: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da22: 100.000MB/s transfers

da22: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)

da23 at twa0 bus 0 target 23 lun 0

da23: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da23: 100.000MB/s transfers

da23: 953664MB (1953103872 512 byte sectors: 255H 63S/T 121575C)

da24 at twa1 bus 0 target 0 lun 0

da24: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da24: 100.000MB/s transfers

da24: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)

da25 at twa1 bus 0 target 1 lun 0

da25: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da25: 100.000MB/s transfers

da25: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)

da26 at twa1 bus 0 target 2 lun 0

da26: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da26: 100.000MB/s transfers

da26: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)

da27 at twa1 bus 0 target 3 lun 0

da27: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da27: 100.000MB/s transfers

da27: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)

da28 at twa1 bus 0 target 4 lun 0

da28: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da28: 100.000MB/s transfers

da28: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)

da29 at twa1 bus 0 target 5 lun 0

da29: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da29: 100.000MB/s transfers

da29: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)

da30 at twa1 bus 0 target 6 lun 0

da30: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da30: 100.000MB/s transfers

da30: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)

da31 at twa1 bus 0 target 7 lun 0

da31: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da31: 100.000MB/s transfers

da31: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)

da32 at twa1 bus 0 target 8 lun 0

da32: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da32: 100.000MB/s transfers

da32: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)

da33 at twa1 bus 0 target 9 lun 0

da33: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da33: 100.000MB/s transfers

da33: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)

da34 at twa1 bus 0 target 10 lun 0

da34: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da34: 100.000MB/s transfers

da34: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)

da35 at twa1 bus 0 target 11 lun 0

da35: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da35: 100.000MB/s transfers

da35: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)

da36 at twa1 bus 0 target 12 lun 0

da36: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da36: 100.000MB/s transfers

da36: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)

da37 at twa1 bus 0 target 13 lun 0

da37: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da37: 100.000MB/s transfers

da37: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)

da38 at twa1 bus 0 target 14 lun 0

da38: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da38: 100.000MB/s transfers

da38: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)

da39 at twa1 bus 0 target 15 lun 0

da39: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da39: 100.000MB/s transfers

da39: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)

da40 at twa1 bus 0 target 16 lun 0

da40: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da40: 100.000MB/s transfers

da40: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)

da41 at twa1 bus 0 target 17 lun 0

da41: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da41: 100.000MB/s transfers

da41: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)

da42 at twa1 bus 0 target 18 lun 0

da42: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da42: 100.000MB/s transfers

da42: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)

da43 at twa1 bus 0 target 19 lun 0

da43: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da43: 100.000MB/s transfers

da43: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)

da44 at twa1 bus 0 target 20 lun 0

da44: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da44: 100.000MB/s transfers

da44: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)

da45 at twa1 bus 0 target 21 lun 0

da45: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da45: 100.000MB/s transfers

da45: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)

da46 at twa1 bus 0 target 22 lun 0

da46: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da46: 100.000MB/s transfers

da46: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)

da47 at twa1 bus 0 target 23 lun 0

da47: <AMCC 9650SE-24M DISK 4.06> Fixed Direct Access SCSI-5 device

da47: 100.000MB/s transfers

da47: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)

SMP: AP CPU #5 Launched!

SMP: AP CPU #6 Launched!

SMP: AP CPU #1 Launched!

SMP: AP CPU #7 Launched!

SMP: AP CPU #2 Launched!

SMP: AP CPU #4 Launched!

SMP: AP CPU #3 Launched!