Pack of CAM improvements

[Jeremy Chadwick]

On Fri, Jan 22, 2010 at 11:23:55AM +0100, Harald Schmalzbauer wrote:
> But I had some questions about zfs raidz states. I think that isn't
> a matter of atacam but if I removed one disk, zpool status still
> showed me the ada3 device "online".
> After reinserting (and proper detection/initialisazion with cam,
> ada3 was present again) and zpool clean, it set the devicea as
> UNAVAIL sinve I/O errors.
> I coudn't get the device into the pool again, no matter what I tried.
> Only rebooting the machine helped. Then I could clean and scrub.
> 
> What are the needed steps to provide a reinsterted hard disk to
> geom? With the latest patches I don't need to issue any reset/rescan
> comman, right?
> So it's a zfs problem, right? My mistake in understanding?

I can't speak with regards to the new ATA-via-CAM stuff, but with
the classic AHCI (meaning ataahci(4)), the procedure I've used
reliably for quite some time on Intel ICHx controllers is this:

For SATA disks that are purely UFS/UFS2:

- Single-user mode might be required here; it varies
- Terminate any processes which rely on filesystems on that disk
- umount /filesystem
- atacontrol detach ataX (where X = channel associated with disk)
- Physically remove bad disk
- Physically insert new disk
- Wait 15 seconds for stuff to settle
- atacontrol attach ataX (where X = previous channel detached)
- sade / sysinstall / gpart / whatever you like
- Restore data...  :-)

For SATA disks part of a ZFS mirror or raidz[123] pool:

- zpool offline <pool> <disk>
- atacontrol detach ataX (where X = channel associated with disk)
- Physically remove bad disk
- Physically insert new disk
- Wait 15 seconds for stuff to settle
- atacontrol attach ataX (where X = previous channel detached)
- zpool replace <pool> <disk>
- zpool online <pool> <disk>

-- 
| Jeremy Chadwick                                   jdc at parodius.com |
| Parodius Networking                       http://www.parodius.com/ |
| UNIX Systems Administrator                  Mountain View, CA, USA |
| Making life hard for others since 1977.              PGP: 4BD6C0CB |