Upgrading from 8.3 to 9.1 zfs pool can't attach
Jeremy Chadwick
jdc at koitsu.org
Wed May 22 18:30:45 UTC 2013
On Wed, May 22, 2013 at 07:54:34PM +0200, Leslie Jensen wrote:
>
>
> 2013-05-22 17:02, Ronald Klop skrev:
> >On Wed, 22 May 2013 12:53:25 +0200, Leslie Jensen <leslie at eskk.nu> wrote:
> >
> >>
> >>I've upgraded a machine with freebsd-update from 8.3 to 9.1.
> >>
> >>After the first restart I edited /etc/fstab in single user mode
> >>because the names on the disks had changed. But the zfs pool I have
> >>seem to have a problem and I'm sure on how to recover it.
> >>
> >>
> >>May 22 12:00:39 kernel: ZFS WARNING: Unable to attach to ada0.
> >>May 22 12:00:39 kernel: ZFS WARNING: Unable to attach to ada1.
> >>May 22 12:00:39 kernel: ZFS WARNING: Unable to attach to ada4.
> >>May 22 12:00:40 kernel: ZFS WARNING: Unable to attach to ada5.
> >>
> >>
> >>camcontrol devlist
> >><ST3320620AS 3.AAE> at scbus0 target 0 lun 0 (ada0,pass0)
> >><ST3320620AS 3.AAE> at scbus1 target 0 lun 0 (ada1,pass1)
> >><WDC WD360GD-00FNA0 35.06K35> at scbus2 target 0 lun 0 (ada2,pass2)
> >><WDC WD360GD-00FNA0 35.06K35> at scbus2 target 1 lun 0 (ada3,pass3)
> >><ST3320620AS 3.AAE> at scbus3 target 0 lun 0 (ada4,pass4)
> >><ST3320620AS 3.AAE> at scbus3 target 1 lun 0 (ada5,pass5)
> >>
> >>ada2 and ada3 are configured as raid/r0 holding the system.
> >>
> >>ada0, ada1, ada4 and ada5 should be a zfs pool with /home
> >>
> >>
> >>cat /etc/fstab
> >>
> >># Device Mountpoint FStype Options Dump Pass#
> >>/dev/raid/r0s1b none swap sw 0 0
> >>/dev/raid/r0s1a / ufs rw 1 1
> >>/dev/raid/r0s1e /tmp ufs rw 2 2
> >>/dev/raid/r0s1f /usr ufs rw 2 2
> >>/dev/raid/r0s1d /var ufs rw 2 2
> >>
> >>
> >>I'm not sure but shouldn't the mount point for /home also appear in
> >>fstab?
> >>
> >>Maybe I messed up during the manual editing of files during the update
> >>process?
> >>
> >>
> >>Can I recover or shall I destroy the pool and rebuild it?
> >>
> >>Thanks
> >>
> >>/Leslie
> >
> >Mounts from zfs do not have to be in fstab. You can set a mountpoint on
> >a zfs volume which kind of mounts itself.
> >Can you send more information about the 'Unable to attach to ada[0-4]'
> >messages? Maybe a complete dmesg (/var/run/dmesg.boot).
> >And how are these disks configured/partitioned? Output from 'gpart list
> >ada0' or 'gpart show ada0' and for the other disks.
> >Are you using stuff like glabel?
> >
> >Ronald.
>
>
> cat /var/run/dmesg.boot
> -----------------------------------------------
> Copyright (c) 1992-2012 The FreeBSD Project.
> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
> The Regents of the University of California. All rights reserved.
> FreeBSD is a registered trademark of The FreeBSD Foundation.
> FreeBSD 9.1-RELEASE-p3 #0: Mon Apr 29 18:27:25 UTC 2013
> root at amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64
> link_elf_obj: symbol ata_controlcmd undefined
> KLD file atapicam.ko - could not finalize loading
> CPU: Intel(R) Pentium(R) 4 CPU 3.20GHz (3192.08-MHz K8-class CPU)
> Origin = "GenuineIntel" Id = 0xf43 Family = f Model = 4 Stepping = 3
>
> Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
> Features2=0x649d<SSE3,DTES64,MON,DS_CPL,EST,CNXT-ID,CX16,xTPR>
> AMD Features=0x20100800<SYSCALL,NX,LM>
> TSC: P-state invariant
> real memory = 4294967296 (4096 MB)
> avail memory = 4084785152 (3895 MB)
> Event timer "LAPIC" quality 400
> ACPI APIC Table: <IntelR AWRDACPI>
> FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
> FreeBSD/SMP: 1 package(s) x 1 core(s) x 2 HTT threads
> cpu0 (BSP): APIC ID: 0
> cpu1 (AP/HT): APIC ID: 1
> ioapic0: Changing APIC ID to 4
> ioapic0 <Version 2.0> irqs 0-23 on motherboard
> kbd1 at kbdmux0
> acpi0: <IntelR AWRDACPI> on motherboard
> acpi0: Power Button (fixed)
> acpi0: reservation of 0, a0000 (3) failed
> acpi0: reservation of 100000, bfde0000 (3) failed
> cpu0: <ACPI CPU> on acpi0
> cpu1: <ACPI CPU> on acpi0
> attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
> Timecounter "i8254" frequency 1193182 Hz quality 0
> Event timer "i8254" frequency 1193182 Hz quality 100
> atrtc0: <AT realtime clock> port 0x70-0x73 irq 8 on acpi0
> Event timer "RTC" frequency 32768 Hz quality 0
> Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
> acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
> acpi_button0: <Power Button> on acpi0
> pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
> pci0: <ACPI PCI bus> on pcib0
> pcib1: <PCI-PCI bridge> irq 16 at device 1.0 on pci0
> pci1: <PCI bus> on pcib1
> vgapci0: <VGA-compatible display> port 0xbe00-0xbeff mem
> 0xd0000000-0xdfffffff,0xfd9e0000-0xfd9effff irq 16 at device 0.0 on
> pci1
> hdac0: <ATI RV610 HDA Controller> mem 0xfd9fc000-0xfd9fffff irq 17
> at device 0.1 on pci1
> pcib2: <ACPI PCI-PCI bridge> irq 16 at device 28.0 on pci0
> pci2: <ACPI PCI bus> on pcib2
> siis0: <SiI3132 SATA controller> port 0xaf00-0xaf7f mem
> 0xfd6ff000-0xfd6ff07f,0xfd6f8000-0xfd6fbfff irq 16 at device 0.0 on
> pci2
> siisch0: <SIIS channel> at channel 0 on siis0
> siisch1: <SIIS channel> at channel 1 on siis0
> pcib3: <ACPI PCI-PCI bridge> irq 17 at device 28.1 on pci0
> pci3: <ACPI PCI bus> on pcib3
> bge0: <Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev.
> 0x004101> mem 0xfddf0000-0xfddfffff irq 17 at device 0.0 on pci3
> bge0: CHIP ID 0x00004101; ASIC REV 0x04; CHIP REV 0x41; PCI-E
> miibus0: <MII bus> on bge0
> brgphy0: <BCM5750 1000BASE-T media interface> PHY 1 on miibus0
> brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT,
> 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto,
> auto-flow
> bge0: Ethernet address: 00:50:8d:7c:c2:60
> pcib4: <ACPI PCI-PCI bridge> irq 16 at device 28.4 on pci0
> pci4: <ACPI PCI bus> on pcib4
> bge1: <Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev.
> 0x004101> mem 0xfdbf0000-0xfdbfffff irq 16 at device 0.0 on pci4
> bge1: CHIP ID 0x00004101; ASIC REV 0x04; CHIP REV 0x41; PCI-E
> miibus1: <MII bus> on bge1
> brgphy1: <BCM5750 1000BASE-T media interface> PHY 1 on miibus1
> brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT,
> 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto,
> auto-flow
> bge1: Ethernet address: 00:50:8d:7c:c2:61
> uhci0: <Intel 82801G (ICH7) USB controller USB-A> port 0xff00-0xff1f
> irq 23 at device 29.0 on pci0
> usbus0 on uhci0
> uhci1: <Intel 82801G (ICH7) USB controller USB-B> port 0xfe00-0xfe1f
> irq 19 at device 29.1 on pci0
> usbus1 on uhci1
> uhci2: <Intel 82801G (ICH7) USB controller USB-C> port 0xfd00-0xfd1f
> irq 18 at device 29.2 on pci0
> usbus2 on uhci2
> uhci3: <Intel 82801G (ICH7) USB controller USB-D> port 0xfc00-0xfc1f
> irq 16 at device 29.3 on pci0
> usbus3 on uhci3
> ehci0: <Intel 82801GB/R (ICH7) USB 2.0 controller> mem
> 0xfdfff000-0xfdfff3ff irq 23 at device 29.7 on pci0
> usbus4: EHCI version 1.0
> usbus4 on ehci0
> pcib5: <ACPI PCI-PCI bridge> at device 30.0 on pci0
> pci5: <ACPI PCI bus> on pcib5
> fwohci0: <Texas Instruments TSB82AA2> mem
> 0xfd8ff000-0xfd8ff7ff,0xfd8f8000-0xfd8fbfff irq 17 at device 2.0 on
> pci5
> fwohci0: OHCI version 1.10 (ROM=1)
> fwohci0: No. of Isochronous channels is 4.
> fwohci0: EUI64 00:50:8d:00:00:72:25:9e
> fwohci0: invalid speed 7 (fixed to 3).
> fwohci0: Phy 1394a available S800, 3 ports.
> fwohci0: Link S800, max_rec 4096 bytes.
> firewire0: <IEEE1394(FireWire) bus> on fwohci0
> fwe0: <Ethernet over FireWire> on firewire0
> if_fwe0: Fake Ethernet address: 02:50:8d:72:25:9e
> fwe0: Ethernet address: 02:50:8d:72:25:9e
> fwip0: <IP over FireWire> on firewire0
> fwip0: Firewire address: 00:50:8d:00:00:72:25:9e @ 0xfffe00000000,
> S800, maxrec 4096
> dcons_crom0: <dcons configuration ROM> on firewire0
> dcons_crom0: bus_addr 0x2a20000
> fwohci0: Initiate bus reset
> fwohci0: fwohci_intr_core: BUS reset
> fwohci0: fwohci_intr_core: node_id=0x00000000, SelfID Count=1,
> CYCLEMASTER mode
> isab0: <PCI-ISA bridge> at device 31.0 on pci0
> isa0: <ISA bus> on isab0
> atapci0: <Intel ICH7 SATA300 controller> port
> 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xfb00-0xfb0f mem
> 0xfdffe000-0xfdffe3ff at device 31.2 on pci0
> ata0: <ATA channel> at channel 0 on atapci0
> ata1: <ATA channel> at channel 1 on atapci0
> pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
> acpi_tz0: <Thermal Zone> on acpi0
> acpi_tz0: _CRT value is absurd, ignored (255.0C)
> fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0
> orm0: <ISA Option ROMs> at iomem 0xc0000-0xcffff,0xd0000-0xd3fff on isa0
> sc0: <System console> at flags 0x100 on isa0
> sc0: VGA <16 virtual consoles, flags=0x300>
> vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
> atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
> atkbd0: <AT Keyboard> irq 1 on atkbdc0
> kbd0 at atkbd0
> atkbd0: [GIANT-LOCKED]
> ppc0: cannot reserve I/O port range
> ctl: CAM Target Layer loaded
> acpi_perf0: <ACPI CPU Frequency Control> on cpu0
> p4tcc0: <CPU Frequency Thermal Control> on cpu0
> est1: <Enhanced SpeedStep Frequency Control> on cpu1
> est: CPU supports Enhanced Speedstep, but is not recognized.
> est: cpu_vendor GenuineIntel, msr 102d0000102d
> device_attach: est1 attach returned 6
> p4tcc1: <CPU Frequency Thermal Control> on cpu1
> (noperiph:siisch0:0:-1:-1): rescan already queued
> (noperiph:siisch1:0:-1:-1): rescan already queued
> firewire0: 1 nodes, maxhop <= 0 cable IRM irm(0) (me)
> firewire0: bus manager 0
> fwohci0: phy int
> Timecounters tick every 1.000 msec
> hdacc0: <ATI R6xx HDA CODEC> at cad 0 on hdac0
> hdaa0: <ATI R6xx Audio Function Group> at nid 1 on hdacc0
> pcm0: <ATI R6xx (HDMI)> at nid 3 on hdaa0
> usbus0: 12Mbps Full Speed USB v1.0
> usbus1: 12Mbps Full Speed USB v1.0
> usbus2: 12Mbps Full Speed USB v1.0
> usbus3: 12Mbps Full Speed USB v1.0
> usbus4: 480Mbps High Speed USB v2.0
> acpi_tz0: _CRT value is absurd, ignored (255.0C)
> acpi_tz0: failed to set new freq, disabling passive cooling
> ugen0.1: <Intel> at usbus0
> uhub0: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
> ugen1.1: <Intel> at usbus1
> uhub1: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1
> ugen2.1: <Intel> at usbus2
> uhub2: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2
> ugen3.1: <Intel> at usbus3
> uhub3: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus3
> ugen4.1: <Intel> at usbus4
> uhub4: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus4
> (aprobe2:ata0:0:0:0): SETFEATURES DISABLE SATA FEATURE. ACB: ef 90
> 00 00 00 40 00 00 00 00 03 00
> (aprobe2:ata0:0:0:0): CAM status: ATA Status Error
> (aprobe2:ata0:0:0:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT )
> (aprobe2:ata0:0:0:0): RES: 51 04 00 00 00 00 00 00 00 03 00
> (aprobe2:ata0:0:0:0): Retrying command
> (aprobe2:ata0:0:0:0): SETFEATURES DISABLE SATA FEATURE. ACB: ef 90
> 00 00 00 40 00 00 00 00 03 00
> (aprobe2:ata0:0:0:0): CAM status: ATA Status Error
> (aprobe2:ata0:0:0:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT )
> (aprobe2:ata0:0:0:0): RES: 51 04 00 00 00 00 00 00 00 03 00
> (aprobe2:ata0:0:0:0): Error 5, Retries exhausted
> (aprobe0:ata0:0:1:0): SETFEATURES DISABLE SATA FEATURE. ACB: ef 90
> 00 00 00 40 00 00 00 00 03 00
> (aprobe0:ata0:0:1:0): CAM status: ATA Status Error
> (aprobe0:ata0:0:1:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT )
> (aprobe0:ata0:0:1:0): RES: 51 04 00 00 00 00 00 00 00 03 00
> (aprobe0:ata0:0:1:0): Retrying command
> (aprobe0:ata0:0:1:0): SETFEATURES DISABLE SATA FEATURE. ACB: ef 90
> 00 00 00 40 00 00 00 00 03 00
> (aprobe0:ata0:0:1:0): CAM status: ATA Status Error
> (aprobe0:ata0:0:1:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT )
> (aprobe0:ata0:0:1:0): RES: 51 04 00 00 00 00 00 00 00 03 00
> (aprobe0:ata0:0:1:0): Error 5, Retries exhausted
> acpi_tz0: _CRT value is absurd, ignored (255.0C)
> uhub0: 2 ports with 2 removable, self powered
> uhub1: 2 ports with 2 removable, self powered
> uhub2: 2 ports with 2 removable, self powered
> uhub3: 2 ports with 2 removable, self powered
> uhub4: 8 ports with 8 removable, self powered
> ada0 at siisch0 bus 0 scbus0 target 0 lun 0
> ada0: <ST3320620AS 3.AAE> ATA-7 SATA 1.x device
> ada0: 150.000MB/s transfers (SATA 1.x, UDMA6, PIO 8192bytes)
> ada0: Command Queueing enabled
> ada0: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C)
> ada0: Previously was known as ad4
> ada1 at siisch1 bus 0 scbus1 target 0 lun 0
> ada1: <ST3320620AS 3.AAE> ATA-7 SATA 1.x device
> ada1: 150.000MB/s transfers (SATA 1.x, UDMA6, PIO 8192bytes)
> ada1: Command Queueing enabled
> ada1: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C)
> ada1: Previously was known as ad6
> ada2 at ata0 bus 0 scbus2 target 0 lun 0
> ada2: <WDC WD360GD-00FNA0 35.06K35> ATA-6 SATA 1.x device
> ada2: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes)
> ada2: 35304MB (72303840 512 byte sectors: 16H 63S/T 16383C)
> ada2: Previously was known as ad0
> ada3 at ata0 bus 0 scbus2 target 1 lun 0
> ada3: <WDC WD360GD-00FNA0 35.06K35> ATA-6 SATA 1.x device
> ada3: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes)
> ada3: 35304MB (72303840 512 byte sectors: 16H 63S/T 16383C)
> ada3: Previously was known as ad1
> ada4 at ata1 bus 0 scbus3 target 0 lun 0
> ada4: <ST3320620AS 3.AAE> ATA-7 SATA 1.x device
> ada4: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes)
> ada4: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C)
> ada4: Previously was known as ad2
> ada5 at ata1 bus 0 scbus3 target 1 lun 0
> ada5: <ST3320620AS 3.AAE> ATA-7 SATA 1.x device
> ada5: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes)
> ada5: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C)
> ada5: Previously was known as ad3
> SMP: AP CPU #1 Launched!
> Timecounter "TSC-low" frequency 12469046 Hz quality 1000
> GEOM_RAID: Intel-165267ff: Array Intel-165267ff created.
> GEOM_RAID: Intel-841a1a00: Array Intel-841a1a00 created.
> GEOM_RAID: Intel-841a1a00: Disk ada2 state changed from NONE to ACTIVE.
> GEOM_RAID: Intel-841a1a00: Subdisk Volume0:0-ada2 state changed from
> NONE to ACTIVE.
> GEOM_RAID: Intel-841a1a00: Disk ada3 state changed from NONE to ACTIVE.
> GEOM_RAID: Intel-841a1a00: Subdisk Volume0:1-ada3 state changed from
> NONE to ACTIVE.
> GEOM_RAID: Intel-841a1a00: Array started.
> GEOM_RAID: Intel-841a1a00: Volume Volume0 state changed from
> STARTING to OPTIMAL.
> GEOM_RAID: Intel-841a1a00: Provider raid/r0 for volume Volume0 created.
> GEOM_RAID: SiI-100311082330: Array SiI-100311082330 created.
> Root mount waiting for: GRAID-SiI GRAID-Intel
> ugen1.2: <Dell> at usbus1
> ukbd0: <Dell Dell QuietKey Keyboard, class 0/0, rev 1.10/1.50, addr
> 2> on usbus1
> kbd2 at ukbd0
> Root mount waiting for: GRAID-SiI GRAID-Intel
> Root mount waiting for: GRAID-SiI GRAID-Intel
> Root mount waiting for: GRAID-SiI GRAID-Intel
> Root mount waiting for: GRAID-SiI GRAID-Intel
> Root mount waiting for: GRAID-SiI GRAID-Intel
> Root mount waiting for: GRAID-SiI GRAID-Intel
> Root mount waiting for: GRAID-SiI GRAID-Intel
> Root mount waiting for: GRAID-SiI GRAID-Intel
> Root mount waiting for: GRAID-SiI GRAID-Intel
> Root mount waiting for: GRAID-SiI GRAID-Intel
> Root mount waiting for: GRAID-SiI GRAID-Intel
> Root mount waiting for: GRAID-SiI GRAID-Intel
> Root mount waiting for: GRAID-SiI GRAID-Intel
> Root mount waiting for: GRAID-SiI GRAID-Intel
> Root mount waiting for: GRAID-SiI GRAID-Intel
> Root mount waiting for: GRAID-SiI GRAID-Intel
> Root mount waiting for: GRAID-SiI GRAID-Intel
> Root mount waiting for: GRAID-SiI GRAID-Intel
> Root mount waiting for: GRAID-SiI GRAID-Intel
> Root mount waiting for: GRAID-SiI GRAID-Intel
> Root mount waiting for: GRAID-SiI GRAID-Intel
> Root mount waiting for: GRAID-SiI GRAID-Intel
> Root mount waiting for: GRAID-SiI GRAID-Intel
> Root mount waiting for: GRAID-SiI GRAID-Intel
> Root mount waiting for: GRAID-SiI GRAID-Intel
> Root mount waiting for: GRAID-SiI GRAID-Intel
> Root mount waiting for: GRAID-SiI GRAID-Intel
> Root mount waiting for: GRAID-SiI GRAID-Intel
> Root mount waiting for: GRAID-SiI GRAID-Intel
> GEOM_RAID: Intel-165267ff: Force array start due to timeout.
> GEOM_RAID: Intel-165267ff: Disk ada0 state changed from NONE to ACTIVE.
> GEOM_RAID: Intel-165267ff: Subdisk Volume0:1-ada0 state changed from
> NONE to ACTIVE.
> GEOM_RAID: Intel-165267ff: Disk ada1 state changed from NONE to ACTIVE.
> GEOM_RAID: Intel-165267ff: Subdisk Volume0:3-ada1 state changed from
> NONE to ACTIVE.
> GEOM_RAID: Intel-165267ff: Array started.
> GEOM_RAID: Intel-165267ff: Volume Volume0 state changed from
> STARTING to BROKEN.
> GEOM_RAID: SiI-100311082330: Force array start due to timeout.
> GEOM_RAID: SiI-100311082330: Disk ada4 state changed from NONE to ACTIVE.
> GEOM_RAID: SiI-100311082330: Subdisk SiI Raid5 Set:0-ada4 state
> changed from NONE to STALE.
> GEOM_RAID: SiI-100311082330: Disk ada5 state changed from NONE to ACTIVE.
> GEOM_RAID: SiI-100311082330: Subdisk SiI Raid5 Set:1-ada5 state
> changed from NONE to STALE.
> GEOM_RAID: SiI-100311082330: Array started.
> GEOM_RAID: SiI-100311082330: Volume SiI Raid5 Set state changed from
> STARTING to BROKEN.
> Trying to mount root from ufs:/dev/raid/r0s1a [rw]...
> ZFS filesystem version 5
> ZFS storage pool version 28
> ZFS WARNING: Unable to attach to ada0.
> ZFS WARNING: Unable to attach to ada1.
> ZFS WARNING: Unable to attach to ada4.
> ZFS WARNING: Unable to attach to ada5.
> ZFS WARNING: Unable to attach to ada0.
> ZFS WARNING: Unable to attach to ada1.
> ZFS WARNING: Unable to attach to ada4.
> ZFS WARNING: Unable to attach to ada5.
> ZFS WARNING: Unable to attach to ada0.
> ZFS WARNING: Unable to attach to ada1.
> ZFS WARNING: Unable to attach to ada4.
> ZFS WARNING: Unable to attach to ada5.
> ----------------------------------------------------
>
>
>
> gpart list
> ------------------------------------------------
>
> Geom name: raid/r0
> modified: false
> state: OK
> fwheads: 255
> fwsectors: 63
> last: 72298495
> first: 63
> entries: 4
> scheme: MBR
> Providers:
> 1. Name: raid/r0s1
> Mediasize: 37013727744 (34G)
> Sectorsize: 512
> Stripesize: 0
> Stripeoffset: 32256
> Mode: r5w5e9
> attrib: active
> rawtype: 165
> length: 37013727744
> offset: 32256
> type: freebsd
> index: 1
> end: 72292499
> start: 63
> Consumers:
> 1. Name: raid/r0
> Mediasize: 37016829952 (34G)
> Sectorsize: 512
> Mode: r5w5e14
>
> Geom name: raid/r0s1
> modified: false
> state: OK
> fwheads: 255
> fwsectors: 63
> last: 72292436
> first: 0
> entries: 8
> scheme: BSD
> Providers:
> 1. Name: raid/r0s1a
> Mediasize: 1073741824 (1.0G)
> Sectorsize: 512
> Stripesize: 0
> Stripeoffset: 32256
> Mode: r1w1e1
> rawtype: 7
> length: 1073741824
> offset: 0
> type: freebsd-ufs
> index: 1
> end: 2097151
> start: 0
> 2. Name: raid/r0s1b
> Mediasize: 4294967296 (4.0G)
> Sectorsize: 512
> Stripesize: 0
> Stripeoffset: 1073774080
> Mode: r1w1e0
> rawtype: 1
> length: 4294967296
> offset: 1073741824
> type: freebsd-swap
> index: 2
> end: 10485759
> start: 2097152
> 3. Name: raid/r0s1d
> Mediasize: 5368709120 (5.0G)
> Sectorsize: 512
> Stripesize: 0
> Stripeoffset: 1073774080
> Mode: r1w1e1
> rawtype: 7
> length: 5368709120
> offset: 5368709120
> type: freebsd-ufs
> index: 4
> end: 20971519
> start: 10485760
> 4. Name: raid/r0s1e
> Mediasize: 2147483648 (2.0G)
> Sectorsize: 512
> Stripesize: 0
> Stripeoffset: 2147515904
> Mode: r1w1e1
> rawtype: 7
> length: 2147483648
> offset: 10737418240
> type: freebsd-ufs
> index: 5
> end: 25165823
> start: 20971520
> 5. Name: raid/r0s1f
> Mediasize: 24128825856 (22G)
> Sectorsize: 512
> Stripesize: 0
> Stripeoffset: 32256
> Mode: r1w1e1
> rawtype: 7
> length: 24128825856
> offset: 12884901888
> type: freebsd-ufs
> index: 6
> end: 72292436
> start: 25165824
> Consumers:
> 1. Name: raid/r0s1
> Mediasize: 37013727744 (34G)
> Sectorsize: 512
> Stripesize: 0
> Stripeoffset: 32256
> Mode: r5w5e9
>
> ----------------------------------------
>
> gpart list ada0
> gpart: No such geom: ada0.
>
> gpart list ada1
> gpart: No such geom: ada1.
>
> gpart list ada4
> gpart: No such geom: ada4.
>
> gpart list ada5
> gpart: No such geom: ada5.
>
> gpart list ad0
> gpart: No such geom: ad0.
>
> gpart list ad1
> gpart: No such geom: ad1.
>
> gpart list ad4
> gpart: No such geom: ad4.
>
> gpart list ad5
> gpart: No such geom: ad5
>
> --------------------------------------
>
>
> gpart show (This is the system raid)
>
> -------------------------------------
>
> => 63 72298433 raid/r0 MBR (34G)
> 63 72292437 1 freebsd [active] (34G)
> 72292500 5996 - free - (3M)
>
> => 0 72292437 raid/r0s1 BSD (34G)
> 0 2097152 1 freebsd-ufs (1.0G)
> 2097152 8388608 2 freebsd-swap (4.0G)
> 10485760 10485760 4 freebsd-ufs (5.0G)
> 20971520 4194304 5 freebsd-ufs (2.0G)
> 25165824 47126613 6 freebsd-ufs (22G)
>
> --------------------------------------------
>
> gpart show ada0
> gpart: No such geom: ada0
>
> gpart show ada1
> gpart: No such geom: ada1
>
> gpart show ada4
> gpart: No such geom: ada4
>
> gpart show ada5
> gpart: No such geom: ada5
>
>
>
> I'm not sure about glabel, Sorry.
I think I see what the issue is here. This is going to be a long Email.
You have 6 disks:
ada0 -- attached to siisch0 (Silicon Image 3132)
ada1 -- attached to siisch1 (Silicon Image 3132)
ada2 -- attached to ata0 (Intel ICH7, non-RAID) as master
ada3 -- attached to ata0 (Intel ICH7, non-RAID) as slave
ada4 -- attached to ata1 (Intel ICH7, non-RAID) as master
ada5 -- attached to ata1 (Intel ICH7, non-RAID) as slave
However, **BEFORE** ZFS starts, we can clearly see this (take the time
to read it slowly and carefully)
> GEOM_RAID: Intel-165267ff: Array Intel-165267ff created.
> GEOM_RAID: Intel-841a1a00: Array Intel-841a1a00 created.
> GEOM_RAID: Intel-841a1a00: Disk ada2 state changed from NONE to ACTIVE.
> GEOM_RAID: Intel-841a1a00: Subdisk Volume0:0-ada2 state changed from NONE to ACTIVE.
> GEOM_RAID: Intel-841a1a00: Disk ada3 state changed from NONE to ACTIVE.
> GEOM_RAID: Intel-841a1a00: Subdisk Volume0:1-ada3 state changed from NONE to ACTIVE.
> GEOM_RAID: Intel-841a1a00: Array started.
> GEOM_RAID: Intel-841a1a00: Volume Volume0 state changed from STARTING to OPTIMAL.
> GEOM_RAID: Intel-841a1a00: Provider raid/r0 for volume Volume0 created.
> GEOM_RAID: SiI-100311082330: Array SiI-100311082330 created.
> Root mount waiting for: GRAID-SiI GRAID-Intel
> ---
> --- this message repeats a very long time due to a problem
> --- I'm about to try and explain
> ---
> GEOM_RAID: Intel-165267ff: Force array start due to timeout.
> GEOM_RAID: Intel-165267ff: Disk ada0 state changed from NONE to ACTIVE.
> GEOM_RAID: Intel-165267ff: Subdisk Volume0:1-ada0 state changed from NONE to ACTIVE.
> GEOM_RAID: Intel-165267ff: Disk ada1 state changed from NONE to ACTIVE.
> GEOM_RAID: Intel-165267ff: Subdisk Volume0:3-ada1 state changed from NONE to ACTIVE.
> GEOM_RAID: Intel-165267ff: Array started.
> GEOM_RAID: Intel-165267ff: Volume Volume0 state changed from STARTING to BROKEN.
> GEOM_RAID: SiI-100311082330: Force array start due to timeout.
> GEOM_RAID: SiI-100311082330: Disk ada4 state changed from NONE to ACTIVE.
> GEOM_RAID: SiI-100311082330: Subdisk SiI Raid5 Set:0-ada4 state changed from NONE to STALE.
> GEOM_RAID: SiI-100311082330: Disk ada5 state changed from NONE to ACTIVE.
> GEOM_RAID: SiI-100311082330: Subdisk SiI Raid5 Set:1-ada5 state changed from NONE to STALE.
> GEOM_RAID: SiI-100311082330: Array started.
> GEOM_RAID: SiI-100311082330: Volume SiI Raid5 Set state changed from STARTING to BROKEN.
> Trying to mount root from ufs:/dev/raid/r0s1a [rw]...
This shows ***THREE*** graid(8) arrays being probed during disk tasting
for RAID metadata.
Remember two things:
1) graid(8) metadata tasting happens ***BEFORE*** ZFS metadata tasting.
2) The metadata is stored *on the disks themselves*.
- Intel-165267ff
-- disk ada0
-- disk ada1
- Intel-841a1a00
-- disk ada2
-- disk ada3
- SiI-100311082330 (used to be part of a RAID-5 set)
-- disk ada4
-- disk ada5
All of these disks become "locked" by graid because of the discovery of
metadata on them.
Plain and simple:
1) ada0 and ada1 were previously used on an Intel controller that had
BIOS-level RAID capability, and (presumably you) never cleared the
metadata from the disks before making use of ZFS,
2) ada4 and ada5 were previously used on a Silicon Image controller that
had RAID capability, and (presumably you) never cleared the metadata
from the disks before making use of ZFS.
Your next question is going to be "how to do I do that?"
There are many ways to do this, and the one I'm going to tell you is the
one that will work reliably without having to ""fight"" with GEOM or
graid(8). So hear me loud and clear.
1. Boot a FreeBSD installation CD/DVD/USB stick
2. Escape to the loader prompt
3. Type: set kern.geom.raid.enable="0"
4. Type: boot
5. Go to a "Fixit" shell (the one that has all the useful utilities)
6. Zero the first and last 8MBytes or so (I'm not going to be accurate)
of all the disks whose RAID controller metadata you want to remove.
Apologies for the caps, but I need to make this clear:
BE AWARE THIS WILL DESTROY PARTITION TABLES AND OVERWRITE/NUKE ANY OTHER
KIND OF METADATA (INCLUDING ZFS), SO IF YOU HAVE DATA ON THESE DRIVES
THAT YOU CARE ABOUT (SOMEHOW?), YOU HAD BETTER GET THE DATA OFF FIRST.
FOR ZFS YOU WILL ALSO NEED TO RE-CREATE THE POOL AFTER THIS (AND IF THEY
ARE 4K SECTOR DISKS YOU WILL NEED TO USE THE GNOP METHOD TO ENSURE
PROPER ALIGNMENT; SEE IVAN VORAS' BLOG).
The procedure to do that, for your disks (given their capacities) is
this:
> ada0: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C)
> ada1: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C)
> ada2: 35304MB (72303840 512 byte sectors: 16H 63S/T 16383C)
> ada3: 35304MB (72303840 512 byte sectors: 16H 63S/T 16383C)
> ada4: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C)
> ada5: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C)
sysctl kern.geom.debugflags=16
zpool import -a
-- I am hoping your pool get imported successfully at this point.
-- This might not even be needed depending upon what the kernel
-- does during its earlier boot.
zpool destroy {poolname}
-- Yes, you really do want to do this. This is how you will get
-- ZFS to "unlock" the disks that you're about to nuke metadata
-- on. If you have multiple pools, you need to do this for
-- all of them.
dd if=/dev/zero of=/dev/ada0 bs=65536 count=128
dd if=/dev/zero of=/dev/ada0 bs=65536 count=128 oseek=4880000
dd if=/dev/zero of=/dev/ada1 bs=65536 count=128
dd if=/dev/zero of=/dev/ada1 bs=65536 count=128 oseek=4880000
dd if=/dev/zero of=/dev/ada4 bs=65536 count=128
dd if=/dev/zero of=/dev/ada4 bs=65536 count=128 oseek=4880000
dd if=/dev/zero of=/dev/ada5 bs=65536 count=128
dd if=/dev/zero of=/dev/ada5 bs=65536 count=128 oseek=4880000
On the commands with "oseek", you will probably receive an error that
says something about "short end of device" or what not; that is
perfectly fine.
After this, I strongly recommend rebooting the system ("reboot")
and booting off of your standard hard disks. Your system should
start in multi-user, but obviously there will be no ZFS pools.
At this point, recreate your ZFS pools as you did originally, and you
should be good to go going forward.
--
| Jeremy Chadwick jdc at koitsu.org |
| UNIX Systems Administrator http://jdc.koitsu.org/ |
| Mountain View, CA, US |
| Making life hard for others since 1977. PGP 4BD6C0CB |
More information about the freebsd-fs
mailing list