Upgrading from 8.3 to 9.1 zfs pool can't attach
Leslie Jensen
leslie at eskk.nu
Wed May 22 20:02:28 UTC 2013
2013-05-22 20:30, Jeremy Chadwick skrev:
> On Wed, May 22, 2013 at 07:54:34PM +0200, Leslie Jensen wrote:
>>
>>
>> 2013-05-22 17:02, Ronald Klop skrev:
>>> On Wed, 22 May 2013 12:53:25 +0200, Leslie Jensen <leslie at eskk.nu> wrote:
>>>
>>>>
>>>> I've upgraded a machine with freebsd-update from 8.3 to 9.1.
>>>>
>>>> After the first restart I edited /etc/fstab in single user mode
>>>> because the names on the disks had changed. But the zfs pool I have
>>>> seem to have a problem and I'm sure on how to recover it.
>>>>
>>>>
>>>> May 22 12:00:39 kernel: ZFS WARNING: Unable to attach to ada0.
>>>> May 22 12:00:39 kernel: ZFS WARNING: Unable to attach to ada1.
>>>> May 22 12:00:39 kernel: ZFS WARNING: Unable to attach to ada4.
>>>> May 22 12:00:40 kernel: ZFS WARNING: Unable to attach to ada5.
>>>>
>>>>
>>>> camcontrol devlist
>>>> <ST3320620AS 3.AAE> at scbus0 target 0 lun 0 (ada0,pass0)
>>>> <ST3320620AS 3.AAE> at scbus1 target 0 lun 0 (ada1,pass1)
>>>> <WDC WD360GD-00FNA0 35.06K35> at scbus2 target 0 lun 0 (ada2,pass2)
>>>> <WDC WD360GD-00FNA0 35.06K35> at scbus2 target 1 lun 0 (ada3,pass3)
>>>> <ST3320620AS 3.AAE> at scbus3 target 0 lun 0 (ada4,pass4)
>>>> <ST3320620AS 3.AAE> at scbus3 target 1 lun 0 (ada5,pass5)
>>>>
>>>> ada2 and ada3 are configured as raid/r0 holding the system.
>>>>
>>>> ada0, ada1, ada4 and ada5 should be a zfs pool with /home
>>>>
>>>>
>>>> cat /etc/fstab
>>>>
>>>> # Device Mountpoint FStype Options Dump Pass#
>>>> /dev/raid/r0s1b none swap sw 0 0
>>>> /dev/raid/r0s1a / ufs rw 1 1
>>>> /dev/raid/r0s1e /tmp ufs rw 2 2
>>>> /dev/raid/r0s1f /usr ufs rw 2 2
>>>> /dev/raid/r0s1d /var ufs rw 2 2
>>>>
>>>>
>>>> I'm not sure but shouldn't the mount point for /home also appear in
>>>> fstab?
>>>>
>>>> Maybe I messed up during the manual editing of files during the update
>>>> process?
>>>>
>>>>
>>>> Can I recover or shall I destroy the pool and rebuild it?
>>>>
>>>> Thanks
>>>>
>>>> /Leslie
>>>
>>> Mounts from zfs do not have to be in fstab. You can set a mountpoint on
>>> a zfs volume which kind of mounts itself.
>>> Can you send more information about the 'Unable to attach to ada[0-4]'
>>> messages? Maybe a complete dmesg (/var/run/dmesg.boot).
>>> And how are these disks configured/partitioned? Output from 'gpart list
>>> ada0' or 'gpart show ada0' and for the other disks.
>>> Are you using stuff like glabel?
>>>
>>> Ronald.
>>
>>
>> cat /var/run/dmesg.boot
>> -----------------------------------------------
>> Copyright (c) 1992-2012 The FreeBSD Project.
>> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
>> The Regents of the University of California. All rights reserved.
>> FreeBSD is a registered trademark of The FreeBSD Foundation.
>> FreeBSD 9.1-RELEASE-p3 #0: Mon Apr 29 18:27:25 UTC 2013
>> root at amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64
>> link_elf_obj: symbol ata_controlcmd undefined
>> KLD file atapicam.ko - could not finalize loading
>> CPU: Intel(R) Pentium(R) 4 CPU 3.20GHz (3192.08-MHz K8-class CPU)
>> Origin = "GenuineIntel" Id = 0xf43 Family = f Model = 4 Stepping = 3
>>
>> Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
>> Features2=0x649d<SSE3,DTES64,MON,DS_CPL,EST,CNXT-ID,CX16,xTPR>
>> AMD Features=0x20100800<SYSCALL,NX,LM>
>> TSC: P-state invariant
>> real memory = 4294967296 (4096 MB)
>> avail memory = 4084785152 (3895 MB)
>> Event timer "LAPIC" quality 400
>> ACPI APIC Table: <IntelR AWRDACPI>
>> FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
>> FreeBSD/SMP: 1 package(s) x 1 core(s) x 2 HTT threads
>> cpu0 (BSP): APIC ID: 0
>> cpu1 (AP/HT): APIC ID: 1
>> ioapic0: Changing APIC ID to 4
>> ioapic0 <Version 2.0> irqs 0-23 on motherboard
>> kbd1 at kbdmux0
>> acpi0: <IntelR AWRDACPI> on motherboard
>> acpi0: Power Button (fixed)
>> acpi0: reservation of 0, a0000 (3) failed
>> acpi0: reservation of 100000, bfde0000 (3) failed
>> cpu0: <ACPI CPU> on acpi0
>> cpu1: <ACPI CPU> on acpi0
>> attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
>> Timecounter "i8254" frequency 1193182 Hz quality 0
>> Event timer "i8254" frequency 1193182 Hz quality 100
>> atrtc0: <AT realtime clock> port 0x70-0x73 irq 8 on acpi0
>> Event timer "RTC" frequency 32768 Hz quality 0
>> Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
>> acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
>> acpi_button0: <Power Button> on acpi0
>> pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
>> pci0: <ACPI PCI bus> on pcib0
>> pcib1: <PCI-PCI bridge> irq 16 at device 1.0 on pci0
>> pci1: <PCI bus> on pcib1
>> vgapci0: <VGA-compatible display> port 0xbe00-0xbeff mem
>> 0xd0000000-0xdfffffff,0xfd9e0000-0xfd9effff irq 16 at device 0.0 on
>> pci1
>> hdac0: <ATI RV610 HDA Controller> mem 0xfd9fc000-0xfd9fffff irq 17
>> at device 0.1 on pci1
>> pcib2: <ACPI PCI-PCI bridge> irq 16 at device 28.0 on pci0
>> pci2: <ACPI PCI bus> on pcib2
>> siis0: <SiI3132 SATA controller> port 0xaf00-0xaf7f mem
>> 0xfd6ff000-0xfd6ff07f,0xfd6f8000-0xfd6fbfff irq 16 at device 0.0 on
>> pci2
>> siisch0: <SIIS channel> at channel 0 on siis0
>> siisch1: <SIIS channel> at channel 1 on siis0
>> pcib3: <ACPI PCI-PCI bridge> irq 17 at device 28.1 on pci0
>> pci3: <ACPI PCI bus> on pcib3
>> bge0: <Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev.
>> 0x004101> mem 0xfddf0000-0xfddfffff irq 17 at device 0.0 on pci3
>> bge0: CHIP ID 0x00004101; ASIC REV 0x04; CHIP REV 0x41; PCI-E
>> miibus0: <MII bus> on bge0
>> brgphy0: <BCM5750 1000BASE-T media interface> PHY 1 on miibus0
>> brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT,
>> 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto,
>> auto-flow
>> bge0: Ethernet address: 00:50:8d:7c:c2:60
>> pcib4: <ACPI PCI-PCI bridge> irq 16 at device 28.4 on pci0
>> pci4: <ACPI PCI bus> on pcib4
>> bge1: <Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev.
>> 0x004101> mem 0xfdbf0000-0xfdbfffff irq 16 at device 0.0 on pci4
>> bge1: CHIP ID 0x00004101; ASIC REV 0x04; CHIP REV 0x41; PCI-E
>> miibus1: <MII bus> on bge1
>> brgphy1: <BCM5750 1000BASE-T media interface> PHY 1 on miibus1
>> brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT,
>> 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto,
>> auto-flow
>> bge1: Ethernet address: 00:50:8d:7c:c2:61
>> uhci0: <Intel 82801G (ICH7) USB controller USB-A> port 0xff00-0xff1f
>> irq 23 at device 29.0 on pci0
>> usbus0 on uhci0
>> uhci1: <Intel 82801G (ICH7) USB controller USB-B> port 0xfe00-0xfe1f
>> irq 19 at device 29.1 on pci0
>> usbus1 on uhci1
>> uhci2: <Intel 82801G (ICH7) USB controller USB-C> port 0xfd00-0xfd1f
>> irq 18 at device 29.2 on pci0
>> usbus2 on uhci2
>> uhci3: <Intel 82801G (ICH7) USB controller USB-D> port 0xfc00-0xfc1f
>> irq 16 at device 29.3 on pci0
>> usbus3 on uhci3
>> ehci0: <Intel 82801GB/R (ICH7) USB 2.0 controller> mem
>> 0xfdfff000-0xfdfff3ff irq 23 at device 29.7 on pci0
>> usbus4: EHCI version 1.0
>> usbus4 on ehci0
>> pcib5: <ACPI PCI-PCI bridge> at device 30.0 on pci0
>> pci5: <ACPI PCI bus> on pcib5
>> fwohci0: <Texas Instruments TSB82AA2> mem
>> 0xfd8ff000-0xfd8ff7ff,0xfd8f8000-0xfd8fbfff irq 17 at device 2.0 on
>> pci5
>> fwohci0: OHCI version 1.10 (ROM=1)
>> fwohci0: No. of Isochronous channels is 4.
>> fwohci0: EUI64 00:50:8d:00:00:72:25:9e
>> fwohci0: invalid speed 7 (fixed to 3).
>> fwohci0: Phy 1394a available S800, 3 ports.
>> fwohci0: Link S800, max_rec 4096 bytes.
>> firewire0: <IEEE1394(FireWire) bus> on fwohci0
>> fwe0: <Ethernet over FireWire> on firewire0
>> if_fwe0: Fake Ethernet address: 02:50:8d:72:25:9e
>> fwe0: Ethernet address: 02:50:8d:72:25:9e
>> fwip0: <IP over FireWire> on firewire0
>> fwip0: Firewire address: 00:50:8d:00:00:72:25:9e @ 0xfffe00000000,
>> S800, maxrec 4096
>> dcons_crom0: <dcons configuration ROM> on firewire0
>> dcons_crom0: bus_addr 0x2a20000
>> fwohci0: Initiate bus reset
>> fwohci0: fwohci_intr_core: BUS reset
>> fwohci0: fwohci_intr_core: node_id=0x00000000, SelfID Count=1,
>> CYCLEMASTER mode
>> isab0: <PCI-ISA bridge> at device 31.0 on pci0
>> isa0: <ISA bus> on isab0
>> atapci0: <Intel ICH7 SATA300 controller> port
>> 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xfb00-0xfb0f mem
>> 0xfdffe000-0xfdffe3ff at device 31.2 on pci0
>> ata0: <ATA channel> at channel 0 on atapci0
>> ata1: <ATA channel> at channel 1 on atapci0
>> pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
>> acpi_tz0: <Thermal Zone> on acpi0
>> acpi_tz0: _CRT value is absurd, ignored (255.0C)
>> fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0
>> orm0: <ISA Option ROMs> at iomem 0xc0000-0xcffff,0xd0000-0xd3fff on isa0
>> sc0: <System console> at flags 0x100 on isa0
>> sc0: VGA <16 virtual consoles, flags=0x300>
>> vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
>> atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
>> atkbd0: <AT Keyboard> irq 1 on atkbdc0
>> kbd0 at atkbd0
>> atkbd0: [GIANT-LOCKED]
>> ppc0: cannot reserve I/O port range
>> ctl: CAM Target Layer loaded
>> acpi_perf0: <ACPI CPU Frequency Control> on cpu0
>> p4tcc0: <CPU Frequency Thermal Control> on cpu0
>> est1: <Enhanced SpeedStep Frequency Control> on cpu1
>> est: CPU supports Enhanced Speedstep, but is not recognized.
>> est: cpu_vendor GenuineIntel, msr 102d0000102d
>> device_attach: est1 attach returned 6
>> p4tcc1: <CPU Frequency Thermal Control> on cpu1
>> (noperiph:siisch0:0:-1:-1): rescan already queued
>> (noperiph:siisch1:0:-1:-1): rescan already queued
>> firewire0: 1 nodes, maxhop <= 0 cable IRM irm(0) (me)
>> firewire0: bus manager 0
>> fwohci0: phy int
>> Timecounters tick every 1.000 msec
>> hdacc0: <ATI R6xx HDA CODEC> at cad 0 on hdac0
>> hdaa0: <ATI R6xx Audio Function Group> at nid 1 on hdacc0
>> pcm0: <ATI R6xx (HDMI)> at nid 3 on hdaa0
>> usbus0: 12Mbps Full Speed USB v1.0
>> usbus1: 12Mbps Full Speed USB v1.0
>> usbus2: 12Mbps Full Speed USB v1.0
>> usbus3: 12Mbps Full Speed USB v1.0
>> usbus4: 480Mbps High Speed USB v2.0
>> acpi_tz0: _CRT value is absurd, ignored (255.0C)
>> acpi_tz0: failed to set new freq, disabling passive cooling
>> ugen0.1: <Intel> at usbus0
>> uhub0: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
>> ugen1.1: <Intel> at usbus1
>> uhub1: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1
>> ugen2.1: <Intel> at usbus2
>> uhub2: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2
>> ugen3.1: <Intel> at usbus3
>> uhub3: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus3
>> ugen4.1: <Intel> at usbus4
>> uhub4: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus4
>> (aprobe2:ata0:0:0:0): SETFEATURES DISABLE SATA FEATURE. ACB: ef 90
>> 00 00 00 40 00 00 00 00 03 00
>> (aprobe2:ata0:0:0:0): CAM status: ATA Status Error
>> (aprobe2:ata0:0:0:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT )
>> (aprobe2:ata0:0:0:0): RES: 51 04 00 00 00 00 00 00 00 03 00
>> (aprobe2:ata0:0:0:0): Retrying command
>> (aprobe2:ata0:0:0:0): SETFEATURES DISABLE SATA FEATURE. ACB: ef 90
>> 00 00 00 40 00 00 00 00 03 00
>> (aprobe2:ata0:0:0:0): CAM status: ATA Status Error
>> (aprobe2:ata0:0:0:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT )
>> (aprobe2:ata0:0:0:0): RES: 51 04 00 00 00 00 00 00 00 03 00
>> (aprobe2:ata0:0:0:0): Error 5, Retries exhausted
>> (aprobe0:ata0:0:1:0): SETFEATURES DISABLE SATA FEATURE. ACB: ef 90
>> 00 00 00 40 00 00 00 00 03 00
>> (aprobe0:ata0:0:1:0): CAM status: ATA Status Error
>> (aprobe0:ata0:0:1:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT )
>> (aprobe0:ata0:0:1:0): RES: 51 04 00 00 00 00 00 00 00 03 00
>> (aprobe0:ata0:0:1:0): Retrying command
>> (aprobe0:ata0:0:1:0): SETFEATURES DISABLE SATA FEATURE. ACB: ef 90
>> 00 00 00 40 00 00 00 00 03 00
>> (aprobe0:ata0:0:1:0): CAM status: ATA Status Error
>> (aprobe0:ata0:0:1:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT )
>> (aprobe0:ata0:0:1:0): RES: 51 04 00 00 00 00 00 00 00 03 00
>> (aprobe0:ata0:0:1:0): Error 5, Retries exhausted
>> acpi_tz0: _CRT value is absurd, ignored (255.0C)
>> uhub0: 2 ports with 2 removable, self powered
>> uhub1: 2 ports with 2 removable, self powered
>> uhub2: 2 ports with 2 removable, self powered
>> uhub3: 2 ports with 2 removable, self powered
>> uhub4: 8 ports with 8 removable, self powered
>> ada0 at siisch0 bus 0 scbus0 target 0 lun 0
>> ada0: <ST3320620AS 3.AAE> ATA-7 SATA 1.x device
>> ada0: 150.000MB/s transfers (SATA 1.x, UDMA6, PIO 8192bytes)
>> ada0: Command Queueing enabled
>> ada0: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C)
>> ada0: Previously was known as ad4
>> ada1 at siisch1 bus 0 scbus1 target 0 lun 0
>> ada1: <ST3320620AS 3.AAE> ATA-7 SATA 1.x device
>> ada1: 150.000MB/s transfers (SATA 1.x, UDMA6, PIO 8192bytes)
>> ada1: Command Queueing enabled
>> ada1: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C)
>> ada1: Previously was known as ad6
>> ada2 at ata0 bus 0 scbus2 target 0 lun 0
>> ada2: <WDC WD360GD-00FNA0 35.06K35> ATA-6 SATA 1.x device
>> ada2: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes)
>> ada2: 35304MB (72303840 512 byte sectors: 16H 63S/T 16383C)
>> ada2: Previously was known as ad0
>> ada3 at ata0 bus 0 scbus2 target 1 lun 0
>> ada3: <WDC WD360GD-00FNA0 35.06K35> ATA-6 SATA 1.x device
>> ada3: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes)
>> ada3: 35304MB (72303840 512 byte sectors: 16H 63S/T 16383C)
>> ada3: Previously was known as ad1
>> ada4 at ata1 bus 0 scbus3 target 0 lun 0
>> ada4: <ST3320620AS 3.AAE> ATA-7 SATA 1.x device
>> ada4: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes)
>> ada4: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C)
>> ada4: Previously was known as ad2
>> ada5 at ata1 bus 0 scbus3 target 1 lun 0
>> ada5: <ST3320620AS 3.AAE> ATA-7 SATA 1.x device
>> ada5: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes)
>> ada5: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C)
>> ada5: Previously was known as ad3
>> SMP: AP CPU #1 Launched!
>> Timecounter "TSC-low" frequency 12469046 Hz quality 1000
>> GEOM_RAID: Intel-165267ff: Array Intel-165267ff created.
>> GEOM_RAID: Intel-841a1a00: Array Intel-841a1a00 created.
>> GEOM_RAID: Intel-841a1a00: Disk ada2 state changed from NONE to ACTIVE.
>> GEOM_RAID: Intel-841a1a00: Subdisk Volume0:0-ada2 state changed from
>> NONE to ACTIVE.
>> GEOM_RAID: Intel-841a1a00: Disk ada3 state changed from NONE to ACTIVE.
>> GEOM_RAID: Intel-841a1a00: Subdisk Volume0:1-ada3 state changed from
>> NONE to ACTIVE.
>> GEOM_RAID: Intel-841a1a00: Array started.
>> GEOM_RAID: Intel-841a1a00: Volume Volume0 state changed from
>> STARTING to OPTIMAL.
>> GEOM_RAID: Intel-841a1a00: Provider raid/r0 for volume Volume0 created.
>> GEOM_RAID: SiI-100311082330: Array SiI-100311082330 created.
>> Root mount waiting for: GRAID-SiI GRAID-Intel
>> ugen1.2: <Dell> at usbus1
>> ukbd0: <Dell Dell QuietKey Keyboard, class 0/0, rev 1.10/1.50, addr
>> 2> on usbus1
>> kbd2 at ukbd0
>> Root mount waiting for: GRAID-SiI GRAID-Intel
>> Root mount waiting for: GRAID-SiI GRAID-Intel
>> Root mount waiting for: GRAID-SiI GRAID-Intel
>> Root mount waiting for: GRAID-SiI GRAID-Intel
>> Root mount waiting for: GRAID-SiI GRAID-Intel
>> Root mount waiting for: GRAID-SiI GRAID-Intel
>> Root mount waiting for: GRAID-SiI GRAID-Intel
>> Root mount waiting for: GRAID-SiI GRAID-Intel
>> Root mount waiting for: GRAID-SiI GRAID-Intel
>> Root mount waiting for: GRAID-SiI GRAID-Intel
>> Root mount waiting for: GRAID-SiI GRAID-Intel
>> Root mount waiting for: GRAID-SiI GRAID-Intel
>> Root mount waiting for: GRAID-SiI GRAID-Intel
>> Root mount waiting for: GRAID-SiI GRAID-Intel
>> Root mount waiting for: GRAID-SiI GRAID-Intel
>> Root mount waiting for: GRAID-SiI GRAID-Intel
>> Root mount waiting for: GRAID-SiI GRAID-Intel
>> Root mount waiting for: GRAID-SiI GRAID-Intel
>> Root mount waiting for: GRAID-SiI GRAID-Intel
>> Root mount waiting for: GRAID-SiI GRAID-Intel
>> Root mount waiting for: GRAID-SiI GRAID-Intel
>> Root mount waiting for: GRAID-SiI GRAID-Intel
>> Root mount waiting for: GRAID-SiI GRAID-Intel
>> Root mount waiting for: GRAID-SiI GRAID-Intel
>> Root mount waiting for: GRAID-SiI GRAID-Intel
>> Root mount waiting for: GRAID-SiI GRAID-Intel
>> Root mount waiting for: GRAID-SiI GRAID-Intel
>> Root mount waiting for: GRAID-SiI GRAID-Intel
>> Root mount waiting for: GRAID-SiI GRAID-Intel
>> GEOM_RAID: Intel-165267ff: Force array start due to timeout.
>> GEOM_RAID: Intel-165267ff: Disk ada0 state changed from NONE to ACTIVE.
>> GEOM_RAID: Intel-165267ff: Subdisk Volume0:1-ada0 state changed from
>> NONE to ACTIVE.
>> GEOM_RAID: Intel-165267ff: Disk ada1 state changed from NONE to ACTIVE.
>> GEOM_RAID: Intel-165267ff: Subdisk Volume0:3-ada1 state changed from
>> NONE to ACTIVE.
>> GEOM_RAID: Intel-165267ff: Array started.
>> GEOM_RAID: Intel-165267ff: Volume Volume0 state changed from
>> STARTING to BROKEN.
>> GEOM_RAID: SiI-100311082330: Force array start due to timeout.
>> GEOM_RAID: SiI-100311082330: Disk ada4 state changed from NONE to ACTIVE.
>> GEOM_RAID: SiI-100311082330: Subdisk SiI Raid5 Set:0-ada4 state
>> changed from NONE to STALE.
>> GEOM_RAID: SiI-100311082330: Disk ada5 state changed from NONE to ACTIVE.
>> GEOM_RAID: SiI-100311082330: Subdisk SiI Raid5 Set:1-ada5 state
>> changed from NONE to STALE.
>> GEOM_RAID: SiI-100311082330: Array started.
>> GEOM_RAID: SiI-100311082330: Volume SiI Raid5 Set state changed from
>> STARTING to BROKEN.
>> Trying to mount root from ufs:/dev/raid/r0s1a [rw]...
>> ZFS filesystem version 5
>> ZFS storage pool version 28
>> ZFS WARNING: Unable to attach to ada0.
>> ZFS WARNING: Unable to attach to ada1.
>> ZFS WARNING: Unable to attach to ada4.
>> ZFS WARNING: Unable to attach to ada5.
>> ZFS WARNING: Unable to attach to ada0.
>> ZFS WARNING: Unable to attach to ada1.
>> ZFS WARNING: Unable to attach to ada4.
>> ZFS WARNING: Unable to attach to ada5.
>> ZFS WARNING: Unable to attach to ada0.
>> ZFS WARNING: Unable to attach to ada1.
>> ZFS WARNING: Unable to attach to ada4.
>> ZFS WARNING: Unable to attach to ada5.
>> ----------------------------------------------------
>>
>>
>>
>> gpart list
>> ------------------------------------------------
>>
>> Geom name: raid/r0
>> modified: false
>> state: OK
>> fwheads: 255
>> fwsectors: 63
>> last: 72298495
>> first: 63
>> entries: 4
>> scheme: MBR
>> Providers:
>> 1. Name: raid/r0s1
>> Mediasize: 37013727744 (34G)
>> Sectorsize: 512
>> Stripesize: 0
>> Stripeoffset: 32256
>> Mode: r5w5e9
>> attrib: active
>> rawtype: 165
>> length: 37013727744
>> offset: 32256
>> type: freebsd
>> index: 1
>> end: 72292499
>> start: 63
>> Consumers:
>> 1. Name: raid/r0
>> Mediasize: 37016829952 (34G)
>> Sectorsize: 512
>> Mode: r5w5e14
>>
>> Geom name: raid/r0s1
>> modified: false
>> state: OK
>> fwheads: 255
>> fwsectors: 63
>> last: 72292436
>> first: 0
>> entries: 8
>> scheme: BSD
>> Providers:
>> 1. Name: raid/r0s1a
>> Mediasize: 1073741824 (1.0G)
>> Sectorsize: 512
>> Stripesize: 0
>> Stripeoffset: 32256
>> Mode: r1w1e1
>> rawtype: 7
>> length: 1073741824
>> offset: 0
>> type: freebsd-ufs
>> index: 1
>> end: 2097151
>> start: 0
>> 2. Name: raid/r0s1b
>> Mediasize: 4294967296 (4.0G)
>> Sectorsize: 512
>> Stripesize: 0
>> Stripeoffset: 1073774080
>> Mode: r1w1e0
>> rawtype: 1
>> length: 4294967296
>> offset: 1073741824
>> type: freebsd-swap
>> index: 2
>> end: 10485759
>> start: 2097152
>> 3. Name: raid/r0s1d
>> Mediasize: 5368709120 (5.0G)
>> Sectorsize: 512
>> Stripesize: 0
>> Stripeoffset: 1073774080
>> Mode: r1w1e1
>> rawtype: 7
>> length: 5368709120
>> offset: 5368709120
>> type: freebsd-ufs
>> index: 4
>> end: 20971519
>> start: 10485760
>> 4. Name: raid/r0s1e
>> Mediasize: 2147483648 (2.0G)
>> Sectorsize: 512
>> Stripesize: 0
>> Stripeoffset: 2147515904
>> Mode: r1w1e1
>> rawtype: 7
>> length: 2147483648
>> offset: 10737418240
>> type: freebsd-ufs
>> index: 5
>> end: 25165823
>> start: 20971520
>> 5. Name: raid/r0s1f
>> Mediasize: 24128825856 (22G)
>> Sectorsize: 512
>> Stripesize: 0
>> Stripeoffset: 32256
>> Mode: r1w1e1
>> rawtype: 7
>> length: 24128825856
>> offset: 12884901888
>> type: freebsd-ufs
>> index: 6
>> end: 72292436
>> start: 25165824
>> Consumers:
>> 1. Name: raid/r0s1
>> Mediasize: 37013727744 (34G)
>> Sectorsize: 512
>> Stripesize: 0
>> Stripeoffset: 32256
>> Mode: r5w5e9
>>
>> ----------------------------------------
>>
>> gpart list ada0
>> gpart: No such geom: ada0.
>>
>> gpart list ada1
>> gpart: No such geom: ada1.
>>
>> gpart list ada4
>> gpart: No such geom: ada4.
>>
>> gpart list ada5
>> gpart: No such geom: ada5.
>>
>> gpart list ad0
>> gpart: No such geom: ad0.
>>
>> gpart list ad1
>> gpart: No such geom: ad1.
>>
>> gpart list ad4
>> gpart: No such geom: ad4.
>>
>> gpart list ad5
>> gpart: No such geom: ad5
>>
>> --------------------------------------
>>
>>
>> gpart show (This is the system raid)
>>
>> -------------------------------------
>>
>> => 63 72298433 raid/r0 MBR (34G)
>> 63 72292437 1 freebsd [active] (34G)
>> 72292500 5996 - free - (3M)
>>
>> => 0 72292437 raid/r0s1 BSD (34G)
>> 0 2097152 1 freebsd-ufs (1.0G)
>> 2097152 8388608 2 freebsd-swap (4.0G)
>> 10485760 10485760 4 freebsd-ufs (5.0G)
>> 20971520 4194304 5 freebsd-ufs (2.0G)
>> 25165824 47126613 6 freebsd-ufs (22G)
>>
>> --------------------------------------------
>>
>> gpart show ada0
>> gpart: No such geom: ada0
>>
>> gpart show ada1
>> gpart: No such geom: ada1
>>
>> gpart show ada4
>> gpart: No such geom: ada4
>>
>> gpart show ada5
>> gpart: No such geom: ada5
>>
>>
>>
>> I'm not sure about glabel, Sorry.
>
> I think I see what the issue is here. This is going to be a long Email.
>
> You have 6 disks:
>
> ada0 -- attached to siisch0 (Silicon Image 3132)
> ada1 -- attached to siisch1 (Silicon Image 3132)
> ada2 -- attached to ata0 (Intel ICH7, non-RAID) as master
> ada3 -- attached to ata0 (Intel ICH7, non-RAID) as slave
> ada4 -- attached to ata1 (Intel ICH7, non-RAID) as master
> ada5 -- attached to ata1 (Intel ICH7, non-RAID) as slave
>
> However, **BEFORE** ZFS starts, we can clearly see this (take the time
> to read it slowly and carefully)
>
>> GEOM_RAID: Intel-165267ff: Array Intel-165267ff created.
>> GEOM_RAID: Intel-841a1a00: Array Intel-841a1a00 created.
>> GEOM_RAID: Intel-841a1a00: Disk ada2 state changed from NONE to ACTIVE.
>> GEOM_RAID: Intel-841a1a00: Subdisk Volume0:0-ada2 state changed from NONE to ACTIVE.
>> GEOM_RAID: Intel-841a1a00: Disk ada3 state changed from NONE to ACTIVE.
>> GEOM_RAID: Intel-841a1a00: Subdisk Volume0:1-ada3 state changed from NONE to ACTIVE.
>> GEOM_RAID: Intel-841a1a00: Array started.
>> GEOM_RAID: Intel-841a1a00: Volume Volume0 state changed from STARTING to OPTIMAL.
>> GEOM_RAID: Intel-841a1a00: Provider raid/r0 for volume Volume0 created.
>> GEOM_RAID: SiI-100311082330: Array SiI-100311082330 created.
>> Root mount waiting for: GRAID-SiI GRAID-Intel
>> ---
>> --- this message repeats a very long time due to a problem
>> --- I'm about to try and explain
>> ---
>> GEOM_RAID: Intel-165267ff: Force array start due to timeout.
>> GEOM_RAID: Intel-165267ff: Disk ada0 state changed from NONE to ACTIVE.
>> GEOM_RAID: Intel-165267ff: Subdisk Volume0:1-ada0 state changed from NONE to ACTIVE.
>> GEOM_RAID: Intel-165267ff: Disk ada1 state changed from NONE to ACTIVE.
>> GEOM_RAID: Intel-165267ff: Subdisk Volume0:3-ada1 state changed from NONE to ACTIVE.
>> GEOM_RAID: Intel-165267ff: Array started.
>> GEOM_RAID: Intel-165267ff: Volume Volume0 state changed from STARTING to BROKEN.
>> GEOM_RAID: SiI-100311082330: Force array start due to timeout.
>> GEOM_RAID: SiI-100311082330: Disk ada4 state changed from NONE to ACTIVE.
>> GEOM_RAID: SiI-100311082330: Subdisk SiI Raid5 Set:0-ada4 state changed from NONE to STALE.
>> GEOM_RAID: SiI-100311082330: Disk ada5 state changed from NONE to ACTIVE.
>> GEOM_RAID: SiI-100311082330: Subdisk SiI Raid5 Set:1-ada5 state changed from NONE to STALE.
>> GEOM_RAID: SiI-100311082330: Array started.
>> GEOM_RAID: SiI-100311082330: Volume SiI Raid5 Set state changed from STARTING to BROKEN.
>> Trying to mount root from ufs:/dev/raid/r0s1a [rw]...
>
> This shows ***THREE*** graid(8) arrays being probed during disk tasting
> for RAID metadata.
>
> Remember two things:
>
> 1) graid(8) metadata tasting happens ***BEFORE*** ZFS metadata tasting.
> 2) The metadata is stored *on the disks themselves*.
>
> - Intel-165267ff
> -- disk ada0
> -- disk ada1
> - Intel-841a1a00
> -- disk ada2
> -- disk ada3
> - SiI-100311082330 (used to be part of a RAID-5 set)
> -- disk ada4
> -- disk ada5
>
> All of these disks become "locked" by graid because of the discovery of
> metadata on them.
>
> Plain and simple:
>
> 1) ada0 and ada1 were previously used on an Intel controller that had
> BIOS-level RAID capability, and (presumably you) never cleared the
> metadata from the disks before making use of ZFS,
>
> 2) ada4 and ada5 were previously used on a Silicon Image controller that
> had RAID capability, and (presumably you) never cleared the metadata
> from the disks before making use of ZFS.
>
> Your next question is going to be "how to do I do that?"
>
> There are many ways to do this, and the one I'm going to tell you is the
> one that will work reliably without having to ""fight"" with GEOM or
> graid(8). So hear me loud and clear.
>
> 1. Boot a FreeBSD installation CD/DVD/USB stick
> 2. Escape to the loader prompt
> 3. Type: set kern.geom.raid.enable="0"
> 4. Type: boot
> 5. Go to a "Fixit" shell (the one that has all the useful utilities)
> 6. Zero the first and last 8MBytes or so (I'm not going to be accurate)
> of all the disks whose RAID controller metadata you want to remove.
>
> Apologies for the caps, but I need to make this clear:
>
> BE AWARE THIS WILL DESTROY PARTITION TABLES AND OVERWRITE/NUKE ANY OTHER
> KIND OF METADATA (INCLUDING ZFS), SO IF YOU HAVE DATA ON THESE DRIVES
> THAT YOU CARE ABOUT (SOMEHOW?), YOU HAD BETTER GET THE DATA OFF FIRST.
>
> FOR ZFS YOU WILL ALSO NEED TO RE-CREATE THE POOL AFTER THIS (AND IF THEY
> ARE 4K SECTOR DISKS YOU WILL NEED TO USE THE GNOP METHOD TO ENSURE
> PROPER ALIGNMENT; SEE IVAN VORAS' BLOG).
>
> The procedure to do that, for your disks (given their capacities) is
> this:
>
>> ada0: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C)
>> ada1: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C)
>> ada2: 35304MB (72303840 512 byte sectors: 16H 63S/T 16383C)
>> ada3: 35304MB (72303840 512 byte sectors: 16H 63S/T 16383C)
>> ada4: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C)
>> ada5: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C)
>
> sysctl kern.geom.debugflags=16
>
> zpool import -a
> -- I am hoping your pool get imported successfully at this point.
> -- This might not even be needed depending upon what the kernel
> -- does during its earlier boot.
>
> zpool destroy {poolname}
> -- Yes, you really do want to do this. This is how you will get
> -- ZFS to "unlock" the disks that you're about to nuke metadata
> -- on. If you have multiple pools, you need to do this for
> -- all of them.
>
> dd if=/dev/zero of=/dev/ada0 bs=65536 count=128
> dd if=/dev/zero of=/dev/ada0 bs=65536 count=128 oseek=4880000
> dd if=/dev/zero of=/dev/ada1 bs=65536 count=128
> dd if=/dev/zero of=/dev/ada1 bs=65536 count=128 oseek=4880000
> dd if=/dev/zero of=/dev/ada4 bs=65536 count=128
> dd if=/dev/zero of=/dev/ada4 bs=65536 count=128 oseek=4880000
> dd if=/dev/zero of=/dev/ada5 bs=65536 count=128
> dd if=/dev/zero of=/dev/ada5 bs=65536 count=128 oseek=4880000
>
> On the commands with "oseek", you will probably receive an error that
> says something about "short end of device" or what not; that is
> perfectly fine.
>
> After this, I strongly recommend rebooting the system ("reboot")
> and booting off of your standard hard disks. Your system should
> start in multi-user, but obviously there will be no ZFS pools.
>
> At this point, recreate your ZFS pools as you did originally, and you
> should be good to go going forward.
>
Thank you very much Jeremy.
Yes you are right, the machine was a Windows home Server and I changed
it to BSD :-)
In that process I rearranged the disks because they where set up in a
strange way, and YES they where already in a RAID configuration when I
reused and reconnected the disks to have a more "correct" setup.
The machine has been working for more that two years but when I started
upgrading the Motherboard battery was out and that of course reset the
BIOS. I do not remember how I configured the disks then (which I will
document now so that I'll not make the same mistake once more)
Luckily the system RAID is working so I can try to fix the problem as
you describe.
I might even get new and larger disks in the same time, and maybe even a
genuine SATA III controller.
/Leslie
More information about the freebsd-fs
mailing list