Corrupt GPT header on disk from twa array - fixable?

Sun Jun 2 22:07:48 UTC 2013

On Jun 2, 2013, at 17:48, Jeremy Chadwick <jdc at koitsu.org> wrote:

> On Sun, Jun 02, 2013 at 05:12:48PM +0200, Alban Hertroys wrote:
>> 
>> On Jun 2, 2013, at 16:46, Warren Block <wblock at wonkity.com> wrote:
>> 
>>> On Sun, 2 Jun 2013, Alban Hertroys wrote:
>>> 
>>>> On Jun 2, 2013, at 16:12, Kimmo Paasiala <kpaasial at gmail.com> wrote:
>>>>> 
>>>>> Looking at the gpart(8) output it seems that only 20GBs of the disk is
>>>>> recognized by the disk driver but the GPT table still shows the full
>>>>> capacity 910GB. I'd say that the GPT table is in fact correct and if
>>>>> you can somehow get the disks to be recognized with full capacity they
>>>>> should be usable as they are. What does dmesg(8) say about the disks?
>>>> 
>>>> From dmesg:
>>>> 
>>>> ada2 at ahcich2 bus 0 scbus2 target 0 lun 0
>>>> ada2: usb_alloc_device: set address 2 failed (USB_ERR_IOERROR, ignored)
>>>> <ST3500418AS CC34> ATA-8 SATA 2.x device
>>>> usbd_setup_device_desc: getting device descriptor at addr 2 failed, USB_ERR_IOERROR
>>>> ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
>>>> ada2: Command Queueing enabled
>>>> ada2: 476940MB (976773168 512 byte sectors: 16H 63S/T 16383C)
>>>> ada2: Previously was known as ad8
>>>> ada3 at ahcich3 bus 0 scbus3 target 0 lun 0
>>>> ada3: <ST3500418AS CC34> ATA-8 SATA 2.x device
>>>> ada3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
>>>> ada3: Command Queueing enabled
>>>> ada3: 476940MB (976773168 512 byte sectors: 16H 63S/T 16383C)
>>>> ada3: Previously was known as ad10
>>>> ada4 at ahcich4 bus 0 scbus4 target 0 lun 0
>>>> ada4: <Hitachi HDS721010CLA332 JP4OA39C> ATA-8 SATA 2.x device
>>>> usbd_req_re_enumerate: addr=2, set address failed! (USB_ERR_IOERROR, ignored)
>>>> ada4: 300.000MB/s transfers (SATA 2.x, usbd_setup_device_desc: getting device descriptor at addr 2 failed, USB_ERR_IOERROR
>>>> UDMA6, PIO 8192bytes)
>>>> ada4: Command Queueing enabled
>>>> ada4: 953869MB (1953525168 512 byte sectors: 16H 63S/T 16383C)
>>>> ada4: Previously was known as ad12
>>>> ada5 at ahcich5 bus 0 scbus5 target 0 lun 0
>>>> ada5: <WDC WD1002FAEX-00Z3A0 05.01D05> ATA-8 SATA 3.x device
>>>> ada5: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
>>>> ada5: Command Queueing enabled
>>>> ada5: 953869MB (1953525168 512 byte sectors: 16H 63S/T 16383C)
>>>> ada5: Previously was known as ad14
>>>> SMP: AP CPU #1 Launched!
>>>> Timecounter "TSC-low" frequency 13371081 Hz quality 800
>>>> GEOM: ada2: the secondary GPT header is not in the last LBA.
>>>> GEOM: ada3: the secondary GPT header is not in the last LBA.
>>>> GEOM_MIRROR: Device mirror/boot launched (2/2).
>>>> GEOM_MIRROR: Device mirror/swap launched (2/2).
>>>> GEOM_MIRROR: Device mirror/root launched (2/2).
>>>> GEOM: ada4: the secondary GPT header is not in the last LBA.
>>>> GEOM: ada5: the secondary GPT header is not in the last LBA.
>>> 

> I think you're missing what Warren is telling you, because you have
> multiple things going on/complexities to deal with simultaneously.
> 
> You haven't provided any details about your gmirror setup either.  All
> we know at this point:
> 
>>>> GEOM_MIRROR: Device mirror/boot launched (2/2).
>>>> GEOM_MIRROR: Device mirror/swap launched (2/2).
>>>> GEOM_MIRROR: Device mirror/root launched (2/2).
> 
> My gut feeling is ada2 and ada3 make up the mirror, and the mirror is at
> the disk level (ada2 and ada3).  I'm basing this on past evidence
> presented in the thread, and having to make assumptions.  No "gmirror
> status" output = we have to make assumptions.

The gmirror is actually on ada0+ada1, and not at all on the disks that I copied the dmesg information of. Those disks weren't in the hardware RAID array of the old server, and the gmirror isn't on the disks that were in that RAID controller. I took care to keep those separate until I can erase them and add them as "normal" disks.

> Now, what Warren is telling you: gmirror + GPT do not play well
> together.  This is a design flaw** on the part of gmirror.  If you want
> to use gmirror with disks using GPT, your only solutions are to mirror
> the partitions (adaXpX) and not the disk (adaX), which has its own set
> of caveats, or to use the MBR scheme (and if these are 4K sectors disks,
> or you plan on using those, you're even more screwed).

I didn't know that, but just incidentally mirrored my partitions on ada0 and ada1 instead of the entire disks. For the sake of removing the confusion from this thread:

# gmirror status
       Name    Status  Components
mirror/boot  COMPLETE  ada0p1 (ACTIVE)
                       ada1p1 (ACTIVE)
mirror/swap  COMPLETE  ada0p2 (ACTIVE)
                       ada1p2 (ACTIVE)
mirror/root  COMPLETE  ada0p3 (ACTIVE)
                       ada1p3 (ACTIVE)

> The errors you see on ada4 and ada5 about the backup GPT header can be
> dealt with in a different manner.
> 
> But for (again, assuming) ada2 and ada3, you will see GPT "backup header
> corruption" messages indefinitely because of the above flaw.

I think that those messages actually stem from the same issue as I'm having with the (hardware-)RAID volumes on ada4 and ada5: Apparently the RAID controller reserved some space at the end of those volumes to store information about the volume layout, very similar to how geom does that.

The geom labels that I put on the partitions inside those volumes are therefore not in the last sector of the disk, but they were in the last sector of each volume while the disks were still attached to the controller.

Those messages on ada2 & 3 don't really bother me. I can read what's on those disks the way they are (unlike the second volume on disks ada4 & 5). Once I'm confident that I don't need anything that's on them anymore, they'll be repartitioned and the problem will be gone.

Alban Hertroys
--
If you can't see the forest for the trees,
cut the trees and you'll find there is no forest.