problem with geli and LSI controller

[Dewayne Geraghty]

On 20/07/2015 5:56 PM, Marc UBM Bocklet wrote:
> On Sun, 19 Jul 2015 09:16:51 -0700
> John-Mark Gurney <jmg at funkthat.com> wrote:
>
>> Marc UBM via freebsd-stable wrote this message on Sun, Jul 19, 2015 at 17:34 +0200:
>>> a few weeks ago our Highpoint Rocket Raid controller (hptrr) started
>>> biting the dust (spurious channel resets). We bought a LSI 9201-16i
>>> (mps) to replace it. Connected to the hptrr were 4 external e-sata
>>> enclosures, configured in JBOD mode. Together with two disks connected
>>> to the onboard SATA controller, this formed a geli encrypted raidz-2
>>> zpool. 
>>> Just now, I connected the disks to the mps controller. They show
>>> up fine in dmesg. The problem is, when trying to attach the disks
>>> formerly connected to the hptrr controller, geli is unable to find the
>>> metadata on the disks and errors out with:
>>>
>>> "geli: Cannot read metadata from /dev/da4: Invalid argument"
>>>
>>> gpart show says "gpart: No such geom: da4"
>>>
>>> Trying to restore the geli metadata gives:
>>> "geli: Provider size mismatch: wrong backup file?"
>>>
>>> Is it possible that the hptrr controller handled the disks in some
>>> special way and it's only possible to read them there?
>> This sounds like the drives were in raid0 mode, and not raw disk
>> mode...  You might be able to recover the disk w/ geli resize,
>> assuming only space was added at the end, not at the begining, but
>> I have never personally tried that myself...  I'd recommend trying
>> on a copy of the drive so you don't loose data if that is possible..
>>
>> You can also try to find the old geli label w/ a command like:
>> dd if=<disk> bs=1m iseek=<location> count=5 | strings -n 9 -td
>>
>> And get current disk size using diskinfo...
>>
>> Something like:
>> #diskinfo /dev/ada0
>> /dev/ada0       512     3000592982016   5860533168      4096    0       5814021 16      63
>> #dd if=/dev/ada0 iseek=$((3000592982016/1024/1024-1)) bs=1m 2>/dev/null | strings -n 6 -td | grep GEOM::
>> 1530880 GEOM::ELI
>> 1531392 GEOM::LABEL
>>
>> You might have to go farther back than 1 MB...
>>
>> Good luck...
> No luck so far, diskinfo gives me:
>
> #diskinfo da4
> da4     512     2000398934016   3907029168      4096    0       243201 255     63
>
> but
> #dd if=/dev/da4 iseek=$((2000398934016/1024/1024-1)) bs=1m 2>/dev/null | strings -n 6 -td | grep GEOM::
> yields nothing.
Marc,
Sometimes shell scripts calculations can be daunting.  Refer to this

$((2000398934016/1024/1024-1))

Now the following portion which converts bytes to MB

2000398934016/1024/1024
that is, dividing bytes by 1M (first by 1KB=1024 then again, means its effectively 1Meg)

and you have then subtracted 1 MB from it.  To step further back from
the end of the hard-disk, you could, for example go back 2MB

$((2000398934016/1024/1024-2))