using gmirror and zfs mirror on the same box -- thoughts?

[Frank Leonhardt]

On 17/08/2017 23:48, Shamim Shahriar wrote:
> Good evening all, hope everyone is well.
>
> I have a strange requirement for a particular system that will sit at 
> a remote location. I intend to use mirror, but at the same time 
> encrypt the system. Boot time encryption is not an option -- I need 
> the system to boot up normally (with network and ssh running, so I can 
> do the rest remotely) and do not wish to risk the normal bootup due to 
> some issues with either geli or other matters (fsck after a power out 
> comes to mind). I would like to have the OS part mirrored as well the 
> data part. As for the data part -- I definitely wish to use zfs with 
> encryption. Encrypting OS is not necessary (but if can be done safely, 
> ideas are welcome)
>
> Now, I can use multiple zpool, but then all of them will try to be 
> active/functional when the machine boots. If I intend to encrypt the 
> data pool (geli), then it needs to wait until the encryption part is 
> taken care of.
>
> So, I am thinking (probably in a very wrong way, corrections welcome), 
> if I get the OS part gmirror-ed, then that comes up with the OS, I 
> have network and ssh to get into the system, and then manually run the 
> encryption and zfs part.
>
> The system has 8GB RAM, which I am assuming should be good enough for 
> geli, gmirror and zfs parts.
>
> If anyone has any better suggestion/scenerio to share, that is greatly 
> welcome. If you think this might actually be disfunctional, please 
> share your thoughts on that (preferably with explanation as to why 
> this is a bad idea). if you have any suggestion that you think is a 
> much better option, please do feel free to share. 

Hi Shamim,

This sounds like a very good idea to me. I often go for mixed systems; 
boot off geom mirrored UFS drives and use ZFS for storage. At one time 
you had to boot from UFS, and it's only been simple to boot from ZFS 
since 10.0 (IIRC).

Although you can boot from a complex raidz array it has problems. For 
example, when you swap a failed drive you don't get the boot code back 
unless you put it there. And there's also more to go wrong (HBA, SAS 
expander and so on). If you boot from a pair of SATA drives directly 
connected to the motherboard it's just more likely to work.

And the final "good thing" about booting from a mirrored UFS is that you 
can use the drive for faster database storage - eli a partition for this 
if needed.

Booting from a geom mirror seems to have broken since 11 - you might 
want to read this:

http://blog.frankleonhardt.com/2017/zfs-is-not-always-the-answer-bring-back-gmirror/

Regards, Frank.