EBS snapshot backups from a FreeBSD zfs file system: zpool freeze?

[Jeremy Chadwick]

On Tue, Jul 09, 2013 at 10:25:55AM +1200, Berend de Boer wrote:
> >>>>> "Jeremy" == Jeremy Chadwick <jdc at koitsu.org> writes:
> 
>     Jeremy> Therefore, the answer/solution for you at this stage seems
>     Jeremy> to be: use Linux.  Linux does what you need -- it offers
>     Jeremy> you guest-level utilities that interface with the
>     Jeremy> proprietary storage system back-end (EBS) that is offered
>     Jeremy> by your choice of hosting vendor (Amazon).
> 
> Ding dong, nothing to do with guest level utilities. Completely
> irrelevant, and I've repeated that numerous times here. All Amazon's
> guest level tools work perfectly fine on FreeBSD. Amazon has ZERO
> Linux/Windows/FreeBSD/whatever specific stuff.

Then please explain this comment:

> On Tue, Jul 09, 2013 at 08:06:25AM +1200, Berend de Boer wrote:
> > >>>>> "Markus" == Markus Gebert <markus.gebert at hostpoint.ch> writes:
> >     Markus> What you really need in that case is the ability to
> >     Markus> snapshot all EBS disks as group.
> >
> > Which Linux offers.

Because what you're saying here, vs. what you said above, is highly
conflicting as I read it.  I do not understand how you can say "Amazon's
guest-level tools work on FreeBSD", then immediately say "Amazon has
zero OS-specific stuff", *while* saying "Linux offers {said thing I
want}".  These statements are extremely confusing.

I think it would help if you would really start to provide *actual
commands* of what you're doing to accomplish the tasks you want to
accomplish -- both as you do them now on Linux, as well as the
ZFS-related stuff you've done on FreeBSD.  To my knowledge, no where in
this thread have you actually shown scrollback/proof/etc. of what you've
been doing, just magical one-liners that seem strange (for example, the
"empty disk" thing would imply that EBS isn't even doing storage caching
correctly, which makes no sense and cannot be true else *nothing* would
work!).

It might turn out to be that what you're seeing can be traced down to
user error, or it can be traced back to the controller driver (on
FreeBSD) not behaving how you want (might require SCSI quirks, etc.),
etc...  The possibilities are endless, and without hard data, I think
everyone is struggling/throwing their arms up in the air.

The reason I mention storage controller drivers is, for example, on ESXi
(I forget what versions) it was discovered that cache flushing (the OS's
way of telling the storage driver, which then tells the disks, "make
sure everything is written to the platters") was a no-op of sorts --
ESXi would say "yep I've done it" when in fact it hadn't yet.

With virtualisation this is a common situation, **solely because** of
all the abstraction and caching (at so many layers that it's almost
unfathomable -- no joke).  I put EBS into the same category.  Because
really, all you know ***TRULY*** is "I see disk daX using controller
xyzX", and the rest you have to pray/hope works (from the OS driver
level all the way down to the physical media this stuff is written to on
the back end / within EBS).  The more abstraction = the more chances
something will not behave how bare metal expects.

>     Jeremy> There is absolutely no shame in any way/shape/form in
>     Jeremy> using an OS that meets your needs/requirements.
> 
> And that's why I'm trying to use FreeBSD as it has features that Linux
> doesn't have. Trust me, if Linux had everything FreeBSD has, why would
> I use FreeBSD? It's even more expensive on AWS!

I'm glad you're trying it, but if you're at the stage where you're
saying "Linux can do X/Y/Z", then it's up to **you** to weigh the pros
and cons of moving away from Linux.  Nobody here can make that decision
for you, whether it be based on technical need or money or whatever
else.

>     Jeremy> EBS is a closed, black-box-proprietary thing.
> 
> Ding dong, it's just block storage like every other block storage out
> there. Sigh.

You once again generalise.  Every kind of storage back-end is different,
I don't know why you have such trouble understanding this.  You're
quite literally telling me that EBS is the same as iSCSI is the same as
an ESXi file-based backing store is the same as a physical disk
(remember your words: "disks are just software"), which is completely
and entirely wrong -- their behaviours differ severely, not just as
solutions/methods, but in *how they behave* when submit with I/O
requests.

All that "black magic" that goes on under the hood?  It matters.  Every
single bit of it.

> I fail to see that mentioning something Linux can do on this mailing
> list would upset you? You don't want to know or read that??

I am one of the most "anti-FreeBSD" people you will ever encounter -- go
ahead, ask anyone on this list, you will find that I am the one who is
usually quick to take my "well then FreeBSD needs to get it's sh**
together on a server level" soapbox.  And that doesn't come lightly
given that I've been using/working professionally with FreeBSD since the
2.2.x days.  Prior to that I ran Linux (0.99pl45 until early 1.3.x).
I'm just trying to give you some perspective into how I am, if it
matters.

What upsets me is that you're saying "Linux has XYZ", in turn creating a
"FreeBSD vs. Linux" conflict, when I personally strive to see that kind
of "OS war" advocacy end.  I want to see people use whatever
tool/OS/thing solves their dilemmas.  If that thing is Linux, cool.  If
it's FreeBSD, cool.  If it's Windows, cool.  The dilemmas, limitations,
and needs/etc. vary per person, per org, per company.

I am against all forms of OS advocacy.  I prefer to keep an open mind.
So while I give you two thumbs up for giving FreeBSD a try, and ZFS too,
but if Linux does something that you can't get on FreeBSD, then to me it
seems like the choice is obvious.  But that's me -- I love
oversimplifying.  ;-)

-- 
| Jeremy Chadwick                                   jdc at koitsu.org |
| UNIX Systems Administrator                http://jdc.koitsu.org/ |
| Making life hard for others since 1977.             PGP 4BD6C0CB |