Jail hot migration

[Julian Fagir]

Hi,

correct me if I'm wrong, I never actually worked with VMware (only a bit
with Xen) but read a bit about the theory behind it. I don't think this can
be as easilly achieved as it is with VMware (or Xen).

> I was lately thinking around jail hot-migration feature where one jail
> could be moved from one host to another without
> shutting it down, something like vmotion in VMware world.
With paravirtualisation, you have the machine in a 'container', the host
system does not know except for the interfaces what the guest system is doing
with the ram it has reserved.
So moving a machine is 'not more' than copying a bunch of memory to another
machine and transferring the interfaces.

Jails are no virtualisation, in the end, what makes up a jail, are only a few
data structures.
Moving a jail would mean to transfer all these single fragments of data
reasonably to another machine. I.e. you would have to transfer all the
kernel-things a process does to another machine. How sould locks or access
to system ressources be managed when transferring to another machine?
I don't even know if the kernel does know all the stuff about a process on a
central place to do this transfer.

> The storage layer should be easy with zfs send and receive or some form
> of shared storage. The tricky part would
> be a memory copy from one node to another and also the CPU handling.
Afaik you have containers for the machines with VMware/Xen, as the host
system does not have to know the file system underneath. Again,
transferring/sharing those is simple as it is only one file to the host
system.
With jails, you only have mount points. What happens to locks and other
fs-specific stuff, ACLs etc (ok, zfs perhaps handles that)? But you can't rely
on the other system having the same filesystem underneath. Also, sharing file
systems is not as easy as sharing a single file (without caring for
permissions). You see the problem when you just look at all the different
ways for locking over the network...

> Anyone has an idea how this could be achieved? I guess it would require
> a kernel module which could take care of memory
> reservations and a daemon to copy and incrementally sync the jails
> memory across.
In case you could build a jail-transportation, a framework for transferring a
single process to another machine would be more appropriate imho as more
generic and simpler. Then just adding the possibility of jailing such a
process is not much work.

A single example why the whole target cannot be achieved easily: Imagine a
program which writes its pid into a lockfile. As jails and host share the
pids, this pid could be already in use by the other host you want to transfer
the jail to.
So what to do know? Just change the pid? Then the process would be confused
and possibly die or malfunction. Not changing the pid would not be possible
as then there would be two processes with the same pid.
Sure, remapping pids would be possible but not within the current jails and
the complications could be great (if the process sees another pid than the
host-system, what will rc-files do which check lockfiles?).
And there are other cases where processes do things the kernel cannot
predict and so cannot transfer that easily.

Also, the nic-transfer would make problems. When you usually have a bridge or
so with Xen/VMware, you have real interfaces or aliases on a BSD-machine, how
should they be taken out of service in time and back to service without
problems (I often have cases when just taking up an alias does not work)?


Jails are no virtualisation. They are an 'extended chroot'-environment and
thus moving a jail is like moving a working machine except for the system
drivers.


I hope this was not all too wrong.

Regards, Julian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-jail/attachments/20100910/ff784cd8/signature.pgp