Vimage project status
Julian Elischer
julian at elischer.org
Tue Jun 10 19:37:47 UTC 2008
Vimage is a project to add virtual networking images to FreeBSD.
This iresults in an exteded Jail functionality.
contact: julian@ zec@
1) What is vimage?
Vimage started out as an attempt to virtualise the network stack to
the point where 'jails' could run with completely separate network
stacks. It was originally available in FreeBSD 4.x and considerable
work and thought has been dome by Marko to rewrite it to cope with
FreeBSD 7/8.
Improvements include:
* Ability to compile it OUT. This allows an integration period with
much reduced risk.
* Ability to copy with loadable kernel modules. Modules can be loaded
individually and still be virtualised. The technique for this was
inspired by the TLS (thread local Storage) techniuque, but we do not
use linker/compiler support. (It would be cute however)
* Fully hierarchical vimage jails.
* 'wormholes' that allow (with (real)admin support) virtual machines
to communicate with each other with high efficiency.
* General purpose framework to allow virtualisation of modules other
than networking components.
It was discussed in the last Dev-summit extensively and is approaching
the tree on an intercept course.. Collision in about 1 week for first
commits and 8 weeks for final functional merge. It is expected that
the tree will be fully functional at all points..
At the same time work is underway to make sure that jail improvements
and vimage play nicely together.
Note, that not all the changes in the development branch will be
integrated into -current immediately. For example changes to allow
per-jail load averages to be kept will not be committed in this pass.
The Milestones included something like:
#1) June 8 (today) Headsup....
#2) June 15 Commit changes that add macros for vnet
(network module) and vinet(inet virtualisation)
with macros defined in such a way to make 0 actual
differences. provable by md5 etc.
Documentat
s/hostname/g//V_hostname/
#define V_hostname hostname
2 weeks settle time, next step prepared, tested
and reviewed.
A script that did 99% of this work is available
(see commit messages below):
#3) June 29 Add changes to convert all globals to members of
per-module structures. Done in a reversible way
(i.e. compilable out). Macros defined so that
depending on compile options structures or globals
are used (one global structure).
Performance implications of using structures are
evaluated. Structures possibly tuned.
Initialisation routines added, checked and tuned.
example:
#if VIMAGE_USE_STRUCTS
#define V_hostname sys_globals.hostname
...
#else
#define V_hostname hostname
...
#endif
#4) July 6 Some Macros added at this point with no resulting
generated code changes in preparation for later stages.
These patches are not strictly needed for #5 yet but
adding them here massively reduces the clutter in the
later diffs allowing the "meat" of the later changes to
be seen and understood better.
#5) July 13 Globals removed in vnet, vinet.
ifdefs and compile option removed or scaled back
to make code clean to read again.
Destructor routines added where needed.
Remaining "NULL Macros" (compile to nothing at this
point) committed to reduce the size of the
MEAT diffs. Review of Meat diffs formally under way
for final comment.
example:
#define INIT_VNET_INET(x) /* nothing */
add "INIT_VNET_INET(curvnet);"(and similar)
where needed.
remove globals (e.g. 'hostname')
#6) July 21 JAIL+Vimage framework committed.
e.g. add new syscall, program, etc.
(part one of meat diffs) structures still only
global instances. vimage inhansed jails can be created
but act jus tlike normal jails?
#7) July 28 Ability to created > 1 vimage enabled.
Vimage enhanced jails now have private network
stacks etc.
#8+) August Start on converting more modules as needed and time
allows.
Marko and I have been working towards splitting up the current diffs
(which do the whole thing) so allow this schedule to be followed.
We may or may not be ready for the June 15 step by then, but if not
it may be a week there-after. So this should be considered the
heads-up. discussion will be on freebsd-virtualization@
and the perforce branch that we have as a current working system
is branch 'vimage'. //depot/projects/vimage/...
diffs for -current can be found at:
http://www.freebsd.org/~julian/vimage.diff and it are usually
fairly up to date.
=================================================================
commit message for stage #2 script.
This script is in TCL specifically to annoy perl and python
users. They are welcome to submit a perl/python/ruby/sh version :-)
=================================================================
Change 143245 by zec at zec_tpx32 on 2008/06/10 17:55:27
Introduce a script for automated renaming of global variables.
The script searches for references of variable names in .c and
.h files in the srcdir hierarchy, and prepends them with a "V_"
prefix. The script also creates or updates a "vimage.h" file
in the current directory containing macros which resolve
V_ instances back to their original counterparts (e.g.
V_rt_tables -> rt_tables).
Best effort is made not to change any variable declarations nor
field names in structure declarations and / or C-style comments.
The script attempts to avoid changing structure names if they
happen to have the same name as any of the requested variables
(e.g. ifnet).
So far the only manual intervention that has to be done on
resulting sources is placement of #include <sys/vimage.h> lines
in .c files for which the script couldn't figure out
automatically that they require this #include.
The resulting diff is ~550K in size, roughly half of the
current delta between HEAD and projects/vimage in p4. The
kernel and modules compile cleanly, as does the buildworld.
Note that the script is _slow_ when operating over the entire
sys tree.
Affected files ...
.. //depot/projects/vimage/var_rename.tcl#1 add
Differences ...
--------------------------------------------------------------------------
http://perforce.freebsd.org/chv.cgi?CH=143246
Change 143246 by zec at zec_tpx32 on 2008/06/10 17:57:06
Add a list of global variables which can be used as input
to the var_rename.tcl script.
Affected files ...
.. //depot/projects/vimage/vimage_globals#1 add
--------------------------------------------------------------------------
http://perforce.freebsd.org/chv.cgi?CH=143250
Change 143250 by zec at zec_tpx32 on 2008/06/10 18:33:50
Add two diffs:
autorename.diff is a result of running var_rename.tcl over
-CURRENT tree, using vimage_globals variable list as input.
manualadjust.diff is a patch that needs to be applied
after autorename.diff in order for the sys tree to compile
cleanly.
Affected files ...
.. //depot/projects/vimage/misc/autorename.diff#1 add
.. //depot/projects/vimage/misc/manualadjust.diff#1 add
Differences ...
--------------------------------------------------------------------------
There are a number of perforce branches in play:
//depot/projects/vimage/
vimage running in -current.. development/debug tree
//depot/user/zec/vimage_7/
"stable" version of vimage integrated into a FreeBSD 7 tree.
works and seems stable. if you want to test drive it,
this may be the tree to use as it doesn't have to cope with
the shifting sands of -current.
//depot/projects/vimage-commit2/
Integration tree for developing commit for the next stage
(currently stage 2)
//depot/projects/vimage-commit/
Integration tree for generating various integration diffs
etc.
More information about the freebsd-wip-status
mailing list