From sfourman at gmail.com Wed Jul 2 08:14:45 2008 From: sfourman at gmail.com (Sam Fourman Jr.) Date: Wed Jul 2 08:23:13 2008 Subject: Vimage headsup.. revised. In-Reply-To: <20080618202212.GF46885@obiwan.tataz.chchile.org> References: <484CC690.9020303@elischer.org> <48536C9A.8020801@elischer.org> <20080618151911.GB46885@obiwan.tataz.chchile.org> <485935FF.8040308@elischer.org> <20080618192903.O83875@maildrop.int.zabbadoz.net> <20080618202212.GF46885@obiwan.tataz.chchile.org> Message-ID: <11167f520807020046n3a4008a4m2bfc0a72e69b6d91@mail.gmail.com> Does anyone Know if vimage works with pf? Sam Fourman Jr. From julian at elischer.org Wed Jul 2 16:28:00 2008 From: julian at elischer.org (Julian Elischer) Date: Wed Jul 2 16:28:04 2008 Subject: Vimage headsup.. revised. In-Reply-To: <11167f520807020046n3a4008a4m2bfc0a72e69b6d91@mail.gmail.com> References: <484CC690.9020303@elischer.org> <48536C9A.8020801@elischer.org> <20080618151911.GB46885@obiwan.tataz.chchile.org> <485935FF.8040308@elischer.org> <20080618192903.O83875@maildrop.int.zabbadoz.net> <20080618202212.GF46885@obiwan.tataz.chchile.org> <11167f520807020046n3a4008a4m2bfc0a72e69b6d91@mail.gmail.com> Message-ID: <486BACA4.7020809@elischer.org> Sam Fourman Jr. wrote: > Does anyone Know if vimage works with pf? > > Sam Fourman Jr. > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" It did, then pf got updated. we have not redone the port of the new pf. From julian at elischer.org Thu Jul 3 20:51:36 2008 From: julian at elischer.org (Julian Elischer) Date: Thu Jul 3 20:51:40 2008 Subject: vimage (and vimage-devel) branches In-Reply-To: <200806192147.22285.zec@freebsd.org> References: <485AADB0.8080006@elischer.org> <200806192147.22285.zec@freebsd.org> Message-ID: <486D3BEC.1070404@elischer.org> Marko Zec wrote: >> >> Currently they are compiling GENERIC fine, and VIMAGE >> ok, but failing to compile LINT > > LINT compiles fine on i386 / vimage-commit2, but not on other branches. > > Marko > >> LINT now compiled for me in the vimage-commit2 and vimage-commi3 branches. VIMAGE compiles for me in the vimage branch. I have not run it yet. From julian at elischer.org Sun Jul 6 08:10:01 2008 From: julian at elischer.org (Julian Elischer) Date: Sun Jul 6 08:10:08 2008 Subject: porting to vimage document In-Reply-To: <9a542da30807050419q5b2fd68fj3229b199f0032def@mail.gmail.com> References: <4863F1B0.9060603@elischer.org> <4863F509.2010408@elischer.org> <4863F5A3.9060703@elischer.org> <9a542da30806261543x667875eej8a5b6fa9da773b12@mail.gmail.com> <486443D1.40302@elischer.org> <9a542da30807050419q5b2fd68fj3229b199f0032def@mail.gmail.com> Message-ID: <48707DD7.9080905@elischer.org> I've started on a document about porting to vimage. http://perforce.freebsd.org/fileViewer.cgi?FSPC=//depot/projects/vimage/porting%5fto%5fvimage.txt It's not finished by a long way but I'm off to bed now.. Marko, feel free to add to this.. I'll take up work on thi sagain tomorrow if I can.. Julian From julian at elischer.org Sun Jul 6 16:54:13 2008 From: julian at elischer.org (Julian Elischer) Date: Sun Jul 6 16:54:21 2008 Subject: porting to vimage document In-Reply-To: <48707DD7.9080905@elischer.org> References: <4863F1B0.9060603@elischer.org> <4863F509.2010408@elischer.org> <4863F5A3.9060703@elischer.org> <9a542da30806261543x667875eej8a5b6fa9da773b12@mail.gmail.com> <486443D1.40302@elischer.org> <9a542da30807050419q5b2fd68fj3229b199f0032def@mail.gmail.com> <48707DD7.9080905@elischer.org> Message-ID: <4870F8B4.2060203@elischer.org> Julian Elischer wrote: > > I've started on a document about porting to vimage. > > http://perforce.freebsd.org/fileViewer.cgi?FSPC=//depot/projects/vimage/porting%5fto%5fvimage.txt > > > It's not finished by a long way but I'm off to bed now.. > > Marko, feel free to add to this.. I'll take up work on this again > tomorrow if I can.. (some more added) > > Julian > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscribe@freebsd.org" From jamie at gritton.org Wed Jul 9 21:02:39 2008 From: jamie at gritton.org (James Gritton) Date: Wed Jul 9 21:02:46 2008 Subject: jail_set (pretty much) done Message-ID: <48752768.9080106@gritton.org> The name-based FreeBSD Jail extensions are now ready for use. I've added support to the user-space programs (jail, jls, jexec), and added some fluff to the relevant man pages. This still stands alone - I haven't yet integrated it with Vimage, which is the next step. But it's there for anyone who wants to take a look at it on its own. I've put diffs against Current in http://gritton.org/jail_set.diff, and I have a perforce branch in //depot/user/jamie/jail_set/. In a nutshell, this adds three system calls: jail_get, jail_set, and jail_remove. Jail_remove is a no-brainer. The other two allow jails to be created and existing jails to be modified in a manner similar to the nmount system call. The user-level jail and jls programs work with these system calls to set/get an arbitrary set of name-based jail parameters. The system has a certain set of parameters, key among them the members of original jail structure, and modules are free to add their own parameters. The linux OS info has been changed to use this setup as a demonstration (and because it's just better that way). Hierarchical jails are supported (though turned off by default). It's all in the updated jail(2) and jail(8) man pages. - Jamie From jamie at gritton.org Fri Jul 11 16:18:27 2008 From: jamie at gritton.org (James Gritton) Date: Fri Jul 11 16:18:33 2008 Subject: Simpler Vimage sysctls Message-ID: <487787CC.6020302@gritton.org> While working on combining jail_set and Vimage, I found that the sysctl virtualization hacks were more complicated than they needed to be. The extra "subs" and "mod" arguments in SYSCTL_HANDLER_V_ARGS don't need to be explicitly passed because they're members of the sysctl_v_oid structure passed in the oidp argument. By using oidp->oid_v_subs instead of subs (and same for mod), SYSCTL_HANDLER_V_ARGS becomes the same as SYSCTL_HANDLER_ARGS, and no longer need to be defined. With the handlers now taking the same arguments, the sysctl_oid and sysctl_v_oid structures become identical and sysctl_v_oid can go away. Unrelated to this is the various SYSCTL_V_XXX macros that refer to either SYSCTL_V_OID or SYSCTL_OID depending on the VIMAGE define. Since SYSCTL_V_OID already reduces to SYSCTL_OID if VIMAGE is undefined, those further switches are unnecessary. I'm including a diff that trims all this away, while keeping the same functionality. - Jamie -------------- next part -------------- diff -r -u ov/src/sys/kern/kern_sysctl.c src/sys/kern/kern_sysctl.c --- ov/src/sys/kern/kern_sysctl.c Wed Jul 9 14:14:02 2008 +++ src/sys/kern/kern_sysctl.c Fri Jul 11 00:53:50 2008 @@ -849,7 +849,7 @@ #ifdef VIMAGE int -sysctl_handle_v_int(SYSCTL_HANDLER_V_ARGS) +sysctl_handle_v_int(SYSCTL_HANDLER_ARGS) { int tmpout, error = 0; @@ -1009,7 +1009,7 @@ #ifdef VIMAGE int -sysctl_handle_v_string(SYSCTL_HANDLER_V_ARGS) +sysctl_handle_v_string(SYSCTL_HANDLER_ARGS) { int error=0; char *tmparg; @@ -1088,7 +1088,7 @@ #ifdef VIMAGE int -sysctl_handle_v_opaque(SYSCTL_HANDLER_V_ARGS) +sysctl_handle_v_opaque(SYSCTL_HANDLER_ARGS) { int error, tries; u_int generation; @@ -1421,17 +1421,7 @@ if (error != 0) return (error); #endif -#ifndef VIMAGE error = oid->oid_handler(oid, arg1, arg2, req); -#else - if (oid->oid_v_subs) { - struct sysctl_v_oid *v_oid = (struct sysctl_v_oid *) oid; - error = v_oid->oid_handler(oid, arg1, arg2, - req, oid->oid_v_subs, - oid->oid_v_mod); - } else - error = oid->oid_handler(oid, arg1, arg2, req); -#endif return (error); } diff -r -u ov/src/sys/netinet/in_pcb.c src/sys/netinet/in_pcb.c --- ov/src/sys/netinet/in_pcb.c Wed Jul 9 14:14:48 2008 +++ src/sys/netinet/in_pcb.c Fri Jul 11 00:27:08 2008 @@ -121,11 +121,7 @@ else if ((var) > (max)) { (var) = (max); } static int -#ifndef VIMAGE sysctl_net_ipport_check(SYSCTL_HANDLER_ARGS) -#else -sysctl_net_ipport_check(SYSCTL_HANDLER_V_ARGS) -#endif { #ifdef VIMAGE INIT_VNET_INET(curvnet); diff -r -u ov/src/sys/netinet/ip_fw2.c src/sys/netinet/ip_fw2.c --- ov/src/sys/netinet/ip_fw2.c Wed Jul 9 14:14:49 2008 +++ src/sys/netinet/ip_fw2.c Fri Jul 11 00:27:29 2008 @@ -157,11 +157,7 @@ static int autoinc_step; #endif -#ifdef VIMAGE -extern int ipfw_chg_hook(SYSCTL_HANDLER_V_ARGS); -#else extern int ipfw_chg_hook(SYSCTL_HANDLER_ARGS); -#endif #ifdef SYSCTL_NODE SYSCTL_NODE(_net_inet_ip, OID_AUTO, fw, CTLFLAG_RW, 0, "Firewall"); diff -r -u ov/src/sys/netinet/ip_fw_pfil.c src/sys/netinet/ip_fw_pfil.c --- ov/src/sys/netinet/ip_fw_pfil.c Wed Jul 9 14:14:49 2008 +++ src/sys/netinet/ip_fw_pfil.c Fri Jul 11 00:27:54 2008 @@ -75,11 +75,7 @@ # endif #endif -#ifdef VIMAGE -int ipfw_chg_hook(SYSCTL_HANDLER_V_ARGS); -#else int ipfw_chg_hook(SYSCTL_HANDLER_ARGS); -#endif /* Dummynet hooks. */ ip_dn_ruledel_t *ip_dn_ruledel_ptr = NULL; @@ -493,11 +489,7 @@ #endif /* INET6 */ int -#ifdef VIMAGE -ipfw_chg_hook(SYSCTL_HANDLER_V_ARGS) -#else ipfw_chg_hook(SYSCTL_HANDLER_ARGS) -#endif { #ifdef VIMAGE INIT_VNET_IPFW(curvnet); diff -r -u ov/src/sys/netinet6/in6_proto.c src/sys/netinet6/in6_proto.c --- ov/src/sys/netinet6/in6_proto.c Wed Jul 9 14:14:55 2008 +++ src/sys/netinet6/in6_proto.c Fri Jul 11 00:28:18 2008 @@ -450,11 +450,7 @@ /* net.inet6.ip6 */ static int -#ifdef VIMAGE -sysctl_ip6_temppltime(SYSCTL_HANDLER_V_ARGS) -#else sysctl_ip6_temppltime(SYSCTL_HANDLER_ARGS) -#endif { INIT_VNET_INET6(curvnet); #ifdef VIMAGE @@ -477,11 +473,7 @@ } static int -#ifdef VIMAGE -sysctl_ip6_tempvltime(SYSCTL_HANDLER_V_ARGS) -#else sysctl_ip6_tempvltime(SYSCTL_HANDLER_ARGS) -#endif { INIT_VNET_INET6(curvnet); #ifdef VIMAGE diff -r -u ov/src/sys/sys/sysctl.h src/sys/sys/sysctl.h --- ov/src/sys/sys/sysctl.h Wed Jul 9 14:15:24 2008 +++ src/sys/sys/sysctl.h Fri Jul 11 00:53:23 2008 @@ -115,9 +115,6 @@ #define SYSCTL_HANDLER_ARGS struct sysctl_oid *oidp, void *arg1, int arg2, \ struct sysctl_req *req -#define SYSCTL_HANDLER_V_ARGS struct sysctl_oid *oidp, void *arg1, int arg2, \ - struct sysctl_req *req, int subs, int mod - /* definitions for sysctl_req 'lock' member */ #define REQ_UNLOCKED 0 /* not locked and not wired */ #define REQ_LOCKED 1 /* locked and not wired */ @@ -169,22 +166,6 @@ short oid_v_mod; }; -struct sysctl_v_oid { - struct sysctl_oid_list *oid_parent; - SLIST_ENTRY(sysctl_oid) oid_link; - int oid_number; - u_int oid_kind; - void *oid_arg1; - int oid_arg2; - const char *oid_name; - int (*oid_handler)(SYSCTL_HANDLER_V_ARGS); - const char *oid_fmt; - int oid_refcnt; - const char *oid_descr; - short oid_v_subs; - short oid_v_mod; -}; - #define SYSCTL_IN(r, p, l) (r->newfunc)(r, p, l) #define SYSCTL_OUT(r, p, l) (r->oldfunc)(r, p, l) @@ -196,9 +177,9 @@ int sysctl_handle_string(SYSCTL_HANDLER_ARGS); int sysctl_handle_opaque(SYSCTL_HANDLER_ARGS); -int sysctl_handle_v_int(SYSCTL_HANDLER_V_ARGS); -int sysctl_handle_v_string(SYSCTL_HANDLER_V_ARGS); -int sysctl_handle_v_opaque(SYSCTL_HANDLER_V_ARGS); +int sysctl_handle_v_int(SYSCTL_HANDLER_ARGS); +int sysctl_handle_v_string(SYSCTL_HANDLER_ARGS); +int sysctl_handle_v_opaque(SYSCTL_HANDLER_ARGS); /* * These functions are used to add/remove an oid from the mib. @@ -247,7 +228,7 @@ #ifdef VIMAGE #define SYSCTL_V_OID(subs, mod, parent, nbr, name, kind, a1, a2, \ handler, fmt, descr) \ - static struct sysctl_v_oid sysctl__##parent##_##name = { \ + static struct sysctl_oid sysctl__##parent##_##name = { \ &sysctl_##parent##_children, { 0 }, nbr, kind, \ (void *) offsetof(struct mod, _##a1), a2, #name, \ handler, fmt, 0, __DESCR(descr), subs, V_MOD_##mod }; \ @@ -277,15 +258,9 @@ SYSCTL_OID(parent, nbr, name, CTLTYPE_STRING|(access), \ arg, len, sysctl_handle_string, "A", descr) -#ifdef VIMAGE #define SYSCTL_V_STRING(subs, mod, parent, nbr, name, access, sym, len, descr) \ SYSCTL_V_OID(subs, mod, parent, nbr, name, CTLTYPE_STRING|(access), \ sym, len, sysctl_handle_v_string, "A", descr) -#else -#define SYSCTL_V_STRING(subs, mod, parent, nbr, name, access, sym, len, descr) \ - SYSCTL_OID(parent, nbr, name, CTLTYPE_STRING|(access), \ - &sym, len, sysctl_handle_string, "A", descr) -#endif #define SYSCTL_ADD_STRING(ctx, parent, nbr, name, access, arg, len, descr) \ sysctl_add_oid(ctx, parent, nbr, name, CTLTYPE_STRING|(access), \ @@ -296,15 +271,9 @@ SYSCTL_OID(parent, nbr, name, CTLTYPE_INT|(access), \ ptr, val, sysctl_handle_int, "I", descr) -#ifdef VIMAGE #define SYSCTL_V_INT(subs, mod, parent, nbr, name, access, sym, val, descr) \ SYSCTL_V_OID(subs, mod, parent, nbr, name, CTLTYPE_INT|(access), \ sym, val, sysctl_handle_v_int, "I", descr) -#else -#define SYSCTL_V_INT(subs, mod, parent, nbr, name, access, sym, val, descr) \ - SYSCTL_OID(parent, nbr, name, CTLTYPE_INT|(access), \ - &sym, val, sysctl_handle_int, "I", descr) -#endif #define SYSCTL_ADD_INT(ctx, parent, nbr, name, access, ptr, val, descr) \ sysctl_add_oid(ctx, parent, nbr, name, CTLTYPE_INT|(access), \ @@ -368,19 +337,11 @@ ptr, sizeof(struct type), sysctl_handle_opaque, \ "S," #type, descr) -#ifdef VIMAGE #define SYSCTL_V_STRUCT(subs, mod, parent, nbr, name, access, sym, \ type, descr) \ SYSCTL_V_OID(subs, mod, parent, nbr, name, CTLTYPE_OPAQUE|(access), \ sym, sizeof(struct type), sysctl_handle_v_opaque, \ "S," #type, descr) -#else -#define SYSCTL_V_STRUCT(subs, mod, parent, nbr, name, access, sym, \ - type, descr) \ - SYSCTL_OID(parent, nbr, name, CTLTYPE_OPAQUE|(access), \ - &sym, sizeof(struct type), sysctl_handle_opaque, \ - "S," #type, descr) -#endif #define SYSCTL_ADD_STRUCT(ctx, parent, nbr, name, access, ptr, type, descr) \ sysctl_add_oid(ctx, parent, nbr, name, CTLTYPE_OPAQUE|(access), \ @@ -391,17 +352,10 @@ SYSCTL_OID(parent, nbr, name, (access), \ ptr, arg, handler, fmt, descr) -#ifdef VIMAGE #define SYSCTL_V_PROC(subs, mod, parent, nbr, name, access, sym, arg, \ handler, fmt, descr) \ SYSCTL_V_OID(subs, mod, parent, nbr, name, (access), \ sym, arg, handler, fmt, descr) -#else -#define SYSCTL_V_PROC(subs, mod, parent, nbr, name, access, sym, arg, \ - handler, fmt, descr) \ - SYSCTL_OID(parent, nbr, name, (access), \ - &sym, arg, handler, fmt, descr) -#endif #define SYSCTL_ADD_PROC(ctx, parent, nbr, name, access, ptr, arg, handler, fmt, descr) \ sysctl_add_oid(ctx, parent, nbr, name, (access), \ @@ -420,9 +374,10 @@ #ifdef VIMAGE #define SYSCTL_RESOLVE_V_ARG1() do { \ char *cp; \ - switch (subs) { \ + switch (oidp->oid_v_subs) { \ case V_NET: \ - cp = (char *) TD_TO_VNET(curthread)->mod_data[mod]; \ + cp = (char *) \ + TD_TO_VNET(curthread)->mod_data[oidp->oid_v_mod]; \ break; \ case V_PROCG: \ cp = (char *) TD_TO_VPROCG(curthread); \ @@ -431,7 +386,7 @@ cp = (char *) TD_TO_VCPU(curthread); \ break; \ default: \ - panic("unsupported module id %d", subs); \ + panic("unsupported module id %d", oidp->oid_v_subs); \ } \ arg1 = cp + (size_t) arg1; \ } while (0) From jamie at gritton.org Fri Jul 11 20:24:09 2008 From: jamie at gritton.org (James Gritton) Date: Fri Jul 11 20:24:15 2008 Subject: Simpler Vimage sysctls In-Reply-To: <487787CC.6020302@gritton.org> References: <487787CC.6020302@gritton.org> Message-ID: <4877C161.3080400@gritton.org> OK, forget that last part about the SYSCTL_V_XXX macros not needing to be redefined for VIMAGE - I didn't notice that they link to the sysctl_handle_v_xxx functions. - Jamie James Gritton wrote: > While working on combining jail_set and Vimage, I found that the > sysctl virtualization hacks were more complicated than they needed to be. > > The extra "subs" and "mod" arguments in SYSCTL_HANDLER_V_ARGS don't > need to be explicitly passed because they're members of the > sysctl_v_oid structure passed in the oidp argument. By using > oidp->oid_v_subs instead of subs (and same for mod), > SYSCTL_HANDLER_V_ARGS becomes the same as SYSCTL_HANDLER_ARGS, and no > longer need to be defined. > > With the handlers now taking the same arguments, the sysctl_oid and > sysctl_v_oid structures become identical and sysctl_v_oid can go away. > > Unrelated to this is the various SYSCTL_V_XXX macros that refer to > either SYSCTL_V_OID or SYSCTL_OID depending on the VIMAGE define. > Since SYSCTL_V_OID already reduces to SYSCTL_OID if VIMAGE is > undefined, those further switches are unnecessary. > > I'm including a diff that trims all this away, while keeping the same > functionality. From julian at elischer.org Sat Jul 12 01:55:26 2008 From: julian at elischer.org (Julian Elischer) Date: Sat Jul 12 01:55:33 2008 Subject: Simpler Vimage sysctls In-Reply-To: <487787CC.6020302@gritton.org> References: <487787CC.6020302@gritton.org> Message-ID: <48780EF3.6030802@elischer.org> James Gritton wrote: > While working on combining jail_set and Vimage, I found that the sysctl > virtualization hacks were more complicated than they needed to be. > > The extra "subs" and "mod" arguments in SYSCTL_HANDLER_V_ARGS don't need > to be explicitly passed because they're members of the sysctl_v_oid > structure passed in the oidp argument. By using oidp->oid_v_subs > instead of subs (and same for mod), SYSCTL_HANDLER_V_ARGS becomes the > same as SYSCTL_HANDLER_ARGS, and no longer need to be defined. I've only looked at the sysctl macros supreficially but assumed that the redundant information was forward looking, and that there was support for future changes there. > > With the handlers now taking the same arguments, the sysctl_oid and > sysctl_v_oid structures become identical and sysctl_v_oid can go away. > > Unrelated to this is the various SYSCTL_V_XXX macros that refer to > either SYSCTL_V_OID or SYSCTL_OID depending on the VIMAGE define. Since > SYSCTL_V_OID already reduces to SYSCTL_OID if VIMAGE is undefined, those > further switches are unnecessary. umm but they call different code.. > > I'm including a diff that trims all this away, while keeping the same > functionality. > > - Jamie > > > ------------------------------------------------------------------------ > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" From julian at elischer.org Sat Jul 12 07:53:33 2008 From: julian at elischer.org (Julian Elischer) Date: Sat Jul 12 07:53:43 2008 Subject: Simpler Vimage sysctls In-Reply-To: <487787CC.6020302@gritton.org> References: <487787CC.6020302@gritton.org> Message-ID: <487862DB.1040904@elischer.org> James Gritton wrote: > While working on combining jail_set and Vimage, I found that the sysctl > virtualization hacks were more complicated than they needed to be. > > The extra "subs" and "mod" arguments in SYSCTL_HANDLER_V_ARGS don't need > to be explicitly passed because they're members of the sysctl_v_oid > structure passed in the oidp argument. By using oidp->oid_v_subs > instead of subs (and same for mod), SYSCTL_HANDLER_V_ARGS becomes the > same as SYSCTL_HANDLER_ARGS, and no longer need to be defined. > > With the handlers now taking the same arguments, the sysctl_oid and > sysctl_v_oid structures become identical and sysctl_v_oid can go away. looking at this I see what you are saying, and it looks odd that there is a short oid_v_subs; short oid_v_mod; even in the non VIMAGE case.. and in the non vimage struct. I do wonder if Marko started moving towards this already and got stopped for some reason. Marko? From bzeeb-lists at lists.zabbadoz.net Sat Jul 12 16:55:06 2008 From: bzeeb-lists at lists.zabbadoz.net (Bjoern A. Zeeb) Date: Sat Jul 12 16:55:18 2008 Subject: vimage(8) cleanup, what about submitting and to where? Message-ID: <20080712165025.N57089@maildrop.int.zabbadoz.net> Hi, I am not sure about the plans for vimage(8) but it lives in usr.sbin and installs itself into /sbin. the .include is not last in the Makefile, etc. In case this will go in with any of the steps could you please fix it in at least //depot/projects/vimage/... I got a kernel compiled and booted today from that branch. Last I heard was that I should wait with any cleanup/corrections/style patches. Has that changed? What is the current state and what can I do if I'd like to have changes? I am asking because someone has to keep the 4(or more?) branches in synch that you have already created. /bz -- Bjoern A. Zeeb Stop bit received. Insert coin for new game. From julian at elischer.org Sat Jul 12 20:27:50 2008 From: julian at elischer.org (Julian Elischer) Date: Sat Jul 12 20:27:57 2008 Subject: vimage(8) cleanup, what about submitting and to where? In-Reply-To: <20080712165025.N57089@maildrop.int.zabbadoz.net> References: <20080712165025.N57089@maildrop.int.zabbadoz.net> Message-ID: <48791396.2010404@elischer.org> Bjoern A. Zeeb wrote: > Hi, > > I am not sure about the plans for vimage(8) but it lives in usr.sbin > and installs itself into /sbin. the .include is not last in the > Makefile, etc. > > In case this will go in with any of the steps could you please fix it > in at least //depot/projects/vimage/... > > I got a kernel compiled and booted today from that branch. > > Last I heard was that I should wait with any cleanup/corrections/style > patches. Has that changed? What is the current state and what can I > do if I'd like to have changes? I am asking because someone has to > keep the 4(or more?) branches in synch that you have already created. > > /bz > Sorry, I have been busy with the birth of my son as you may have guessed, and Marko has been inundated by $WORK for a while. Marko has been committing a lot of style fixes to the p4 branch "vimage" and I have been integrating those changes across to the 'vimage-devel" branch. if you have p4 acccess you should feel free to commit any "obvious" fixes yourself and just post a summary to this mailing list. bigger changes we can discuss here. For example the changes suggested by James Gritton.. From julian at elischer.org Mon Jul 14 07:34:24 2008 From: julian at elischer.org (Julian Elischer) Date: Mon Jul 14 07:34:30 2008 Subject: doc on porting code to Vimage slight updates. Message-ID: <487B0129.5070000@elischer.org> The file visible from http://perforce.freebsd.org/fileDownLoad.cgi?FSPC=//depot/projects/vimage/porting%5fto%5fvimage.txt has been updated to clarify a few points. more changes will come. From jamie at gritton.org Mon Jul 14 22:52:52 2008 From: jamie at gritton.org (James Gritton) Date: Mon Jul 14 22:52:59 2008 Subject: IFNET_WLOCK missing from if_reassign_common Message-ID: <487BD8BE.1040609@gritton.org> In testing jail_set_vimage, I found that moving a network interface cause a assertion failure in ifnet_setbyindex. It turns out that if_reassign_common in kern_vimage.c should be locking IFNET_WLOCK. I'm including a patch that locks it in the same way it's done in if_alloc (which seems to be the inspiration for much of this code). - Jamie -------------- next part -------------- --- ov/src/sys/kern/kern_vimage.c Wed Jul 9 14:14:03 2008 +++ src/sys/kern/kern_vimage.c Mon Jul 14 16:44:18 2008 @@ -283,10 +283,12 @@ do { INIT_VNET_NET(curvnet); + IFNET_WLOCK(); ifnet_setbyindex(ifp->if_index, NULL); /* XXX: should be locked with if_findindex() */ while (V_if_index > 0 && ifnet_byindex(V_if_index) == NULL) V_if_index--; + IFNET_WUNLOCK(); } while (0); CURVNET_SET_QUIET(new_vnet); @@ -309,7 +311,9 @@ V_if_index = ifp->if_index; if (V_if_index >= V_if_indexlim) if_grow(); + IFNET_WLOCK(); ifnet_setbyindex(ifp->if_index, ifp); + IFNET_WUNLOCK(); /* Rename the ifnet */ if (new_vnet == ifp->if_home_vnet) { From julian at elischer.org Mon Jul 14 23:10:27 2008 From: julian at elischer.org (Julian Elischer) Date: Mon Jul 14 23:10:33 2008 Subject: IFNET_WLOCK missing from if_reassign_common In-Reply-To: <487BD8BE.1040609@gritton.org> References: <487BD8BE.1040609@gritton.org> Message-ID: <487BDC7A.8040606@elischer.org> James Gritton wrote: > In testing jail_set_vimage, I found that moving a network interface > cause a assertion failure in ifnet_setbyindex. It turns out that > if_reassign_common in kern_vimage.c should be locking IFNET_WLOCK. I'm > including a patch that locks it in the same way it's done in if_alloc > (which seems to be the inspiration for much of this code). > > - Jamie cool, Jamie, do you want (write) access to the actual vimage tree in p4? > > > ------------------------------------------------------------------------ > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" From jamie at gritton.org Mon Jul 14 23:13:23 2008 From: jamie at gritton.org (James Gritton) Date: Mon Jul 14 23:13:35 2008 Subject: IFNET_WLOCK missing from if_reassign_common In-Reply-To: <487BDC7A.8040606@elischer.org> References: <487BD8BE.1040609@gritton.org> <487BDC7A.8040606@elischer.org> Message-ID: <487BDD8A.9060300@gritton.org> Sure - hopefully the soon to be announced jail_set changes for vimage will be pronounced acceptable, and then I can merge them in. - Jamie Julian Elischer wrote: > James Gritton wrote: >> In testing jail_set_vimage, I found that moving a network interface >> cause a assertion failure in ifnet_setbyindex. It turns out that >> if_reassign_common in kern_vimage.c should be locking IFNET_WLOCK. >> I'm including a patch that locks it in the same way it's done in >> if_alloc (which seems to be the inspiration for much of this code). >> >> - Jamie > > cool, > > Jamie, do you want (write) access to the actual vimage tree in p4? From jamie at gritton.org Mon Jul 14 23:46:23 2008 From: jamie at gritton.org (James Gritton) Date: Mon Jul 14 23:46:30 2008 Subject: jail_set_vimage - Vimage under new jails Message-ID: <487BE548.3050500@gritton.org> I've finished the merge of jail_set and Vimage. This uses the name-based jails instead of the jail-similar vimage frameworks, with Vimage's VNET stuff being enabled in a jail with the "vnet" parameter (in this scenario, it's optional whether a jail has its own network stack or just inherits its parent's). Once such a jail is set up, it behaves in the same way as a vimage does, as far as the network stack separation goes. The only difference is in administration, which uses the jail framework. In addition to the main changes of moving vnet from struct vimage to a prison service, some related changes are: * Future-compat hooks for the vprocg and vcpu stuff has been removed - when such stuff is added, it would belong under the jail umbrella. This means that the three subsystems V_NET, V_PROCG, and V_CPU are reduced to one subsystem V_VNET, which actually amounts to no subsystems at all anymore. * The IMUNES_SYMLINK_HACK has gone away, though I suppose it could come back. * The V_hostname (and G_hostname and *_domainname) stuff has been removed, in favor of the way jail_set handles virtual hostnames. * The jail_set userspace changes to jail programs have been added. * The vimage program has been superseded by the vifmove program. It uses a struct vifmovereq, which replaces the obsolete struct vi_req. * Some other bits I mentioned (simpler sysctls and a locking fix) have found their way in. Probably also some other bits I haven't mentioned. The VNET modularization is still that way it was. While vnet has become a prison service, essentially a jail module, the network modules that plug in to vnet know nothing of the jail situation, and remain VNET modules. The vnet pointers still live in interfaces, sockets, threads, wherever they used to be. The places that had vimage pointers now have prison pointers, but there weren't very many of those. This is in the perforce tree //depot/user/jamie/jail_set_vimage, and a patch is at http://gritton.org/jail_set_vimage.diff. This is my vision of the future direction of Vimage, and of course I hope it becomes "the" vision. In other words: Marko and Julian, give it a try and let me know what you think. - Jamie From julian at elischer.org Tue Jul 15 00:12:59 2008 From: julian at elischer.org (Julian Elischer) Date: Tue Jul 15 00:13:05 2008 Subject: jail_set_vimage - Vimage under new jails In-Reply-To: <487BE548.3050500@gritton.org> References: <487BE548.3050500@gritton.org> Message-ID: <487BEB21.6040407@elischer.org> James Gritton wrote: > I've finished the merge of jail_set and Vimage. This uses the > name-based jails instead of the jail-similar vimage frameworks, with > Vimage's VNET stuff being enabled in a jail with the "vnet" parameter > (in this scenario, it's optional whether a jail has its own network > stack or just inherits its parent's). Once such a jail is set up, it > behaves in the same way as a vimage does, as far as the network stack > separation goes. The only difference is in administration, which uses > the jail framework. > > In addition to the main changes of moving vnet from struct vimage to a > prison service, some related changes are: > > * Future-compat hooks for the vprocg and vcpu stuff has been removed - > when such stuff is added, it would belong under the jail umbrella. > This means that the three subsystems V_NET, V_PROCG, and V_CPU are > reduced to one subsystem V_VNET, which actually amounts to no > subsystems at all anymore. > * The IMUNES_SYMLINK_HACK has gone away, though I suppose it could > come back. > * The V_hostname (and G_hostname and *_domainname) stuff has been > removed, in favor of the way jail_set handles virtual hostnames. > * The jail_set userspace changes to jail programs have been added. > * The vimage program has been superseded by the vifmove program. It > uses a struct vifmovereq, which replaces the obsolete struct vi_req. > * Some other bits I mentioned (simpler sysctls and a locking fix) have > found their way in. Probably also some other bits I haven't > mentioned. > > The VNET modularization is still that way it was. While vnet has > become a prison service, essentially a jail module, the network > modules that plug in to vnet know nothing of the jail situation, and > remain VNET modules. The vnet pointers still live in interfaces, > sockets, threads, wherever they used to be. The places that had > vimage pointers now have prison pointers, but there weren't very many > of those. > > This is in the perforce tree //depot/user/jamie/jail_set_vimage, and a > patch is at http://gritton.org/jail_set_vimage.diff. > > This is my vision of the future direction of Vimage, and of course I hope > it becomes "the" vision. In other words: Marko and Julian, give it a try > and let me know what you think. This is cool stuff.. You have no idea how good it is to have other people looking at the Vimage code with fresh eyes and thoughts. Vimage importation was delayed for a number of reasons, some technical as people brought up issues, but ONE of them was I saw this on the side and after thinking about our talk at BSDCan I thought it would be better to see what came from it. (Also because both Marko and I ran out of hours for awhile while $LIFE intervened in one way or another. It was pointed out at BSDCan that "BSD Jails" is a kind of "unofficial trade name" that BSD has that is well known and respected and that keeping The "Jail" name maybe with "new and improved, now with 'VNET' support for whiter whites" might be a smart move from the PR point of view. One question I have is to do with Jails in general. There are a lot of other patches floating around with jails features. How many of those patches are going to be incorporated? Julian > > - Jamie > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscribe@freebsd.org" From julian at elischer.org Tue Jul 15 00:15:04 2008 From: julian at elischer.org (Julian Elischer) Date: Tue Jul 15 00:15:11 2008 Subject: jail_set_vimage - Vimage under new jails In-Reply-To: <487BEB21.6040407@elischer.org> References: <487BE548.3050500@gritton.org> <487BEB21.6040407@elischer.org> Message-ID: <487BEB9F.3000502@elischer.org> Julian Elischer wrote: > James Gritton wrote: >> I've finished the merge of jail_set and Vimage. This uses the >> name-based jails instead of the jail-similar vimage frameworks, with >> Vimage's VNET stuff being enabled in a jail with the "vnet" parameter >> (in this scenario, it's optional whether a jail has its own network >> stack or just inherits its parent's). Once such a jail is set up, it >> behaves in the same way as a vimage does, as far as the network stack >> separation goes. The only difference is in administration, which uses >> the jail framework. I liked the hierarchical feature of the vimage system. when you say "name based", do you mean the code you refer to is not hierarchical? >> >> In addition to the main changes of moving vnet from struct vimage to a >> prison service, some related changes are: >> >> * Future-compat hooks for the vprocg and vcpu stuff has been removed - >> when such stuff is added, it would belong under the jail umbrella. >> This means that the three subsystems V_NET, V_PROCG, and V_CPU are >> reduced to one subsystem V_VNET, which actually amounts to no >> subsystems at all anymore. >> * The IMUNES_SYMLINK_HACK has gone away, though I suppose it could >> come back. >> * The V_hostname (and G_hostname and *_domainname) stuff has been >> removed, in favor of the way jail_set handles virtual hostnames. >> * The jail_set userspace changes to jail programs have been added. >> * The vimage program has been superseded by the vifmove program. It >> uses a struct vifmovereq, which replaces the obsolete struct vi_req. >> * Some other bits I mentioned (simpler sysctls and a locking fix) have >> found their way in. Probably also some other bits I haven't >> mentioned. >> >> The VNET modularization is still that way it was. While vnet has >> become a prison service, essentially a jail module, the network >> modules that plug in to vnet know nothing of the jail situation, and >> remain VNET modules. The vnet pointers still live in interfaces, >> sockets, threads, wherever they used to be. The places that had >> vimage pointers now have prison pointers, but there weren't very many >> of those. >> >> This is in the perforce tree //depot/user/jamie/jail_set_vimage, and a >> patch is at http://gritton.org/jail_set_vimage.diff. >> >> This is my vision of the future direction of Vimage, and of course I hope >> it becomes "the" vision. In other words: Marko and Julian, give it a try >> and let me know what you think. > > This is cool stuff.. > > You have no idea how good it is to have other people looking at > the Vimage code with fresh eyes and thoughts. > > Vimage importation was delayed for a number of reasons, some technical > as people brought up issues, but ONE of them was I saw this on the > side and after thinking about our talk at BSDCan I thought it would > be better to see what came from it. (Also because both Marko and > I ran out of hours for awhile while $LIFE intervened in one way > or another. > > It was pointed out at BSDCan that "BSD Jails" is a kind of > "unofficial trade name" that BSD has that is well known and > respected and that keeping The "Jail" name maybe with > "new and improved, now with 'VNET' support for whiter whites" > might be a smart move from the PR point of view. > > One question I have is to do with Jails in general. > There are a lot of other patches floating around with > jails features. How many of those patches are going to be > incorporated? > > Julian > > > > > > >> >> - Jamie >> _______________________________________________ >> freebsd-virtualization@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >> To unsubscribe, send any mail to >> "freebsd-virtualization-unsubscribe@freebsd.org" > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscribe@freebsd.org" From jamie at gritton.org Tue Jul 15 00:22:42 2008 From: jamie at gritton.org (James Gritton) Date: Tue Jul 15 00:22:54 2008 Subject: jail_set_vimage - Vimage under new jails In-Reply-To: <487BEB9F.3000502@elischer.org> References: <487BE548.3050500@gritton.org> <487BEB21.6040407@elischer.org> <487BEB9F.3000502@elischer.org> Message-ID: <487BEDCA.8090705@gritton.org> These jails are hierarchical. The "named-based" I refer to is the extensibility, where new named parameters can be set in the jail_set system call, rather than relying on a fixed structure. Perhaps I should just say "extensible" instead. - Jamie Julian Elischer wrote: >> James Gritton wrote: >> I've finished the merge of jail_set and Vimage. This uses the >> name-based jails instead of the jail-similar vimage frameworks, with >> Vimage's VNET stuff being enabled in a jail with the "vnet" parameter >> (in this scenario, it's optional whether a jail has its own network >> stack or just inherits its parent's). Once such a jail is set up, it >> behaves in the same way as a vimage does, as far as the network stack >> separation goes. The only difference is in administration, which uses >> the jail framework. > > I liked the hierarchical feature of the vimage system. > when you say "name based", do you mean the code you refer to is > not hierarchical? From jamie at gritton.org Tue Jul 15 01:39:25 2008 From: jamie at gritton.org (James Gritton) Date: Tue Jul 15 01:39:32 2008 Subject: jail_set_vimage - Vimage under new jails In-Reply-To: <487BEB21.6040407@elischer.org> References: <487BE548.3050500@gritton.org> <487BEB21.6040407@elischer.org> Message-ID: <487BFFCA.4090105@gritton.org> I'd like to have "jail_set versions" of a lot of what's out there - certainly what's being currently worked on. My next target is Bjoern Zeeb's multi-ip/no-ip/ipv6 extension (actually, I've got no-ip already). But I don't want to turn it all into one grand jail_set patch without buy-in, as I don't know which patches have what kind of support. Some things like vimage work well as prison services, using the provided jail layering and remaining their own separate things. But most of the jail patches are extensions to the base jail stuff itself, and would make more sense to just add to the existing jail code. Some of these changes re-appear in different forms in different patches. - Jamie Julian Elischer wrote: > One question I have is to do with Jails in general. > There are a lot of other patches floating around with > jails features. How many of those patches are going to be > incorporated? From bzeeb-lists at lists.zabbadoz.net Tue Jul 15 11:35:08 2008 From: bzeeb-lists at lists.zabbadoz.net (Bjoern A. Zeeb) Date: Tue Jul 15 11:35:15 2008 Subject: jail_set_vimage - Vimage under new jails In-Reply-To: <487BFFCA.4090105@gritton.org> References: <487BE548.3050500@gritton.org> <487BEB21.6040407@elischer.org> <487BFFCA.4090105@gritton.org> Message-ID: <20080715111109.K57089@maildrop.int.zabbadoz.net> On Mon, 14 Jul 2008, James Gritton wrote: Hi, > Julian Elischer wrote: >> One question I have is to do with Jails in general. >> There are a lot of other patches floating around with >> jails features. How many of those patches are going to be >> incorporated? I am aware of one, mostly to be able to MFC it to 7. For more see http://wiki.freebsd.org/Jails which is a good overview but also has to be read with a pinch of salt for some items. > I'd like to have "jail_set versions" of a lot of what's out there - certainly > what's being currently worked on. My next target is Bjoern Zeeb's > multi-ip/no-ip/ipv6 extension (actually, I've got no-ip already). But I > don't want to turn it all into one grand jail_set patch without buy-in, as I > don't know which patches have what kind of support. I'll reply to your other private mail on how we can do that later today. > Some things like vimage work well as prison services, using the provided jail > layering and remaining their own separate things. But most of the jail Remaining separate things sounds very good to me; I was a bit worried before that too tight integration might happen. > patches are extensions to the base jail stuff itself, and would make more > sense to just add to the existing jail code. Some of these changes re-appear > in different forms in different patches. For prison services, there are possible bugs in the current implementation as is once there will be more than one serivce. Pawel said he had a rewrite (or something) in p4 already with his ZFS work. /bz -- Bjoern A. Zeeb Stop bit received. Insert coin for new game. From zec at icir.org Wed Jul 16 11:09:58 2008 From: zec at icir.org (Marko Zec) Date: Wed Jul 16 11:10:04 2008 Subject: Simpler Vimage sysctls In-Reply-To: <487787CC.6020302@gritton.org> References: <487787CC.6020302@gritton.org> Message-ID: <200807161309.49269.zec@icir.org> On Friday 11 July 2008 18:18:20 James Gritton wrote: > While working on combining jail_set and Vimage, I found that the > sysctl virtualization hacks were more complicated than they needed to > be. > > The extra "subs" and "mod" arguments in SYSCTL_HANDLER_V_ARGS don't > need to be explicitly passed because they're members of the > sysctl_v_oid structure passed in the oidp argument. By using > oidp->oid_v_subs instead of subs (and same for mod), > SYSCTL_HANDLER_V_ARGS becomes the same as SYSCTL_HANDLER_ARGS, and no > longer need to be defined. > > With the handlers now taking the same arguments, the sysctl_oid and > sysctl_v_oid structures become identical and sysctl_v_oid can go > away. > > Unrelated to this is the various SYSCTL_V_XXX macros that refer to > either SYSCTL_V_OID or SYSCTL_OID depending on the VIMAGE define. > Since SYSCTL_V_OID already reduces to SYSCTL_OID if VIMAGE is > undefined, those further switches are unnecessary. > > I'm including a diff that trims all this away, while keeping the same > functionality. > > - Jamie Good catch, thanks! I just submitted a slightly modified version of your patch to p4/vimage branch, which allows for this to work with both options VIMAGE and nooptions VIMAGE configurations -> SYSCTL_V_* macros need to automatically fall back to their "plain" SYSCTL_* counterparts for nooptions VIMAGE builds. In retrospect, I really cannot recall why I introduced this SYSCTL_HANDLER_V_ARGS special casing in the first place, besides perhaps having it as an explicit reminder that in functions acting as handlers for virtualized objects, it is required to look up and dereference the address of such object in the appropriate resource container structure, which can / must be accomplished via SYSCTL_RESOLVE_V_ARG1() macro. Thanks, Marko