VIMAGE

Fri May 31 08:53:27 UTC 2013

On May 30, 2013, at 6:38 PM, "Teske, Devin" <Devin.Teske at fisglobal.com> wrote:

> 
> On May 30, 2013, at 9:18 AM, Joe wrote:
> 
>> Pietro Paolini wrote:
>>> On May 30, 2013, at 2:29 PM, Joe <fbsd8 at a1poweruser.com> wrote:
>>>> Pietro Paolini wrote:
>>>>> Hello all,
>>>>> I am a new bye on the FreeBSD and I am looking at the VIMAGE features experiencing some problems.
>>>>> I added the options :
>>>>> VIMAGE
>>>>> if_bridge
>>>>> and I removed
>>>>> STCP
>>>>> then I recompiled my kernel and install it.
>>>>> After that, following this tutorial http://imunes.tel.fer.hr/virtnet/eurobsdcon07_tutorial.pdf I tried the "Exercise 2" which consist on the following commands:
>>>>> vimage -c n1
>>>>> vimage -c n2
>>>>> ngctl mkpeer efface ether ether
>>>>> ngctl mkpeer efface ether ether
>>>>> ngctl mkpeer em0: bridge lower link0
>>>>> ngctl name em0:lower bridge0
>>>>> ngctl connect em0: bridge0: upper link1
>>>>> ngctl connect ngeth0: bridge0: ether link2
>>>>> ngctl connect ngeth1: bridge0: ether link3
>>>>> vimage -i n1 ngeth0 e0
>>>>> But my virtual interface on the n1 vimage does not receive any packet from the external network while I can see the packet go out from it.
>>>>> For instance using DHCP, e0 on n1 sends DHCP packets but it does not receive the answers (which are send, I verified it from wireshark), in adding
>>>>> the ARP request for his IP address (if I try to add it statically) are not received then it can not answer.
>>>>> At the end of the line the question is: how can I make this "virtual network" and the external real network be able to communicate ?
>>>>> Thanks in advance.
>>>>> Pietro.
>>>> 1. That link is from 2007. So very much has changed since then.
>>>> There are more current links on the internet about this subject. Most are for 8.X releases.
>>>> 
>>>> 2. If your running 8.2-RELEASE or 9.1-RELEASE all you need to add is "options vimage" statement to your kernel source and recompile.
>>>> 
>>>> 3. There are 2 networking methods available for creating vnet/vimage jail networks, if_bridge/epair and netgraph. The if_bridge/epair method is far simpler to config and use then the netgraph method.
>>>> 
>>>> 4. There are 2 methods of jail setup, the rc.d method where your jail definition parameters go into the hosts rc.conf and the jail(8) method where you can place each jails definition parameter in separate files.
>>>> 
>>>> 5. There are two very important show stopper PRs on vimage,
>>>> 164763 memory leak and 149050 the rc.d keyword "nojail" problem.
>>>> Vimage is a very long way from prime time usage, thats why it's labeled as highly experimental. Host system freezes and page faults are common.
>>>> 
>>>> 6. When it comes to running a firewall in a vnet/vimage jail your limited to IPFW and it has limitations. Dummynet and in kernel NAT cause system freezes. IPFILTER causes page fault at boot time. PF will run on the host but not run in the vnet/vimage jail. Here are a bunch of PRs on vimage firewall problems, 143621, 176092, 161094, 176992, 143808, 148155, 165252, 178480, 178482
>>>> 
>>>> 
>>>> Check out these links
>>>> 
>>>> http://druidbsd.sourceforge.net/vimage.shtml
>>>> http://devinteske.com/vimage-jails-on-freebsd-8
>>>> http://lists.freebsd.org/pipermail/freebsd-virtualization/2011-September/000747.html
>>>> 
>>>> http://bsdbased.com/2009/12/06/freebsd-8-vimage-epair-howto
>>>> http://zewaren.net/site/?q=node/78
>>>> 
>>>> 
>>>> _______________________________________________
>>>> freebsd-questions at freebsd.org mailing list
>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-questions
>>>> To unsubscribe, send any mail to "freebsd-questions-unsubscribe at freebsd.org"
>>> Thanks so much, really interesting and good links but I can't again accomplish my task - I followed http://bsdbased.com/2009/12/06/freebsd-8-vimage-epair-howto as a base tutorial - I would like simulate more client for my PC using this technology, and that's my topology:
>>> 1 - Two epairs created 2 - Two jail created
>>> 3 - Assign one intf for jail
>>> 4 - Add the real interface to a bridge where I put all the interfaces
>>> -------
>>> JAIL1
>>> 0b
>>> -------
>>> -------
>>> JAIL2          1b
>>> -------
>>> -------
>>> HOST
>>> 0a
>>> 1a
>>> em0 --> REAL
>>> -------
>>> Where {0,1}a is the first pair and {0,1}b is the second and em0 is my real interface (it has an IP address), then I got to the problem, I tried to :
>>> jexec 2 dhclient epair1b  And I can see the DHCP packet with the correct MAC address going out, the server reply (I have a sniffer pc) but the transaction does not end successfully, what is really strange is that if I attach tcpdump on em0 I can NOT see the answer server sends while when I try
>>> dhclient em0 I can see the packet going in and out and the DHCP transaction finish successfully. Do you have any idea about how can I accomplish my target ? Maybe I am using the wrong technology ?
>>> I would not surprised if I make a error on my configuration but what really interest me is if I CAN do that using jail.
>>> Thanks a lot, and in advance ! :-)
>>> Pietro.
>> 
>> 
>> Pietro;
>> 
>> You really have to provide a lot more info about your host system and jail configuration.
>> 
>> What version of FreeBSD are you running?
>> 
>> Which method do you use to create your jails, rc.d/rc.conf or jail(8).
>> 
>> How are your creating the if_bridge/epair network, IE entering commands by hand or using a script? Post the script.
>> 
>> Can you post the output of ifconfig command after you have your jail network created?
>> 
>> How are you handling the "rc.d nojail keyword" problem?
>> 
>> Have you manually assigned a private LAN ip address and default route to the epairXb interface inside of the vnet jail?
>> 
>> DHCP will not work from inside of the vnet jail. Your ISP only assigns a single dymamic IP address per each account. You already used your assigned ip address for your host. If the computer your running the vnet/vimage jail on is on a local area network and the gateway host is running a DHCP server to dynamically assign private ip address to computers on the LAN, then DHCP in the vnet/vimage jail may work.
>> 
> 
> I imagine that most vimage solutions do not do what mine does… (http://druidbsd.sourceforge.net/download.shtml#vimage)
> 
> Which is to actually give each bridged interface a unique MAC address.
> 
> This makes each vimage truly appear as a separate host. This makes DHCP from a vimage possible because the DHCP server sees discretely unique systems, through-and-through.
> 
> 
> 
>> To the broader question, why do you think you need a vnet/vimage jail in the first place. You wrote "simulate more client for my PC", If client means hosting paying clients then you sure don't want to be using vimage because it highly experimental and NOT reliable.
>> 
> 
> The reports of vimages unreliability must be coupled with the feature-set you desire.
> 
> I say this because yesterday afternoon… we shipped our first-ever production platform to a customer -- based entirely on FreeBSD 8.1 + vimage + zfs + sysutils/zxfer + multipath + nullfs + nfs. No single point of failure.
> 
> We ran vimage in the lab for about 2.5 years before arriving at the point that we were ready to put it into production. (NOTE: FreeBSD-8.1 is about 36 months old).
> 
> So to say that it's not reliable enough for production… it depends on your definition of production. From the FreeBSD Foundation side… I'd say that this sounds about correct (because "production" means to the FreeBSD Foundation… able to work with all GENERIC features). However, in our case "production" means… able to work with the features *we* need.
> 
> The features VIMAGE doesn't work with today are PF and a few other things (I saw Joe post in a different reply that "nooptions SCTP" may not be required anymore… so progress seems to be afoot).
> 
> But with respect to stability and reliability… if you can get them to run… they're solid…
> 
> Just don't do any of the following:
> 
> 1. Expose /dev/mem to the jail via a ruleset
> 
> 2. Expose /dev/kmem (same problem as #1 … you could find yourself in a kernel panic)
> 
> NOTE: Both of the above are self-inflicted gunshot wounds. These devices are not exposed by default (and the only reason to do so is to try to get things like Xorg or "netstat -nr" to work within the vimage; side-note: "route -n get default" works).
> 
> Oh… and then there's the issue that when you stop a vimage, some KVA pages are supposedly lost.
> 
> Well… in true production, we don't make it a habit of stopping vimages (once they are up, they stay up). Usually the only reason for a vimage to go down is if the base machine goes down or you need to change the mounts… in either case, the KVA pages will be refreshed on a reboot of the base machine… so it's really not an issue to the way we run production.
> -- 
> Devin
> 
> 
>> Jails are a security feature that encloses a process in a container. By process I mean "postfix email server" or "apache web server". If your "client" means "processes", then this is what the non-vnet/vimage jail is for.
>> 
>> _______________________________________________
>> freebsd-questions at freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-questions
>> To unsubscribe, send any mail to "freebsd-questions-unsubscribe at freebsd.org"
> 
> _____________
> The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you.

I am running a 9.1-RELEASE with VIMAGE option enabled, I don't see any problem with my DHCP server because I am in a custom env then DHCP is running on a machine than I configured.

I used both ways

http://imunes.tel.fer.hr/virtnet/eurobsdcon07_tutorial.pdf 

or

http://bsdbased.com/2009/12/06/freebsd-8-vimage-epair-howto

All I wish is to connect my external network on the real ethernet interface to my virtual network stack (jail, visage, as you wish call it).