VIMAGE
Teske, Devin
Devin.Teske at fisglobal.com
Mon Jun 3 16:31:28 UTC 2013
On Jun 3, 2013, at 6:33 AM, Pietro Paolini wrote:
>
> On Jun 1, 2013, at 5:26 AM, "Teske, Devin" <Devin.Teske at fisglobal.com> wrote:
>
>>
>> On May 31, 2013, at 3:05 PM, Teske, Devin wrote:
>>
>>>
>>> On May 31, 2013, at 1:48 AM, Pietro Paolini wrote:
>>>
>>>>
>>>> On May 30, 2013, at 6:25 PM, "Teske, Devin" <Devin.Teske at fisglobal.com> wrote:
>>>>
>>>>>
>>>>> On May 30, 2013, at 3:35 AM, Pietro Paolini wrote:
>>>>>
>>>>>> Hello all,
>>>>>>
>>>>>> I am a new bye on the FreeBSD and I am looking at the VIMAGE features experiencing some problems.
>>>>>> I added the options :
>>>>>> VIMAGE
>>>>>> if_bridge
>>>>>>
>>>>>> and I removed
>>>>>> STCP
>>>>>>
>>>>>> then I recompiled my kernel and install it.
>>>>>>
>>>>>> After that, following this tutorial https://urldefense.proofpoint.com/v1/url?u=http://imunes.tel.fer.hr/virtnet/eurobsdcon07_tutorial.pdf&k=%2FbkpAUdJWZuiTILCq%2FFnQg%3D%3D%0A&r=Mrjs6vR4%2Faj2Ns9%2FssHJjg%3D%3D%0A&m=xe0XNgnKBiT9v8HzxwWwnNMOVN3YdEHmTsIZfFoQA9Y%3D%0A&s=b271820faf310ac274ded8c3135b4931f2a5278e78ec1a66ce6a5ab971ff64f5 I tried the "Exercise 2" which consist on
>>>>>> the following commands:
>>>>>>
>>>>>> vimage -c n1
>>>>>> vimage -c n2
>>>>>> ngctl mkpeer efface ether ether
>>>>>> ngctl mkpeer efface ether ether
>>>>>
>>>>> Don't you just love autocorrect? (does the same thing to me… turns "eiface" into "efface")
>>>>>
>>>>>
>>>>>> ngctl mkpeer em0: bridge lower link0
>>>>>
>>>>> Looks good.
>>>>>
>>>>>
>>>>>> ngctl name em0:lower bridge0
>>>>>
>>>>> I usually do my "connect" before the "name"… but shouldn't matter. Should work all the same.
>>>>>
>>>>>
>>>>>> ngctl connect em0: bridge0: upper link1
>>>>>
>>>>> This looks wrong to me.
>>>>>
>>>>> I'd expect:
>>>>>
>>>>> ngctl connect em0: bridge0:lower upper link1
>>>>>
>>>>
>>>>
>>>> Many thanks for the answer Devin,
>>>> when I try to use that last command I receive:
>>>>
>>>> ngctl connect em0: bridge0:lower upper link1
>>>> ngctl: send msg: Invalid argument
>>>>
>>>> What's wrong ?
>>>>
>>>
>>> Let's start from scratch on a freshly booted box…
>>>
>>> dteske at scu0a.jbsd.vicor.com ~ $ sudo ngctl ls -l
>>> [sudo] Password:
>>> There are 4 total nodes:
>>> Name: em0 Type: ether ID: 00000002 Num hooks: 0
>>> Name: em1 Type: ether ID: 00000003 Num hooks: 0
>>> Name: ngctl1719 Type: socket ID: 00000004 Num hooks: 0
>>> Name: msk0 Type: ether ID: 00000001 Num hooks: 0
>>>
>>> Ok… we have an "ether" type node for each of our physical adapters (these are provided by ng_ether(4); you didn't have to do anything to get these nodes).
>>>
>>> We also have a single "socket" type node. This is the "ngctl" connection to the netgraph subsystem (you can learn more by reading ng_socket(4)).
>>>
>>> Here's the corresponding hardware behind em0, em1, and msk0:
>>>
>>> ===
>>>
>>> dteske at scu0a.jbsd.vicor.com ~ $ grep '\(em\|e1000phy\|mskc\?\)[[:digit:]]' /var/run/dmesg.boot
>>> mskc0: <Marvell Yukon 88E8050 Gigabit Ethernet> port 0xdc00-0xdcff mem 0xfcffc000-0xfcffffff irq 16 at device 0.0 on pci5
>>> msk0: <Marvell Technology Group Ltd. Yukon EC Id 0xb6 Rev 0x02> on mskc0
>>> msk0: Ethernet address: xx:xx:xx:xx:xx:xx
>>> miibus0: <MII bus> on msk0
>>> e1000phy0: <Marvell 88E1111 Gigabit PHY> PHY 0 on miibus0
>>> e1000phy0: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto
>>> mskc0: [ITHREAD]
>>> em0: <Intel(R) PRO/1000 Legacy Network Connection 1.0.3> port 0xec80-0xecbf mem 0xfebe0000-0xfebfffff irq 16 at device 4.0 on pci7
>>> em0: [FILTER]
>>> em0: Ethernet address: xx:xx:xx:xx:xx:xx
>>> em1: <Intel(R) PRO/1000 Legacy Network Connection 1.0.3> port 0xec00-0xec3f mem 0xfeba0000-0xfebbffff,0xfeb80000-0xfeb9ffff irq 18 at device 6.0 on pci7
>>> em1: [FILTER]
>>> em1: Ethernet address: xx:xx:xx:xx:xx:xx
>>> em0: link state changed to UP
>>>
>>> ===
>>>
>>> Next, let's make a bridge (think of it as a big software switch that we're going to hook a bunch of interfaces; created, physical, or otherwise).
>>>
>>> Since I'm doing this over an SSH connection (a mistake I made earlier today), I'm not going to touch em0 (the adapter my SSH connection is using). Creating the bridge on an actively configured PHY will knock it off the net. This is not to say you can't have an active configuration on a bridged interface… just that the creation of the bridge (something you should only do once each time you boot) will disrupt an active connection.
>>>
>>> So…
>>>
>>> dteske at scu0a.jbsd.vicor.com ~ $ sudo ngctl mkpeer em1: bridge lower link0
>>>
>>> NOTE: No output == Success.
>>>
>>> ===
>>>
>>> Now let's look at our handiwork…
>>>
>>> dteske at scu0a.jbsd.vicor.com ~ $ sudo ngctl info em1:lower
>>> Name: <unnamed> Type: bridge ID: 00000007 Num hooks: 1
>>> Local hook Peer name Peer type Peer ID Peer hook
>>> ---------- --------- --------- ------- ---------
>>> link0 em1 ether 00000003 lower
>>>
>>>
>>> Ok, we see that the lower peer hook of the em1 ether-node goes off to something named "link0".
>>>
>>> To see where link0 is off-to… we need a full listing (back to "ngctl ls -l").
>>>
>>>
>>> dteske at scu0a.jbsd.vicor.com ~ $ sudo ngctl ls -l
>>> There are 5 total nodes:
>>> Name: <unnamed> Type: bridge ID: 00000007 Num hooks: 1
>>> Local hook Peer name Peer type Peer ID Peer hook
>>> ---------- --------- --------- ------- ---------
>>> link0 em1 ether 00000003 lower
>>> Name: em0 Type: ether ID: 00000002 Num hooks: 0
>>> Name: em1 Type: ether ID: 00000003 Num hooks: 1
>>> Local hook Peer name Peer type Peer ID Peer hook
>>> ---------- --------- --------- ------- ---------
>>> lower <unnamed> bridge 00000007 link0
>>> Name: ngctl1762 Type: socket ID: 0000000b Num hooks: 0
>>> Name: msk0 Type: ether ID: 00000001 Num hooks: 0
>>>
>>>
>>> Matching "link0" in the first column to "link0" in the last-column, we can see that this lower-link0 is to a bridge (with no name).
>>>
>>> NOTE: When you're digesting the above output… it helps to imagine whitespace in between the nodes with their respective hooks and other nodes. Future pastes below will introduce such whitespace to make it easier to read.
>>>
>>> ===
>>>
>>> Right now, the only way to refer to the bridge is by way of "em1:lower" (because we created the bridge right on the lower hook of the em1 ether-node).
>>>
>>> At this point, let's talk about naming. Giving our bridge a name is entirely optional, but greatly clarifies the output of both "ngctl ls -l" and "ngctl dot".
>>>
>>> dteske at scu0a.jbsd.vicor.com ~ $ sudo ngctl name em1:lower em1bridge
>>> dteske at scu0a.jbsd.vicor.com ~ $ sudo ngctl ls -l
>>> There are 5 total nodes:
>>> Name: em0 Type: ether ID: 00000002 Num hooks: 0
>>>
>>> Name: em1 Type: ether ID: 00000003 Num hooks: 1
>>> Local hook Peer name Peer type Peer ID Peer hook
>>> ---------- --------- --------- ------- ---------
>>> lower em1bridge bridge 00000007 link0
>>>
>>> Name: ngctl1831 Type: socket ID: 0000001a Num hooks: 0
>>>
>>> Name: em1bridge Type: bridge ID: 00000007 Num hooks: 1
>>> Local hook Peer name Peer type Peer ID Peer hook
>>> ---------- --------- --------- ------- ---------
>>> link0 em1 ether 00000003 lower
>>>
>>> Name: msk0 Type: ether ID: 00000001 Num hooks: 0
>>>
>>> The new "em1bridge" name acts as an alias to "em1:lower" in future ngctl commands. For example, "ngctl info em1:lower" and "ngctl info em1bridge" can now be used interchangeably and produce the same results.
>>>
>>> dteske at scu0a.jbsd.vicor.com ~ $ sudo ngctl info em1bridge:
>>> Name: em1bridge Type: bridge ID: 00000007 Num hooks: 1
>>> Local hook Peer name Peer type Peer ID Peer hook
>>> ---------- --------- --------- ------- ---------
>>> link0 em1 ether 00000003 lower
>>> dteske at scu0a.jbsd.vicor.com ~ $ sudo ngctl info em1:lower
>>> Name: em1bridge Type: bridge ID: 00000007 Num hooks: 1
>>> Local hook Peer name Peer type Peer ID Peer hook
>>> ---------- --------- --------- ------- ---------
>>> link0 em1 ether 00000003 lower
>>>
>>> ===
>>>
>>> We're not done with the bridge yet. Because we foresee the possibility that it might be nice to be able to communicate with the jail that we're going to later hook into this bridge… we should hook the physical adapter's "upper" hook into the bridge.
>>>
>>> If you don't do this, you won't be able to (for example) ping a jail from the host where the host has only the PHY and the jail has only a (yet uncreated) eiface. Regardless of the fact that the bridge uses the PHY and the jail uses the bridge, to communicate with an IP that is configured on the base host, you must hook the upper.
>>>
>>> dteske at scu0a.jbsd.vicor.com ~ $ sudo ngctl connect em1: em1:lower upper link1
>>>
>>> If you want to use the alias I set up earlier (of "em1bridge") that works too (just don't forget the colon at the end of the alias):
>>>
>>> dteske at scu0a.jbsd.vicor.com ~ $ sudo ngctl connect em1: em1bridge: upper link1
>>>
>>> Here's the results:
>>>
>>> dteske at scu0a.jbsd.vicor.com ~ $ sudo ngctl ls -l
>>> There are 5 total nodes:
>>> Name: em0 Type: ether ID: 00000002 Num hooks: 0
>>>
>>> Name: em1 Type: ether ID: 00000003 Num hooks: 2
>>> Local hook Peer name Peer type Peer ID Peer hook
>>> ---------- --------- --------- ------- ---------
>>> upper em1bridge bridge 0000002a link1
>>> lower em1bridge bridge 0000002a link0
>>>
>>> Name: ngctl1874 Type: socket ID: 00000030 Num hooks: 0
>>>
>>> Name: em1bridge Type: bridge ID: 0000002a Num hooks: 2
>>> Local hook Peer name Peer type Peer ID Peer hook
>>> ---------- --------- --------- ------- ---------
>>> link1 em1 ether 00000003 upper
>>> link0 em1 ether 00000003 lower
>>>
>>> Name: msk0 Type: ether ID: 00000001 Num hooks: 0
>>>
>>>
>>> NOTE: Some of the Peer ID's have changed, because I wanted to test that the alias could be used; I used "sudo ngctl shutdown em1bridge:" and re-executed up to the point where I connect the em1:upper into the bridge… except this time using the alias of "em1bridge" instead of "em1:lower" (indeed, you can use them interchangeably).
>>>
>>> ===
>>>
>>> Ok… We've now done the hard part… which was to create and configure a bridge that is usable by any new nodes we connect to it and also (if you hooked the upper portion of em1 back into its own lower which is acting as the bridge) the base machine can communicate with any of the forth-coming jails (if on the same subnet at least).
>>>
>>> There's an easy step that shouldn't be skipped though…
>>>
>>> Before you can truly use this bridge with any other interfaces…
>>>
>>> dteske at scu0a.jbsd.vicor.com ~ $ sudo ifconfig em1 up
>>> dteske at scu0a.jbsd.vicor.com ~ $ sudo ngctl msg em1: setpromisc 1
>>> dteske at scu0a.jbsd.vicor.com ~ $ sudo ngctl msg em1: setautosrc 0
>>>
>>> A bridge cannot send packets out if the interface is down.
>>> A bridge cannot work properly without promiscuous mode.
>>> A bridge cannot send out packets for different addresses unless you turn off "setautosrc"
>>>
>>> ===
>>>
>>> Let's create our first virtual NIC and connect it to the bridge.
>>>
>>> dteske at scu0a.jbsd.vicor.com ~ $ sudo ngctl mkpeer em1bridge: eiface link2 ether
>>>
>>> This command did two things. It created a new "eiface" node (see ng_eiface(4)), and connected it to the bridge.
>>>
>>> Let's have a look:
>>>
>>> dteske at scu0a.jbsd.vicor.com ~ $ sudo ngctl ls -l
>>> There are 6 total nodes:
>>> Name: em0 Type: ether ID: 00000002 Num hooks: 0
>>>
>>> Name: em1 Type: ether ID: 00000003 Num hooks: 2
>>> Local hook Peer name Peer type Peer ID Peer hook
>>> ---------- --------- --------- ------- ---------
>>> upper em1bridge bridge 0000002a link1
>>> lower em1bridge bridge 0000002a link0
>>>
>>> Name: ngeth0 Type: eiface ID: 00000035 Num hooks: 1
>>> Local hook Peer name Peer type Peer ID Peer hook
>>> ---------- --------- --------- ------- ---------
>>> ether em1bridge bridge 0000002a link2
>>>
>>> Name: ngctl2800 Type: socket ID: 00000036 Num hooks: 0
>>>
>>> Name: em1bridge Type: bridge ID: 0000002a Num hooks: 3
>>> Local hook Peer name Peer type Peer ID Peer hook
>>> ---------- --------- --------- ------- ---------
>>> link2 ngeth0 eiface 00000035 ether
>>> link1 em1 ether 00000003 upper
>>> link0 em1 ether 00000003 lower
>>>
>>> Name: msk0 Type: ether ID: 00000001 Num hooks: 0
>>>
>>> The list of hooks for our bridge (em1bridge) is growing, and now we see a new node (ngeth0) with one hook into that bridge.
>>>
>>> ===
>>>
>>> ASIDE: If you wanted to script this… here's how you can test for an unused link:
>>>
>>> Right now, we have link0, link1, and link2 for the bridge. If a link exists for a bridge, the following command will return some info about the link and return success (whereas if the link does not exist, the command will return an error and exit with error-status):
>>>
>>> dteske at scu0a.jbsd.vicor.com ~ $ sudo ngctl msg em1bridge: getstats 0
>>> Rec'd response "getstats" (4) from "[2a]:":
>>> Args:
>>> {}
>>> dteske at scu0a.jbsd.vicor.com ~ $ sudo ngctl msg em1bridge: getstats 1
>>> Rec'd response "getstats" (4) from "[2a]:":
>>> Args:
>>> {}
>>> dteske at scu0a.jbsd.vicor.com ~ $ sudo ngctl msg em1bridge: getstats 2
>>> Rec'd response "getstats" (4) from "[2a]:":
>>> Args:
>>> {}
>>> dteske at scu0a.jbsd.vicor.com ~ $ sudo ngctl msg em1bridge: getstats 3
>>> ngctl: send msg: Socket is not connected
>>> dteske at scu0a.jbsd.vicor.com ~ $ sudo ngctl msg em1bridge: getstats 4
>>> ngctl: send msg: Socket is not connected
>>> dteske at scu0a.jbsd.vicor.com ~ $ sudo ngctl msg em1bridge: getstats 5
>>> ngctl: send msg: Socket is not connected
>>>
>>> As you can see from the above output… we get errors for link3, link4, and link5, because they don't exist. Naturally, testing $? exit status after each of these commands would show how this can be scripted (HINT: throw stdout/stderr to /dev/null and test $?).
>>>
>>> ===
>>>
>>> At this point… you say "ifconfig":
>>>
>>> dteske at oos0a.lbxrich.vicor.com ~ $ ifconfig
>>> msk0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
>>> options=c011a<TXCSUM,VLAN_MTU,VLAN_HWTAGGING,TSO4,VLAN_HWTSO,LINKSTATE>
>>> ether xx:xx:xx:xx:xx:xx
>>> media: Ethernet autoselect
>>> em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
>>> options=209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC>
>>> ether xx:xx:xx:xx:xx:xx
>>> inet xx.xx.xx.xx netmask 0xffffff80 broadcast xx.xx.xx.xx
>>> media: Ethernet autoselect (1000baseT <full-duplex>)
>>> status: active
>>> em1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
>>> options=209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC>
>>> ether xx:xx:xx:xx:xx:xx
>>> media: Ethernet autoselect
>>> status: no carrier
>>> ipfw0: flags=8801<UP,SIMPLEX,MULTICAST> metric 0 mtu 65536
>>> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
>>> options=3<RXCSUM,TXCSUM>
>>> inet 127.0.0.1 netmask 0xff000000
>>> ngeth0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
>>> ether 00:00:00:00:00:00
>>>
>>> ===
>>>
>>> Ok, there are two problems with the network interface.
>>>
>>> 1. It has a NULL MAC address (00:00:00:00:00:00). Good luck communicating on the Internet (remember, we disabled setautosrc -- we intend to make up a MAC address that is unique).
>>>
>>> 2. The name leaves something to be desired (if we're going to use this with a vimage jail, it would be nice if the interface had the jail name in it, so that when you do an "ngctl ls -l" or an "ngctl dot" … you're going to see the jail name so it becomes clear which jails are hooked to which PHY's through which bridges).
>>>
>>> ===
>>>
>>> Let's tackle the easier one first… let's rename this new interface.
>>>
>>> You and I already know that this interface that we want to rename is "ngeth0"… but you can actually extract the name from the link in the bridge.
>>>
>>> dteske at scu0a.jbsd.vicor.com ~ $ sudo ngctl show -n em1bridge:link2
>>> Name: ngeth0 Type: eiface ID: 00000035 Num hooks: 1
>>>
>>>
>>> First, we rename it in netgraph (this does not affect the output of ifconfig -- and again, we do this to make "ngctl ls -l" and "ngctl dot" more palatable):
>>>
>>> dteske at scu0a.jbsd.vicor.com ~ $ sudo ngctl name em1bridge:link2 ng0_myjail
>>> dteske at scu0a.jbsd.vicor.com ~ $ sudo ngctl ls -l
>>> There are 6 total nodes:
>>> Name: em0 Type: ether ID: 00000002 Num hooks: 0
>>>
>>> Name: em1 Type: ether ID: 00000003 Num hooks: 2
>>> Local hook Peer name Peer type Peer ID Peer hook
>>> ---------- --------- --------- ------- ---------
>>> upper em1bridge bridge 0000002a link1
>>> lower em1bridge bridge 0000002a link0
>>>
>>> Name: ngctl2843 Type: socket ID: 00000046 Num hooks: 0
>>>
>>> Name: ng0_myjail Type: eiface ID: 00000035 Num hooks: 1
>>> Local hook Peer name Peer type Peer ID Peer hook
>>> ---------- --------- --------- ------- ---------
>>> ether em1bridge bridge 0000002a link2
>>>
>>> Name: em1bridge Type: bridge ID: 0000002a Num hooks: 3
>>> Local hook Peer name Peer type Peer ID Peer hook
>>> ---------- --------- --------- ------- ---------
>>> link2 ng0_myjail eiface 00000035 ether
>>> link1 em1 ether 00000003 upper
>>> link0 em1 ether 00000003 lower
>>>
>>> Name: msk0 Type: ether ID: 00000001 Num hooks: 0
>>>
>>>
>>> Looking good. However, ifconfig hasn't changed…
>>>
>>> dteske at scu0a.jbsd.vicor.com ~ $ ifconfig
>>> ...
>>> ngeth0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
>>> ether 00:00:00:00:00:00
>>>
>>> We want to rename the interface with ifconfig for a different reason.
>>>
>>> We renamed the interface with netgraph earlier so that netgraph outputs would be nice and easy to digest.
>>>
>>> This time, we rename with ifconfig so that we can layer jails onto the same rootdir.
>>>
>>> The naming convention (which is the same naming convention I use for renaming on the netgraph side) is:
>>>
>>> ng#_name
>>>
>>> The # always starts at zero for each jail where "name" is the name of the jail.
>>>
>>> Again… I use this scheme so that I can layer jails onto the same root-dir; /etc/rc.conf is then populated with things like:
>>>
>>> ifconfig_ng0_myjail=...
>>> ifconfig_ng0_myrouter=...
>>> ifconfig_ng1_myrouter=...
>>> ifconfig_ng0_anotherjail=...
>>>
>>> So that when you say "service netif start" inside the vnet jail… it applies the right settings.
>>>
>>> So… we rename with ifconfig:
>>>
>>> dteske at scu0a.jbsd.vicor.com ~ $ sudo ifconfig ngeth0 name ng0_myjail
>>> dteske at scu0a.jbsd.vicor.com ~ $ ifconfig
>>> ...
>>> ng0_myjail: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
>>> ether 00:00:00:00:00:00
>>>
>>> ===
>>>
>>> We're almost ready to shove this interface into a jail (which we haven't created yet).
>>>
>>> But… we come back to that NULL MAC address.
>>>
>>> NOTE: Forming your own MAC address, or even coming up with your own formula should not be taken lightly.
>>>
>>> Here's a formula I use (which is based on several RFC's for MAC address formation):
>>>
>>> NOTE: In this context, ${_bridge} is em1 and $LINKNUM is 2
>>>
>>> # Set the MAC address of the new interface
>>> # using a sensible algorithm to prevent
>>> # conflicts on the network.
>>> #
>>> # MAC LP:LL:LB:BB:BB:BB
>>> # P 2, 6, A, or E but usually 2
>>> # NOTE: Indicates "privately administered" MAC
>>> # L ng_bridge(4) link number (1-65535)
>>> # B Same as bridged interface
>>> #
>>> _bridge_ether=$( ifconfig ${_bridge} ether |
>>> awk '/ether/{print $2}' )
>>> _ether_devid="${_bridge_ether#??:??:?}"
>>> n=$LINKNUM
>>> _quad=$(($n & 15))
>>> case "${_quad}" in
>>> 10) _quad=a;; 11) _quad=b;; 12) _quad=c;;
>>> 13) _quad=d;; 14) _quad=e;; 15) _quad=f;;
>>> esac
>>> _ether_devid=":${_quad}${_ether_devid}"
>>> n=$(($n >> 4))
>>> _quad=$(($n & 15))
>>> case "${_quad}" in
>>> 10) _quad=a;; 11) _quad=b;; 12) _quad=c;;
>>> 13) _quad=d;; 14) _quad=e;; 15) _quad=f;;
>>> esac
>>> _ether_devid="${_quad}${_ether_devid}"
>>> n=$(($n >> 4))
>>> _quad=$(($n & 15))
>>> case "${_quad}" in
>>> 10) _quad=a;; 11) _quad=b;; 12) _quad=c;;
>>> 13) _quad=d;; 14) _quad=e;; 15) _quad=f;;
>>> esac
>>> _ether_devid="2:${_quad}${_ether_devid}"
>>> n=$(($n >> 4))
>>> _quad=$(($n & 15))
>>> case "${_quad}" in
>>> 10) _quad=a;; 11) _quad=b;; 12) _quad=c;;
>>> 13) _quad=d;; 14) _quad=e;; 15) _quad=f;;
>>> esac
>>> _ether_devid="${_quad}${_ether_devid}"
>>> n=$(($n >> 4))
>>>
>>> After which… ${_ether_devid} holds a properly formed MAC address that can (in every case I've tested) "get out".
>>>
>>> Here's what I do to set it:
>>>
>>> ifconfig ng0_myjail ether "${_ether_devid}"
>>>
>>> Here's an example of how the MAC address was translated from the physical adapter to the ng_eiface(4) interface:
>>>
>>> dteske at scu0a.jbsd.vicor.com ~ $ ifconfig em1; ifconfig ng0_myjail
>>> em1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
>>> options=209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC>
>>> ether 00:0e:0c:ab:1b:76
>>> media: Ethernet autoselect
>>> status: no carrier
>>> ng0_myjail: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
>>> ether 02:00:2c:ab:1b:76
>>>
>>> ===
>>>
>>> OK… we're now ready to shove that interface into a vimage jail.
>>>
>>> But…
>>>
>>> First we need a vimage jail. (this is not a tutorial on how to create, manage, build, or do anything else with jails, vimage-jails, or vps-jails *other* than give it a netgraph based interface)
>>>
>>> I'm going to use my existing base machine as a fake jail (by pointing my jail's rootdir at "/").
>>>
>>> NOTE: Certain sysctl's have to be set appropriately before you fire up the jail to make this vimage jail able to do "more" on the net.
>>>
>>> dteske at scu0a.jbsd.vicor.com ~ $ sudo sysctl security.jail.set_hostname_allowed=1 security.jail.sysvipc_allowed=1 security.jail.socket_unixiproute_only=1
>>> security.jail.set_hostname_allowed: 1 -> 1
>>> security.jail.sysvipc_allowed: 1 -> 1
>>> security.jail.socket_unixiproute_only: 0 -> 1
>>>
>>> NOTE: Unless you intend to reboot to restore the defaults later… you might want to take down those previous values for restoration *after* we fire up the "vimage" jail.
>>>
>>> dteske at scu0a.jbsd.vicor.com ~ $ sudo jail -i -c vnet name=myjail host.hostname=myjail path=/ persist
>>> 1
>>> dteske at scu0a.jbsd.vicor.com ~ $ jls
>>> JID IP Address Hostname Path
>>> 1 - myjail /
>>>
>>> OK… we have a running jail (with the vnet property, making it a "vimage" jail -- which can accept network interfaces).
>>>
>>> ===
>>>
>>> Right now our jail has no network interfaces (well, it has an unconfigured lo0).
>>>
>>> dteske at scu0a.jbsd.vicor.com ~ $ sudo jexec myjail ifconfig
>>> lo0: flags=8008<LOOPBACK,MULTICAST> metric 0 mtu 16384
>>> options=3<RXCSUM,TXCSUM>
>>>
>>> So let's pass the netgraph created interface into the jail…
>>>
>>> dteske at scu0a.jbsd.vicor.com ~ $ sudo ifconfig ng0_myjail vnet 1
>>> dteske at scu0a.jbsd.vicor.com ~ $ sudo jexec myjail ifconfig
>>> lo0: flags=8008<LOOPBACK,MULTICAST> metric 0 mtu 16384
>>> options=3<RXCSUM,TXCSUM>
>>> ng0_myjail: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
>>> ether 02:00:2c:ab:1b:76
>>>
>>> Sweet!
>>>
>>> ===
>>>
>>> Almost there…
>>>
>>> Let's go into /etc/rc.conf, give it an IP, and start the network…
>>>
>>> dteske at scu0a.jbsd.vicor.com ~ $ sudo sysrc ifconfig_ng0_myjail="inet 192.168.1.1 netmask 255.255.255.0"
>>> /etc/rc.conf: ifconfig_ng0_myjail: -> inet 192.168.1.1 netmask 255.255.255.0
>>> dteske at scu0a.jbsd.vicor.com ~ $ grep ng0 /etc/rc.conf
>>> ifconfig_ng0_myjail="inet 192.168.1.1 netmask 255.255.255.0"
>>> dteske at scu0a.jbsd.vicor.com ~ $ sudo jexec myjail service netif start
>>> Starting Network: lo0 ng0_myjail.
>>> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
>>> options=3<RXCSUM,TXCSUM>
>>> inet 127.0.0.1 netmask 0xff000000
>>> ng0_myjail: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
>>> ether 02:00:2c:ab:1b:76
>>> inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255
>>>
>>> Now we're cookin' with gasoline!
>>>
>>> ===
>>>
>>> Optionally go configure your base machine with an IP and have fun.
>>
>> A quick conclusion…
>>
>> Because we've built this all on top of netgraph… we can … graph it.
>>
>> dteske at scu0a.jbsd.vicor.com ~ $ sudo ngctl dot | dot -Tsvg -o netgraph-scu0a.svg
>>
>> I then uploaded the file to the web and here it is:
>>
>> https://urldefense.proofpoint.com/v1/url?u=http://druidbsd.sourceforge.net/download/netgraph-scu0a.svg&k=%2FbkpAUdJWZuiTILCq%2FFnQg%3D%3D%0A&r=Mrjs6vR4%2Faj2Ns9%2FssHJjg%3D%3D%0A&m=xe0XNgnKBiT9v8HzxwWwnNMOVN3YdEHmTsIZfFoQA9Y%3D%0A&s=0a6ee990c4871ecc2a5b69879d11f65798fe70d8948fb9d4592bd915b1fd0199
>>
>> You should compare this directly to the output of "ngctl ls -l":
>>
>> dteske at scu0a.jbsd.vicor.com ~ $ sudo ngctl ls -l
>> There are 6 total nodes:
>> Name: em0 Type: ether ID: 00000002 Num hooks: 0
>>
>> Name: em1 Type: ether ID: 00000003 Num hooks: 2
>> Local hook Peer name Peer type Peer ID Peer hook
>> ---------- --------- --------- ------- ---------
>> upper em1bridge bridge 0000002a link1
>> lower em1bridge bridge 0000002a link0
>>
>> Name: ng0_myjail Type: eiface ID: 00000035 Num hooks: 1
>> Local hook Peer name Peer type Peer ID Peer hook
>> ---------- --------- --------- ------- ---------
>> ether em1bridge bridge 0000002a link2
>>
>> Name: em1bridge Type: bridge ID: 0000002a Num hooks: 3
>> Local hook Peer name Peer type Peer ID Peer hook
>> ---------- --------- --------- ------- ---------
>> link2 ng0_myjail eiface 00000035 ether
>> link1 em1 ether 00000003 upper
>> link0 em1 ether 00000003 lower
>>
>> Name: ngctl8676 Type: socket ID: 00000049 Num hooks: 0
>>
>> Name: msk0 Type: ether ID: 00000001 Num hooks: 0
>>
>> You'll notice that when you graph the layout with "ngctl dot", the nodes are rendered as boxes displaying their "Peer Name" up top, their "Peer Type" in the lower-left, and their "Peer ID" in the bottom-right.
>>
>> The edges from one node to another contains two octagons. These are the "Local hook" and "Peer hook".
>> --
>> Devin
>>
>> _____________
>> The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you.
>
> Hello Devin,
>
> If you live in the same city I will invite you for a couple of beer (I have to pay of course!) - I live in the Netherlands then let me know -
>
If I ever make it out there, I'll certainly shoot you an e-mail.
> I followed your tutorial with the expected result: I can ping the em1 interface but I still have the same problems of before regarding external to internal networks communication.
>
> Please note that on my original host (no jail) the default gateway is 192.168.1.254, that's important for what I am going to do.
>
> I added a default route like :
>
> route add default -interface ng0_myjail
>
Why not:
jexec myjail route add -net default 192.168.1.254
??
--
Devin
> and then I try to :
>
> jexec myjail ping 8.8.8.8
>
> I analyzed the wireshark capture and I can see that an ARP request for the 192.168.254 (with the MAC address of our virtual NIC as source, as expected) go out my freebsd env - which is on a virtualbox - and it gets the answer but when I read the ARP table of the jail I can see it as incomplete. It seems that packet going out but the answer is not received and it is confirmed when I try to sniff with tcpdump on my FREEBSD, I can't see any ARP request going in both from my physical and virtual NIC, the same if I try :
>
> ping 192.168.1.254
>
> Then I can see ping reply coming from wireshark but not from tcpdump on FreeBDS.
>
> Wireshark is attached on the host machine on the physical interface where VB is attached in Bridged Mode, my original physical interface on FreeBSD took the IP address from DHCP without problem then the problem is related to the jail.
>
> do I try do accomplish a task which is not possible with JAIL or something is wrong in my configuration or worst, in my brain :P ?
>
> Thanks in advance,
> Pietro.
>
>
>
>
>
>
_____________
The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you.
More information about the freebsd-questions
mailing list