VIMAGE

Teske, Devin Devin.Teske at fisglobal.com
Sat Jun 1 03:26:19 UTC 2013


On May 31, 2013, at 3:05 PM, Teske, Devin wrote:


On May 31, 2013, at 1:48 AM, Pietro Paolini wrote:


On May 30, 2013, at 6:25 PM, "Teske, Devin" <Devin.Teske at fisglobal.com<mailto:Devin.Teske at fisglobal.com>> wrote:


On May 30, 2013, at 3:35 AM, Pietro Paolini wrote:

Hello all,

I am a new bye on the FreeBSD and I am looking at the VIMAGE features experiencing some problems.
I added the options :
VIMAGE
if_bridge

and I removed
STCP

then I recompiled my kernel and install it.

After that, following this tutorial http://imunes.tel.fer.hr/virtnet/eurobsdcon07_tutorial.pdf I tried the "Exercise 2" which consist on
the following commands:

vimage -c n1
vimage -c n2
ngctl mkpeer efface ether ether
ngctl mkpeer efface ether ether

Don't you just love autocorrect? (does the same thing to me… turns "eiface" into "efface")


ngctl mkpeer em0: bridge lower link0

Looks good.


ngctl name em0:lower bridge0

I usually do my "connect" before the "name"… but shouldn't matter. Should work all the same.


ngctl connect em0: bridge0: upper link1

This looks wrong to me.

I'd expect:

ngctl connect em0: bridge0:lower upper link1



Many thanks for the answer Devin,
when I try to use that last command I receive:

ngctl connect em0: bridge0:lower upper link1
ngctl: send msg: Invalid argument

What's wrong ?


Let's start from scratch on a freshly booted box…

dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ sudo ngctl ls -l
[sudo] Password:
There are 4 total nodes:
  Name: em0             Type: ether           ID: 00000002   Num hooks: 0
  Name: em1             Type: ether           ID: 00000003   Num hooks: 0
  Name: ngctl1719       Type: socket          ID: 00000004   Num hooks: 0
  Name: msk0            Type: ether           ID: 00000001   Num hooks: 0

Ok… we have an "ether" type node for each of our physical adapters (these are provided by ng_ether(4); you didn't have to do anything to get these nodes).

We also have a single "socket" type node. This is the "ngctl" connection to the netgraph subsystem (you can learn more by reading ng_socket(4)).

Here's the corresponding hardware behind em0, em1, and msk0:

===

dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ grep '\(em\|e1000phy\|mskc\?\)[[:digit:]]' /var/run/dmesg.boot
mskc0: <Marvell Yukon 88E8050 Gigabit Ethernet> port 0xdc00-0xdcff mem 0xfcffc000-0xfcffffff irq 16 at device 0.0 on pci5
msk0: <Marvell Technology Group Ltd. Yukon EC Id 0xb6 Rev 0x02> on mskc0
msk0: Ethernet address: xx:xx:xx:xx:xx:xx
miibus0: <MII bus> on msk0
e1000phy0: <Marvell 88E1111 Gigabit PHY> PHY 0 on miibus0
e1000phy0:  none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto
mskc0: [ITHREAD]
em0: <Intel(R) PRO/1000 Legacy Network Connection 1.0.3> port 0xec80-0xecbf mem 0xfebe0000-0xfebfffff irq 16 at device 4.0 on pci7
em0: [FILTER]
em0: Ethernet address: xx:xx:xx:xx:xx:xx
em1: <Intel(R) PRO/1000 Legacy Network Connection 1.0.3> port 0xec00-0xec3f mem 0xfeba0000-0xfebbffff,0xfeb80000-0xfeb9ffff irq 18 at device 6.0 on pci7
em1: [FILTER]
em1: Ethernet address: xx:xx:xx:xx:xx:xx
em0: link state changed to UP

===

Next, let's make a bridge (think of it as a big software switch that we're going to hook a bunch of interfaces; created, physical, or otherwise).

Since I'm doing this over an SSH connection (a mistake I made earlier today), I'm not going to touch em0 (the adapter my SSH connection is using). Creating the bridge on an actively configured PHY will knock it off the net. This is not to say you can't have an active configuration on a bridged interface… just that the creation of the bridge (something you should only do once each time you boot) will disrupt an active connection.

So…

dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ sudo ngctl mkpeer em1: bridge lower link0

NOTE: No output == Success.

===

Now let's look at our handiwork…

dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ sudo ngctl info em1:lower
  Name: <unnamed>       Type: bridge          ID: 00000007   Num hooks: 1
  Local hook      Peer name       Peer type    Peer ID         Peer hook
  ----------      ---------       ---------    -------         ---------
  link0           em1             ether        00000003        lower


Ok, we see that the lower peer hook of the em1 ether-node goes off to something named "link0".

To see where link0 is off-to… we need a full listing (back to "ngctl ls -l").


dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ sudo ngctl ls -l
There are 5 total nodes:
  Name: <unnamed>       Type: bridge          ID: 00000007   Num hooks: 1
  Local hook      Peer name       Peer type    Peer ID         Peer hook
  ----------      ---------       ---------    -------         ---------
  link0           em1             ether        00000003        lower
  Name: em0             Type: ether           ID: 00000002   Num hooks: 0
  Name: em1             Type: ether           ID: 00000003   Num hooks: 1
  Local hook      Peer name       Peer type    Peer ID         Peer hook
  ----------      ---------       ---------    -------         ---------
  lower           <unnamed>       bridge       00000007        link0
  Name: ngctl1762       Type: socket          ID: 0000000b   Num hooks: 0
  Name: msk0            Type: ether           ID: 00000001   Num hooks: 0


Matching "link0" in the first column to "link0" in the last-column, we can see that this lower-link0 is to a bridge (with no name).

NOTE: When you're digesting the above output… it helps to imagine whitespace in between the nodes with their respective hooks and other nodes. Future pastes below will introduce such whitespace to make it easier to read.

===

Right now, the only way to refer to the bridge is by way of "em1:lower" (because we created the bridge right on the lower hook of the em1 ether-node).

At this point, let's talk about naming. Giving our bridge a name is entirely optional, but greatly clarifies the output of both "ngctl ls -l" and "ngctl dot".

dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ sudo ngctl name em1:lower em1bridge
dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ sudo ngctl ls -l
There are 5 total nodes:
  Name: em0             Type: ether           ID: 00000002   Num hooks: 0

  Name: em1             Type: ether           ID: 00000003   Num hooks: 1
  Local hook      Peer name       Peer type    Peer ID         Peer hook
  ----------      ---------       ---------    -------         ---------
  lower           em1bridge       bridge       00000007        link0

  Name: ngctl1831       Type: socket          ID: 0000001a   Num hooks: 0

  Name: em1bridge       Type: bridge          ID: 00000007   Num hooks: 1
  Local hook      Peer name       Peer type    Peer ID         Peer hook
  ----------      ---------       ---------    -------         ---------
  link0           em1             ether        00000003        lower

  Name: msk0            Type: ether           ID: 00000001   Num hooks: 0

The new "em1bridge" name acts as an alias to "em1:lower" in future ngctl commands. For example, "ngctl info em1:lower" and "ngctl info em1bridge" can now be used interchangeably and produce the same results.

dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ sudo ngctl info em1bridge:
  Name: em1bridge       Type: bridge          ID: 00000007   Num hooks: 1
  Local hook      Peer name       Peer type    Peer ID         Peer hook
  ----------      ---------       ---------    -------         ---------
  link0           em1             ether        00000003        lower
dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ sudo ngctl info em1:lower
  Name: em1bridge       Type: bridge          ID: 00000007   Num hooks: 1
  Local hook      Peer name       Peer type    Peer ID         Peer hook
  ----------      ---------       ---------    -------         ---------
  link0           em1             ether        00000003        lower

===

We're not done with the bridge yet. Because we foresee the possibility that it might be nice to be able to communicate with the jail that we're going to later hook into this bridge… we should hook the physical adapter's "upper" hook into the bridge.

If you don't do this, you won't be able to (for example) ping a jail from the host where the host has only the PHY and the jail has only a (yet uncreated) eiface. Regardless of the fact that the bridge uses the PHY and the jail uses the bridge, to communicate with an IP that is configured on the base host, you must hook the upper.

dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ sudo ngctl connect em1: em1:lower upper link1

If you want to use the alias I set up earlier (of "em1bridge") that works too (just don't forget the colon at the end of the alias):

dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ sudo ngctl connect em1: em1bridge: upper link1

Here's the results:

dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ sudo ngctl ls -l
There are 5 total nodes:
  Name: em0             Type: ether           ID: 00000002   Num hooks: 0

  Name: em1             Type: ether           ID: 00000003   Num hooks: 2
  Local hook      Peer name       Peer type    Peer ID         Peer hook
  ----------      ---------       ---------    -------         ---------
  upper           em1bridge       bridge       0000002a        link1
  lower           em1bridge       bridge       0000002a        link0

  Name: ngctl1874       Type: socket          ID: 00000030   Num hooks: 0

  Name: em1bridge       Type: bridge          ID: 0000002a   Num hooks: 2
  Local hook      Peer name       Peer type    Peer ID         Peer hook
  ----------      ---------       ---------    -------         ---------
  link1           em1             ether        00000003        upper
  link0           em1             ether        00000003        lower

  Name: msk0            Type: ether           ID: 00000001   Num hooks: 0


NOTE: Some of the Peer ID's have changed, because I wanted to test that the alias could be used; I used "sudo ngctl shutdown em1bridge:" and re-executed up to the point where I connect the em1:upper into the bridge… except this time using the alias of "em1bridge" instead of "em1:lower" (indeed, you can use them interchangeably).

===

Ok… We've now done the hard part… which was to create and configure a bridge that is usable by any new nodes we connect to it and also (if you hooked the upper portion of em1 back into its own lower which is acting as the bridge) the base machine can communicate with any of the forth-coming jails (if on the same subnet at least).

There's an easy step that shouldn't be skipped though…

Before you can truly use this bridge with any other interfaces…

dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ sudo ifconfig em1 up
dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ sudo ngctl msg em1: setpromisc 1
dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ sudo ngctl msg em1: setautosrc 0

A bridge cannot send packets out if the interface is down.
A bridge cannot work properly without promiscuous mode.
A bridge cannot send out packets for different addresses unless you turn off "setautosrc"

===

Let's create our first virtual NIC and connect it to the bridge.

dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ sudo ngctl mkpeer em1bridge: eiface link2 ether

This command did two things. It created a new "eiface" node (see ng_eiface(4)), and connected it to the bridge.

Let's have a look:

dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ sudo ngctl ls -l
There are 6 total nodes:
  Name: em0             Type: ether           ID: 00000002   Num hooks: 0

  Name: em1             Type: ether           ID: 00000003   Num hooks: 2
  Local hook      Peer name       Peer type    Peer ID         Peer hook
  ----------      ---------       ---------    -------         ---------
  upper           em1bridge       bridge       0000002a        link1
  lower           em1bridge       bridge       0000002a        link0

  Name: ngeth0          Type: eiface          ID: 00000035   Num hooks: 1
  Local hook      Peer name       Peer type    Peer ID         Peer hook
  ----------      ---------       ---------    -------         ---------
  ether           em1bridge       bridge       0000002a        link2

  Name: ngctl2800       Type: socket          ID: 00000036   Num hooks: 0

  Name: em1bridge       Type: bridge          ID: 0000002a   Num hooks: 3
  Local hook      Peer name       Peer type    Peer ID         Peer hook
  ----------      ---------       ---------    -------         ---------
  link2           ngeth0          eiface       00000035        ether
  link1           em1             ether        00000003        upper
  link0           em1             ether        00000003        lower

  Name: msk0            Type: ether           ID: 00000001   Num hooks: 0

The list of hooks for our bridge (em1bridge) is growing, and now we see a new node (ngeth0) with one hook into that bridge.

===

ASIDE: If you wanted to script this… here's how you can test for an unused link:

Right now, we have link0, link1, and link2 for the bridge. If a link exists for a bridge, the following command will return some info about the link and return success (whereas if the link does not exist, the command will return an error and exit with error-status):

dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ sudo ngctl msg em1bridge: getstats 0
Rec'd response "getstats" (4) from "[2a]:":
Args: {}
dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ sudo ngctl msg em1bridge: getstats 1
Rec'd response "getstats" (4) from "[2a]:":
Args: {}
dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ sudo ngctl msg em1bridge: getstats 2
Rec'd response "getstats" (4) from "[2a]:":
Args: {}
dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ sudo ngctl msg em1bridge: getstats 3
ngctl: send msg: Socket is not connected
dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ sudo ngctl msg em1bridge: getstats 4
ngctl: send msg: Socket is not connected
dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ sudo ngctl msg em1bridge: getstats 5
ngctl: send msg: Socket is not connected

As you can see from the above output… we get errors for link3, link4, and link5, because they don't exist. Naturally, testing $? exit status after each of these commands would show how this can be scripted (HINT: throw stdout/stderr to /dev/null and test $?).

===

At this point… you say "ifconfig":

dteske at oos0a.lbxrich.vicor.com<mailto:dteske at oos0a.lbxrich.vicor.com> ~ $ ifconfig
msk0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=c011a<TXCSUM,VLAN_MTU,VLAN_HWTAGGING,TSO4,VLAN_HWTSO,LINKSTATE>
        ether xx:xx:xx:xx:xx:xx
        media: Ethernet autoselect
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC>
        ether xx:xx:xx:xx:xx:xx
        inet xx.xx.xx.xx netmask 0xffffff80 broadcast xx.xx.xx.xx
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
em1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC>
        ether xx:xx:xx:xx:xx:xx
        media: Ethernet autoselect
        status: no carrier
ipfw0: flags=8801<UP,SIMPLEX,MULTICAST> metric 0 mtu 65536
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=3<RXCSUM,TXCSUM>
        inet 127.0.0.1 netmask 0xff000000
ngeth0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 00:00:00:00:00:00

===

Ok, there are two problems with the network interface.

1. It has a NULL MAC address (00:00:00:00:00:00). Good luck communicating on the Internet (remember, we disabled setautosrc -- we intend to make up a MAC address that is unique).

2. The name leaves something to be desired (if we're going to use this with a vimage jail, it would be nice if the interface had the jail name in it, so that when you do an "ngctl ls -l" or an "ngctl dot" … you're going to see the jail name so it becomes clear which jails are hooked to which PHY's through which bridges).

===

Let's tackle the easier one first… let's rename this new interface.

You and I already know that this interface that we want to rename is "ngeth0"… but you can actually extract the name from the link in the bridge.

dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ sudo ngctl show -n em1bridge:link2
  Name: ngeth0          Type: eiface          ID: 00000035   Num hooks: 1


First, we rename it in netgraph (this does not affect the output of ifconfig -- and again, we do this to make "ngctl ls -l" and "ngctl dot" more palatable):

dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ sudo ngctl name em1bridge:link2 ng0_myjail
dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ sudo ngctl ls -l
There are 6 total nodes:
  Name: em0             Type: ether           ID: 00000002   Num hooks: 0

  Name: em1             Type: ether           ID: 00000003   Num hooks: 2
  Local hook      Peer name       Peer type    Peer ID         Peer hook
  ----------      ---------       ---------    -------         ---------
  upper           em1bridge       bridge       0000002a        link1
  lower           em1bridge       bridge       0000002a        link0

  Name: ngctl2843       Type: socket          ID: 00000046   Num hooks: 0

  Name: ng0_myjail      Type: eiface          ID: 00000035   Num hooks: 1
  Local hook      Peer name       Peer type    Peer ID         Peer hook
  ----------      ---------       ---------    -------         ---------
  ether           em1bridge       bridge       0000002a        link2

  Name: em1bridge       Type: bridge          ID: 0000002a   Num hooks: 3
  Local hook      Peer name       Peer type    Peer ID         Peer hook
  ----------      ---------       ---------    -------         ---------
  link2           ng0_myjail      eiface       00000035        ether
  link1           em1             ether        00000003        upper
  link0           em1             ether        00000003        lower

  Name: msk0            Type: ether           ID: 00000001   Num hooks: 0


Looking good. However, ifconfig hasn't changed…

dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ ifconfig
...
ngeth0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 00:00:00:00:00:00

We want to rename the interface with ifconfig for a different reason.

We renamed the interface with netgraph earlier so that netgraph outputs would be nice and easy to digest.

This time, we rename with ifconfig so that we can layer jails onto the same rootdir.

The naming convention (which is the same naming convention I use for renaming on the netgraph side) is:

ng#_name

The # always starts at zero for each jail where "name" is the name of the jail.

Again… I use this scheme so that I can layer jails onto the same root-dir; /etc/rc.conf is then populated with things like:

ifconfig_ng0_myjail=...
ifconfig_ng0_myrouter=...
ifconfig_ng1_myrouter=...
ifconfig_ng0_anotherjail=...

So that when you say "service netif start" inside the vnet jail… it applies the right settings.

So… we rename with ifconfig:

dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ sudo ifconfig ngeth0 name ng0_myjail
dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ ifconfig
...
ng0_myjail: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 00:00:00:00:00:00

===

We're almost ready to shove this interface into a jail (which we haven't created yet).

But… we come back to that NULL MAC address.

NOTE: Forming your own MAC address, or even coming up with your own formula should not be taken lightly.

Here's a formula I use (which is based on several RFC's for MAC address formation):

NOTE: In this context, ${_bridge} is em1 and $LINKNUM is 2

                                # Set the MAC address of the new interface
                                # using a sensible algorithm to prevent
                                # conflicts on the network.
                                #
                                # MAC  LP:LL:LB:BB:BB:BB
                                # P    2, 6, A, or E but usually 2
                                # NOTE: Indicates "privately administered" MAC
                                # L    ng_bridge(4) link number (1-65535)
                                # B    Same as bridged interface
                                #
                                _bridge_ether=$( ifconfig ${_bridge} ether |
                                        awk '/ether/{print $2}' )
                                _ether_devid="${_bridge_ether#??:??:?}"
                                n=$LINKNUM
                                _quad=$(($n & 15))
                                case "${_quad}" in
                                10) _quad=a;; 11) _quad=b;; 12) _quad=c;;
                                13) _quad=d;; 14) _quad=e;; 15) _quad=f;;
                                esac
                                _ether_devid=":${_quad}${_ether_devid}"
                                n=$(($n >> 4))
                                _quad=$(($n & 15))
                                case "${_quad}" in
                                10) _quad=a;; 11) _quad=b;; 12) _quad=c;;
                                13) _quad=d;; 14) _quad=e;; 15) _quad=f;;
                                esac
                                _ether_devid="${_quad}${_ether_devid}"
                                n=$(($n >> 4))
                                _quad=$(($n & 15))
                                case "${_quad}" in
                                10) _quad=a;; 11) _quad=b;; 12) _quad=c;;
                                13) _quad=d;; 14) _quad=e;; 15) _quad=f;;
                                esac
                                _ether_devid="2:${_quad}${_ether_devid}"
                                n=$(($n >> 4))
                                _quad=$(($n & 15))
                                case "${_quad}" in
                                10) _quad=a;; 11) _quad=b;; 12) _quad=c;;
                                13) _quad=d;; 14) _quad=e;; 15) _quad=f;;
                                esac
                                _ether_devid="${_quad}${_ether_devid}"
                                n=$(($n >> 4))

After which… ${_ether_devid}  holds a properly formed MAC address that can (in every case I've tested) "get out".

Here's what I do to set it:

ifconfig ng0_myjail ether "${_ether_devid}"

Here's an example of how the MAC address was translated from the physical adapter to the ng_eiface(4) interface:

dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ ifconfig em1; ifconfig ng0_myjail
em1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC>
ether 00:0e:0c:ab:1b:76
media: Ethernet autoselect
status: no carrier
ng0_myjail: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 02:00:2c:ab:1b:76

===

OK… we're now ready to shove that interface into a vimage jail.

But…

First we need a vimage jail. (this is not a tutorial on how to create, manage, build, or do anything else with jails, vimage-jails, or vps-jails *other* than give it a netgraph based interface)

I'm going to use my existing base machine as a fake jail (by pointing my jail's rootdir at "/").

NOTE: Certain sysctl's have to be set appropriately before you fire up the jail to make this vimage jail able to do "more" on the net.

dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ sudo sysctl security.jail.set_hostname_allowed=1 security.jail.sysvipc_allowed=1 security.jail.socket_unixiproute_only=1
security.jail.set_hostname_allowed: 1 -> 1
security.jail.sysvipc_allowed: 1 -> 1
security.jail.socket_unixiproute_only: 0 -> 1

NOTE: Unless you intend to reboot to restore the defaults later… you might want to take down those previous values for restoration *after* we fire up the "vimage" jail.

dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ sudo jail -i -c vnet name=myjail host.hostname=myjail path=/ persist
1
dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ jls
   JID  IP Address      Hostname                      Path
     1  -               myjail                        /

OK… we have a running jail (with the vnet property, making it a "vimage" jail -- which can accept network interfaces).

===

Right now our jail has no network interfaces (well, it has an unconfigured lo0).

dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ sudo jexec myjail ifconfig
lo0: flags=8008<LOOPBACK,MULTICAST> metric 0 mtu 16384
options=3<RXCSUM,TXCSUM>

So let's pass the netgraph created interface into the jail…

dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ sudo ifconfig ng0_myjail vnet 1
dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ sudo jexec myjail ifconfig
lo0: flags=8008<LOOPBACK,MULTICAST> metric 0 mtu 16384
options=3<RXCSUM,TXCSUM>
ng0_myjail: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 02:00:2c:ab:1b:76

Sweet!

===

Almost there…

Let's go into /etc/rc.conf, give it an IP, and start the network…

dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ sudo sysrc ifconfig_ng0_myjail="inet 192.168.1.1 netmask 255.255.255.0"
/etc/rc.conf: ifconfig_ng0_myjail:  -> inet 192.168.1.1 netmask 255.255.255.0
dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ grep ng0 /etc/rc.conf
ifconfig_ng0_myjail="inet 192.168.1.1 netmask 255.255.255.0"
dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ sudo jexec myjail service netif start
Starting Network: lo0 ng0_myjail.
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=3<RXCSUM,TXCSUM>
inet 127.0.0.1 netmask 0xff000000
ng0_myjail: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 02:00:2c:ab:1b:76
inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255

Now we're cookin' with gasoline!

===

Optionally go configure your base machine with an IP and have fun.

A quick conclusion…

Because we've built this all on top of netgraph… we can … graph it.

dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ sudo ngctl dot | dot -Tsvg -o netgraph-scu0a.svg

I then uploaded the file to the web and here it is:

http://druidbsd.sourceforge.net/download/netgraph-scu0a.svg

You should compare this directly to the output of "ngctl ls -l":

dteske at scu0a.jbsd.vicor.com<mailto:dteske at scu0a.jbsd.vicor.com> ~ $ sudo ngctl ls -l
There are 6 total nodes:
  Name: em0             Type: ether           ID: 00000002   Num hooks: 0

  Name: em1             Type: ether           ID: 00000003   Num hooks: 2
  Local hook      Peer name       Peer type    Peer ID         Peer hook
  ----------      ---------       ---------    -------         ---------
  upper           em1bridge       bridge       0000002a        link1
  lower           em1bridge       bridge       0000002a        link0

  Name: ng0_myjail      Type: eiface          ID: 00000035   Num hooks: 1
  Local hook      Peer name       Peer type    Peer ID         Peer hook
  ----------      ---------       ---------    -------         ---------
  ether           em1bridge       bridge       0000002a        link2

  Name: em1bridge       Type: bridge          ID: 0000002a   Num hooks: 3
  Local hook      Peer name       Peer type    Peer ID         Peer hook
  ----------      ---------       ---------    -------         ---------
  link2           ng0_myjail      eiface       00000035        ether
  link1           em1             ether        00000003        upper
  link0           em1             ether        00000003        lower

  Name: ngctl8676       Type: socket          ID: 00000049   Num hooks: 0

  Name: msk0            Type: ether           ID: 00000001   Num hooks: 0

You'll notice that when you graph the layout with "ngctl dot", the nodes are rendered as boxes displaying their "Peer Name" up top, their "Peer Type" in the lower-left, and their "Peer ID" in the bottom-right.

The edges from one node to another contains two octagons. These are the "Local hook" and "Peer hook".
--
Devin

_____________
The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you.


More information about the freebsd-questions mailing list