Redundant Trunked VLANs Revisited

[Michael Jeung]

Just replying to myself on this one in case someone else is trying to  
do something similar.

The long and short of this is that while this setup appears to work  
for the first few hours of operation, over the course of time the  
server begins to have severe latency problems on all the VLAN  
interfaces.  I'm talking about 10000-20000 ms delays.  I'm currently  
eliminating the VLAN interfaces and just testing with ng_one2many,  
but I'm very curious if anyone else has successfully solved this  
problem in production.

I'm basically trying to trunk two NICs together for redundancy and  
place the trunk on two separate VLANs for access.  Seems like a  
simple problem, but I can't figure it out. =)

Regards,
Michael Jeung


On May 23, 2006, at 1:23 PM, Michael Jeung wrote:

> Regarding: http://lists.freebsd.org/pipermail/freebsd-net/2004- 
> March/003210.html
>
> I'm trying to implement a similar solution, but instead of using  
> ng_bridge, I'm using ng_one2many.
>
> sw1--em0--\                                  /--default(ng_eiface)-- 
> ngeth0
>   |        multi0(ng_one2many)--vlt0(ng_vlan)--vlan10(ng_eiface)-- 
> ngeth1
> sw2--em1--/                                  \--vlan20(ng_eiface)-- 
> ngeth2
>
> Here's my netgraph script:
>
> #!/bin/sh
> # Configure NICs as up and load kernel module
> ifconfig em0 up
> ifconfig em1 up
>
> kldload ng_ether.ko
>
> # Plumb nodes together
> ngctl -f- << EOF
> mkpeer em0: one2many upper one
> name em0:upper multi0
> connect em0: multi0: lower many0
> connect em1: multi0: lower many1
>
> # Allow em1 to xmit/recv em0 frames
> msg em1: setpromisc 1
> msg em1: setautosrc 0
> msg em0: setpromisc 1
> msg em0: setautosrc 0
>
> # Reconnect the one hook to the vlan interface (vlt0)
> rmhook multi0: one
> mkpeer multi0: vlan one downstream
> name multi0:one vlt0
>
> # VLAN Default (ngeth0)
> mkpeer vlt0: eiface nomatch ether
> name vlt0:nomatch default
>
> # VLAN 10 (ngeth1)
> mkpeer vlt0: eiface vlan10 ether
> msg vlt0: addfilter { vlan=10 hook="vlan10" }
> name vlt0:vlan10 vlan10
>
> # VLAN 20 (ngeth2)
> mkpeer vlt0: eiface vlan20 ether
> msg vlt0: addfilter { vlan=20 hook="vlan20" }
> name vlt0:vlan20 vlan20
> EOF
>
> # Configure all links as up, set xmit/failover policy
> ngctl msg multi0: setconfig "{ xmitAlg=1 failAlg=2 enabledLinks=[ 1  
> 1 ] }"
>
> # Assign IP and default route
> ifconfig ngeth0 inet 192.168.45.70 netmask 255.255.255.0
> ifconfig ngeth1 inet 192.168.10.70 netmask 255.255.255.0
> ifconfig ngeth2 inet 192.168.20.70 netmask 255.255.255.0
> route add default 192.168.45.1
>
>
> It seems to be working pretty well, but something that's confusing  
> me is this:  When I go and put IP addresses on ngeth0, ngeth1 and  
> ngeth2 I can ping those IP addresses without much difficulty.   
> However, by default, the mac addresses for these virtual interfaces  
> are all zeroed out (See below).  Plus, when I go into the switch  
> and search for the IP addresses, I can't find the MAC addresses  
> associated with them -- even though I can ping them!  How can I  
> ping an IP address that doesn't have a MAC address associated with  
> it in the switch?
>
> ngeth0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
>         inet 192.168.45.70 netmask 0xffffff00 broadcast 192.168.45.255
>         ether 00:00:00:00:00:00
> ngeth1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
>         inet 192.168.10.70 netmask 0xffffff00 broadcast 192.168.10.255
>         ether 00:00:00:00:00:00
> ngeth2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
>         inet 192.168.20.70 netmask 0xffffff00 broadcast 192.168.20.255
>         ether 00:00:00:00:00:00
>
> Also, I'm sort of new at working with netgraph so if you spot  
> anything weird with my script, I'd really like to know. =)  I'll  
> probably be putting this up onto a non-critical production machine  
> in a few days.
>
> Thanks,
> Michael Jeung