Problem with Bridging ... and bge devices under FreeBSD 7.x?

Tue Oct 28 20:35:46 PDT 2008

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

- --On Tuesday, October 28, 2008 22:08:18 -0400 Michael Proto 
<mike at jellydonut.org> wrote:

>
>
>
> On Tue, Oct 28, 2008 at 7:56 PM, Marc G. Fournier <freebsd at hub.org> wrote:
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
> I'm trying to run a QEMU VM on top of a FreeBSD 7.x server ... I've tried the
> exact same setup on my desktop, using 192.168.1.x and an fxp device, and it
> all
> works perfectly, but as soon as I do this on another machine on a public IP,
> I'm not getting any routing, I can't even ping it from the same machine ...
>
> My first thought was  that there was an issue with IP aliases already on the
> bge device, but tried doing the following:
>
> ifconfig bridge0 destroy
> ifconfig tap0 destroy
> ifconfig fxp0 -alias 192.168.1.101
> ifconfig fxp0 alias 192.168.1.101 netmask 255.255.255.255
> ifconfig bridge0 create
> ifconfig tap0 create
> ifconfig bridge0 addm fxp0 addm tap0 up
>
> on my desktop here and then starting up the qemu image, and all worked as
> expected, so having an alias on the interface, before or after, doesn't make a
> difference ... at least with the fxp device ...
>
> Using VNC to connect to the VM, I can look at the interface, and it says it is
> connected ... and the IP/Gateway are all set right for the network I'm on,
> netmask is set to 255.255.255.0, same as on the 'private network' ...
>
> Please note that when I say "it works" on my private network / desktop, I'm
> using it to connect to my work computer, across the Internet, via Windows RDP,
> and it works flawlessly ...
>
> Looking at /var/log/messages, you can see the bridge being setup:
>
>
> Oct 27 18:53:21 io kernel: bridge0: Ethernet address: ce:44:c7:1b:47:40
>
> as well as the tap device:
>
> Oct 27 18:53:25 io kernel: tap0: Ethernet address: 00:bd:96:ae:67:00
> Oct 27 18:53:41 io kernel: tap0: promiscuous mode enabled
>
> and the ethernet going promiscuous:
>
> Oct 26 20:53:56 ganymede kernel: fxp0: promiscuous mode enabled
>
> So, all I have left is that everything is being setup okay, but there is
> something I'm missing here ... something with bridge<->bge, maybe?  I've even
> tries to compare the output of 'ifconfig -a' as far as the bridge0 and tap0
> devices are concerned, and other then the mac address, they look identical
> also
> ...
>
> So, pointers to what I may be missing here?  a sysctl value that I need to set
> for this interface?
>
>
>
>
> I'm having a little trouble understanding the setup you have. In your test
> case, is the IP of your VM 192.168.1.101? If so, then I don't think you want
> that IP aliased on the physical interface of your bridge. The VM NIC will
> answer for packets destined on your local segment, which the bridge would
> forward to the physical interface. If you assign the VM's IP to that physical
> interface, then your host would think that traffic is destined for itself and
> not pass it to the bridge.
>
> If I'm misunderstanding and the 192.168.1.101 alias (or whatever the equiv in
> your production setup) isn't being used by your VM then I would start looking
> at the ARP traffic crossing both the tap0, lo0, and physical interfaces.
>
> What does an 'ifconfig -a' look like on both systems? netstat -rn? Any packet
> filtering?

I always fear I'm going to send more info then I should, and generate chaos and 
confusion :)

On my test box, the VM is set to 192.168.1.100 ... the alias I added to fxp0 
was to simulate what I have on the "public server", where there is a bge0 
device with n aliases attached to it ... in no case is the IP assigned to the 
VM actually aliased onto any interface on the network itself

Now, to try and answer your other questions ...

netstat -nr on the 192 server shows the IP to be at:

> netstat -nr | grep 168.1.100
192.168.1.100      52:54:00:12:34:56  UHLW        1        1   fxp0   1128

which is very odd, as that MAC address is not found via ifconfig -a:

> ifconfig -a | grep 52
>

while arp -a also shows the 52:54 MAC, although MACs for the ifconfig -a are, 
in fact:

> ifconfig -a | grep ether
        ether 00:02:b3:ee:da:3e
        ether 5e:d1:e6:8b:55:50
        ether 00:bd:25:18:6d:00

On the server, I'm getting nothing in arp or netstat for the IP in question:

io# arp -a | grep 204.213
io# netstat -nr | grep 204.213
io#

I've even tried doing a ping *from* the VM (logged in with VNC) to see if it 
will broadcast itself out, and nothing ...

I'm starting QEMU on both servers with the same options as well:

qemu -m 512M -net nic -net tap winxp.img

just to confirm that I'm not doing anything different for attaching to the 
network ...

So, right now, all I can see as being "different" is bge vs fxp interfaces ... 
both machines are running 7.x ...

- -- 
Marc G. Fournier        Hub.Org Hosting Solutions S.A. (http://www.hub.org)
Email . scrappy at hub.org                              MSN . scrappy at hub.org
Yahoo . yscrappy               Skype: hub.org        ICQ . 7615664
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (FreeBSD)

iEYEARECAAYFAkkH2gcACgkQ4QvfyHIvDvNHUgCgtQORpycxkREQuiogWWOwydWG
WfUAoOlRghz5Iy7XYWwwpOI5JgMjmBfi
=3Q5f
-----END PGP SIGNATURE-----