Bridge woes

[Michael Gmelin]

> On 28. Oct 2020, at 12:32, D'Arcy Cain <darcy at druid.net> wrote:
> 
> On 10/27/20 2:58 PM, Michael Gmelin wrote:
> 
> I hope you don't mind but I reverted this conversation back to the list in case it gives someone else any ideas.
> 
>> Hi,
>> I tried to reproduce the problem on my home network, but things just
>> work as expected.
>> I could run VMs with IPs off the local network, fixed ones as well as
>> DHCP.
>> The topology looks a bit different:
>> vm->server->router ->(nat)-> internet
>>      |
>>      + dhcp/dns
> 
> I suppose that that is essentially the same but let me see if I get it.  You have a network, say 192.168.1.0/24, behind your NAT router.  You have physical servers like 192.168.1.1 and 192.168.1.2 on this network.  You then put a VM on the  .1 host numbered 192.168.1.3 and it can connect to 192.168.1.2.  Is that correct?
> 
>> I would speculate that there's either something going on with
>> the switch (you might want to take a look at it), or you're experiencing
>> some sort of asymmetric routing issue (ping/icmp is usually just fine
> 
> Not sure what that could be.  It's not just a problem with external hosts. Hosts on the same network are also showing the symptoms.  Another point is that I can access it inbound.  It's only outbound connections that don't work.
> 
>> with that). Or it might be something with the bge driver (I'm using em
> 
> The only server that it can connect to is running bce.  I have some em servers but it doesn't connect to those.
> 
>> here). I assume you already tried disabling all sorts of offloading to
>> see if it makes a difference?
> 
> Yep.  I tried -tso -lro -rxcsum -rxcsum6 -txcsum -txcsum6 -vlanhwtag -vlanhwtso and subsets of that.
> 
>> Other than that I would suggest to play with tcpdump to see if packets
>> are returned on the same interface they've been sent out on or not.
> 
> Here is an example packet seen on the host:
> 
> 11:20:40.397067 IP 98.158.139.71.44448 > 98.158.139.66.22: Flags [S], seq 3285763868, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 3003762262 ecr 0], length 0
> 
> The .66 never sees the packet and the host never sees a return packet.  On the other hand, a connection attempt from .66 to the VM shows up properly.
> 
> 
>> Proxy arp might play a role on a local network, that's something I've
>> seen in the past when I has hosts with multiple interfaces on the same
>> (multiple) networks. If you can afford to try it, I would see if
>> shutting down eth1 (and then flushing all arp tables on all
>> hosts/devices involved in your test) makes a difference[0].
> 
> I want to be careful about dropping eth1 as it is the only way in if I mess up eth0.
> 

Can you (afford to) reboot the machine reliably? If so, schedule a reboot using "shutdown -r +10" and then bring down the the interface to see if it makes a difference.

-m
> -- 
> D'Arcy J.M. Cain <darcy at druid.net>         |  Democracy is three wolves
> http://www.druid.net/darcy/                |  and a sheep voting on
> +1 416 788 2246     (DoD#0082)    (eNTP)   |  what's for dinner.
> IM: darcy at VybeNetworks.com, VoIP: sip:darcy at druid.net
> 
> Disclaimer: By sending an email to ANY of my addresses you
> are agreeing that:
> 
> 1.  I am by definition, "the intended recipient".
> 2.  All information in the email is mine to do with as I see
>    fit and make such financial profit, political mileage, or
>    good joke as it lends itself to. In particular, I may quote
>    it where I please.
> 3.  I may take the contents as representing the views of
>    your company if I so wish.
> 4.  This overrides any disclaimer or statement of
>    confidentiality that may be included or implied in
>    your message.