bhyve and arp problem

[Daniel Braniss]

> On 2 Apr 2018, at 15:33, Harry Schmalzbauer <freebsd at omnilan.de> wrote:
> 
> Bezüglich Daniel Braniss's Nachricht vom 30.03.2018 13:16 (localtime):
>> hi,
>> this is my first attempt at bhyve, and so far all seems ok, except
>> in my guest, the mac address of the hosting keeps flipping, ie, every 20 minutes
>> i see a message :
>> 	… arp: nnn (the hosting ip)  moved from xxxx to yyyy
>> on both the host and guest I’m running a very resent -stable.
>> the yyyy is the mac of the host nic, while the xxxx is the tap0
>> 
>> i know this looks harmless, but it’s annoying
> 
> You can calm it with
> 'sysctl net.link.ether.inet.log_arp_movements=0'
> 
> There's also "net.link.ether.inet.log_arp_wrong_iface" and
> "net.inet.ip.check_interface" which influence related behaviour.
> 
> You also posted (documentationized IP-addresses):
>> I think the problem starts with the host seeing the client/guest on 2 interfaces, the nic (mlnxen0) and the tap(tap0)
>> on the host:
>> 
>> arp -a
>> ...
>> bhv-00.cs.huji.ac.il (192.0.2.246) at xx.xx.xx.xx.xx on tap0 expires in 1001 seconds [ethernet]
>> bhv-00.cs.huji.ac.il (192.0.2.246) at xx.xx.xx.xx.xx on mlxen0 expires in 644 seconds [ethernet]
> 

the above 2 lines are on the host running bhyve (server?) and the MACs belong to the client, and they are identical,
there is no complaints.
(BTW, did you change the ip’s?)

> Initially, you reference two MAC-addresses with xxxx and yyyy.
this is on the client, where the MAC are different (it’s of the hosting computer).

> The recent post indicates non-different MAC-addresses.
> 
> If xxxx and yyyy - resp. xx.xx.xx.xx.xx - are equal (but seen on
> different interfaces), this wouldn't get logged I think.
> But it was the only harmless case for straight forward setups.
> Even with STP/LACP/CARP/etc. in place, "arp: IP-address moved" always
> indicates a misconfiguration and I don't know any example where the two
> different MAC-Addresses for one IP-address were harmless.
> While using a single (locally administrated?) MAC address more than once
> sitewide _can_ make sense, having two interfaces on one host which both
> are on the same ethernet segment like the two interfaces with the same
> MAC address, looks like an unintended setup.
> 
> So I strongly suggest to analyze your setup before altering the
> mentioned sysctl!!!
> 
I do want to know if there are ip/mac issues, it usually happens when more than one host has the same ip,
which is not the case here :-(

thanks,
	danny

> -harry