Intermittent connectivity loss with em(4)

[Morgan Wesström]

On 2019-10-08 07:25, Doug Hardie wrote:
> It sounds like your cable modem is not using NAT for the various machines, but is using one local address for both.  Hence, it depends on the response to the arp requests which one will receive the packets.  In my experience you have to configure cable modems for the local IP addresses of the machines on your network.  However, I have only dealt with a couple of cable modem connections.  You might want to contact the internet provider and explain the situation to them.  They can probably identify the cause.
> 
> -- Doug
> 

That is almost correct. The cable modem is a SagemCom 2864 with tailored 
firmware for my ISP and it has a setting to switch between NAT/bridge 
mode. I've put it in bridge mode and the ISP provides 4 public 
IP-addresses through DHCP.

Some kind of arp screwup is my suspicion too although it perplexes me 
this doesn't happen when I boot the machine on Linux. I've used FreeBSD 
for 15+ years as my router and have never seen this before. I'll see if 
I can boot it on OpenBSD later and compare the behaviour. Being able to 
get more debug info from em(4) somehow might have been useful.

The ISP in question is Sweden's ComHem and when it comes to them having 
competence to resolve this... well, let's not go there... :)

 > Your choices then to confirm are to run the linux solution which
 > worked perfectly for an extended period to catch the intermittent
 > problem or setup a host on one the spare nics and ping a minute to
 > that host to see if all interfaces drop at the same time or just the
 > cablemodem interface, that would give you a more realistic view of
 > kernel / driver / apm issues.
 >
 > If you are running routed through your cablemodem, ping a minute to
 > the cablemodem also, that will test the actual interface.
 >
 > Horses for courses, but IMHO you are nearly always better off 
bridging isp cpe
 >
 > Harry.

Thank you, Harry. No, Linux isn't an alternative. I run it on my clients 
but I feel much more comfortable with FreeBSD protecting my network. The 
Linux iptables/networking learning curve would be to much for me. :)

Only what is currently the cable modem interface is experiencing this 
behaviour. I connect one of the others to my LAN and ssh to the machine 
that way and I never lose connection through ssh. As I explained earlier 
I've rotated through all three interfaces and whatever is connected to 
the cable modem fails.

The behaviour is strangely regular. It starts within minutes after boot 
and then goes through 5-7 minutes of lost connectivity and 5-7 minutes 
of restored connectivity and just cycles through this pattern. Link LED 
remains on and activity LED is flashing. I forced the interface to 
100Nbit/s for one test but behaviour didn't change.

/Morgan