Intermittent connectivity loss with em(4)

[Morgan Wesström]

On 2019-10-08 13:17, Harry Duncan wrote:
> Interesting.... ssh ... so if ssh is failing through the cablemodem,
> that points to something very very different. SSH is very fault
> tolerant, as long as you don't try data entry, you can take down
> networks and put them back up and have it resume as before.
> 
> If ssh session failure is what sparked this, then it is more likely
> that you are falling victim to your ISP doing something like 4 to 6
> NAT or 6 to 4 or even 4 to 4 NAT with RFC1913 on one side, and the far
> side of the NAT not maintaining a constant IP through some load
> balancing setup.
> 
> I've had some really weird experiences recently with the whole
> ip4/ip6, like travelling to SE Asia and not able to connect into my
> static ipv4 office network until I got a native to call the ISP and
> then made some change, and I got onto the "whole" internet then. I
> didn't look into it when I got what I wanted, but put it down to 6 to
> 4 nat/
> 
> Harry.

I normally ssh to the test machine through another interface connected 
to my LAN. At one point I kept an ssh session alive to my VPS through 
the cable modem and this was the only time I didn't lose connectivity 
within 5 minutes - it took a little over 30 minutes that time. This was 
when I suspected that network activity prevented some NIC power saving 
to kick in but test results were inconclusive.

I have now tested OpenBSD on the machine and the loss of connectivity 
pattern is there too.

I re-tested with Linux for almost an hour and there was no loss of 
connectivity during that time. (I can't wrap my head around why the 
different OSes behaves differently.)

I tcpdumped all traffic from the dhcp handshake until I had gone through 
a whole cycle of lost/restored connectivity. Scrutinized it in wireshark 
but couldn't find any abnormal packets sent to/from my machine neither 
before loss of connectivity nor before restore of connectivity. (I did 
notice though that the main router's sent packets were always present. I 
got another impression yesterday but it was probably just a fluke that 
machine didn't send any data while I tcpdumped yesterday.)
I then compared it to a tcpdump of the Linux-traffic. Noticed a small 
difference in the dhcp handshake so I replaced dhclient on FreeBSD with 
dhcpcd used on Linux but it made no difference. (It was a long shot I know.)
Another difference I noticed was that Linux made frequent ARP requests 
for the upstream gateway during the whole session while FreeBSD did not. 
I doubt it matters because my main router is running the exact same 
setup as the test router and it doesn't lose connectivity.

Last thing to test is to unplug the main router and only run the test 
router but it'll have to wait. This isn't a big problem for me. Just an 
annoyance and I'm curious to figure out why it behaves the way it does. 
Usually I assume it's something I have done wrong and it's an 
opportunity to learn.

On 2019-10-08 15:07, Sergio de Almeida Lenzi wrote:
 > Hello... in Brazil I solved the problem when I put a small 5 port switch
 > between the modem and the FreeBSD machine..
 > one cable FreeBSD -> small switch -> modem instead of : Freebsd -> modem
 > since than no connection drop at all
 >
 > I do not know why this happens.. must be something about hardware,
 > timing...
 >
 > Hope this helps...

Thank you Sergio. Good to hear I'm not alone with my problem. Your 
suggestion is excellent and I already considered it as a way to narrow 
down the problem to the cable modem.

/Morgan