Intermittent connectivity loss with em(4)

[Morgan Wesström]

I'd appreciate some help with the following problem because I'm probably 
blind to the obvious from all my trouble shooting.

I have a small Supermicro Atom-based ITX-board with 3x Intel 82574L NICs 
(2 on mainboard, 1 in PCIe slot) and I experience intermittent network 
connectivity loss every few minutes. The machine is currently running 
FreeBSD 12.0-RELEASE-p10. I loop a single ping once per minute to 
dns.google (8.8.8.8) and output looks like this:

20:16:35 1 packets transmitted, 1 packets received, 0.0% packet loss
20:17:35 1 packets transmitted, 1 packets received, 0.0% packet loss
20:18:35 1 packets transmitted, 1 packets received, 0.0% packet loss
20:19:35 1 packets transmitted, 1 packets received, 0.0% packet loss
20:20:35 1 packets transmitted, 0 packets received, 100.0% packet loss
20:21:47 1 packets transmitted, 0 packets received, 100.0% packet loss
20:22:58 1 packets transmitted, 0 packets received, 100.0% packet loss
20:24:09 1 packets transmitted, 0 packets received, 100.0% packet loss
20:25:20 1 packets transmitted, 1 packets received, 0.0% packet loss
20:26:20 1 packets transmitted, 1 packets received, 0.0% packet loss
20:27:20 1 packets transmitted, 1 packets received, 0.0% packet loss
20:28:20 1 packets transmitted, 1 packets received, 0.0% packet loss
20:29:20 1 packets transmitted, 1 packets received, 0.0% packet loss
20:30:21 1 packets transmitted, 0 packets received, 100.0% packet loss
20:31:32 1 packets transmitted, 0 packets received, 100.0% packet loss
20:32:43 1 packets transmitted, 0 packets received, 100.0% packet loss
20:33:54 1 packets transmitted, 0 packets received, 100.0% packet loss
20:35:05 1 packets transmitted, 0 packets received, 100.0% packet loss
20:36:16 1 packets transmitted, 1 packets received, 0.0% packet loss
20:37:16 1 packets transmitted, 1 packets received, 0.0% packet loss
20:38:17 1 packets transmitted, 1 packets received, 0.0% packet loss
20:39:17 1 packets transmitted, 1 packets received, 0.0% packet loss
20:40:17 1 packets transmitted, 1 packets received, 0.0% packet loss
20:41:17 1 packets transmitted, 0 packets received, 100.0% packet loss

THINGS I'VE TESTED WITHOUT RESOLVING THE PROBLEM

- Tried all three NICs in the computer but they all show the same behaviour.
- Tried a different ping target.
- Tried the LiveCD environment from the older FreeBSD 11.3-RELEASE memstick.
- Disabled MSI-X interrupts.
- Additionally disabled MSI interrupts.
- Recompiled em(4) and enabled DEBUG_INIT, DEBUG_IOCTL and DEBUG_HW but 
this only generates a few more message in dmesg during boot. There is 
nothing shown in dmesg or otherwise when connectivity is lost.

THING'S I'VE TRIED THAT SEEMINGLY RESOLVES OR ALLEVIATES THE PROBLEM

- Booting the system on Linux (4.19 kernel) with just a simple command 
prompt and running the same ping test does _NOT_ show any connectivity 
loss. At least not during the hour or so I tested. To me this rules out 
any hardware related problems as well as the network connection itself.
- Generating small amounts of network traffic on the interface (like an 
ssh session) seems to reduce the problem. I can then run the ping test 
for maybe 30 minutes without loss of connectivity in FreeBSD but 
eventually it fails too.

I really need help to understand what's going on here. I have a gut 
feeling some power saving is playing a trick on me but 
hw.em.smart_pwr_down is set to 0 default and I have no indication of any 
power saving function kicking in. How can I debug what's going on in 
em(4)? Am I just stupid and missing something obvious here?

Kind regards
Morgan Wesström