em device hangs on ifconfig alias ...

[Atanas]

User Freebsd said the following on 6/29/06 9:29 PM:
> 
> The other funny thing about the current em driver is that if you move an 
> IP to it from a different server, the appropriate ARP packets aren't 
> sent out to redirect the IP traffic .. recently, someone pointed me to 
> arping, which has solved my problem *external* to the driver ...
> 
That's the second reason why I (still) avoid em in mass-aliased systems.

I have a single pool of IP addresses shared by many servers with 
multiple aliases each. When someone leaves and frees an IP, it gets 
reused and brought up on a different server. In case it was previously 
handled by em, the traffic doesn't get redirected to the new server.

Similar thing happens even with machines with single static IPs. For 
instance when retiring an old production system, I usually request a new 
box to be brought up on a different IP, make a fresh install on 
everything and test, swap IP addresses and reboot. In case of em, after 
a soft reboot both systems are inaccessible.

A workaround is to power both of the systems down and then power them 
up. This however cannot be done remotely and in case there were IP 
aliases, they still don't get any traffic.

> I have a third machine that uses an em driver, but its an older 4.x 
> kernel, and it operates perfectly ... no timeouts/hangs and sends out 
> the appropriate ARP packet ... all three servers are connected to the 
> same Cisco switch, with all ports configured identically, so it isn't a 
> switch issue, as someone else intimated ...
> 
This seems strange, could depend on the chip version, who knows.

I still have many 4.x based machines, and both em issues (the card reset 
on each alias and the arp packets not been sent when going down) were 
present when I was doing my tests.

I check for these once in a while (a year or so), usually with the 
latest major release branch.

We had a compatibility issue about a year ago with a (rather exotic?) 
fiber NIC - 82545GM, where FreeBSD-4.x did better. The em driver coming 
with 5.x didn't support that (or wasn't working as expected, I don't 
remember the specifics), while the one coming with 4.x did, so we ended 
up installing 4.11 then.

Regards,
Atanas