Using any network interface whatsoever

[M. Warner Losh]

eth0 works well for the degenerate case where there's a network card
in the system, and nothing else.  It works less well for systmes where
there are more than one card, and where the hardware changes a lot for
all the reasons discussed in this thread.  It is too generic.

Of course, when you have N copies of the same card on FreeBSD, it is
as degenerate as ethN is.

You can use devd right now to tie the device instance to a
configuration using bus topology.  That information is available to
the devd script that's forked.  However, experiments in doing that
work only so long as chaning one card doesn't change the hardware
topology of the other cards.  For single-banger NICs, it won't.
However, when you have multi-port NIC cards that are a bunch of
individual chips behind a PCI bridge, the PCI bus numbers that they
live on change as you insert/remove these multi-port NIC cards, so
hardware topology fails you.

Then you might think of tying the MAC address to an IP
address/configuration.  This works well in the above scenario since
you can now remove cards that change topology.  However, when you
replace a defective card with another, you need to reconfigure because
the MAC address changes.

I've actually run into all of these problems on a machine we have at
work that acts as a gateway to about 10-20 private networks.  It has
had between 2-4 dual cards and 2-4 quad cards, in various mix and
match flavors over the years.  We've replaced dual fxp cards with quad
de cards that were replaced with quad dc cards, etc.  Pairs of dual
cards were replaced with a quad card at times.  We've run a single fxp
card in place of a failed quad fxp card for a few days too.  The mobo
has been replaced a few times as well, which also changes bus
topology, as well as introduces new built-in NIC cards.

The only solution that works for us has been to carefully desk check
the results of each hardware change after the system has been
rebooted.  After the desk check, a ping of systems ensures that
nothing has fallen through the cracks (usually).  If there's a way
that we can manage this chaotic system better, I'd love to hear about
it.

Warner

P.S. There's conflicting netequite about what I should do with a long
thread.  some folks say quote the whole thing, others say quote tiny
bits.  I've decided to offend both camps by quoting nothing :-).