netstat interface output

[Gary Aitken]

On 02/24/17 18:09, Doug Hardie wrote:
> 
>> On 22 February 2017, at 15:38, Doug Hardie <bc979 at lafn.org> wrote:
>> 
>> I am starting to develop a nagios check for errors on interfaces.
>> However, there are some issues with the entries in netstat that I
>> don't understand.  Each interface has multiple entries:  one for
>> each address and one for the link.  It would seem to me that the
>> link counts would be the sum of the other address entries, but it
>> is not.  Often it is way off.  At first I thought it was possibly
>> caused by overflows of the counters since most systems have been
>> running for months.  However, checking one system that was only up
>> for one day, the ip4 count was considerably larger than the link
>> count.  This is shown in one of the examples below.
>> 
>> The other issue is one system seems to lose addresses.  The address
>> quits responding, although one address remains and does work.  That
>> interface has a DHPC assigned address along with several fixed
>> addresses.  For the first few days, all addresses work fine.  Then
>> all the fixed addresses disappear and no longer work.  The netstat
>> output for this is shown below.  The first is approximately 1 day
>> after the system was booted and all addresses are working.  The
>> second is the next day and only the DHCP assigned address remains.
>> Unfortunately that system is remote and without the fixed addresses
>> I can't access it to get to the messages file.
>> 
>> Network interface status: Name    Mtu Network       Address
>> Ipkts Ierrs Idrop    Opkts Oerrs  Coll  Drop bge0   1500 <Link#1>
>> 68:5b:35:ab:96:52  3703649     0     0  1911094     0     0     0 
>> bge0      - 192.168.1.205 192.168.1.205            0     -     -
>> 0     -     -     - bge0      - 10.0.1.205/32 10.0.1.205
>> 0     -     -        0     -     -     - bge0      - 192.168.0.205
>> 192.168.0.205          498     -     -        0     -     -     - 
>> bge0      - 192.168.0.0/2 192.168.0.7        3700267     -     -
>> 1912398     -     -     - lo0   16384 <Link#2>      lo0
>> 0     0     0        0     0     0     0 lo0       - localhost
>> localhost                0     -     -        0     -     -     - 
>> lo0       - fe80::%lo0/64 fe80::1%lo0              0     -     -
>> 0     -     -     - lo0       - your-net      localhost
>> 0     -     -        0     -     -     -
>> 
>> Local system status: 3:01AM  up 1 day,  8:08, 0 users, load
>> averages: 0.18, 0.16, 0.11
>> 
>> 
>> 
>> Network interface status: Name    Mtu Network       Address
>> Ipkts Ierrs Idrop    Opkts Oerrs  Coll  Drop bge0   1500 <Link#1>
>> 68:5b:35:ab:96:52  6420868     0     0  3313113     0     0     0 
>> bge0      - 192.168.0.0/2 192.168.0.7        1809545     -     -
>> 934183     -     -     - lo0   16384 <Link#2>      lo0
>> 0     0     0        0     0     0     0 lo0       - localhost
>> localhost                0     -     -        0     -     -     - 
>> lo0       - fe80::%lo0/64 fe80::1%lo0              0     -     -
>> 0     -     -     - lo0       - your-net      localhost
>> 0     -     -        0     -     -     -
>> 
>> Local system status: 3:01AM  up 2 days,  8:08, 0 users, load
>> averages: 0.04, 0.11, 0.08
>> 
>> 
>> Any ideas what could cause this?
>> 
> 
> I finally got the logs.  They show periodic bge0:link state changed
> to DOWN and then UP  Then dhclient gets a new IP address and mask
> (although its always the same address).
> 
> I get the impression that when the link goes down, all the IP
> addresses are lost.  Dhclient replaces the DHCP address, but the
> static addresses are not added back in.  I don't see anything similar
> on identical hardware so I suspect the downs are being caused by the
> router.  However, I need to have the static addresses return when it
> comes back up.  Is there a way to make that happen.

I'm not an expert on this, but my experience is that individual host
addresses show up in the routing table when the remote machine contacts
the machine you're interested in.  

So one way to get them to reappear would be to run a cron job on the
remote machines which periodically ping the host you're interested in.
You could also run a cron job on the host you're interested in which
periodically pings all the others.  This sounds like a classic case
for use of the broadcast address, i.e. the host you're interested in
sends out a broadcast asking everyone to check in, and they would all 
respond.  I have no experience with using broadcast addrs.

There may be others, but I'm pretty sure one of those would work.

Gary