packet loss to firewall while Internet link is down

[D G Teed]

Hi all,

When the Internet link goes down, ssh refuses
to allow connection from within the LAN to our BSD
firewall/gateway.  An existing ssh connection might stay
up, but be very sluggish.  We run our own DNS, so that
can't be the reason for timeouts.

When the Internet is down, the CPU load factor on the
FreeBSD firewall is low, but the number of TCP packets
that can't get past the first hop is likely high, which
might cause some sort of congestion on the machine.

The console is very responsive.  mtr to any point
on the local LAN from the firewall sees 50 to 80%
packet loss.  However, there is no packet loss between
other machines on the lan and our network guy says
the router port and cable check out fine.

There are no console error messages providing a clue.
netstat -m shows the mb_map is about 26% in use
while the Internet is down.  The machine in question
is FreeBSD 4.11, running ipfw and acting as a gateway
(not NAT).

Once the Internet comes back up, ssh in works, and
ssh sessions are very responsive again.

Is there some kernel variable I can tweak, or some tests I
can try the next time the Internet goes down and the
gateway/firewall drop packets on connections to our LAN?

Our operations manager is a Windows guy, and every time
he can't ssh in, he thinks the firewall needs a reboot, when
the real problem is that the Internet is down and
there is something we need to tweak to make it
better able to survive local LAN traffic.

--Donald