UDP flooding / Ethernet issues? WAS Re: named "error sending response: not enough free resources"

[James Smallacombe]

> On Thu, Jan 28, 2010 at 12:59 PM, James Smallacombe <up at 3.am> wrote:
>
>> To follow up on this: Noticed the issue again this morning, which also was
>> accompanied by latency so high that I could not connect (some pings got
>> through at very high latency).  I emailed the provider and they told me that
>> they had my port on their Ether switch set to 10Mbs.  They switched it to
>> 100Mbs and only time will tell if that fixes it.
>>
>> Does this sound like it could be the entire cause?  I ask because I've
>> maxed out pipes before, but never seen it shut all traffic down this much.
>> One key difference that I forgot to mention is that this server is running
>> TWO instances of named, on two different IPs (for different domains), each
>> running a few hundred zones.
>>
>> Bottom line:  Would congestion cause this issue, or would this issue cause
>> congestion?

Some updates that may confuse more than inform: I caught this while it 
was happening yesterday and was able to do a tcpdump.  I saw a ton of UDP 
traffic outbound to one IP that turned out to be a colocated server in 
Chicago.  I put that IP in my ipfw rules and once I blocked "any to" that 
IP, it seemed to stop.  Since then however, the logs have show the same 
issue again and there have been a few brief service disruptions.

Today's security run output showed this:

+(RULE NUMBER) 16054161 131965203420 deny ip from any to (blocked IP)

and more alarmingly, this:

kernel log messages:
+++ /tmp/security.BErFHSS3      2010-01-29 03:09:32.000000000 -0500
+re0: link state changed to DOWN
+re0: link state changed to UP
+re0: promiscuous mode enabled
+re0: promiscuous mode disabled
+re0: promiscuous mode enabled
+re0: promiscuous mode disabled
+re0: promiscuous mode enabled
+re0: promiscuous mode disabled

re0 obviously being the Realtek Ethernet driver.  The server itself never 
went down during this time, but the Ethernet did.  Is there any DOS type 
of event that could cause this, or could the root of the problem be an 
Ethernet hardware or driver issue?  Again, it is not clear to me which is 
the cause and which is the effect.

Last bit of info:  I just did a: 'tcpdump -n | grep -i udp' and saw a 
bunch of these, coming up a couple of times per second:

11:31:59.387561 IP (IP REMOVED) > (IP REMOVED): NBT UDP PACKET(137): 
QUERY; REQUEST; BROADCAST

Where the source and destination IPs vary, but are NOT one of mine, but DO 
appear to belong to my colo/dedicated server provider and their customers. 
Is my server being used to DDOS others?  If so, how?

TIA,

James Smallacombe		      PlantageNet, Inc. CEO and Janitor
up at 3.am							    http://3.am
=========================================================================