named: error sending response: not enough free resources

[Chris St Denis]

Steve Bertrand wrote:
> Chris St Denis wrote:
>   
>> Wojciech Puchar wrote:
>>     
>>> possible reasons
>>> - your firewall rules are the cause - check it.
>>>       
>>    Nope
>>
>>    eureka# ipfw list
>>
>>     
>>> - your network card produce problems (REALLY i have that case)
>>>       
>>    I have had this kind of error on multiple servers over the years, so
>>    i don't think it's a hardware problem.
>>
>>     
>>> - the network/LAN named tries to sent UDP packet is somehow flooded.
>>>       
>>    Dns is probably fairly busy. It's the primary authorative dns for
>>    some busy domains. Is there a setting I can do to increase the
>>    limits of UDP packets to keep it from causing problems?
>>
>>    The server is approaching it's 10 mbps interface speed during peak
>>    hours, I may need to upgrade it to 100mbps.
>>     
>
> The 10Mb ceiling (provided by your ifconfig output) could be a damper on
> this.
>
> What type of device is em1 attached to? Is it a switch or a hub? Is it
> possible to upgrade this? You should upgrade it to 100 (or 1000)
> anyways. Does this device show any collisions?
>   
This is a dedicated server in a datacenter. I don't know the exact 
switch specs but it's likely a
layer 2/3 managed switch. Probably a 1U catalyst.

I can upgrade the connection to 100mbps for a small monthly fee. I've 
left it at 10 because I haven't
had a need, but with traffic recently growing, this is probably the problem.

> Can you do the following for a few minutes (until at least the problem
> is triggered):
>
> # tcpdump -n -i em1 proto 17 port 53 -s -w /var/log/dns.pcap
>
> ...and then:
>
> # mail -s "tcpdump output" steve at ipv6canada.com < /var/log/dns.pcap
>   
I don't think this is necessary. If cutting down the http traffic or 
raising the port speed doesn't
fix it, I'll look into further debugging with this.
> Is this server a caching recursive server for internal clients, or an
> authoritative server?
>   
An authoritative for some moderately busy domains. Also recursive for 
some jails on this and another server (main recursive is on a private 
(10.0.0.0/24 on em0) network, and this server predates multi-ip jails)

A "tcpdump -n -i em1 -s 0 port 53 > packets.txt" for 1 minute shows

eureka# wc -l packets.txt
     359 packets.txt

So about 350 dns packets a minute, at least in this particular minute. 
Less than I expected, I guess most is going to the other dns server at 
the moment.
> What else runs on this box?
>   
Web hosting. Thats where the full 10mbps comes from.

> If you generate further network traffic over the interface, do the log
> entries pile up faster?
>
> What does:
>
> # netstat -s -p udp
>   
eureka# netstat -s -p udp
udp:
        194973570 datagrams received
        0 with incomplete header
        13 with bad data length field
        884 with bad checksum
        68521 with no checksum
        669174 dropped due to no socket
        17 broadcast/multicast datagrams dropped due to no socket
        733 dropped due to full socket buffers
        0 not for hashed pcb
        194302749 delivered
        195188906 datagrams output

Fyi, if these are since last reboot, this server has been up 381 days.
> say?
>
> I'd focus squarely on the 10Mbps cap first. That should be easy to test
> and eliminate. Then, once that is rectified, we can find out whether
> it's an inherent problem with the system.
>   
Yes, I'll deal with this, then reply again if the problem is not resolved.

Thanks for the suggestions.
> Steve
>