FreeBSD: syslog-ng: I/O error occurred while writing; fd='xx', error='No buffer space available (yy)'

[Traiano Welcome]

Hi Mark


On 22/03/2012 11:52, "Mark Blackman" <mark at exonetric.com> wrote:

>
>On 22 Mar 2012, at 09:00, Traiano Welcome wrote:
>
>> 
>> 
>> My question is: What does this error mean, and how can I resolve it?
>
>
>From a very casual inspection of the problem, I'd say you're pushing out
>syslog messages faster than the kernel can get them out the interface.
>How many syslog messages are going in (per second) and what kind of
>network interface are you trying to send them out through?


That's what I thought as well, but it's the details that evade me. Almost
all traffic to and from this server is UDP (syslog), the graph I sent
earlier shows the kind of volumes and trends that are typical: Peak
traffic during the problem periods averages at about 1 Mbps outbound and
200 Kbps inbound to/from the interface. The interface itself is a
Embedded Broadcom 5708 NIC on a Dell PowerEdge 1950.


Here are a couple of netstat polls during one of the problem periods:

----
[root at syslog2]# date;netstat -p udp -s |egrep -w
"(received|delivered|dropped)"
Thu Mar 22 12:11:34 SAST 2012
        19969 datagrams received
        2 dropped due to no socket
        0 dropped due to full socket buffers
        19967 delivered
.
.
.
[root at syslog2~]# date;netstat -p udp -s |egrep -w
"(received|delivered|dropped)"
Thu Mar 22 13:36:46 SAST 2012
        662385 datagrams received
        118 dropped due to no socket
        0 dropped due to full socket buffers
        662267 delivered
---


Somehow this doesn't strike  me as a large volume of throughput ...





>
>> 
>> I have tried to frame this as an operating system kernel resource issue,
>> and experimented with increasing the freebsd kernel sysctls for UDP
>> performance:
>
>
>I think you can push nmbclusters up to about 600k, but if your input is
>running faster than your output, no amount of buffering will permanently
>stave off this problem.



I've done that just in the last 2 hours, though I agree with you that this
is probably a (very) temporary imrovement.



>
>- Mark
>
>
>