sonewconn: pcb 0xfffff801efdd1000: Listen queue overflow: 31 already in queue awaiting acceptance

[Charles Swiger]

On Mar 16, 2015, at 2:30 PM, David Benfell <benfell at parts-unknown.org> wrote:
> I'm seeing these messages in my nightly security log output. Googling
> around, I find I should increase kern.ipc.somaxconn. I have. Now to
> 4096. This in turn seems to mean I need to increase apc.shm_size in
> /usr/local/etc/php.ini and I have, now, to 1024M.
[ ... ]
> But 1) I have no idea how to set them properly, and 2) I don't
> understand the relationship.
> 
> What should I be doing?

They aren't directly related.  You generally need to increase apc.shm_size
suitable for however many PHP processes you're running-- generally via mod_php
in Apache or fastcgi for nginx and the like.

The listen queue overflow means that your system is failing to process
incoming requests fast enough to keep up.  The bottleneck could be
on a resource like CPU or memory, or serialization against a database
table, or something else.  (You'll want to identify the bottleneck.)

Having a bigger listen queue can be helpful if you see transient bursts
of high activity.  However, if the load is not a brief spike, then
you've only got 90 seconds or so with typical web clients before they
decide to time out and drop the connection.

Don't try growing the listen queue longer than that, because it won't help--
in fact, it's counterproductive to system resources on an already-busy system.
(Thus notions like "surge queue tuning", "surge protection", etc on
load-balancers commonly used for large sites.)

Regards,
-- 
-Chuck