too many open file descriptors messages since bind 9.4.2-P1
bakul at bitblocks.com
Tue Jul 15 22:12:33 UTC 2008
On Tue, 15 Jul 2008 14:18:41 PDT JINMEI Tatuya / =?ISO-2022-JP?B?GyRCP0BMQEMjOkgbKEI=?= <Jinmei_Tatuya at isc.org> wrote:
> At Tue, 15 Jul 2008 23:09:30 +0200,
> Kris Kennaway <kris at FreeBSD.org> wrote:
> > > If that's regularly happening, I'm afraid recent P1 versions don't
> > > handle that well, and recommend you try 9.4.3b2 ore 9.5.1b1.
> > Or increase the number of file descriptors as a workaround, per my email :)
> Does FreeBSD allow an application to increase FD_SETSIZE (at its
> compilation time)? I thought FD_SETSIZE defaults to 1024. Any
> 9.x.y-P1 versions can only open FD_SETSIZE sockets, regardless of the
> # FDs limit.
$ cvs log /sys/kern/kern_generic.c
date: 1996/08/20 07:17:48; author: smpatel; state: Exp; lines: +43 -15
Remove the kernel FD_SETSIZE limit for select().
Unless things have changed, you can completely ignore
FD_SETSIZE (& struct fd_set) and decide at runtime how many
fds you want in a select read/write set (subject to the
Hmm... things have reverted back....
cvs blame -r1.71 /sys/kern/kern_generic.c # the earliest reversal I can find
1.71 (peter 07-Feb-01): * This is kinda bogus. We have fd limi
ts, but that doesn't
1.71 (peter 07-Feb-01): * map too well to the size of the pfd array. Make sure
1.71 (peter 07-Feb-01): * we let the process use at least FD_SETSIZE entries.
1.71 (peter 07-Feb-01): * The specs say we only have to support OPEN_MAX entries (64).
1.71 (peter 07-Feb-01): */
1.71 (peter 07-Feb-01): lim = min((int)p->p_rlimit[RLIMIT_NOFILE].rlim_cur, maxfilesperproc);
1.71 (peter 07-Feb-01): lim = min(lim, FD_SETSIZE);
1.71 (peter 07-Feb-01): if (nfds > lim)
1.71 (peter 07-Feb-01): return (EINVAL);
Sigh.... This is a mistake. I don't see why a user is not
allowed to select on all the fds he can open. The
corresponding log indicates perhaps the author didn't know
select used to work for # of fds > FD_SETSIZE.
date: 2001/02/07 23:28:01; author: peter; state: Exp; lines: +16 -8
The code I picked up from NetBSD in '97 had a nasty bug. It limited
the index of the pollfd array to the number of fd's currently open, not
the maximum number of fd's. ie: if you had 0,1,2 open, you could not
use pollfd slots higher than 20. The specs say we only have to support
OPEN_MAX  entries but we allow way more than that.
> Besides, I guess that the P1 versions severely suffer from heavy
> overhead of select(2) when it regularly opens more than 1000 sockets.
> Even if 'too many open file' messages are gone, many users won't
> accept the increased load due to the overhead. Beta versions use
> kqueue, eliminating the fundamental overhead as well as the (too low)
> limitation of # of descriptors.
Or more portably you can use poll(2).
More information about the freebsd-net