sysctl maxfiles

[Jeremy Chadwick]

On Sat, Sep 27, 2008 at 10:14:09PM +0200, Miroslav Lachman wrote:
> Jeremy Chadwick wrote:
>> On Sat, Sep 27, 2008 at 11:10:01AM +1000, Aristedes Maniatis wrote:
>>
>>> By default FreeBSD 7.0 shipped with the sysctls set to:
>>>
>>> kern.maxfiles: 12328
>>> kern.maxfilesperproc: 11095
>
> [...]
>
>> Anyway, I'd like to know why you have so many fds open simultaneously in
>> the first place.  We're talking over 11,000 fds actively open at once --
>> this is not a small number.  What exactly is this machine doing?  Are
>> you absolutely certain tuning this higher is justified?  Have you looked
>> into the possibility that you have a program which is exhausting fds by
>> not closing them when finished?  (Yes, this is quite common; I've seen
>> bad Java code cause this problem on Solaris.)
>
> I can imagine some webhosting machine running Apache virtualhosts. Each  
> virtual host using 3 logfiles (access log, error log, IO log) so it is  
> "only" about 4000 domains (virtualhosts) which is not so uncommon in  
> these days ;)

We're a web/shell hosting provider who used to do it that way.  It
became unreasonable/impossible to manage.  Also, if said logfiles are
being placed in directories where users of those virtualhosts can remove
the files (and make symlinks to other places), that's a security hole
(because Apache opens webserver logfiles as root).

The way we do it is much more resource-friendly: log everything to a
single logfile, then every night split the logfile up (based on the
CustomLog %v parameter into per-vhost log files.  Apache comes with a
script to do this called split-logfile.

> I don't know what files are "really" open in the meaning of  
> kern.maxfiles. I have webserver with about 100 hosted domains and there  
> is some numbers:
>
> root at roxy ~/# fstat -u www | wc -l
>     9931

I don't think this is an accurate portrait of the number of open files.
The number is going to be too high; I believe entries that contain
FD=jail/mmap/root/text/tr/wd are not actual descriptors (are they?)

> root at roxy ~/# fstat -u root | wc -l
>      718
> root at roxy ~/# fstat | grep httpd | wc -l
>     6379
> root at roxy ~/# fstat | grep httpd | wc -l
>     6002
> root at roxy ~/# fstat -u www | wc -l
>     4691
> root at roxy ~/# sysctl kern.openfiles
> kern.openfiles: 846
>
> All above taken within few seconds.
>
> Can somebody explain the difference between kern.openfiles and fstat?

-- 
| Jeremy Chadwick                                jdc at parodius.com |
| Parodius Networking                       http://www.parodius.com/ |
| UNIX Systems Administrator                  Mountain View, CA, USA |
| Making life hard for others since 1977.              PGP: 4BD6C0CB |