[Bug 203900] Can not increase above 32K max open files descriptor per PID

[bugzilla-noreply at freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203900

            Bug ID: 203900
           Summary: Can not increase above 32K max open files descriptor
                    per PID
           Product: Base System
           Version: 10.1-STABLE
          Hardware: amd64
                OS: Any
            Status: New
          Severity: Affects Some People
          Priority: ---
         Component: kern
          Assignee: freebsd-bugs at FreeBSD.org
          Reporter: ri24l_pwt at yahoo.com

I run application which is sometimes opening >200k files at the same time. But,
I get issue about "Too many open files". To test it, simply I create many dummy
files and open it. Here the steps

Download source code to generate & load files: 
 * https://gist.github.com/juniorh/3b2fb0a80cddb8e407b3
 * https://gist.github.com/juniorh/ef9273911dee551f1048

# create files
> mkdir /tmp/1
> python dummyFileGenerator.py -r 1 -n 1000000 -d /tmp/1

# open all files
> python dummyFileLoader.py -r 1 -n 1000000 -d /tmp/1 

Then, after the script open 32766 files, error occures like:
=================================
...
...
open file number: 30763
retry openfile 32766
Traceback (most recent call last):
  File "dummyFileLoader.py", line 69, in <module>
  File "dummyFileLoader.py", line 54, in openfile
IOError: [Errno 24] Too many open files: '/tmp/1/00/00/7f/fd'
=================================

It seems FreeBSD only alow each PID to open up to 32K files. Changing
kern.maxfiles and kern.maxfilesperproc is also not solved the issue. 

For additional, here my environtment:
=================================
# sysctl -a | grep maxfile
kern.maxfiles: 130305
kern.maxfilesperproc: 117270

# ulimit -a
cpu time               (seconds, -t)  unlimited
file size           (512-blocks, -f)  unlimited
data seg size           (kbytes, -d)  33554432
stack size              (kbytes, -s)  524288
core file size      (512-blocks, -c)  unlimited
max memory size         (kbytes, -m)  unlimited
locked memory           (kbytes, -l)  unlimited
max user processes              (-u)  8514
open files                      (-n)  117270
virtual mem size        (kbytes, -v)  unlimited
swap limit              (kbytes, -w)  unlimited
sbsize                   (bytes, -b)  unlimited
pseudo-terminals                (-p)  unlimited

# uname -a
FreeBSD fbd1 10.1-RELEASE FreeBSD 10.1-RELEASE #0 r274401: Tue Nov 11 21:02:49
UTC 2014     root at releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC  amd64
=================================

-- 
You are receiving this mail because:
You are the assignee for the bug.