svn commit: r188727 - in stable/7: sys sys/contrib/pf sys/dev/ath/ath_hal sys/dev/cxgb sys/kern sys/modules/sem sys/sys tools/regression/posixsem usr.bin/procstat

[Bruce Simpson]

John Baldwin wrote:
> Hmm, by symbols I meant having things built with debug symbols (i.e. "-g").
>   

OK... I will try to build a Python interpreter w/debug symbols next time 
around.

I am just giving this a dry run for now -- I am collaborating on a 
project where we've ended up implementing stuff on Linux using Python 
multiprocessing as it seemed the best fit for the team's capabilities.

So I'm posting this here for lists to pick up, hopefully someone looking 
to use multiprocessing seriously on FreeBSD will come across this.

It would be nice to port to FreeBSD at a later date, unfortunately, 
we're a bit further down this road now...

> Also, do you have 'sem.ko' loaded?  The only reason I can think of why you 
> would get a core dump in ksem_open() itself would be if you got a SIGSYS 
> because the module wasn't loaded.
>   

That seems to have been the issue! Thanks.
If I cd src/modules/sem && make && make install, and kldload the module 
before running Python, then I do not see the core dump.
I didn't realise a separate kernel module was needed for POSIX 
semaphores, the option seems to be P1003_1B_SEMAPHORES according to 
sem(4). It is quite possible the Python guys totally missed this...

It seems that whilst ipcs(1) is the appropriate tool to examine SYSV IPC 
facilities, there is no such tool for POSIX semaphores on FreeBSD, or am 
I wrong? When the Python process is running and using sem(4), I see a 
segment of shared memory in ipcs(1) output.

Perhaps the libc wrapper should be taught to return ENOSYS if the system 
call(s) it requires are not present, rather than SIGSYS?

However, the mp_benchmarks.py script just hangs when testing 
multiprocessing.Queue. A ktrace reveals that a child process is simply 
waiting for data on a pipe which doesn't arrive.

Killing Python at this point and restarting results in future attempts 
to open a semaphore by the module failing with ENFILE after the 4th 
ksem_open() call:
%%%
Traceback (most recent call last):
  File "Doc/includes/mp_benchmarks.py", line 238, in <module>
    test()
  File "Doc/includes/mp_benchmarks.py", line 192, in test
    multiprocessing.Condition())
  File 
"/usr/local/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-freebsd-7.1-STABLE-amd64.egg/multiprocessing/__init__.py", 
line 190, in Condition
    return Condition(lock)
  File 
"/usr/local/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-freebsd-7.1-STABLE-amd64.egg/multiprocessing/synchronize.py", 
line 168, in __init__
    self._lock = lock or RLock()
  File 
"/usr/local/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-freebsd-7.1-STABLE-amd64.egg/multiprocessing/synchronize.py", 
line 142, in __init__
    SemLock.__init__(self, RECURSIVE_MUTEX, 1, 1)
  File 
"/usr/local/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-freebsd-7.1-STABLE-amd64.egg/multiprocessing/synchronize.py", 
line 49, in __init__
    sl = self._semlock = _multiprocessing.SemLock(kind, value, maxvalue)
OSError: [Errno 23] Too many open files in system
Exit 1
anglepoise:~/p/multiprocessing-2.6.1.1 %
%%%