From stephleg at free.fr Sun Mar 1 02:00:13 2009 From: stephleg at free.fr (Stephane Legrand) Date: Sun Mar 1 02:00:20 2009 Subject: threads/132215: Crash after running ppp Message-ID: <200903010956.n219uLdu083246@www.freebsd.org> >Number: 132215 >Category: threads >Synopsis: Crash after running ppp >Confidential: no >Severity: serious >Priority: low >Responsible: freebsd-threads >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Sun Mar 01 10:00:09 UTC 2009 >Closed-Date: >Last-Modified: >Originator: Stephane Legrand >Release: 7.1-STABLE >Organization: >Environment: 7.1-STABLE i386 >Description: Hi, After trying to update my system to the 7.1-STABLE of 28th february 2009, when i run ppp, it crashes the system. Here is the backtrace : sequoia# kgdb kernel.debug /var/crash/vmcore.0 GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd"... Unread portion of the kernel message buffer: Sleeping thread (tid 100053, pid 516) owns a non-sleepable lock panic: sleeping thread cpuid = 0 Uptime: 37s Physical memory: 1002 MB Dumping 158 MB: 143 127 111 95 79 63 47 31 15 Reading symbols from /boot/kernel/linprocfs.ko...Reading symbols from /boot/kernel/linprocfs.ko.symbols...done. done. Loaded symbols for /boot/kernel/linprocfs.ko Reading symbols from /boot/kernel/linux.ko...Reading symbols from /boot/kernel/linux.ko.symbols...done. done. Loaded symbols for /boot/kernel/linux.ko Reading symbols from /boot/kernel/linsysfs.ko...Reading symbols from /boot/kernel/linsysfs.ko.symbols...done. done. Loaded symbols for /boot/kernel/linsysfs.ko Reading symbols from /boot/kernel/snd_hda.ko...Reading symbols from /boot/kernel/snd_hda.ko.symbols...done. done. Loaded symbols for /boot/kernel/snd_hda.ko Reading symbols from /boot/kernel/sound.ko...Reading symbols from /boot/kernel/sound.ko.symbols...done. done. Loaded symbols for /boot/kernel/sound.ko Reading symbols from /boot/kernel/acpi.ko...Reading symbols from /boot/kernel/acpi.ko.symbols...done. done. Loaded symbols for /boot/kernel/acpi.ko Reading symbols from /boot/kernel/netgraph.ko...Reading symbols from /boot/kernel/netgraph.ko.symbols...done. done. Loaded symbols for /boot/kernel/netgraph.ko Reading symbols from /boot/kernel/ng_ether.ko...Reading symbols from /boot/kernel/ng_ether.ko.symbols...done. done. Loaded symbols for /boot/kernel/ng_ether.ko Reading symbols from /boot/kernel/ng_pppoe.ko...Reading symbols from /boot/kernel/ng_pppoe.ko.symbols...done. done. Loaded symbols for /boot/kernel/ng_pppoe.ko Reading symbols from /boot/kernel/ng_socket.ko...Reading symbols from /boot/kernel/ng_socket.ko.symbols...done. done. Loaded symbols for /boot/kernel/ng_socket.ko Reading symbols from /boot/kernel/ipfw.ko...Reading symbols from /boot/kernel/ipfw.ko.symbols...done. done. Loaded symbols for /boot/kernel/ipfw.ko Reading symbols from /usr/local/modules/fuse.ko...done. Loaded symbols for /usr/local/modules/fuse.ko Reading symbols from /boot/kernel/accf_http.ko...Reading symbols from /boot/kernel/accf_http.ko.symbols...done. done. Loaded symbols for /boot/kernel/accf_http.ko Reading symbols from /boot/kernel/i915.ko...done. Loaded symbols for /boot/kernel/i915.ko Reading symbols from /boot/kernel/drm.ko...done. Loaded symbols for /boot/kernel/drm.ko #0 doadump () at pcpu.h:196 196 __asm __volatile("movl %%fs:0,%0" : "=r" (td)); (kgdb) bt #0 doadump () at pcpu.h:196 #1 0xc075c6df in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418 #2 0xc075c9a4 in panic (fmt=Variable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:574 #3 0xc078eeec in propagate_priority (td=0xc444a690) at /usr/src/sys/kern/subr_turnstile.c:222 #4 0xc078fd11 in turnstile_wait (ts=0xc4260e60, owner=0xc444a690, queue=Variable "queue" is not available. ) at /usr/src/sys/kern/subr_turnstile.c:740 #5 0xc074f6b3 in _mtx_lock_sleep (m=0xc4267e7c, tid=3297722368, opts=0, file=0x0, line=0) at /usr/src/sys/kern/kern_mutex.c:420 #6 0xc0800fbe in rtalloc1_fib (dst=0xe6a1caa8, report=0, ignflags=0, fibnum=0) at /usr/src/sys/net/route.c:296 #7 0xc080194c in rtalloc1 (dst=0xe6a1caa8, report=0, ignflags=0) at /usr/src/sys/net/route.c:266 #8 0xc08b8c74 in selectroute (dstsock=0xc48367c0, opts=0x0, mopts=Variable "mopts" is not available. ) at /usr/src/sys/netinet6/in6_src.c:597 #9 0xc08b8ec8 in in6_selectif (dstsock=0xc48367c0, opts=0x0, mopts=0x0, ro=0x0, retifp=0xe6a1cb54) at /usr/src/sys/netinet6/in6_src.c:673 #10 0xc08b936d in in6_selectsrc (dstsock=0xc48367c0, opts=0x0, inp=0xc450a168, ro=0x0, cred=0xc49d1600, ifpp=0xe6a1cb9c, errorp=0xe6a1cba0) at /usr/src/sys/netinet6/in6_src.c:261 #11 0xc08b6acb in in6_pcbladdr (inp=0xc450a168, nam=0xc48367c0, plocal_addr6=0xe6a1cbd8) at /usr/src/sys/netinet6/in6_pcb.c:325 #12 0xc08b705d in in6_pcbconnect (inp=0xc450a168, nam=0xc48367c0, cred=0xc49d1600) at /usr/src/sys/netinet6/in6_pcb.c:370 #13 0xc08d0b4c in udp6_connect (so=0xc4a3c000, nam=0xc48367c0, td=0xc48f4000) at /usr/src/sys/netinet6/udp6_usrreq.c:867 #14 0xc07acda7 in soconnect (so=0xc4a3c000, nam=0xc48367c0, td=0xc48f4000) at /usr/src/sys/kern/uipc_socket.c:768 #15 0xc07b3ebc in kern_connect (td=0xc48f4000, fd=18, sa=0xc48367c0) at /usr/src/sys/kern/uipc_syscalls.c:570 #16 0xc07b4041 in connect (td=0xc48f4000, uap=0xe6a1ccfc) at /usr/src/sys/kern/uipc_syscalls.c:534 #17 0xc09fdd08 in syscall (frame=0xe6a1cd38) at /usr/src/sys/i386/i386/trap.c:1090 #18 0xc09e4910 in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:255 #19 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) With a -STABLE kernel of 10th january 2009, ppp works fine. Regards, Stephane Legrand. >How-To-Repeat: - Install a -STABLE kernel of 28th february 2009. - run ppp. - after a few seconds, the system crashes. >Fix: Use a kernel of 10th january 2009. >Release-Note: >Audit-Trail: >Unformatted: From bugmaster at FreeBSD.org Mon Mar 2 03:07:45 2009 From: bugmaster at FreeBSD.org (FreeBSD bugmaster) Date: Mon Mar 2 03:12:39 2009 Subject: Current problem reports assigned to freebsd-threads@FreeBSD.org Message-ID: <200903021107.n22B71H6057469@freefall.freebsd.org> Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o threa/132215 threads Crash after running ppp o threa/128922 threads threads hang with xorg running o threa/127225 threads bug in lib/libthr/thread/thr_init.c o threa/122923 threads 'nice' does not prevent background process from steali o threa/121336 threads lang/neko threading ok on UP, broken on SMP (FreeBSD 7 o threa/118715 threads kse problem o threa/116668 threads can no longer use jdk15 with libthr on -stable SMP o threa/116181 threads /dev/io-related io access permissions are not propagat o threa/115211 threads pthread_atfork misbehaves in initial thread o threa/110636 threads [request] gdb(1): using gdb with multi thread applicat o threa/110306 threads apache 2.0 segmentation violation when calling gethost o threa/103975 threads Implicit loading/unloading of libpthread.so may crash o threa/101323 threads [patch] fork(2) in threaded programs broken. s threa/100815 threads FBSD 5.5 broke nanosleep in libc_r s threa/94467 threads send(), sendto() and sendmsg() are not correct in libc s threa/84483 threads problems with devel/nspr and -lc_r on 4.x o threa/83914 threads [libc] popen() doesn't work in static threaded program o threa/80992 threads abort() sometimes not caught by gdb depending on threa o threa/80435 threads panic on high loads o threa/79887 threads [patch] freopen() isn't thread-safe o threa/79683 threads svctcp_create() fails if multiple threads call at the s threa/76694 threads fork cause hang in dup()/close() function in child (-l s threa/76690 threads fork hang in child for -lc_r o threa/75374 threads pthread_kill() ignores SA_SIGINFO flag o threa/75273 threads FBSD 5.3 libpthread (KSE) bug o threa/72953 threads fork() unblocks blocked signals w/o PTHREAD_SCOPE_SYST o threa/70975 threads [sysvipc] unexpected and unreliable behaviour when usi s threa/69020 threads pthreads library leaks _gc_mutex s threa/49087 threads Signals lost in programs linked with libc_r s threa/48856 threads Setting SIGCHLD to SIG_IGN still leaves zombies under s threa/40671 threads pthread_cancel doesn't remove thread from condition qu s threa/39922 threads [threads] [patch] Threaded applications executed with s threa/37676 threads libc_r: msgsnd(), msgrcv(), pread(), pwrite() need wra s threa/34536 threads accept() blocks other threads s threa/32295 threads [libc_r] [patch] pthread(3) dont dequeue signals s threa/30464 threads pthread mutex attributes -- pshared s threa/24632 threads libc_r delicate deviation from libc in handling SIGCHL s threa/24472 threads libc_r does not honor SO_SNDTIMEO/SO_RCVTIMEO socket o 38 problems total. From jhb at freebsd.org Mon Mar 2 08:11:57 2009 From: jhb at freebsd.org (John Baldwin) Date: Mon Mar 2 08:12:11 2009 Subject: threads/132215: Crash after running ppp In-Reply-To: <200903010956.n219uLdu083246@www.freebsd.org> References: <200903010956.n219uLdu083246@www.freebsd.org> Message-ID: <200903021111.32643.jhb@freebsd.org> On Sunday 01 March 2009 4:56:21 am Stephane Legrand wrote: > > >Number: 132215 > >Category: threads > >Synopsis: Crash after running ppp > >Confidential: no > >Severity: serious > >Priority: low > >Responsible: freebsd-threads > >State: open > >Quarter: > >Keywords: > >Date-Required: > >Class: sw-bug > >Submitter-Id: current-users > >Arrival-Date: Sun Mar 01 10:00:09 UTC 2009 > >Closed-Date: > >Last-Modified: > >Originator: Stephane Legrand > >Release: 7.1-STABLE > >Organization: > >Environment: > 7.1-STABLE i386 > >Description: > Hi, > > After trying to update my system to the 7.1-STABLE of 28th february 2009, when i run ppp, it crashes the system. Here is the backtrace : > > sequoia# kgdb kernel.debug /var/crash/vmcore.0 > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you are > welcome to change it and/or distribute copies of it under certain conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for details. > This GDB was configured as "i386-marcel-freebsd"... > > Unread portion of the kernel message buffer: > Sleeping thread (tid 100053, pid 516) owns a non-sleepable lock > panic: sleeping thread > cpuid = 0 > Uptime: 37s > Physical memory: 1002 MB > Dumping 158 MB: 143 127 111 95 79 63 47 31 15 > > Reading symbols from /boot/kernel/linprocfs.ko...Reading symbols from /boot/kernel/linprocfs.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/linprocfs.ko > Reading symbols from /boot/kernel/linux.ko...Reading symbols from /boot/kernel/linux.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/linux.ko > Reading symbols from /boot/kernel/linsysfs.ko...Reading symbols from /boot/kernel/linsysfs.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/linsysfs.ko > Reading symbols from /boot/kernel/snd_hda.ko...Reading symbols from /boot/kernel/snd_hda.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/snd_hda.ko > Reading symbols from /boot/kernel/sound.ko...Reading symbols from /boot/kernel/sound.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/sound.ko > Reading symbols from /boot/kernel/acpi.ko...Reading symbols from /boot/kernel/acpi.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/acpi.ko > Reading symbols from /boot/kernel/netgraph.ko...Reading symbols from /boot/kernel/netgraph.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/netgraph.ko > Reading symbols from /boot/kernel/ng_ether.ko...Reading symbols from /boot/kernel/ng_ether.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/ng_ether.ko > Reading symbols from /boot/kernel/ng_pppoe.ko...Reading symbols from /boot/kernel/ng_pppoe.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/ng_pppoe.ko > Reading symbols from /boot/kernel/ng_socket.ko...Reading symbols from /boot/kernel/ng_socket.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/ng_socket.ko > Reading symbols from /boot/kernel/ipfw.ko...Reading symbols from /boot/kernel/ipfw.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/ipfw.ko > Reading symbols from /usr/local/modules/fuse.ko...done. > Loaded symbols for /usr/local/modules/fuse.ko > Reading symbols from /boot/kernel/accf_http.ko...Reading symbols from /boot/kernel/accf_http.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/accf_http.ko > Reading symbols from /boot/kernel/i915.ko...done. > Loaded symbols for /boot/kernel/i915.ko > Reading symbols from /boot/kernel/drm.ko...done. > Loaded symbols for /boot/kernel/drm.ko > #0 doadump () at pcpu.h:196 > 196 __asm __volatile("movl %%fs:0,%0" : "=r" (td)); > (kgdb) bt > #0 doadump () at pcpu.h:196 > #1 0xc075c6df in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418 > #2 0xc075c9a4 in panic (fmt=Variable "fmt" is not available. > ) at /usr/src/sys/kern/kern_shutdown.c:574 > #3 0xc078eeec in propagate_priority (td=0xc444a690) at /usr/src/sys/kern/subr_turnstile.c:222 > #4 0xc078fd11 in turnstile_wait (ts=0xc4260e60, owner=0xc444a690, queue=Variable "queue" is not available. > ) at /usr/src/sys/kern/subr_turnstile.c:740 > #5 0xc074f6b3 in _mtx_lock_sleep (m=0xc4267e7c, tid=3297722368, opts=0, file=0x0, line=0) > at /usr/src/sys/kern/kern_mutex.c:420 Can you do something like this in kgdb: frame 5 set $td = (struct thread *)(m->mtx_lock & ~3) p $td->td_tid (This should print out a thread ID) thread $td->td_tid where This should give you the stack trace of the misbehaving thread which slept while holding a route lock. Also, this is a kernel bug, not a thread issue. -- John Baldwin From jhb at FreeBSD.org Mon Mar 2 08:13:10 2009 From: jhb at FreeBSD.org (jhb@FreeBSD.org) Date: Mon Mar 2 08:13:22 2009 Subject: kern/132215: [route locking] Crash after running ppp Message-ID: <200903021613.n22GD7Xq096055@freefall.freebsd.org> Old Synopsis: Crash after running ppp New Synopsis: [route locking] Crash after running ppp Responsible-Changed-From-To: freebsd-threads->freebsd-bugs Responsible-Changed-By: jhb Responsible-Changed-When: Mon Mar 2 16:12:07 UTC 2009 Responsible-Changed-Why: This not a threads-specific bug, but a bug in the kernel's routing table locking. http://www.freebsd.org/cgi/query-pr.cgi?pr=132215 From rwatson at FreeBSD.org Mon Mar 2 08:52:40 2009 From: rwatson at FreeBSD.org (Robert Watson) Date: Mon Mar 2 08:52:46 2009 Subject: threads/132215: Crash after running ppp In-Reply-To: <200903021111.32643.jhb@freebsd.org> References: <200903010956.n219uLdu083246@www.freebsd.org> <200903021111.32643.jhb@freebsd.org> Message-ID: On Mon, 2 Mar 2009, John Baldwin wrote: >> Unread portion of the kernel message buffer: >> Sleeping thread (tid 100053, pid 516) owns a non-sleepable lock >> panic: sleeping thread ... > Can you do something like this in kgdb: > > frame 5 > set $td = (struct thread *)(m->mtx_lock & ~3) > p $td->td_tid > > (This should print out a thread ID) > > thread $td->td_tid > where > > This should give you the stack trace of the misbehaving thread which slept > while holding a route lock. > > Also, this is a kernel bug, not a thread issue. Actually, just a stack trace of tid 100053/pid 516 should be sufficient. This is the second report I've had of a panic along these lines relating to ppp, and it seems likely to me that the fixes to routing locking have introduced an edge case where the routing code sleeps allocating or waiting for a route while holding a lock where it didn't happen previously. If we can work out the stack trace leading to the sleep, we can probably fix this in short order. Robert N M Watson Computer Laboratory University of Cambridge From pawel.worach at gmail.com Wed Mar 4 11:52:26 2009 From: pawel.worach at gmail.com (Pawel Worach) Date: Wed Mar 4 11:52:33 2009 Subject: libthr does not obey WITHOUT_SYSCALL_COMPAT Message-ID: Hi, If libc is built using WITHOUT_SYSCALL_COMPAT applications linked with libthr end up having unresolved symbols since libthr references __fcntl_compat unconditionally. Here is a patch to make libthr also obey WITHOUT_SYSCALL_COMPAT http://www.vlakno.cz/~pwo/libthr.diff Regards -- Pawel From davidxu at freebsd.org Sun Mar 8 19:35:29 2009 From: davidxu at freebsd.org (David Xu) Date: Sun Mar 8 19:35:34 2009 Subject: libthr does not obey WITHOUT_SYSCALL_COMPAT In-Reply-To: References: Message-ID: <49B480F7.8040800@freebsd.org> Pawel Worach wrote: > Hi, > > If libc is built using WITHOUT_SYSCALL_COMPAT applications linked with > libthr end up having unresolved symbols since libthr references > __fcntl_compat unconditionally. > Here is a patch to make libthr also obey WITHOUT_SYSCALL_COMPAT > http://www.vlakno.cz/~pwo/libthr.diff > > Regards Committed! Thanks, David Xu From deischen at freebsd.org Sun Mar 8 21:28:21 2009 From: deischen at freebsd.org (Daniel Eischen) Date: Sun Mar 8 21:28:27 2009 Subject: libthr does not obey WITHOUT_SYSCALL_COMPAT In-Reply-To: <49B480F7.8040800@freebsd.org> References: <49B480F7.8040800@freebsd.org> Message-ID: On Mon, 9 Mar 2009, David Xu wrote: > Pawel Worach wrote: >> Hi, >> >> If libc is built using WITHOUT_SYSCALL_COMPAT applications linked with >> libthr end up having unresolved symbols since libthr references >> __fcntl_compat unconditionally. >> Here is a patch to make libthr also obey WITHOUT_SYSCALL_COMPAT >> http://www.vlakno.cz/~pwo/libthr.diff >> >> Regards > > Committed! I never got around to replying to this... I don't quite understand why __fcntl_compat is there. We have F_GETFD, F_SETFD, F_DUPFD, F_DUP2FD, F_GETFL, F_SETFL, F_GETOWN, and F_SETOWN according to fcntl(2). But thr_syscalls.c only handles F_DUPFD, F_SETFD, F_SETFL, F_GETFD, and F_GETFL, leaving F_DUP2FD, F_GETOWN, and F_SETOWN to be handled by the default case. And the default case does nothing now if WITHOUT_SYSCALL_COMPAT is defined. So how do F_DUP2FD, F_GETOWN, and F_SETOWN get handled? Do we really need to call __sys_fcntl_compat() from libthr? When did the ABI change, before or after libc.so.7? -- DE From davidxu at freebsd.org Sun Mar 8 22:11:06 2009 From: davidxu at freebsd.org (David Xu) Date: Sun Mar 8 22:11:24 2009 Subject: libthr does not obey WITHOUT_SYSCALL_COMPAT In-Reply-To: References: <49B480F7.8040800@freebsd.org> Message-ID: <49B4A571.7000302@freebsd.org> Daniel Eischen wrote: > On Mon, 9 Mar 2009, David Xu wrote: > >> Pawel Worach wrote: >>> Hi, >>> >>> If libc is built using WITHOUT_SYSCALL_COMPAT applications linked with >>> libthr end up having unresolved symbols since libthr references >>> __fcntl_compat unconditionally. >>> Here is a patch to make libthr also obey WITHOUT_SYSCALL_COMPAT >>> http://www.vlakno.cz/~pwo/libthr.diff >>> >>> Regards >> >> Committed! > > I never got around to replying to this... > > I don't quite understand why __fcntl_compat is there. We have > F_GETFD, F_SETFD, F_DUPFD, F_DUP2FD, F_GETFL, F_SETFL, F_GETOWN, > and F_SETOWN according to fcntl(2). But thr_syscalls.c only > handles F_DUPFD, F_SETFD, F_SETFL, F_GETFD, and F_GETFL, leaving > F_DUP2FD, F_GETOWN, and F_SETOWN to be handled by the default > case. And the default case does nothing now if WITHOUT_SYSCALL_COMPAT > is defined. So how do F_DUP2FD, F_GETOWN, and F_SETOWN get > handled? > > Do we really need to call __sys_fcntl_compat() from libthr? > When did the ABI change, before or after libc.so.7? > I don't know when it appeared. Would this patch eliminate the shit ? Index: thr_syscalls.c =================================================================== --- thr_syscalls.c (revision 189549) +++ thr_syscalls.c (working copy) @@ -258,7 +258,7 @@ #ifdef SYSCALL_COMPAT ret = __fcntl_compat(fd, cmd, va_arg(ap, void *)); #else - ret = EOPNOTSUPP; + ret = __sys_fcntl(fd, cmd, va_arg(ap, void *)); #endif } va_end(ap); From deischen at freebsd.org Sun Mar 8 22:23:25 2009 From: deischen at freebsd.org (Daniel Eischen) Date: Sun Mar 8 22:23:31 2009 Subject: libthr does not obey WITHOUT_SYSCALL_COMPAT In-Reply-To: <49B4A571.7000302@freebsd.org> References: <49B480F7.8040800@freebsd.org> <49B4A571.7000302@freebsd.org> Message-ID: On Mon, 9 Mar 2009, David Xu wrote: > Daniel Eischen wrote: >> On Mon, 9 Mar 2009, David Xu wrote: >> >>> Pawel Worach wrote: >>>> Hi, >>>> >>>> If libc is built using WITHOUT_SYSCALL_COMPAT applications linked with >>>> libthr end up having unresolved symbols since libthr references >>>> __fcntl_compat unconditionally. >>>> Here is a patch to make libthr also obey WITHOUT_SYSCALL_COMPAT >>>> http://www.vlakno.cz/~pwo/libthr.diff >>>> >>>> Regards >>> >>> Committed! >> >> I never got around to replying to this... >> >> I don't quite understand why __fcntl_compat is there. We have >> F_GETFD, F_SETFD, F_DUPFD, F_DUP2FD, F_GETFL, F_SETFL, F_GETOWN, >> and F_SETOWN according to fcntl(2). But thr_syscalls.c only >> handles F_DUPFD, F_SETFD, F_SETFL, F_GETFD, and F_GETFL, leaving >> F_DUP2FD, F_GETOWN, and F_SETOWN to be handled by the default >> case. And the default case does nothing now if WITHOUT_SYSCALL_COMPAT >> is defined. So how do F_DUP2FD, F_GETOWN, and F_SETOWN get >> handled? >> >> Do we really need to call __sys_fcntl_compat() from libthr? >> When did the ABI change, before or after libc.so.7? >> > > I don't know when it appeared. Would this patch eliminate the shit ? I think so. But I think for future ABI changes to cancellation points, wouldn't we need syscall wrappers in libc? Reason, libc and libthr are now symbol-versioned, so there will no longer be library bumps for ABI changes. But if a syscall is a cancellation point and libthr has to play games with it, like fcntl, I think there should be a wrapper around it in libc. If the ABI changes, then both libc and libthr would need to provide a compat version for it. I think. ;-) -- DE From davidxu at freebsd.org Sun Mar 8 23:00:08 2009 From: davidxu at freebsd.org (David Xu) Date: Sun Mar 8 23:00:15 2009 Subject: libthr does not obey WITHOUT_SYSCALL_COMPAT In-Reply-To: References: <49B480F7.8040800@freebsd.org> <49B4A571.7000302@freebsd.org> Message-ID: <49B4B0EF.5080507@freebsd.org> Daniel Eischen wrote: > On Mon, 9 Mar 2009, David Xu wrote: > >> Daniel Eischen wrote: >>> On Mon, 9 Mar 2009, David Xu wrote: >>> >>>> Pawel Worach wrote: >>>>> Hi, >>>>> >>>>> If libc is built using WITHOUT_SYSCALL_COMPAT applications linked with >>>>> libthr end up having unresolved symbols since libthr references >>>>> __fcntl_compat unconditionally. >>>>> Here is a patch to make libthr also obey WITHOUT_SYSCALL_COMPAT >>>>> http://www.vlakno.cz/~pwo/libthr.diff >>>>> >>>>> Regards >>>> >>>> Committed! >>> >>> I never got around to replying to this... >>> >>> I don't quite understand why __fcntl_compat is there. We have >>> F_GETFD, F_SETFD, F_DUPFD, F_DUP2FD, F_GETFL, F_SETFL, F_GETOWN, >>> and F_SETOWN according to fcntl(2). But thr_syscalls.c only >>> handles F_DUPFD, F_SETFD, F_SETFL, F_GETFD, and F_GETFL, leaving >>> F_DUP2FD, F_GETOWN, and F_SETOWN to be handled by the default >>> case. And the default case does nothing now if WITHOUT_SYSCALL_COMPAT >>> is defined. So how do F_DUP2FD, F_GETOWN, and F_SETOWN get >>> handled? >>> >>> Do we really need to call __sys_fcntl_compat() from libthr? >>> When did the ABI change, before or after libc.so.7? >>> >> >> I don't know when it appeared. Would this patch eliminate the shit ? > > I think so. But I think for future ABI changes to cancellation > points, wouldn't we need syscall wrappers in libc? Reason, libc > and libthr are now symbol-versioned, so there will no longer be > library bumps for ABI changes. But if a syscall is a cancellation > point and libthr has to play games with it, like fcntl, I think > there should be a wrapper around it in libc. If the ABI changes, > then both libc and libthr would need to provide a compat version > for it. I think. ;-) > Yes, it is better to use versioning instead, I don't know why fcntl_compat is there without using this feature. From deischen at freebsd.org Mon Mar 9 05:00:27 2009 From: deischen at freebsd.org (Daniel Eischen) Date: Mon Mar 9 05:00:36 2009 Subject: libthr does not obey WITHOUT_SYSCALL_COMPAT In-Reply-To: <49B4B0EF.5080507@freebsd.org> References: <49B480F7.8040800@freebsd.org> <49B4A571.7000302@freebsd.org> <49B4B0EF.5080507@freebsd.org> Message-ID: On Mon, 9 Mar 2009, David Xu wrote: > Daniel Eischen wrote: >> On Mon, 9 Mar 2009, David Xu wrote: >> >>> I don't know when it appeared. Would this patch eliminate the shit ? >> >> I think so. But I think for future ABI changes to cancellation >> points, wouldn't we need syscall wrappers in libc? Reason, libc >> and libthr are now symbol-versioned, so there will no longer be >> library bumps for ABI changes. But if a syscall is a cancellation >> point and libthr has to play games with it, like fcntl, I think >> there should be a wrapper around it in libc. If the ABI changes, >> then both libc and libthr would need to provide a compat version >> for it. I think. ;-) >> > > Yes, it is better to use versioning instead, I don't know why fcntl_compat is > there without using this feature. Hmm, looking a little closer at this... It looks like there is a wrapper for fcntl in libc (libc/sys/fcntl.c). But I think it should always be built. Currently, it is hidden behind WITHOUT_SYSCALL_COMPAT in libc/sys/Makefile. But it never calls an older fcntl syscall. It just transforms the old style struct into new style, then calls the new fcntl syscall. So I think the mistake is building fcntl.c conditionally, when it should always be built regardless of WITHOUT_SYSCALL_COMPAT being defined or not defined. -- DE From deischen at freebsd.org Mon Mar 9 05:15:19 2009 From: deischen at freebsd.org (Daniel Eischen) Date: Mon Mar 9 05:15:25 2009 Subject: libthr does not obey WITHOUT_SYSCALL_COMPAT In-Reply-To: References: <49B480F7.8040800@freebsd.org> <49B4A571.7000302@freebsd.org> <49B4B0EF.5080507@freebsd.org> Message-ID: On Mon, 9 Mar 2009, Daniel Eischen wrote: > On Mon, 9 Mar 2009, David Xu wrote: > >> Daniel Eischen wrote: >>> On Mon, 9 Mar 2009, David Xu wrote: >>> >>>> I don't know when it appeared. Would this patch eliminate the shit ? >>> >>> I think so. But I think for future ABI changes to cancellation >>> points, wouldn't we need syscall wrappers in libc? Reason, libc >>> and libthr are now symbol-versioned, so there will no longer be >>> library bumps for ABI changes. But if a syscall is a cancellation >>> point and libthr has to play games with it, like fcntl, I think >>> there should be a wrapper around it in libc. If the ABI changes, >>> then both libc and libthr would need to provide a compat version >>> for it. I think. ;-) >>> >> >> Yes, it is better to use versioning instead, I don't know why fcntl_compat >> is there without using this feature. > > Hmm, looking a little closer at this... It looks like there > is a wrapper for fcntl in libc (libc/sys/fcntl.c). But I think > it should always be built. Currently, it is hidden behind > WITHOUT_SYSCALL_COMPAT in libc/sys/Makefile. But it never calls > an older fcntl syscall. It just transforms the old style struct ^^^ new > into new style, then calls the new fcntl syscall. ^^^ old > So I think the mistake is building fcntl.c conditionally, when > it should always be built regardless of WITHOUT_SYSCALL_COMPAT > being defined or not defined. I guess if the default is with syscall compat, then it is okay to build optionally without it. -- DE From bugmaster at FreeBSD.org Mon Mar 9 10:15:19 2009 From: bugmaster at FreeBSD.org (FreeBSD bugmaster) Date: Mon Mar 9 10:17:38 2009 Subject: Current problem reports assigned to freebsd-threads@FreeBSD.org Message-ID: <200903091715.n29HFDOB045414@freefall.freebsd.org> Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o threa/128922 threads threads hang with xorg running o threa/127225 threads bug in lib/libthr/thread/thr_init.c o threa/122923 threads 'nice' does not prevent background process from steali o threa/121336 threads lang/neko threading ok on UP, broken on SMP (FreeBSD 7 o threa/118715 threads kse problem o threa/116668 threads can no longer use jdk15 with libthr on -stable SMP o threa/116181 threads /dev/io-related io access permissions are not propagat o threa/115211 threads pthread_atfork misbehaves in initial thread o threa/110636 threads [request] gdb(1): using gdb with multi thread applicat o threa/110306 threads apache 2.0 segmentation violation when calling gethost o threa/103975 threads Implicit loading/unloading of libpthread.so may crash o threa/101323 threads [patch] fork(2) in threaded programs broken. s threa/100815 threads FBSD 5.5 broke nanosleep in libc_r s threa/94467 threads send(), sendto() and sendmsg() are not correct in libc s threa/84483 threads problems with devel/nspr and -lc_r on 4.x o threa/83914 threads [libc] popen() doesn't work in static threaded program o threa/80992 threads abort() sometimes not caught by gdb depending on threa o threa/80435 threads panic on high loads o threa/79887 threads [patch] freopen() isn't thread-safe o threa/79683 threads svctcp_create() fails if multiple threads call at the s threa/76694 threads fork cause hang in dup()/close() function in child (-l s threa/76690 threads fork hang in child for -lc_r o threa/75374 threads pthread_kill() ignores SA_SIGINFO flag o threa/75273 threads FBSD 5.3 libpthread (KSE) bug o threa/72953 threads fork() unblocks blocked signals w/o PTHREAD_SCOPE_SYST o threa/70975 threads [sysvipc] unexpected and unreliable behaviour when usi s threa/69020 threads pthreads library leaks _gc_mutex s threa/49087 threads Signals lost in programs linked with libc_r s threa/48856 threads Setting SIGCHLD to SIG_IGN still leaves zombies under s threa/40671 threads pthread_cancel doesn't remove thread from condition qu s threa/39922 threads [threads] [patch] Threaded applications executed with s threa/37676 threads libc_r: msgsnd(), msgrcv(), pread(), pwrite() need wra s threa/34536 threads accept() blocks other threads s threa/32295 threads [libc_r] [patch] pthread(3) dont dequeue signals s threa/30464 threads pthread mutex attributes -- pshared s threa/24632 threads libc_r delicate deviation from libc in handling SIGCHL s threa/24472 threads libc_r does not honor SO_SNDTIMEO/SO_RCVTIMEO socket o 37 problems total. From nkoch at demig.de Mon Mar 16 01:46:10 2009 From: nkoch at demig.de (Norbert Koch) Date: Mon Mar 16 01:46:17 2009 Subject: 'pthread_mutex_unlock() BUG in libc_r? Message-ID: <49BE0E07.50908@demig.de> Hello, first of all, I know that libc_r is an ancient and no more supported library. But what I found looks like such a huge bug that I would like to hear a different opinion about that. Because, may be I am totally wrong with that. (Btw. I am doing this under FreeBSD 4.11 but libc_r looks the same under 7.1) I am having two threads A (low priority) and B (high priority). This is the pseudo code: Thread A: forever { sleep_some_time() call common_func() set global ponter P to NULL } Thread B: forever { sleep_some_time() set global pointer P to not NULL call common_func() assert P is not NULL } common_func() { pthread_lock_mutex(M) do something completely_unrelated pthread_mutex_unlock(M) } Without calling common_func() all is ok. With common_func the assertion fails from time to time. The problem I found is in pthread_mutex_unlock(). As far as I understand the code, the next thread waiting on the mutex is made runnable, but there is no call of the scheduler! This means, that the lower priority thread will continue running until blocking itself or being switched away by running out of time. I see the same (wrong?) code in pthread_cond_signal() and pthread_cond_broadcast(). So, could anyone please enlighten me if I am totally wrong here or if there is perhaps some weird hidden aspect in the posix standard I missed? If I found a bug here, what about libthr/libpthread? Are they behaving correctly? Best regards, Norbert Koch From deischen at freebsd.org Mon Mar 16 02:32:29 2009 From: deischen at freebsd.org (Daniel Eischen) Date: Mon Mar 16 02:32:35 2009 Subject: 'pthread_mutex_unlock() BUG in libc_r? In-Reply-To: <49BE0E07.50908@demig.de> References: <49BE0E07.50908@demig.de> Message-ID: On Mon, 16 Mar 2009, Norbert Koch wrote: > Hello, > > first of all, I know that libc_r is an ancient and > no more supported library. > But what I found looks like such a huge bug that > I would like to hear a different opinion about > that. Because, may be I am totally wrong with that. > (Btw. I am doing this under FreeBSD 4.11 but libc_r > looks the same under 7.1) > > I am having two threads A (low priority) and B (high priority). > This is the pseudo code: > > Thread A: > forever { > sleep_some_time() > call common_func() > set global ponter P to NULL > } > > Thread B: > forever { > sleep_some_time() > set global pointer P to not NULL > call common_func() > assert P is not NULL > } > > common_func() > { > pthread_lock_mutex(M) > do something completely_unrelated > pthread_mutex_unlock(M) > } There is no guarantee that pthread_mutex_unlock() (or other unlocks of synchronization types) are preemption points. Also, you cannot assume that the run time for each thread is the same. The threads are started at separate times, and there may even be a scheduling preemption point between the two successful calls to pthread_create() causing one thread to start before the next thread is created. There is no guarantee that sleep_some_time() ends exactly the same for both threads. In order for this to work, you need both threads A and B to block on a synchronization variable (mutex, CV, etc) and to have the main thread (or another thread) wake them up after ensuring that they both are waiting. -- DE From bugmaster at FreeBSD.org Mon Mar 16 04:07:05 2009 From: bugmaster at FreeBSD.org (FreeBSD bugmaster) Date: Mon Mar 16 04:09:32 2009 Subject: Current problem reports assigned to freebsd-threads@FreeBSD.org Message-ID: <200903161107.n2GB73hS043416@freefall.freebsd.org> Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o threa/128922 threads threads hang with xorg running o threa/127225 threads bug in lib/libthr/thread/thr_init.c o threa/122923 threads 'nice' does not prevent background process from steali o threa/121336 threads lang/neko threading ok on UP, broken on SMP (FreeBSD 7 o threa/118715 threads kse problem o threa/116668 threads can no longer use jdk15 with libthr on -stable SMP o threa/116181 threads /dev/io-related io access permissions are not propagat o threa/115211 threads pthread_atfork misbehaves in initial thread o threa/110636 threads [request] gdb(1): using gdb with multi thread applicat o threa/110306 threads apache 2.0 segmentation violation when calling gethost o threa/103975 threads Implicit loading/unloading of libpthread.so may crash o threa/101323 threads [patch] fork(2) in threaded programs broken. s threa/100815 threads FBSD 5.5 broke nanosleep in libc_r s threa/94467 threads send(), sendto() and sendmsg() are not correct in libc s threa/84483 threads problems with devel/nspr and -lc_r on 4.x o threa/83914 threads [libc] popen() doesn't work in static threaded program o threa/80992 threads abort() sometimes not caught by gdb depending on threa o threa/80435 threads panic on high loads o threa/79887 threads [patch] freopen() isn't thread-safe o threa/79683 threads svctcp_create() fails if multiple threads call at the s threa/76694 threads fork cause hang in dup()/close() function in child (-l s threa/76690 threads fork hang in child for -lc_r o threa/75374 threads pthread_kill() ignores SA_SIGINFO flag o threa/75273 threads FBSD 5.3 libpthread (KSE) bug o threa/72953 threads fork() unblocks blocked signals w/o PTHREAD_SCOPE_SYST o threa/70975 threads [sysvipc] unexpected and unreliable behaviour when usi s threa/69020 threads pthreads library leaks _gc_mutex s threa/49087 threads Signals lost in programs linked with libc_r s threa/48856 threads Setting SIGCHLD to SIG_IGN still leaves zombies under s threa/40671 threads pthread_cancel doesn't remove thread from condition qu s threa/39922 threads [threads] [patch] Threaded applications executed with s threa/37676 threads libc_r: msgsnd(), msgrcv(), pread(), pwrite() need wra s threa/34536 threads accept() blocks other threads s threa/32295 threads [libc_r] [patch] pthread(3) dont dequeue signals s threa/30464 threads pthread mutex attributes -- pshared s threa/24632 threads libc_r delicate deviation from libc in handling SIGCHL s threa/24472 threads libc_r does not honor SO_SNDTIMEO/SO_RCVTIMEO socket o 37 problems total. From bugmaster at FreeBSD.org Mon Mar 23 04:07:06 2009 From: bugmaster at FreeBSD.org (FreeBSD bugmaster) Date: Mon Mar 23 04:09:30 2009 Subject: Current problem reports assigned to freebsd-threads@FreeBSD.org Message-ID: <200903231107.n2NB74vU004170@freefall.freebsd.org> Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o threa/128922 threads threads hang with xorg running o threa/127225 threads bug in lib/libthr/thread/thr_init.c o threa/122923 threads 'nice' does not prevent background process from steali o threa/121336 threads lang/neko threading ok on UP, broken on SMP (FreeBSD 7 o threa/118715 threads kse problem o threa/116668 threads can no longer use jdk15 with libthr on -stable SMP o threa/116181 threads /dev/io-related io access permissions are not propagat o threa/115211 threads pthread_atfork misbehaves in initial thread o threa/110636 threads [request] gdb(1): using gdb with multi thread applicat o threa/110306 threads apache 2.0 segmentation violation when calling gethost o threa/103975 threads Implicit loading/unloading of libpthread.so may crash o threa/101323 threads [patch] fork(2) in threaded programs broken. s threa/100815 threads FBSD 5.5 broke nanosleep in libc_r s threa/94467 threads send(), sendto() and sendmsg() are not correct in libc s threa/84483 threads problems with devel/nspr and -lc_r on 4.x o threa/83914 threads [libc] popen() doesn't work in static threaded program o threa/80992 threads abort() sometimes not caught by gdb depending on threa o threa/80435 threads panic on high loads o threa/79887 threads [patch] freopen() isn't thread-safe o threa/79683 threads svctcp_create() fails if multiple threads call at the s threa/76694 threads fork cause hang in dup()/close() function in child (-l s threa/76690 threads fork hang in child for -lc_r o threa/75374 threads pthread_kill() ignores SA_SIGINFO flag o threa/75273 threads FBSD 5.3 libpthread (KSE) bug o threa/72953 threads fork() unblocks blocked signals w/o PTHREAD_SCOPE_SYST o threa/70975 threads [sysvipc] unexpected and unreliable behaviour when usi s threa/69020 threads pthreads library leaks _gc_mutex s threa/49087 threads Signals lost in programs linked with libc_r s threa/48856 threads Setting SIGCHLD to SIG_IGN still leaves zombies under s threa/40671 threads pthread_cancel doesn't remove thread from condition qu s threa/39922 threads [threads] [patch] Threaded applications executed with s threa/37676 threads libc_r: msgsnd(), msgrcv(), pread(), pwrite() need wra s threa/34536 threads accept() blocks other threads s threa/32295 threads [libc_r] [patch] pthread(3) dont dequeue signals s threa/30464 threads pthread mutex attributes -- pshared s threa/24632 threads libc_r delicate deviation from libc in handling SIGCHL s threa/24472 threads libc_r does not honor SO_SNDTIMEO/SO_RCVTIMEO socket o 37 problems total. From bugmaster at FreeBSD.org Mon Mar 30 04:07:02 2009 From: bugmaster at FreeBSD.org (FreeBSD bugmaster) Date: Mon Mar 30 04:09:28 2009 Subject: Current problem reports assigned to freebsd-threads@FreeBSD.org Message-ID: <200903301107.n2UB71ZV054926@freefall.freebsd.org> Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o threa/128922 threads threads hang with xorg running o threa/127225 threads bug in lib/libthr/thread/thr_init.c o threa/122923 threads 'nice' does not prevent background process from steali o threa/121336 threads lang/neko threading ok on UP, broken on SMP (FreeBSD 7 o threa/118715 threads kse problem o threa/116668 threads can no longer use jdk15 with libthr on -stable SMP o threa/116181 threads /dev/io-related io access permissions are not propagat o threa/115211 threads pthread_atfork misbehaves in initial thread o threa/110636 threads [request] gdb(1): using gdb with multi thread applicat o threa/110306 threads apache 2.0 segmentation violation when calling gethost o threa/103975 threads Implicit loading/unloading of libpthread.so may crash o threa/101323 threads [patch] fork(2) in threaded programs broken. s threa/100815 threads FBSD 5.5 broke nanosleep in libc_r s threa/94467 threads send(), sendto() and sendmsg() are not correct in libc s threa/84483 threads problems with devel/nspr and -lc_r on 4.x o threa/83914 threads [libc] popen() doesn't work in static threaded program o threa/80992 threads abort() sometimes not caught by gdb depending on threa o threa/80435 threads panic on high loads o threa/79887 threads [patch] freopen() isn't thread-safe o threa/79683 threads svctcp_create() fails if multiple threads call at the s threa/76694 threads fork cause hang in dup()/close() function in child (-l s threa/76690 threads fork hang in child for -lc_r o threa/75374 threads pthread_kill() ignores SA_SIGINFO flag o threa/75273 threads FBSD 5.3 libpthread (KSE) bug o threa/72953 threads fork() unblocks blocked signals w/o PTHREAD_SCOPE_SYST o threa/70975 threads [sysvipc] unexpected and unreliable behaviour when usi s threa/69020 threads pthreads library leaks _gc_mutex s threa/49087 threads Signals lost in programs linked with libc_r s threa/48856 threads Setting SIGCHLD to SIG_IGN still leaves zombies under s threa/40671 threads pthread_cancel doesn't remove thread from condition qu s threa/39922 threads [threads] [patch] Threaded applications executed with s threa/37676 threads libc_r: msgsnd(), msgrcv(), pread(), pwrite() need wra s threa/34536 threads accept() blocks other threads s threa/32295 threads [libc_r] [patch] pthread(3) dont dequeue signals s threa/30464 threads pthread mutex attributes -- pshared s threa/24632 threads libc_r delicate deviation from libc in handling SIGCHL s threa/24472 threads libc_r does not honor SO_SNDTIMEO/SO_RCVTIMEO socket o 37 problems total. From rrs at lakerest.net Mon Mar 30 14:10:28 2009 From: rrs at lakerest.net (Randall Stewart) Date: Mon Mar 30 14:10:34 2009 Subject: A mutex for inter-process ;-) Message-ID: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> Hi all: I have recently written a small set of routines that allow two process to have a "mutex" between them.. actually it allows all of the threads in any set of processes to have mutexes between themselves ;-) Anyway it seems to be working fairly well.. I still have to write a man page for it (documentation always last).. and eventually I would like to port in some of the WITNESS type features since the mutex's have names.. I probably should also think about scaling it up a bit.. right now its really more for a small scale (100 or less mutexes)... Who should I talk to about getting this in... having it reviewed etc. I think it belongs in libthr since it really needs the tid of the pthreads from the pthread_t type... and for now I have a horrible hack in to get it ;-) Thanks R ------------------------------ Randall Stewart 803-317-4952 (cell) 803-345-0391(direct) From delphij at delphij.net Mon Mar 30 14:16:49 2009 From: delphij at delphij.net (Xin LI) Date: Mon Mar 30 14:16:55 2009 Subject: A mutex for inter-process ;-) In-Reply-To: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> Message-ID: <49D136B1.6060809@delphij.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Randall Stewart wrote: > Hi all: > > I have recently written a small set of routines that allow > two process to have a "mutex" between them.. actually it allows > all of the threads in any set of processes to have mutexes between > themselves ;-) > > Anyway it seems to be working fairly well.. I still have to write a man > page > for it (documentation always last).. and eventually I would like to port in > some of the WITNESS type features since the mutex's have names.. > > I probably should also think about scaling it up a bit.. right now its > really > more for a small scale (100 or less mutexes)... > > Who should I talk to about getting this in... having it reviewed etc. I > think > it belongs in libthr since it really needs the tid of the pthreads from the > pthread_t type... and for now I have a horrible hack in to get it ;-) I think davidxu@ deischen@ and julian@? BTW. How do you handle with one process exit (abnormally) without releasing the mutex? Just curious :) Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.11 (FreeBSD) iEYEARECAAYFAknRNrEACgkQi+vbBBjt66DIswCbBWRMJN55c60UTBBIZMRCY4zo 6hcAnixfVXdtdnn0fT/Z31v0EdyVCVlH =JL/U -----END PGP SIGNATURE----- From deischen at freebsd.org Mon Mar 30 14:22:55 2009 From: deischen at freebsd.org (Daniel Eischen) Date: Mon Mar 30 14:23:05 2009 Subject: A mutex for inter-process ;-) In-Reply-To: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> Message-ID: On Mon, 30 Mar 2009, Randall Stewart wrote: > Hi all: > > I have recently written a small set of routines that allow > two process to have a "mutex" between them.. actually it allows > all of the threads in any set of processes to have mutexes between themselves > ;-) > > Anyway it seems to be working fairly well.. I still have to write a man page > for it (documentation always last).. and eventually I would like to port in > some of the WITNESS type features since the mutex's have names.. > > I probably should also think about scaling it up a bit.. right now its really > more for a small scale (100 or less mutexes)... > > Who should I talk to about getting this in... having it reviewed etc. I think > it belongs in libthr since it really needs the tid of the pthreads from the > pthread_t type... and for now I have a horrible hack in to get it ;-) The real way to do this is to support PTHREAD_PROCESS_SHARED mutexes within our normal mutex, and to change our current mutex (and cv) types to be structs instead of pointers. The current API, other than the type change, shouldn't change at all. -- DE From rrs at lakerest.net Mon Mar 30 16:29:22 2009 From: rrs at lakerest.net (Randall Stewart) Date: Mon Mar 30 16:29:28 2009 Subject: A mutex for inter-process ;-) In-Reply-To: <49D136B1.6060809@delphij.net> References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <49D136B1.6060809@delphij.net> Message-ID: <78DBBDDA-5A39-4CEB-8289-F36EFB96461D@lakerest.net> On Mar 30, 2009, at 5:16 PM, Xin LI wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Randall Stewart wrote: >> Hi all: >> >> I have recently written a small set of routines that allow >> two process to have a "mutex" between them.. actually it allows >> all of the threads in any set of processes to have mutexes between >> themselves ;-) >> >> Anyway it seems to be working fairly well.. I still have to write a >> man >> page >> for it (documentation always last).. and eventually I would like to >> port in >> some of the WITNESS type features since the mutex's have names.. >> >> I probably should also think about scaling it up a bit.. right now >> its >> really >> more for a small scale (100 or less mutexes)... >> >> Who should I talk to about getting this in... having it reviewed >> etc. I >> think >> it belongs in libthr since it really needs the tid of the pthreads >> from the >> pthread_t type... and for now I have a horrible hack in to get it ;-) > > I think davidxu@ deischen@ and julian@? > > BTW. How do you handle with one process exit (abnormally) without > releasing the mutex? Just curious :) I have a couple of ways of dealing with this.. 1) Of course the initialization routine calls atexit() to get a "cleanup handler" in place. 2) Often times, of course, this can fail e.g. you get a SIGSEGV.. or some such. When you attach the memory, an audit is done. The audit will validate that the pid is still alive and has the particular tid in it. Of course this is not 100% but as long as the tid's have not rolled over it should work. The function is also public (need to add that and many things to the manual pages ;-D) so that one can call it whenever one wants :-) I will ping Julian and the others... R > > > Cheers, > - -- > Xin LI http://www.delphij.net/ > FreeBSD - The Power to Serve! > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.0.11 (FreeBSD) > > iEYEARECAAYFAknRNrEACgkQi+vbBBjt66DIswCbBWRMJN55c60UTBBIZMRCY4zo > 6hcAnixfVXdtdnn0fT/Z31v0EdyVCVlH > =JL/U > -----END PGP SIGNATURE----- > ------------------------------ Randall Stewart 803-317-4952 (cell) 803-345-0391(direct) From rrs at lakerest.net Mon Mar 30 16:30:49 2009 From: rrs at lakerest.net (Randall Stewart) Date: Mon Mar 30 16:30:56 2009 Subject: A mutex for inter-process ;-) In-Reply-To: References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> Message-ID: On Mar 30, 2009, at 5:22 PM, Daniel Eischen wrote: > On Mon, 30 Mar 2009, Randall Stewart wrote: > >> Hi all: >> >> I have recently written a small set of routines that allow >> two process to have a "mutex" between them.. actually it allows >> all of the threads in any set of processes to have mutexes between >> themselves ;-) >> >> Anyway it seems to be working fairly well.. I still have to write a >> man page >> for it (documentation always last).. and eventually I would like to >> port in >> some of the WITNESS type features since the mutex's have names.. >> >> I probably should also think about scaling it up a bit.. right now >> its really >> more for a small scale (100 or less mutexes)... >> >> Who should I talk to about getting this in... having it reviewed >> etc. I think >> it belongs in libthr since it really needs the tid of the pthreads >> from the >> pthread_t type... and for now I have a horrible hack in to get it ;-) > > The real way to do this is to support PTHREAD_PROCESS_SHARED > mutexes within our normal mutex, and to change our current > mutex (and cv) types to be structs instead of pointers. > The current API, other than the type change, shouldn't > change at all. So how do you propose to name the mutex's so that two disparate process can locate the same mutex? I don't see how a pthread_mutex can suffice... we need more than just the current mutex... What am I missing? R > > > -- > DE > ------------------------------ Randall Stewart 803-317-4952 (cell) 803-345-0391(direct) From deischen at freebsd.org Mon Mar 30 16:56:13 2009 From: deischen at freebsd.org (Daniel Eischen) Date: Mon Mar 30 16:56:20 2009 Subject: A mutex for inter-process ;-) In-Reply-To: References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> Message-ID: On Mon, 30 Mar 2009, Randall Stewart wrote: > > On Mar 30, 2009, at 5:22 PM, Daniel Eischen wrote: > >> On Mon, 30 Mar 2009, Randall Stewart wrote: >> >>> Hi all: >>> >>> I have recently written a small set of routines that allow >>> two process to have a "mutex" between them.. actually it allows >>> all of the threads in any set of processes to have mutexes between >>> themselves ;-) >>> >>> Anyway it seems to be working fairly well.. I still have to write a man >>> page >>> for it (documentation always last).. and eventually I would like to port >>> in >>> some of the WITNESS type features since the mutex's have names.. >>> >>> I probably should also think about scaling it up a bit.. right now its >>> really >>> more for a small scale (100 or less mutexes)... >>> >>> Who should I talk to about getting this in... having it reviewed etc. I >>> think >>> it belongs in libthr since it really needs the tid of the pthreads from >>> the >>> pthread_t type... and for now I have a horrible hack in to get it ;-) >> >> The real way to do this is to support PTHREAD_PROCESS_SHARED >> mutexes within our normal mutex, and to change our current >> mutex (and cv) types to be structs instead of pointers. >> The current API, other than the type change, shouldn't >> change at all. > > > So how do you propose to name the mutex's so that two disparate > process can locate the same mutex? They are placed in shared memory, according to POSIX. > I don't see how a pthread_mutex can suffice... we need more than > just the current mutex... > > What am I missing? As far as I know, David Xu implemented the kernel hooks for umtx (the underlying mutex in pthread mutex) to be shared. As soon as you can place the entire userland pthread_mutex_t struct in shared memory, it should all just work (with probably some trivial changes in libthr). The harder part is versioning all the symbols that currently think pthread_mutex_t, pthread_cond_t, etc, are pointers, and defining the structs with enough foresight so that it is unlikely we have to modify them in the future (causing a future ABI breakage), and also aligning them nicely for the various archs. You should really look at how POSIX defines process shared mutex, cvs, etc. See: pthread_barrierattr_[gs]etpshared() pthread_condattr_[gs]etpshared() pthread_mutexattr_[gs]etpshared() pthread_wrlockattr_[gs]etsphared() You can use this as a starting point: http://www.opengroup.org/onlinepubs/009695399/ http://www.opengroup.org/onlinepubs/009695399/functions/pthread_barrierattr_setpshared.html http://www.opengroup.org/onlinepubs/009695399/functions/pthread_condattr_setpshared.html http://www.opengroup.org/onlinepubs/009695399/functions/pthread_mutexattr_setpshared.html http://www.opengroup.org/onlinepubs/009695399/functions/pthread_rwlockattr_setpshared.html -- DE From julian at elischer.org Mon Mar 30 18:05:33 2009 From: julian at elischer.org (Julian Elischer) Date: Mon Mar 30 18:05:40 2009 Subject: A mutex for inter-process ;-) In-Reply-To: <78DBBDDA-5A39-4CEB-8289-F36EFB96461D@lakerest.net> References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <49D136B1.6060809@delphij.net> <78DBBDDA-5A39-4CEB-8289-F36EFB96461D@lakerest.net> Message-ID: <49D16A0F.4000404@elischer.org> Randall Stewart wrote: > >> >> I think davidxu@ deischen@ and julian@? >> >> BTW. How do you handle with one process exit (abnormally) without >> releasing the mutex? Just curious :) > > I have a couple of ways of dealing with this.. > > 1) Of course the initialization routine calls atexit() to get a > "cleanup handler" in place. this is not really sufficient for a system supplied service. > 2) Often times, of course, this can fail e.g. you get a SIGSEGV.. > or some such. When you attach the memory, an audit is done. The > audit will validate that the pid is still alive and has the > particular tid in it. Of course this is not 100% but as long as the > tid's have not rolled over it should work. The function is also > public (need to add that and many things to the manual pages ;-D) > so that onecan call it whenever one wants :-) TIDs do roll over the last I looked.. (this may have changed) did you say man page? goodie.. lets' see it.. There have been a lot of IPC and mutex implementations but the trick always comes with the requirement that they handle process/thread death. David has done some recent work in this space.. > > I will ping Julian and the others... > From davidxu at freebsd.org Mon Mar 30 19:18:32 2009 From: davidxu at freebsd.org (David Xu) Date: Mon Mar 30 19:18:39 2009 Subject: A mutex for inter-process ;-) In-Reply-To: References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> Message-ID: <49D17D76.5060309@freebsd.org> Daniel Eischen wrote: > On Mon, 30 Mar 2009, Randall Stewart wrote: > >> >> On Mar 30, 2009, at 5:22 PM, Daniel Eischen wrote: >> >>> On Mon, 30 Mar 2009, Randall Stewart wrote: >>> >>>> Hi all: >>>> >>>> I have recently written a small set of routines that allow >>>> two process to have a "mutex" between them.. actually it allows >>>> all of the threads in any set of processes to have mutexes between >>>> themselves ;-) >>>> >>>> Anyway it seems to be working fairly well.. I still have to write a >>>> man page >>>> for it (documentation always last).. and eventually I would like to >>>> port in >>>> some of the WITNESS type features since the mutex's have names.. >>>> >>>> I probably should also think about scaling it up a bit.. right now >>>> its really >>>> more for a small scale (100 or less mutexes)... >>>> >>>> Who should I talk to about getting this in... having it reviewed >>>> etc. I think >>>> it belongs in libthr since it really needs the tid of the pthreads >>>> from the >>>> pthread_t type... and for now I have a horrible hack in to get it ;-) >>> >>> The real way to do this is to support PTHREAD_PROCESS_SHARED >>> mutexes within our normal mutex, and to change our current >>> mutex (and cv) types to be structs instead of pointers. >>> The current API, other than the type change, shouldn't >>> change at all. >> >> >> So how do you propose to name the mutex's so that two disparate >> process can locate the same mutex? > > They are placed in shared memory, according to POSIX. > >> I don't see how a pthread_mutex can suffice... we need more than >> just the current mutex... >> >> What am I missing? > > As far as I know, David Xu implemented the kernel hooks > for umtx (the underlying mutex in pthread mutex) to be > shared. As soon as you can place the entire userland > pthread_mutex_t struct in shared memory, it should all > just work (with probably some trivial changes in libthr). > The harder part is versioning all the symbols that > currently think pthread_mutex_t, pthread_cond_t, etc, > are pointers, and defining the structs with enough > foresight so that it is unlikely we have to modify > them in the future (causing a future ABI breakage), > and also aligning them nicely for the various archs. > > You should really look at how POSIX defines process > shared mutex, cvs, etc. See: > > pthread_barrierattr_[gs]etpshared() > pthread_condattr_[gs]etpshared() > pthread_mutexattr_[gs]etpshared() > pthread_wrlockattr_[gs]etsphared() > > You can use this as a starting point: > > http://www.opengroup.org/onlinepubs/009695399/ > > http://www.opengroup.org/onlinepubs/009695399/functions/pthread_barrierattr_setpshared.html > > > http://www.opengroup.org/onlinepubs/009695399/functions/pthread_condattr_setpshared.html > > > http://www.opengroup.org/onlinepubs/009695399/functions/pthread_mutexattr_setpshared.html > > > http://www.opengroup.org/onlinepubs/009695399/functions/pthread_rwlockattr_setpshared.html > > You are right. umtx is ready for process-shared mutex, condition variable and rwlock. We are blocked by our pthread_mutex_t and pthread_cond_t definitions which are pointers, mmap()ing it into shared memory and calling pthread API will not work correctly, they should be defined as a block of memory. Recent POSIX standard introduces robust mutex type which can detects mutex owner's death, but in theory, shared memory model will never be robust. Regards, David Xu From rrs at lakerest.net Mon Mar 30 21:44:34 2009 From: rrs at lakerest.net (Randall Stewart) Date: Mon Mar 30 21:44:40 2009 Subject: A mutex for inter-process ;-) In-Reply-To: <49D17D76.5060309@freebsd.org> References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <49D17D76.5060309@freebsd.org> Message-ID: On Mar 30, 2009, at 10:18 PM, David Xu wrote: > Daniel Eischen wrote: >> On Mon, 30 Mar 2009, Randall Stewart wrote: >>> >>> On Mar 30, 2009, at 5:22 PM, Daniel Eischen wrote: >>> >>>> On Mon, 30 Mar 2009, Randall Stewart wrote: >>>> >>>>> Hi all: >>>>> >>>>> I have recently written a small set of routines that allow >>>>> two process to have a "mutex" between them.. actually it allows >>>>> all of the threads in any set of processes to have mutexes >>>>> between themselves ;-) >>>>> >>>>> Anyway it seems to be working fairly well.. I still have to >>>>> write a man page >>>>> for it (documentation always last).. and eventually I would like >>>>> to port in >>>>> some of the WITNESS type features since the mutex's have names.. >>>>> >>>>> I probably should also think about scaling it up a bit.. right >>>>> now its really >>>>> more for a small scale (100 or less mutexes)... >>>>> >>>>> Who should I talk to about getting this in... having it reviewed >>>>> etc. I think >>>>> it belongs in libthr since it really needs the tid of the >>>>> pthreads from the >>>>> pthread_t type... and for now I have a horrible hack in to get >>>>> it ;-) >>>> >>>> The real way to do this is to support PTHREAD_PROCESS_SHARED >>>> mutexes within our normal mutex, and to change our current >>>> mutex (and cv) types to be structs instead of pointers. >>>> The current API, other than the type change, shouldn't >>>> change at all. >>> >>> >>> So how do you propose to name the mutex's so that two disparate >>> process can locate the same mutex? >> They are placed in shared memory, according to POSIX. >>> I don't see how a pthread_mutex can suffice... we need more than >>> just the current mutex... >>> >>> What am I missing? >> As far as I know, David Xu implemented the kernel hooks >> for umtx (the underlying mutex in pthread mutex) to be >> shared. As soon as you can place the entire userland >> pthread_mutex_t struct in shared memory, it should all >> just work (with probably some trivial changes in libthr). >> The harder part is versioning all the symbols that >> currently think pthread_mutex_t, pthread_cond_t, etc, >> are pointers, and defining the structs with enough >> foresight so that it is unlikely we have to modify >> them in the future (causing a future ABI breakage), >> and also aligning them nicely for the various archs. >> You should really look at how POSIX defines process >> shared mutex, cvs, etc. See: >> pthread_barrierattr_[gs]etpshared() >> pthread_condattr_[gs]etpshared() >> pthread_mutexattr_[gs]etpshared() >> pthread_wrlockattr_[gs]etsphared() >> You can use this as a starting point: >> http://www.opengroup.org/onlinepubs/009695399/ >> http://www.opengroup.org/onlinepubs/009695399/functions/pthread_barrierattr_setpshared.html >> http://www.opengroup.org/onlinepubs/009695399/functions/pthread_condattr_setpshared.html >> http://www.opengroup.org/onlinepubs/009695399/functions/pthread_mutexattr_setpshared.html >> http://www.opengroup.org/onlinepubs/009695399/functions/pthread_rwlockattr_setpshared.html > > You are right. umtx is ready for process-shared mutex, condition > variable and rwlock. We are blocked by our pthread_mutex_t > and pthread_cond_t definitions which are pointers, mmap()ing it into > shared memory and calling pthread API will not work correctly, they > should be defined as a block of memory. Yes, the stuff I have been playing with uses umtx... it works seamlessly... > > Recent POSIX standard introduces robust mutex type which can detects > mutex owner's death, but in theory, shared memory model will never > be robust. I agree, but its something someone I interviewed with was asking about... and they were busy going about making kernel hacks to add shared mutex's which led me down the path of looking what's there and not there.. I will go poke around and look at the posix stuff... I wonder if some sort of extensions in the kernel might be a good thing to do with a shared memory model to deal with the "robustness" issue. Something to think about for sure. R > > > Regards, > David Xu > ------------------------------ Randall Stewart 803-317-4952 (cell) 803-345-0391(direct) From rrs at lakerest.net Mon Mar 30 21:47:11 2009 From: rrs at lakerest.net (Randall Stewart) Date: Mon Mar 30 21:47:18 2009 Subject: A mutex for inter-process ;-) In-Reply-To: <20090331043245.GZ92757@elvis.mu.org> References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <49D136B1.6060809@delphij.net> <78DBBDDA-5A39-4CEB-8289-F36EFB96461D@lakerest.net> <20090331043245.GZ92757@elvis.mu.org> Message-ID: <59795ADA-892B-4FAF-8506-82007D317C12@lakerest.net> The actual take I took is that both the pid and first tid are recorded. When you start up you validate each pid listed and make sure the main tid matches and the process is alive. Now of course there is a chance that both the tid and pid will wrap.. but both would have to wrap and you would have to have the new pid be assigned the same tid during the wrap. Maybe this would be common.. don't know .. but its a start. Without adding some sort of hook at process death to detect and cleanup the shared memory.. R On Mar 31, 2009, at 12:32 AM, Alfred Perlstein wrote: > One trick to handling pid wrap is to also record process > start time. I sort of wish our signalling code allowed > this as a optional thing to make _really sure_ you weren't > signalling the wrong process. > > -Alfred > > * Randall Stewart [090330 16:29] wrote: >> >> On Mar 30, 2009, at 5:16 PM, Xin LI wrote: >> >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA1 >>> >>> Randall Stewart wrote: >>>> Hi all: >>>> >>>> I have recently written a small set of routines that allow >>>> two process to have a "mutex" between them.. actually it allows >>>> all of the threads in any set of processes to have mutexes between >>>> themselves ;-) >>>> >>>> Anyway it seems to be working fairly well.. I still have to write a >>>> man >>>> page >>>> for it (documentation always last).. and eventually I would like to >>>> port in >>>> some of the WITNESS type features since the mutex's have names.. >>>> >>>> I probably should also think about scaling it up a bit.. right now >>>> its >>>> really >>>> more for a small scale (100 or less mutexes)... >>>> >>>> Who should I talk to about getting this in... having it reviewed >>>> etc. I >>>> think >>>> it belongs in libthr since it really needs the tid of the pthreads >>>> from the >>>> pthread_t type... and for now I have a horrible hack in to get >>>> it ;-) >>> >>> I think davidxu@ deischen@ and julian@? >>> >>> BTW. How do you handle with one process exit (abnormally) without >>> releasing the mutex? Just curious :) >> >> I have a couple of ways of dealing with this.. >> >> 1) Of course the initialization routine calls atexit() to get a >> "cleanup handler" in place. >> 2) Often times, of course, this can fail e.g. you get a SIGSEGV.. or >> some such. When you >> attach the memory, an audit is done. The audit will validate that >> the pid is still alive >> and has the particular tid in it. Of course this is not 100% but >> as long as the tid's have >> not rolled over it should work. The function is also public (need >> to add that and many things >> to the manual pages ;-D) so that one can call it whenever one >> wants :-) >> >> I will ping Julian and the others... >> >> R >> >>> >>> >>> Cheers, >>> - -- >>> Xin LI http://www.delphij.net/ >>> FreeBSD - The Power to Serve! >>> -----BEGIN PGP SIGNATURE----- >>> Version: GnuPG v2.0.11 (FreeBSD) >>> >>> iEYEARECAAYFAknRNrEACgkQi+vbBBjt66DIswCbBWRMJN55c60UTBBIZMRCY4zo >>> 6hcAnixfVXdtdnn0fT/Z31v0EdyVCVlH >>> =JL/U >>> -----END PGP SIGNATURE----- >>> >> >> ------------------------------ >> Randall Stewart >> 803-317-4952 (cell) >> 803-345-0391(direct) >> >> _______________________________________________ >> freebsd-threads@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-threads >> To unsubscribe, send any mail to "freebsd-threads-unsubscribe@freebsd.org >> " > > -- > - Alfred Perlstein > ------------------------------ Randall Stewart 803-317-4952 (cell) 803-345-0391(direct) From alfred at freebsd.org Mon Mar 30 21:52:35 2009 From: alfred at freebsd.org (Alfred Perlstein) Date: Mon Mar 30 21:52:43 2009 Subject: A mutex for inter-process ;-) In-Reply-To: <78DBBDDA-5A39-4CEB-8289-F36EFB96461D@lakerest.net> References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <49D136B1.6060809@delphij.net> <78DBBDDA-5A39-4CEB-8289-F36EFB96461D@lakerest.net> Message-ID: <20090331043245.GZ92757@elvis.mu.org> One trick to handling pid wrap is to also record process start time. I sort of wish our signalling code allowed this as a optional thing to make _really sure_ you weren't signalling the wrong process. -Alfred * Randall Stewart [090330 16:29] wrote: > > On Mar 30, 2009, at 5:16 PM, Xin LI wrote: > > >-----BEGIN PGP SIGNED MESSAGE----- > >Hash: SHA1 > > > >Randall Stewart wrote: > >>Hi all: > >> > >>I have recently written a small set of routines that allow > >>two process to have a "mutex" between them.. actually it allows > >>all of the threads in any set of processes to have mutexes between > >>themselves ;-) > >> > >>Anyway it seems to be working fairly well.. I still have to write a > >>man > >>page > >>for it (documentation always last).. and eventually I would like to > >>port in > >>some of the WITNESS type features since the mutex's have names.. > >> > >>I probably should also think about scaling it up a bit.. right now > >>its > >>really > >>more for a small scale (100 or less mutexes)... > >> > >>Who should I talk to about getting this in... having it reviewed > >>etc. I > >>think > >>it belongs in libthr since it really needs the tid of the pthreads > >>from the > >>pthread_t type... and for now I have a horrible hack in to get it ;-) > > > >I think davidxu@ deischen@ and julian@? > > > >BTW. How do you handle with one process exit (abnormally) without > >releasing the mutex? Just curious :) > > I have a couple of ways of dealing with this.. > > 1) Of course the initialization routine calls atexit() to get a > "cleanup handler" in place. > 2) Often times, of course, this can fail e.g. you get a SIGSEGV.. or > some such. When you > attach the memory, an audit is done. The audit will validate that > the pid is still alive > and has the particular tid in it. Of course this is not 100% but > as long as the tid's have > not rolled over it should work. The function is also public (need > to add that and many things > to the manual pages ;-D) so that one can call it whenever one > wants :-) > > I will ping Julian and the others... > > R > > > > > > >Cheers, > >- -- > >Xin LI http://www.delphij.net/ > >FreeBSD - The Power to Serve! > >-----BEGIN PGP SIGNATURE----- > >Version: GnuPG v2.0.11 (FreeBSD) > > > >iEYEARECAAYFAknRNrEACgkQi+vbBBjt66DIswCbBWRMJN55c60UTBBIZMRCY4zo > >6hcAnixfVXdtdnn0fT/Z31v0EdyVCVlH > >=JL/U > >-----END PGP SIGNATURE----- > > > > ------------------------------ > Randall Stewart > 803-317-4952 (cell) > 803-345-0391(direct) > > _______________________________________________ > freebsd-threads@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > To unsubscribe, send any mail to "freebsd-threads-unsubscribe@freebsd.org" -- - Alfred Perlstein From rrs at lakerest.net Mon Mar 30 22:04:43 2009 From: rrs at lakerest.net (Randall Stewart) Date: Mon Mar 30 22:04:50 2009 Subject: A mutex for inter-process ;-) In-Reply-To: References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> Message-ID: <9157F968-5CCF-451C-9BA0-E12A957D6B38@lakerest.net> Daniel: In-line :-) On Mar 30, 2009, at 7:56 PM, Daniel Eischen wrote: > On Mon, 30 Mar 2009, Randall Stewart wrote: > >> >> On Mar 30, 2009, at 5:22 PM, Daniel Eischen wrote: >> >>> On Mon, 30 Mar 2009, Randall Stewart wrote: >>>> Hi all: >>>> I have recently written a small set of routines that allow >>>> two process to have a "mutex" between them.. actually it allows >>>> all of the threads in any set of processes to have mutexes >>>> between themselves ;-) >>>> Anyway it seems to be working fairly well.. I still have to write >>>> a man page >>>> for it (documentation always last).. and eventually I would like >>>> to port in >>>> some of the WITNESS type features since the mutex's have names.. >>>> I probably should also think about scaling it up a bit.. right >>>> now its really >>>> more for a small scale (100 or less mutexes)... >>>> Who should I talk to about getting this in... having it reviewed >>>> etc. I think >>>> it belongs in libthr since it really needs the tid of the >>>> pthreads from the >>>> pthread_t type... and for now I have a horrible hack in to get >>>> it ;-) >>> The real way to do this is to support PTHREAD_PROCESS_SHARED >>> mutexes within our normal mutex, and to change our current >>> mutex (and cv) types to be structs instead of pointers. >>> The current API, other than the type change, shouldn't >>> change at all. >> >> >> So how do you propose to name the mutex's so that two disparate >> process can locate the same mutex? > > They are placed in shared memory, according to POSIX. > >> I don't see how a pthread_mutex can suffice... we need more than >> just the current mutex... >> >> What am I missing? > > As far as I know, David Xu implemented the kernel hooks > for umtx (the underlying mutex in pthread mutex) to be > shared. As soon as you can place the entire userland > pthread_mutex_t struct in shared memory, it should all > just work (with probably some trivial changes in libthr). > The harder part is versioning all the symbols that > currently think pthread_mutex_t, pthread_cond_t, etc, > are pointers, and defining the structs with enough > foresight so that it is unlikely we have to modify > them in the future (causing a future ABI breakage), > and also aligning them nicely for the various archs. > > You should really look at how POSIX defines process > shared mutex, cvs, etc. See: > > pthread_barrierattr_[gs]etpshared() > pthread_condattr_[gs]etpshared() > pthread_mutexattr_[gs]etpshared() > pthread_wrlockattr_[gs]etsphared() > > You can use this as a starting point: > > http://www.opengroup.org/onlinepubs/009695399/ > http://www.opengroup.org/onlinepubs/009695399/functions/pthread_barrierattr_setpshared.html > http://www.opengroup.org/onlinepubs/009695399/functions/pthread_condattr_setpshared.html > http://www.opengroup.org/onlinepubs/009695399/functions/pthread_mutexattr_setpshared.html > http://www.opengroup.org/onlinepubs/009695399/functions/pthread_rwlockattr_setpshared.html Ok, I have poked around at these... all the mutex attributes defined here do is set the attributes to shared. There does not seem to be any standard naming mechanism. In fact following the set attributes stuff it gives examples of a condition variable and defines "new local methods" to get a shared semaphore. Creating the actual naming semantics in the new local methods. All that they do on the mutex side is set the attributes to "shared" and basically do the very same thing that I was playing with... i.e. mmap() the file after initializing it... Now granted I did not use the pthread_mutex_*() calls themselves but instead used the umtx() calls directly on the shared memory. Not sure if there is much difference there.. but in any event there is no declaration here in posix on calls for setting "names" so one could then expand the stuff and add witness etc. It looks to me like its more or less a "left open" for future work.. I just love standards bodies ;-D R > > > -- > DE > ------------------------------ Randall Stewart 803-317-4952 (cell) 803-345-0391(direct) From eischen at vigrid.com Tue Mar 31 00:01:38 2009 From: eischen at vigrid.com (Daniel Eischen) Date: Tue Mar 31 00:01:45 2009 Subject: A mutex for inter-process ;-) In-Reply-To: <9157F968-5CCF-451C-9BA0-E12A957D6B38@lakerest.net> References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <9157F968-5CCF-451C-9BA0-E12A957D6B38@lakerest.net> Message-ID: On Tue, 31 Mar 2009, Randall Stewart wrote: > Daniel: > > In-line :-) > > > On Mar 30, 2009, at 7:56 PM, Daniel Eischen wrote: > >> On Mon, 30 Mar 2009, Randall Stewart wrote: >> >>> >>> What am I missing? >> >> As far as I know, David Xu implemented the kernel hooks >> for umtx (the underlying mutex in pthread mutex) to be >> shared. As soon as you can place the entire userland >> pthread_mutex_t struct in shared memory, it should all >> just work (with probably some trivial changes in libthr). >> The harder part is versioning all the symbols that >> currently think pthread_mutex_t, pthread_cond_t, etc, >> are pointers, and defining the structs with enough >> foresight so that it is unlikely we have to modify >> them in the future (causing a future ABI breakage), >> and also aligning them nicely for the various archs. >> >> You should really look at how POSIX defines process >> shared mutex, cvs, etc. See: >> >> pthread_barrierattr_[gs]etpshared() >> pthread_condattr_[gs]etpshared() >> pthread_mutexattr_[gs]etpshared() >> pthread_wrlockattr_[gs]etsphared() >> >> You can use this as a starting point: >> >> http://www.opengroup.org/onlinepubs/009695399/ >> http://www.opengroup.org/onlinepubs/009695399/functions/pthread_barrierattr_setpshared.html >> http://www.opengroup.org/onlinepubs/009695399/functions/pthread_condattr_setpshared.html >> http://www.opengroup.org/onlinepubs/009695399/functions/pthread_mutexattr_setpshared.html >> http://www.opengroup.org/onlinepubs/009695399/functions/pthread_rwlockattr_setpshared.html > > Ok, I have poked around at these... all the mutex attributes defined here > do is set the attributes to shared. There does not seem to be any standard > naming mechanism. Naming mechanism for what? Names shouldn't be needed for anything, nor do I think it is desired. > In fact following the set attributes stuff it gives examples of a condition > variable and defines "new local methods" to get a shared semaphore. Creating > the actual naming semantics in the new local methods. All that they > do on the mutex side is set the attributes to "shared" and basically do > the very same thing that I was playing with... i.e. mmap() the file > after initializing it... They define the API. We should not be making new APIs for something that already exists, that applications already know how to use, etc. > Now granted I did not use the pthread_mutex_*() calls themselves but instead > used the umtx() calls directly on the shared memory. Not sure if there is > much difference there.. but in any event there is no declaration here > in posix on calls for setting "names" so one could then expand the stuff > and add witness etc. It looks to me like its more or less a "left open" > for future work.. See above. The proper way to do this is to define the pthread_foo types, mark them as pshared, and have libthr make appropriate umtx calls when they are marked as shared. It is up to the application to define the shared memory segment and place the pthread types in the shared memory. There is no need for "names" on umtx, mutex, whatever. The kernel umtx, as David already pointed out, already handles process shared umtx. -- DE From julian at elischer.org Tue Mar 31 00:29:24 2009 From: julian at elischer.org (Julian Elischer) Date: Tue Mar 31 00:29:30 2009 Subject: A mutex for inter-process ;-) In-Reply-To: References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <49D17D76.5060309@freebsd.org> Message-ID: <49D1C669.5030809@elischer.org> Randall Stewart wrote: > > > > I agree, but its something someone I interviewed with was asking > about... and well I told Huawei to talk to david about it when they asked me. From srinivasganji at gmail.com Tue Mar 31 02:30:29 2009 From: srinivasganji at gmail.com (Srinivas Ganji) Date: Tue Mar 31 02:30:35 2009 Subject: Is it possible to use the libthr.a file on a Redhat Linux? Message-ID: Dear All, I have tried to use the libthr.a library for compiling an application which is working fine on Redhat system with libpthread library. However, I end up with the following errors. ../lib/linux/libthr.a(thr_sem.o): In function `_sem_init': thr_sem.c:(.text+0x100): undefined reference to `ksem_init' thr_sem.c:(.text+0x115): undefined reference to `ksem_destroy' ../lib/linux/libthr.a(thr_sem.o): In function `_sem_destroy': thr_sem.c:(.text+0x216): undefined reference to `ksem_destroy' ../lib/linux/libthr.a(thr_sem.o): In function `_sem_timedwait': thr_sem.c:(.text+0x2ad): undefined reference to `ksem_timedwait' ../lib/linux/libthr.a(thr_sem.o): In function `_sem_wait': .... .... .... collect2: ld returned 1 exit status make: *** [target] Error 1 So, I have also mentioned the libc.so.7(This is also a FreeBSD libc library) library in our application to remove the above undefined references. So, at that time I got the following errors. /usr/bin/ld: errno@@FBSD_1.0: TLS definition in /lib/libc.so.6 section .tbss mismatches non-TLS definition in ../lib/linux/libc.so section .bss /lib/libc.so.6: could not read symbols: Bad value Here, the lib/libc.so.6 is a Redhat libc library where as ../lib/linux/libc.so is a FreeBSD library (libc.so.7). My question is: Is it possible to use the FreeBSD libthr.a library on a Redhat Linux distribution? Thanks in advance. With Regards, Srinivas G From rrs at lakerest.net Tue Mar 31 04:05:21 2009 From: rrs at lakerest.net (Randall Stewart) Date: Tue Mar 31 04:05:47 2009 Subject: A mutex for inter-process ;-) In-Reply-To: References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <9157F968-5CCF-451C-9BA0-E12A957D6B38@lakerest.net> Message-ID: <081E4C0E-4DD0-45DA-BDFE-89FC2388E1AE@lakerest.net> On Mar 31, 2009, at 2:50 AM, Daniel Eischen wrote: > On Tue, 31 Mar 2009, Randall Stewart wrote: > >> Daniel: >> >> In-line :-) >> >> >> On Mar 30, 2009, at 7:56 PM, Daniel Eischen wrote: >> >>> On Mon, 30 Mar 2009, Randall Stewart wrote: >>>> What am I missing? >>> As far as I know, David Xu implemented the kernel hooks >>> for umtx (the underlying mutex in pthread mutex) to be >>> shared. As soon as you can place the entire userland >>> pthread_mutex_t struct in shared memory, it should all >>> just work (with probably some trivial changes in libthr). >>> The harder part is versioning all the symbols that >>> currently think pthread_mutex_t, pthread_cond_t, etc, >>> are pointers, and defining the structs with enough >>> foresight so that it is unlikely we have to modify >>> them in the future (causing a future ABI breakage), >>> and also aligning them nicely for the various archs. >>> You should really look at how POSIX defines process >>> shared mutex, cvs, etc. See: >>> pthread_barrierattr_[gs]etpshared() >>> pthread_condattr_[gs]etpshared() >>> pthread_mutexattr_[gs]etpshared() >>> pthread_wrlockattr_[gs]etsphared() >>> You can use this as a starting point: >>> http://www.opengroup.org/onlinepubs/009695399/ >>> http://www.opengroup.org/onlinepubs/009695399/functions/pthread_barrierattr_setpshared.html >>> http://www.opengroup.org/onlinepubs/009695399/functions/pthread_condattr_setpshared.html >>> http://www.opengroup.org/onlinepubs/009695399/functions/pthread_mutexattr_setpshared.html >>> http://www.opengroup.org/onlinepubs/009695399/functions/pthread_rwlockattr_setpshared.html >> >> Ok, I have poked around at these... all the mutex attributes >> defined here >> do is set the attributes to shared. There does not seem to be any >> standard >> naming mechanism. > > Naming mechanism for what? Names shouldn't be needed for anything, > nor do I think it is desired. > >> In fact following the set attributes stuff it gives examples of a >> condition >> variable and defines "new local methods" to get a shared semaphore. >> Creating >> the actual naming semantics in the new local methods. All that they >> do on the mutex side is set the attributes to "shared" and >> basically do >> the very same thing that I was playing with... i.e. mmap() the file >> after initializing it... > > They define the API. We should not be making new APIs for something > that already exists, that applications already know how to use, etc. So what you are saying is ... just let the application do it. Provide nothing but the ability to "mark" a mutex as shared. And let the app figure it out. Hmm.. If one company is asking for this ability i.e. easily do shared mutexs I am sure other folks have wanted it as well. Now rolling your own is a valid thing to do.. but it seems to me providing something for general use is not a bad idea either. The pages you pointed out even show such a mechanism for semaphores... i.e. there definition of semaphore_create(char *shared_name) semaphore_open(char *shared_name) semaphore_post(..) and kin. Curious place for it though.. under pthread_mutex_destroy() ;-) And of course as pointed out this does not solve the quick death syndrome (for that matter neither did I yet but I am thinking about this one ;D)... which is the real hard problem.. and really does require assistance beyond what an application can generally do... IMO having a general library function available is a good thing. If you are not interested in seeing it in libthr where I think it would belong.. thats fine I can build a port or other such... I will send Julian the manual page after I get it built through :-D R > > >> Now granted I did not use the pthread_mutex_*() calls themselves >> but instead >> used the umtx() calls directly on the shared memory. Not sure if >> there is >> much difference there.. but in any event there is no declaration here >> in posix on calls for setting "names" so one could then expand the >> stuff >> and add witness etc. It looks to me like its more or less a "left >> open" >> for future work.. > > See above. The proper way to do this is to define the pthread_foo > types, mark them as pshared, and have libthr make appropriate umtx > calls when they are marked as shared. It is up to the application > to define the shared memory segment and place the pthread types in > the shared memory. There is no need for "names" on umtx, mutex, > whatever. > > The kernel umtx, as David already pointed out, already handles > process shared umtx. > > -- > DE > > ------------------------------ Randall Stewart 803-317-4952 (cell) 803-345-0391(direct) From ivoras at freebsd.org Tue Mar 31 06:05:12 2009 From: ivoras at freebsd.org (Ivan Voras) Date: Tue Mar 31 06:05:23 2009 Subject: Is it possible to use the libthr.a file on a Redhat Linux? In-Reply-To: References: Message-ID: Srinivas Ganji wrote: > Dear All, > > > > I have tried to use the libthr.a library for compiling an application which > is working fine on Redhat system with libpthread library. However, I end up > with the following errors. > My question is: Is it possible to use the FreeBSD libthr.a library on a > Redhat Linux distribution? Just to clarify things: you are asking if you can use a system library tightly integrated with its operating system on a completely different, unrelated operating system? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 252 bytes Desc: OpenPGP digital signature Url : http://lists.freebsd.org/pipermail/freebsd-threads/attachments/20090331/ac845eba/signature.pgp From deischen at freebsd.org Tue Mar 31 06:12:04 2009 From: deischen at freebsd.org (Daniel Eischen) Date: Tue Mar 31 06:12:10 2009 Subject: A mutex for inter-process ;-) In-Reply-To: <081E4C0E-4DD0-45DA-BDFE-89FC2388E1AE@lakerest.net> References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <9157F968-5CCF-451C-9BA0-E12A957D6B38@lakerest.net> <081E4C0E-4DD0-45DA-BDFE-89FC2388E1AE@lakerest.net> Message-ID: On Tue, 31 Mar 2009, Randall Stewart wrote: > > On Mar 31, 2009, at 2:50 AM, Daniel Eischen wrote: > >> >> They define the API. We should not be making new APIs for something >> that already exists, that applications already know how to use, etc. > > So what you are saying is ... just let the application do it. Provide > nothing but the ability to "mark" a mutex as shared. And let the > app figure it out. Correct. We do not need nor want any more SYS V IPC stuff and have more utilities like ipcrm, ipcs, etc to deal with them. POSIX already defines the API for us and tells us how to use process shared mutexes, et al. > Hmm.. If one company is asking for this ability i.e. easily > do shared mutexs I am sure other folks have wanted it as well. > Now rolling your own is a valid thing to do.. but it seems to > me providing something for general use is not a bad idea either. There already is something for general usage, see POSIX ;-) > The pages you pointed out even show such a mechanism for > semaphores... i.e. there definition of > > semaphore_create(char *shared_name) > semaphore_open(char *shared_name) > semaphore_post(..) > > and kin. > > > Curious place for it though.. under pthread_mutex_destroy() ;-) They are also in the POSIX spec under their own entries. > And of course as pointed out this does not solve the quick death > syndrome (for that matter neither did I yet but I am thinking > about this one ;D)... which is the real hard problem.. and really > does require assistance beyond what an application can generally > do... No, the kernel can do it under the existing umutex API. You should really be asking David Xu this stuff, but the kernel can remove any of its own resources (if it has any allocated) when the shared memory is removed, or it may be possible to have POSIX mutex robustness by having the kernel unlock or deallocate umutex resources upon process termination. The point is, it is possible for the kernel to do this, if it already doesn't, using the existing umutex APIs. > IMO having a general library function available is a good thing. If > you are not interested in seeing it in libthr where I think it would > belong.. thats fine I can build a port or other such... > > I will send Julian the manual page after I get it built through :-D There already is an API for doing what you want, and sorry, but no, I don't think adding a BSD-only API that is different from something POSIX already defines is a good thing ;-) -- DE From jhb at freebsd.org Tue Mar 31 08:03:36 2009 From: jhb at freebsd.org (John Baldwin) Date: Tue Mar 31 08:03:47 2009 Subject: WITNESS for pthreads In-Reply-To: References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <9157F968-5CCF-451C-9BA0-E12A957D6B38@lakerest.net> Message-ID: <200903311038.43401.jhb@freebsd.org> On Tuesday 31 March 2009 2:50:27 am Daniel Eischen wrote: > > Ok, I have poked around at these... all the mutex attributes defined here > > do is set the attributes to shared. There does not seem to be any standard > > naming mechanism. > > Naming mechanism for what? Names shouldn't be needed for anything, > nor do I think it is desired. Off topic: names would be very helpful to port witness to pthreads. The thoughts I have had for doing this though would be to add a new _np attribute to set the name. I actually would like to write a 'libwitness' that basically overrides the various symbols and provides the name_np attribute and implement witness in the shared library on top of whatever pthreads library is in use. This would also allow it to be portable to other OS's. (Well, it could break pshared mutexes, but using the pointer-style types, you could have the libwitness allocate its own "mutex" structure which has a "real" mutex inside of it along with the name and other per-lock data it tracks. It would then forward mutex operations to the real pthreads library after performing LOR checks, etc.). -- John Baldwin From jhb at freebsd.org Tue Mar 31 08:03:36 2009 From: jhb at freebsd.org (John Baldwin) Date: Tue Mar 31 08:03:47 2009 Subject: WITNESS for pthreads In-Reply-To: References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <9157F968-5CCF-451C-9BA0-E12A957D6B38@lakerest.net> Message-ID: <200903311038.43401.jhb@freebsd.org> On Tuesday 31 March 2009 2:50:27 am Daniel Eischen wrote: > > Ok, I have poked around at these... all the mutex attributes defined here > > do is set the attributes to shared. There does not seem to be any standard > > naming mechanism. > > Naming mechanism for what? Names shouldn't be needed for anything, > nor do I think it is desired. Off topic: names would be very helpful to port witness to pthreads. The thoughts I have had for doing this though would be to add a new _np attribute to set the name. I actually would like to write a 'libwitness' that basically overrides the various symbols and provides the name_np attribute and implement witness in the shared library on top of whatever pthreads library is in use. This would also allow it to be portable to other OS's. (Well, it could break pshared mutexes, but using the pointer-style types, you could have the libwitness allocate its own "mutex" structure which has a "real" mutex inside of it along with the name and other per-lock data it tracks. It would then forward mutex operations to the real pthreads library after performing LOR checks, etc.). -- John Baldwin From rrs at lakerest.net Tue Mar 31 08:11:05 2009 From: rrs at lakerest.net (Randall Stewart) Date: Tue Mar 31 08:11:18 2009 Subject: WITNESS for pthreads In-Reply-To: <200903311038.43401.jhb@freebsd.org> References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <9157F968-5CCF-451C-9BA0-E12A957D6B38@lakerest.net> <200903311038.43401.jhb@freebsd.org> Message-ID: This was one of the places I was heading (as I wrote privately to Daniel ;-D) I suppose I can share it all i.e. the pthread mutex stuff will of course work with shared mutexe's but it won't: a) Build an easy to use semantic for the app to agree on sharing memory.. i.e. you have left undefined how the process figure out what they are sharing. There is some value in setting up a easy semantic for app dev's to use. b) What happens when a process exits or hits a core dump while holding one of these mutex's? Is this what you are thinking the PROCESS_SHARED would do?? c) If you build something to do so you have some nice way of naming mutex's you can do something similar to our WITNESS option in the kernel... this is something the few times I have played in user space recently that I have missed... having LOR warnings and such can be a useful tool. You can't have this without IMO. I was am interested in a/b but one of my long term intents is to do ;-) R On Mar 31, 2009, at 10:38 AM, John Baldwin wrote: > On Tuesday 31 March 2009 2:50:27 am Daniel Eischen wrote: >>> Ok, I have poked around at these... all the mutex attributes >>> defined here >>> do is set the attributes to shared. There does not seem to be any >>> standard >>> naming mechanism. >> >> Naming mechanism for what? Names shouldn't be needed for anything, >> nor do I think it is desired. > > Off topic: names would be very helpful to port witness to pthreads. > The > thoughts I have had for doing this though would be to add a new _np > attribute > to set the name. I actually would like to write a 'libwitness' that > basically overrides the various symbols and provides the name_np > attribute > and implement witness in the shared library on top of whatever > pthreads > library is in use. This would also allow it to be portable to other > OS's. > (Well, it could break pshared mutexes, but using the pointer-style > types, you > could have the libwitness allocate its own "mutex" structure which has > a "real" mutex inside of it along with the name and other per-lock > data it > tracks. It would then forward mutex operations to the real pthreads > library > after performing LOR checks, etc.). > > -- > John Baldwin > ------------------------------ Randall Stewart 803-317-4952 (cell) 803-345-0391(direct) From rrs at lakerest.net Tue Mar 31 08:11:05 2009 From: rrs at lakerest.net (Randall Stewart) Date: Tue Mar 31 08:11:18 2009 Subject: WITNESS for pthreads In-Reply-To: <200903311038.43401.jhb@freebsd.org> References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <9157F968-5CCF-451C-9BA0-E12A957D6B38@lakerest.net> <200903311038.43401.jhb@freebsd.org> Message-ID: This was one of the places I was heading (as I wrote privately to Daniel ;-D) I suppose I can share it all i.e. the pthread mutex stuff will of course work with shared mutexe's but it won't: a) Build an easy to use semantic for the app to agree on sharing memory.. i.e. you have left undefined how the process figure out what they are sharing. There is some value in setting up a easy semantic for app dev's to use. b) What happens when a process exits or hits a core dump while holding one of these mutex's? Is this what you are thinking the PROCESS_SHARED would do?? c) If you build something to do so you have some nice way of naming mutex's you can do something similar to our WITNESS option in the kernel... this is something the few times I have played in user space recently that I have missed... having LOR warnings and such can be a useful tool. You can't have this without IMO. I was am interested in a/b but one of my long term intents is to do ;-) R On Mar 31, 2009, at 10:38 AM, John Baldwin wrote: > On Tuesday 31 March 2009 2:50:27 am Daniel Eischen wrote: >>> Ok, I have poked around at these... all the mutex attributes >>> defined here >>> do is set the attributes to shared. There does not seem to be any >>> standard >>> naming mechanism. >> >> Naming mechanism for what? Names shouldn't be needed for anything, >> nor do I think it is desired. > > Off topic: names would be very helpful to port witness to pthreads. > The > thoughts I have had for doing this though would be to add a new _np > attribute > to set the name. I actually would like to write a 'libwitness' that > basically overrides the various symbols and provides the name_np > attribute > and implement witness in the shared library on top of whatever > pthreads > library is in use. This would also allow it to be portable to other > OS's. > (Well, it could break pshared mutexes, but using the pointer-style > types, you > could have the libwitness allocate its own "mutex" structure which has > a "real" mutex inside of it along with the name and other per-lock > data it > tracks. It would then forward mutex operations to the real pthreads > library > after performing LOR checks, etc.). > > -- > John Baldwin > ------------------------------ Randall Stewart 803-317-4952 (cell) 803-345-0391(direct) From jhb at freebsd.org Tue Mar 31 08:34:39 2009 From: jhb at freebsd.org (John Baldwin) Date: Tue Mar 31 08:34:45 2009 Subject: WITNESS for pthreads In-Reply-To: References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <200903311038.43401.jhb@freebsd.org> Message-ID: <200903311127.06447.jhb@freebsd.org> On Tuesday 31 March 2009 11:11:02 am Randall Stewart wrote: > This was one of the places I was heading (as I wrote privately to > Daniel ;-D) > > I suppose I can share it all i.e. the pthread mutex stuff > will of course work with shared mutexe's but it won't: > > a) Build an easy to use semantic for the app to agree on sharing > memory.. i.e. you > have left undefined how the process figure out what they are > sharing. There is > some value in setting up a easy semantic for app dev's to use. You can use shm_open() to share memory regions by name and then create mutexes and condvars in that. > interface> > > b) What happens when a process exits or hits a core dump while holding > one > of these mutex's? Is this what you are thinking the PROCESS_SHARED > would > do?? There is a "robust" mutex extension David Xu mentioned. Presumably though what would happen is that when one thread went to block on a mutex, the kernel (in the umtx code) would see if the current owning thread had exited, and if so, do something "appropriate" (break the lock, etc.) at that time. I think a (pid, tid, process starttime) tuple would work ok for detecting this. > the > PROCESS_SHARED could be made to help here> > > c) If you build something to do so you have some nice way of naming > mutex's you can do something similar to our WITNESS option in the > kernel... this is something the few times I have played in user > space recently that I have missed... having LOR warnings and such > can be a useful tool. You can't have this without IMO. > > > I was am interested in a/b but one of my long term intents is to do > ;-) All my WITNESS thoughts are completely separate from PROCESS_SHARED mutexes and I think actually break PROCESS_SHARED mutexes. (Though perhaps they can still be made to work but using something far more invasive where WITNESS defines its own pthread_mutex structure that the app has to be compiled against.) -- John Baldwin From deischen at freebsd.org Tue Mar 31 10:33:02 2009 From: deischen at freebsd.org (Daniel Eischen) Date: Tue Mar 31 10:33:14 2009 Subject: WITNESS for pthreads In-Reply-To: <200903311038.43401.jhb@freebsd.org> References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <9157F968-5CCF-451C-9BA0-E12A957D6B38@lakerest.net> <200903311038.43401.jhb@freebsd.org> Message-ID: On Tue, 31 Mar 2009, John Baldwin wrote: > On Tuesday 31 March 2009 2:50:27 am Daniel Eischen wrote: >>> Ok, I have poked around at these... all the mutex attributes defined here >>> do is set the attributes to shared. There does not seem to be any standard >>> naming mechanism. >> >> Naming mechanism for what? Names shouldn't be needed for anything, >> nor do I think it is desired. > > Off topic: names would be very helpful to port witness to pthreads. The > thoughts I have had for doing this though would be to add a new _np attribute > to set the name. I actually would like to write a 'libwitness' that > basically overrides the various symbols and provides the name_np attribute > and implement witness in the shared library on top of whatever pthreads > library is in use. This would also allow it to be portable to other OS's. > (Well, it could break pshared mutexes, but using the pointer-style types, you > could have the libwitness allocate its own "mutex" structure which has > a "real" mutex inside of it along with the name and other per-lock data it > tracks. It would then forward mutex operations to the real pthreads library > after performing LOR checks, etc.). I think this is all overkill when we don't even have proper pthread synchronization primitives in libthr that can be used in shared memory. And if we also implement robust mutexes, then you have additional error-checking. -- DE From deischen at freebsd.org Tue Mar 31 10:33:02 2009 From: deischen at freebsd.org (Daniel Eischen) Date: Tue Mar 31 10:33:15 2009 Subject: WITNESS for pthreads In-Reply-To: <200903311038.43401.jhb@freebsd.org> References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <9157F968-5CCF-451C-9BA0-E12A957D6B38@lakerest.net> <200903311038.43401.jhb@freebsd.org> Message-ID: On Tue, 31 Mar 2009, John Baldwin wrote: > On Tuesday 31 March 2009 2:50:27 am Daniel Eischen wrote: >>> Ok, I have poked around at these... all the mutex attributes defined here >>> do is set the attributes to shared. There does not seem to be any standard >>> naming mechanism. >> >> Naming mechanism for what? Names shouldn't be needed for anything, >> nor do I think it is desired. > > Off topic: names would be very helpful to port witness to pthreads. The > thoughts I have had for doing this though would be to add a new _np attribute > to set the name. I actually would like to write a 'libwitness' that > basically overrides the various symbols and provides the name_np attribute > and implement witness in the shared library on top of whatever pthreads > library is in use. This would also allow it to be portable to other OS's. > (Well, it could break pshared mutexes, but using the pointer-style types, you > could have the libwitness allocate its own "mutex" structure which has > a "real" mutex inside of it along with the name and other per-lock data it > tracks. It would then forward mutex operations to the real pthreads library > after performing LOR checks, etc.). I think this is all overkill when we don't even have proper pthread synchronization primitives in libthr that can be used in shared memory. And if we also implement robust mutexes, then you have additional error-checking. -- DE From alfred at freebsd.org Tue Mar 31 10:52:35 2009 From: alfred at freebsd.org (Alfred Perlstein) Date: Tue Mar 31 10:52:43 2009 Subject: WITNESS for pthreads In-Reply-To: <200903311038.43401.jhb@freebsd.org> References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <9157F968-5CCF-451C-9BA0-E12A957D6B38@lakerest.net> <200903311038.43401.jhb@freebsd.org> Message-ID: <20090331174401.GD92757@elvis.mu.org> * John Baldwin [090331 08:03] wrote: > On Tuesday 31 March 2009 2:50:27 am Daniel Eischen wrote: > > > Ok, I have poked around at these... all the mutex attributes defined here > > > do is set the attributes to shared. There does not seem to be any standard > > > naming mechanism. > > > > Naming mechanism for what? Names shouldn't be needed for anything, > > nor do I think it is desired. > > Off topic: names would be very helpful to port witness to pthreads. The > thoughts I have had for doing this though would be to add a new _np attribute > to set the name. I actually would like to write a 'libwitness' that > basically overrides the various symbols and provides the name_np attribute > and implement witness in the shared library on top of whatever pthreads > library is in use. This would also allow it to be portable to other OS's. > (Well, it could break pshared mutexes, but using the pointer-style types, you > could have the libwitness allocate its own "mutex" structure which has > a "real" mutex inside of it along with the name and other per-lock data it > tracks. It would then forward mutex operations to the real pthreads library > after performing LOR checks, etc.). I've heard of this work being done by multiple other places in house. so you have a great idea, if you have time to run with it, it would likely eb greatly appreciated and give FreeBSD a big bump as a development platform. -alfred From alfred at freebsd.org Tue Mar 31 11:22:35 2009 From: alfred at freebsd.org (Alfred Perlstein) Date: Tue Mar 31 11:22:41 2009 Subject: WITNESS for pthreads In-Reply-To: <200903311038.43401.jhb@freebsd.org> References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <9157F968-5CCF-451C-9BA0-E12A957D6B38@lakerest.net> <200903311038.43401.jhb@freebsd.org> Message-ID: <20090331174401.GD92757@elvis.mu.org> * John Baldwin [090331 08:03] wrote: > On Tuesday 31 March 2009 2:50:27 am Daniel Eischen wrote: > > > Ok, I have poked around at these... all the mutex attributes defined here > > > do is set the attributes to shared. There does not seem to be any standard > > > naming mechanism. > > > > Naming mechanism for what? Names shouldn't be needed for anything, > > nor do I think it is desired. > > Off topic: names would be very helpful to port witness to pthreads. The > thoughts I have had for doing this though would be to add a new _np attribute > to set the name. I actually would like to write a 'libwitness' that > basically overrides the various symbols and provides the name_np attribute > and implement witness in the shared library on top of whatever pthreads > library is in use. This would also allow it to be portable to other OS's. > (Well, it could break pshared mutexes, but using the pointer-style types, you > could have the libwitness allocate its own "mutex" structure which has > a "real" mutex inside of it along with the name and other per-lock data it > tracks. It would then forward mutex operations to the real pthreads library > after performing LOR checks, etc.). I've heard of this work being done by multiple other places in house. so you have a great idea, if you have time to run with it, it would likely eb greatly appreciated and give FreeBSD a big bump as a development platform. -alfred From rrs at lakerest.net Tue Mar 31 12:56:12 2009 From: rrs at lakerest.net (Randall Stewart) Date: Tue Mar 31 12:56:17 2009 Subject: WITNESS for pthreads In-Reply-To: <200903311127.06447.jhb@freebsd.org> References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <200903311038.43401.jhb@freebsd.org> <200903311127.06447.jhb@freebsd.org> Message-ID: <989B6D28-243F-4A13-8C9D-F9C1CD5C2D77@lakerest.net> On Mar 31, 2009, at 11:27 AM, John Baldwin wrote: > On Tuesday 31 March 2009 11:11:02 am Randall Stewart wrote: >> This was one of the places I was heading (as I wrote privately to >> Daniel ;-D) >> >> I suppose I can share it all i.e. the pthread mutex stuff >> will of course work with shared mutexe's but it won't: >> >> a) Build an easy to use semantic for the app to agree on sharing >> memory.. i.e. you >> have left undefined how the process figure out what they are >> sharing. There is >> some value in setting up a easy semantic for app dev's to use. > > You can use shm_open() to share memory regions by name and then > create mutexes > and condvars in that. Thats what my little ipc_mutex...() functions do ;-) > > >> > interface> >> >> b) What happens when a process exits or hits a core dump while >> holding >> one >> of these mutex's? Is this what you are thinking the PROCESS_SHARED >> would >> do?? > > There is a "robust" mutex extension David Xu mentioned. Presumably > though > what would happen is that when one thread went to block on a mutex, > the > kernel (in the umtx code) would see if the current owning thread had > exited, > and if so, do something "appropriate" (break the lock, etc.) at that > time. I > think a (pid, tid, process starttime) tuple would work ok for > detecting this. If that is implemented.. I need to go look into this.. what I found when I was doing some digging is that umtx was used.. and I saw no way to make them "robust".. it may be something that needs adding.. > > >> > the >> PROCESS_SHARED could be made to help here> >> >> c) If you build something to do so you have some nice way of >> naming >> mutex's you can do something similar to our WITNESS option in the >> kernel... this is something the few times I have played in user >> space recently that I have missed... having LOR warnings and such >> can be a useful tool. You can't have this without IMO. >> >> >> I was am interested in a/b but one of my long term intents is to do >> ;-) > > All my WITNESS thoughts are completely separate from PROCESS_SHARED > mutexes > and I think actually break PROCESS_SHARED mutexes. (Though perhaps > they can > still be made to work but using something far more invasive where > WITNESS > defines its own pthread_mutex structure that the app has to be > compiled > against.) Which could also be put in shared memory so that you could learn the lock ordering across multiple processes ;-) R > > > -- > John Baldwin > ------------------------------ Randall Stewart 803-317-4952 (cell) 803-345-0391(direct) From davidxu at freebsd.org Tue Mar 31 18:34:43 2009 From: davidxu at freebsd.org (David Xu) Date: Tue Mar 31 18:34:55 2009 Subject: WITNESS for pthreads In-Reply-To: References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <9157F968-5CCF-451C-9BA0-E12A957D6B38@lakerest.net> <200903311038.43401.jhb@freebsd.org> Message-ID: <49D2C4B0.2020805@freebsd.org> Randall Stewart wrote: > This was one of the places I was heading (as I wrote privately to Daniel > ;-D) > > I suppose I can share it all i.e. the pthread mutex stuff > will of course work with shared mutexe's but it won't: > > a) Build an easy to use semantic for the app to agree on sharing > memory.. i.e. you > have left undefined how the process figure out what they are sharing. > There is > some value in setting up a easy semantic for app dev's to use. > > interface> > > b) What happens when a process exits or hits a core dump while holding one > of these mutex's? Is this what you are thinking the PROCESS_SHARED would > do?? > > PROCESS_SHARED could be made to help here> > Yes, kernel has to involve in this area, maybe all locking and unlocking for PROCESS_SHARED mutex should be done in kernel, so kernel can remember a list of mutex the current thread owned, when the thread exits for whatever reason, kernel should reset the mutexes to a state and wake up threads waiting on these mutexes. It seems that Solaris does it in this way, another way is setting a mutex list pointer in kernel, but the list itself is in user address space, it is a bit tricky for kernel to figure out the list's intermediate states when the thread is killed and fix the mutex states, the benefit is locking and unlocking are fast because they are done by userland when possible, it seems Linux does it in this way. > c) If you build something to do so you have some nice way of naming > mutex's you can do something similar to our WITNESS option in the > kernel... this is something the few times I have played in user > space recently that I have missed... having LOR warnings and such > can be a useful tool. You can't have this without IMO. > > > I was am interested in a/b but one of my long term intents is to do ;-) > > > R From davidxu at freebsd.org Tue Mar 31 18:34:43 2009 From: davidxu at freebsd.org (David Xu) Date: Tue Mar 31 18:34:55 2009 Subject: WITNESS for pthreads In-Reply-To: References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <9157F968-5CCF-451C-9BA0-E12A957D6B38@lakerest.net> <200903311038.43401.jhb@freebsd.org> Message-ID: <49D2C4B0.2020805@freebsd.org> Randall Stewart wrote: > This was one of the places I was heading (as I wrote privately to Daniel > ;-D) > > I suppose I can share it all i.e. the pthread mutex stuff > will of course work with shared mutexe's but it won't: > > a) Build an easy to use semantic for the app to agree on sharing > memory.. i.e. you > have left undefined how the process figure out what they are sharing. > There is > some value in setting up a easy semantic for app dev's to use. > > interface> > > b) What happens when a process exits or hits a core dump while holding one > of these mutex's? Is this what you are thinking the PROCESS_SHARED would > do?? > > PROCESS_SHARED could be made to help here> > Yes, kernel has to involve in this area, maybe all locking and unlocking for PROCESS_SHARED mutex should be done in kernel, so kernel can remember a list of mutex the current thread owned, when the thread exits for whatever reason, kernel should reset the mutexes to a state and wake up threads waiting on these mutexes. It seems that Solaris does it in this way, another way is setting a mutex list pointer in kernel, but the list itself is in user address space, it is a bit tricky for kernel to figure out the list's intermediate states when the thread is killed and fix the mutex states, the benefit is locking and unlocking are fast because they are done by userland when possible, it seems Linux does it in this way. > c) If you build something to do so you have some nice way of naming > mutex's you can do something similar to our WITNESS option in the > kernel... this is something the few times I have played in user > space recently that I have missed... having LOR warnings and such > can be a useful tool. You can't have this without IMO. > > > I was am interested in a/b but one of my long term intents is to do ;-) > > > R