ports/128845: devel/scons has race condition

[Steven Kreuzer]

>Number:         128845
>Category:       ports
>Synopsis:       devel/scons has race condition
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-ports-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          change-request
>Submitter-Id:   current-users
>Arrival-Date:   Thu Nov 13 16:40:05 UTC 2008
>Closed-Date:
>Last-Modified:
>Originator:     Steven Kreuzer
>Release:        6.3-STABLE
>Organization:
>Environment:
FreeBSD slurry.exit2shell.com 6.3-STABLE FreeBSD 6.3-STABLE #1: Fri May 23 20:22:28 EDT 2008 root at biggierat.sddi.net:/usr/obj/usr/src/sys/SMP amd64
>Description:
Python's subprocess module has a race condition: Popen() constructor has a call to global "_cleanup()" function on whenever a Popen object gets created, and that call causes a check for all pending Popen objects whether their subprocess has exited - i.e. the poll() method is called for every active Popen object.

See http://bugs.python.org/issue1731717 for addition details

SCon's compat/_scons_subprocess.py module is just a copy of a more recent
stock Python subprocess.py modified so it will work with older Python
versions.

The attached patch will add locks around calls to Popen and change
the compat module in a way that the subprocess module is always used, no matter if Python already ships one.

The rationale behind this decision is that there are many Python versions in the wild with different Popen() race condition problems. 
>How-To-Repeat:
Run scons -j8 on a quad core machine
>Fix:


Patch attached with submission follows:

Index: Makefile
===================================================================
RCS file: /usr/share/cvs/freebsd/ports/devel/scons/Makefile,v
retrieving revision 1.33
diff -u -r1.33 Makefile
--- Makefile	29 Oct 2008 00:52:10 -0000	1.33
+++ Makefile	13 Nov 2008 15:39:15 -0000
@@ -7,6 +7,7 @@
 
 PORTNAME=	scons
 PORTVERSION=	1.1.0
+PORTREVISION=	1
 CATEGORIES=	devel python
 MASTER_SITES=	SF
 
--- /dev/null	2008-11-13 11:09:02.000000000 -0500
+++ files/patch-engine-SCons-compat-__init__.py	2008-11-13 11:03:17.064635000 -0500
@@ -0,0 +1,29 @@
+Index: engine/SCons/compat/__init__.py
+===================================================================
+--- engine/SCons/compat/__init__.py (revision 2695)
++++ engine/SCons/compat/__init__.py (working copy)
+@@ -167,11 +167,17 @@
+     del shlex
+     import_as('_scons_shlex', 'shlex')
+ 
+-try:
+-    import subprocess
+-except ImportError:
+-    # Pre-2.4 Python has no subprocess module.
+-    import_as('_scons_subprocess', 'subprocess')
++#try:
++#    import subprocess
++#except ImportError:
++#    # Pre-2.4 Python has no subprocess module.
++#    import_as('_scons_subprocess', 'subprocess')
++
++# Import subprocess unconditionally to avoid possible race conditions in
++# the official subprocess API. If there are API versions without known
++# problems, we can version-check and use the original subprocess module
++# in these cases.
++import_as('_scons_subprocess', 'subprocess')
+ 
+ import sys
+ try:
+
+
--- /dev/null	2008-11-13 11:18:00.000000000 -0500
+++ files/patch-engine-SCons-compat-_scons_subprocess.py	2008-11-13 11:01:02.109982000 -0500
@@ -0,0 +1,33 @@
+Index: engine/SCons/compat/_scons_subprocess.py
+===================================================================
+--- engine/SCons/compat/_scons_subprocess.py (revision 2695)
++++ engine/SCons/compat/_scons_subprocess.py (working copy)
+@@ -581,13 +581,19 @@
+     class object:
+         pass
+ 
++import thread
++lock = thread.allocate_lock()
++
+ class Popen(object):
+     def __init__(self, args, bufsize=0, executable=None,
+                  stdin=None, stdout=None, stderr=None,
+                  preexec_fn=None, close_fds=False, shell=False,
+                  cwd=None, env=None, universal_newlines=False,
+                  startupinfo=None, creationflags=0):
+-        """Create new Popen instance."""
++        """Create new Popen instance.
++        Popen is not thread-safe and is therefore protected with a lock.
++        """
++        lock.acquire()
+         _cleanup()
+ 
+         self._child_created = False
+@@ -655,6 +661,7 @@
+                 self.stderr = os.fdopen(errread, 'rU', bufsize)
+             else:
+                 self.stderr = os.fdopen(errread, 'rb', bufsize)
++        lock.release()
+ 
+ 
+     def _translate_newlines(self, data):


>Release-Note:
>Audit-Trail:
>Unformatted: