devel/py-Jinja2 build hangs on py27-setuptools 'git archive /usr/ports'

Fri Mar 15 13:17:51 UTC 2019

I'm trying to update my ports tree (git repository tracking 
https://github.com/freebsd/freebsd-ports.git at revision d99460ebe, but 
I've observed this issue in 2018 Oct when I first got my laptop).  
Whenever portmaster(8) tries to update certain Python ports (e.g. 
devel/py-Jinja2), the 'make' process hangs with some sub-process calling 
py-setuptools which calls 'git archive --prefix /usr/ports/ HEAD":

    ===>   py27-Jinja2-2.10 depends on package: py27-setuptools>0 - found
    ===>   py27-Jinja2-2.10 depends on file: /usr/local/bin/python2.7 -
    found
    ===>  Configuring for py27-Jinja2-2.10
    running config
    ===>  Building for py27-Jinja2-2.10
    running build
    running build_py
    creating build
    creating build/lib
    creating build/lib/jinja2
    copying jinja2/runtime.py -> build/lib/jinja2
    copying jinja2/idtracking.py -> build/lib/jinja2
    copying jinja2/lexer.py -> build/lib/jinja2
    copying jinja2/_identifier.py -> build/lib/jinja2
    copying jinja2/exceptions.py -> build/lib/jinja2
    copying jinja2/sandbox.py -> build/lib/jinja2
    copying jinja2/tests.py -> build/lib/jinja2
    copying jinja2/ext.py -> build/lib/jinja2
    copying jinja2/utils.py -> build/lib/jinja2
    copying jinja2/visitor.py -> build/lib/jinja2
    copying jinja2/filters.py -> build/lib/jinja2
    copying jinja2/environment.py -> build/lib/jinja2
    copying jinja2/asyncfilters.py -> build/lib/jinja2
    copying jinja2/nodes.py -> build/lib/jinja2
    copying jinja2/nativetypes.py -> build/lib/jinja2
    copying jinja2/meta.py -> build/lib/jinja2
    copying jinja2/compiler.py -> build/lib/jinja2
    copying jinja2/__init__.py -> build/lib/jinja2
    copying jinja2/defaults.py -> build/lib/jinja2
    copying jinja2/loaders.py -> build/lib/jinja2
    copying jinja2/debug.py -> build/lib/jinja2
    copying jinja2/constants.py -> build/lib/jinja2
    copying jinja2/_compat.py -> build/lib/jinja2
    copying jinja2/optimizer.py -> build/lib/jinja2
    copying jinja2/asyncsupport.py -> build/lib/jinja2
    copying jinja2/parser.py -> build/lib/jinja2
    copying jinja2/bccache.py -> build/lib/jinja2
    running egg_info
    writing requirements to Jinja2.egg-info/requires.txt
    writing Jinja2.egg-info/PKG-INFO
    writing top-level names to Jinja2.egg-info/top_level.txt
    writing dependency_links to Jinja2.egg-info/dependency_links.txt
    writing entry points to Jinja2.egg-info/entry_points.txt

    ****** HANGS HERE ******

    $ pstree -w -g 0 -p 30914
    -+= 00001 root /sbin/init
      \-+= 01867 ajenkins kdeinit5: Running... (kdeinit5)
        \-+- 02515 ajenkins /usr/local/bin/konsole
          \-+= 02516 ajenkins /usr/local/bin/bash
            \-+= 53948 root sudo portmaster -B -R -f --no-confirm -y -r
    libX11 -m DISABLE_VULNERABILITIES=yes
              \-+- 53949 root /bin/sh /usr/local/sbin/portmaster -B -R
    -f --no-confirm -y -r libX11 -m DISABLE_VULNERABILITIES=yes
                \-+- 29518 root /bin/sh /usr/local/sbin/portmaster -m
    DISABLE_VULNERABILITIES=yes -y -R -B cairo-1.15.12,2
                  \-+- 29606 root /bin/sh /usr/local/sbin/portmaster -B
    -R -y -m DISABLE_VULNERABILITIES=yes mesa-libs-18.3.2
                    \-+- 29720 root /bin/sh /usr/local/sbin/portmaster
    -m DISABLE_VULNERABILITIES=yes -y -R -B llvm60-6.0.1_6
                      \-+- 29794 root /bin/sh /usr/local/sbin/portmaster
    -B -R -y -m DISABLE_VULNERABILITIES=yes cmake-3.13.4
                        \-+- 29934 root /bin/sh
    /usr/local/sbin/portmaster -m DISABLE_VULNERABILITIES=yes -y -R -B
    textproc/py-sphinx
                          \-+- 29984 root /bin/sh
    /usr/local/sbin/portmaster -B -R -y -m DISABLE_VULNERABILITIES=yes
    devel/py-Jinja2 at py27
                            \-+- 30882 root /usr/bin/make
    DISABLE_VULNERABILITIES=yes -DNO_DEPENDS stage
                              \-+- 30887 root /usr/bin/make
    CONFIG_DONE_PY27-JINJA2=1
    /usr/ports/devel/py-Jinja2/work-py27/.stage_done.Jinja2._usr_local
                                \-+- 30910 root /usr/local/bin/python2.7
    -c -c install --record
    /usr/ports/devel/py-Jinja2/work-py27/.PLIST.pymodtmp -c -O1
    --prefix=/usr/local --single-version-externally-managed
    --root=/usr/ports/devel/py-Jinja2/work-py27/stage
                                  \--- 30914 root git archive --prefix
    /usr/ports/ HEAD
    $ top -d 1
    last pid: 32085;  load averages:  2.58,  2.85,
    2.69                                                        up
    0+10:53:25  08:55:23
    126 processes: 3 running, 123 sleeping
    CPU:  0.8% user,  1.0% nice, 15.9% system,  0.3% interrupt, 82.0% idle
    Mem: 1024M Active, 2432M Inact, 18M Laundry, 26G Wired, 1363M Free
    ARC: 20G Total, 5954M MFU, 13G MRU, 424K Anon, 239M Header, 1602M Other
          17G Compressed, 28G Uncompressed, 1.64:1 Ratio
    Swap:

       PID USERNAME    THR PRI NICE   SIZE    RES STATE    C   TIME WCPU
    COMMAND
    30910 root          1 113   10   196M   188M CPU5     5  34:13
    100.00% python2.7
    30914 root          1  52   10  1948M   558M CPU0     0  33:58
    100.00% git
      2035 ajenkins      2  27    0    15M  5872K select  11  43:02
    7.08% ksysguardd
      1895 ajenkins     53  21    0   588M   391M select   1  22:45
    2.88% plasmashell
      1309 root          1  20    0    11M  2708K select  11   4:45
    2.88% syslogd

The git(1) process will take several hours (>5), then the build will 
continue until the next Python port that uses py-setuptools builds.  I'm 
running FreeBSD-CURRENT as of 2019/02/12, git commit 68c8581f on a 2018 
Dell XPS 15 laptop w/ Intel i9 processor.

I'll try killing the build (it's been running >2 hours since I 
restarted), moving /usr/ports/.git and restarting the build...

Okay that fixes the hang; aside from some apparently non-fatal build 
errors, 'make' took 1.226 seconds.

    $ for i in /usr/ports/.git*; do mv "${i}" "${i}.bak"; done
    $ $ cd /usr/ports/devel/py-Jinja2 && sudo make clean && time sudo make
    ===>  Cleaning for py27-Jinja2-2.10
    ===>  Cleaning for py36-Jinja2-2.10
    ===>  License BSD3CLAUSE accepted by the user
    ===>   py27-Jinja2-2.10 depends on file: /usr/local/sbin/pkg - found
    ===> Fetching all distfiles required by py27-Jinja2-2.10 for building
    ===>  Extracting for py27-Jinja2-2.10
    => SHA256 Checksum OK for Jinja2-2.10.tar.gz.
    ===>  Patching for py27-Jinja2-2.10
    ===>   py27-Jinja2-2.10 depends on package: py27-setuptools>0 - found
    ===>   py27-Jinja2-2.10 depends on file: /usr/local/bin/python2.7 -
    found
    ===>  Configuring for py27-Jinja2-2.10
    running config
    ===>  Building for py27-Jinja2-2.10
    running build
    running build_py
    creating build
    creating build/lib
    creating build/lib/jinja2
    copying jinja2/sandbox.py -> build/lib/jinja2
    copying jinja2/asyncsupport.py -> build/lib/jinja2
    copying jinja2/runtime.py -> build/lib/jinja2
    copying jinja2/ext.py -> build/lib/jinja2
    copying jinja2/nodes.py -> build/lib/jinja2
    copying jinja2/exceptions.py -> build/lib/jinja2
    copying jinja2/nativetypes.py -> build/lib/jinja2
    copying jinja2/filters.py -> build/lib/jinja2
    copying jinja2/visitor.py -> build/lib/jinja2
    copying jinja2/debug.py -> build/lib/jinja2
    copying jinja2/idtracking.py -> build/lib/jinja2
    copying jinja2/loaders.py -> build/lib/jinja2
    copying jinja2/tests.py -> build/lib/jinja2
    copying jinja2/utils.py -> build/lib/jinja2
    copying jinja2/asyncfilters.py -> build/lib/jinja2
    copying jinja2/environment.py -> build/lib/jinja2
    copying jinja2/meta.py -> build/lib/jinja2
    copying jinja2/optimizer.py -> build/lib/jinja2
    copying jinja2/_identifier.py -> build/lib/jinja2
    copying jinja2/__init__.py -> build/lib/jinja2
    copying jinja2/defaults.py -> build/lib/jinja2
    copying jinja2/bccache.py -> build/lib/jinja2
    copying jinja2/compiler.py -> build/lib/jinja2
    copying jinja2/constants.py -> build/lib/jinja2
    copying jinja2/lexer.py -> build/lib/jinja2
    copying jinja2/_compat.py -> build/lib/jinja2
    copying jinja2/parser.py -> build/lib/jinja2
    running egg_info
    writing requirements to Jinja2.egg-info/requires.txt
    writing Jinja2.egg-info/PKG-INFO
    writing top-level names to Jinja2.egg-info/top_level.txt
    writing dependency_links to Jinja2.egg-info/dependency_links.txt
    writing entry points to Jinja2.egg-info/entry_points.txt
    reading manifest file 'Jinja2.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    warning: no previously-included files matching '*.py[co]' found
    anywhere in distribution
    no previously-included directories found matching 'docs/_build'
    writing manifest file 'Jinja2.egg-info/SOURCES.txt'
    ===>  Staging for py27-Jinja2-2.10
    ===>   py27-Jinja2-2.10 depends on package: py27-MarkupSafe>=0.23 -
    found
    ===>   py27-Jinja2-2.10 depends on package: py27-Babel>=1.3 - found
    ===>   py27-Jinja2-2.10 depends on package: py27-setuptools>0 - found
    ===>   py27-Jinja2-2.10 depends on file: /usr/local/bin/python2.7 -
    found
    ===>   Generating temporary packing list
    running install
    running build
    running build_py
    running egg_info
    writing requirements to Jinja2.egg-info/requires.txt
    writing Jinja2.egg-info/PKG-INFO
    writing top-level names to Jinja2.egg-info/top_level.txt
    writing dependency_links to Jinja2.egg-info/dependency_links.txt
    writing entry points to Jinja2.egg-info/entry_points.txt
    reading manifest file 'Jinja2.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    warning: no previously-included files matching '*.py[co]' found
    anywhere in distribution
    no previously-included directories found matching 'docs/_build'
    writing manifest file 'Jinja2.egg-info/SOURCES.txt'
    running install_lib
    creating
    /usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7
    creating

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages
    creating

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2
    copying build/lib/jinja2/utils.py ->

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2
    copying build/lib/jinja2/parser.py ->

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2
    copying build/lib/jinja2/asyncsupport.py ->

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2
    copying build/lib/jinja2/tests.py ->

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2
    copying build/lib/jinja2/visitor.py ->

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2
    copying build/lib/jinja2/environment.py ->

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2
    copying build/lib/jinja2/filters.py ->

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2
    copying build/lib/jinja2/meta.py ->

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2
    copying build/lib/jinja2/_identifier.py ->

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2
    copying build/lib/jinja2/runtime.py ->

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2
    copying build/lib/jinja2/defaults.py ->

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2
    copying build/lib/jinja2/sandbox.py ->

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2
    copying build/lib/jinja2/__init__.py ->

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2
    copying build/lib/jinja2/lexer.py ->

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2
    copying build/lib/jinja2/compiler.py ->

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2
    copying build/lib/jinja2/constants.py ->

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2
    copying build/lib/jinja2/asyncfilters.py ->

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2
    copying build/lib/jinja2/idtracking.py ->

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2
    copying build/lib/jinja2/_compat.py ->

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2
    copying build/lib/jinja2/optimizer.py ->

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2
    copying build/lib/jinja2/bccache.py ->

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2
    copying build/lib/jinja2/ext.py ->

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2
    copying build/lib/jinja2/exceptions.py ->

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2
    copying build/lib/jinja2/debug.py ->

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2
    copying build/lib/jinja2/nativetypes.py ->

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2
    copying build/lib/jinja2/nodes.py ->

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2
    copying build/lib/jinja2/loaders.py ->

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2
    byte-compiling

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2/utils.py
    to utils.pyc
    byte-compiling

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2/parser.py
    to parser.pyc
    byte-compiling

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2/asyncsupport.py
    to asyncsupport.pyc
       File
    "/usr/local/lib/python2.7/site-packages/jinja2/asyncsupport.py", line 22
         async def concat_async(async_gen):
                 ^
    SyntaxError: invalid syntax

    byte-compiling

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2/tests.py
    to tests.pyc
    byte-compiling

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2/visitor.py
    to visitor.pyc
    byte-compiling

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2/environment.py
    to environment.pyc
    byte-compiling

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2/filters.py
    to filters.pyc
    byte-compiling

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2/meta.py
    to meta.pyc
    byte-compiling

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2/_identifier.py
    to _identifier.pyc
    byte-compiling

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2/runtime.py
    to runtime.pyc
    byte-compiling

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2/defaults.py
    to defaults.pyc
    byte-compiling

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2/sandbox.py
    to sandbox.pyc
    byte-compiling

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2/__init__.py
    to __init__.pyc
    byte-compiling

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2/lexer.py
    to lexer.pyc
    byte-compiling

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2/compiler.py
    to compiler.pyc
    byte-compiling

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2/constants.py
    to constants.pyc
    byte-compiling

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2/asyncfilters.py
    to asyncfilters.pyc
       File
    "/usr/local/lib/python2.7/site-packages/jinja2/asyncfilters.py", line 7
         async def auto_to_seq(value):
                 ^
    SyntaxError: invalid syntax

    byte-compiling

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2/idtracking.py
    to idtracking.pyc
    byte-compiling

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2/_compat.py
    to _compat.pyc
    byte-compiling

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2/optimizer.py
    to optimizer.pyc
    byte-compiling

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2/bccache.py
    to bccache.pyc
    byte-compiling

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2/ext.py
    to ext.pyc
    byte-compiling

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2/exceptions.py
    to exceptions.pyc
    byte-compiling

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2/debug.py
    to debug.pyc
    byte-compiling

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2/nativetypes.py
    to nativetypes.pyc
    byte-compiling

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2/nodes.py
    to nodes.pyc
    byte-compiling

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/jinja2/loaders.py
    to loaders.pyc
    writing byte-compilation script '/tmp/tmpn6f3Nq.py'
    /usr/local/bin/python2.7 -O /tmp/tmpn6f3Nq.py
       File
    "/usr/local/lib/python2.7/site-packages/jinja2/asyncsupport.py", line 22
         async def concat_async(async_gen):
                 ^
    SyntaxError: invalid syntax

       File
    "/usr/local/lib/python2.7/site-packages/jinja2/asyncfilters.py", line 7
         async def auto_to_seq(value):
                 ^
    SyntaxError: invalid syntax

    removing /tmp/tmpn6f3Nq.py
    running install_egg_info
    Copying Jinja2.egg-info to

/usr/ports/devel/py-Jinja2/work-py27/stage/usr/local/lib/python2.7/site-packages/Jinja2-2.10-py2.7.egg-info
    running install_scripts
    writing list of installed files to
    '/usr/ports/devel/py-Jinja2/work-py27/.PLIST.pymodtmp'
    ====> Compressing man pages (compress-man)

    real    0m1.288s
    user    0m0.630s
    sys     0m0.658s

Restoring the git files restores the hang.

What's up with that?  Should I CC @python?

Thanks in advance,
Anthony Jenkins