ports/171155: PKGNG fails to deinstall large ports

[Stefan Esser]

>Number:         171155
>Category:       ports
>Synopsis:       PKGNG fails to deinstall large ports
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-ports-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Wed Aug 29 06:50:06 UTC 2012
>Closed-Date:
>Last-Modified:
>Originator:     Stefan Esser
>Release:        10-CURRENT
>Organization:
FreeBSD
>Environment:
FreeBSD xyzzy 10.0-CURRENT FreeBSD 10.0-CURRENT #33 r239375M: Sun Aug 19 00:18:08 CEST 2012     root at xyzzy:/sys/amd64/compile/SE  amd64
>Description:
PKG_NG seems to have introduced a limit on the size of ports
that can be deinstalled:


I have located the line that is causing this error. It is the
posix_spawn on line 102 of libpkg/script.c. The argument length
is limited to KERN_ARGMAX (sysctl kern.argmax) which seems to
be a R/O value initialized to 256KB.

AFAICT this is not even a loader tuneable. A new kernel must
be built, or pkg must be modified to honor the argument length
limit.

The argument is the string argument of "sh -c <string>", which
can not be split in the general case.

In the specific case of math/lapack, the arguments to execve()
(called by posix_spawn()) are:

argv[0] = "sh"
argv[1] = "-c"
argv[2] = " set -- lapack-3.4.0_2
rm -f /usr/local/man/cat3/INSTALL_ilaver.f.3.gz ...
/usr/local/man/cat3/zupmtr.f.3.gz.bz2" (==> ~330KB total!)

The argument list must be broken into pieces of less than 256KB,
since rising KERN_ARGMAX to >330KB will lead to the same problem
with the next port with even longer PLIST.
>How-To-Repeat:
On a system with pkg-1.0.r6_1 installed and WITH_PKGNG=yes in /etc/make.conf:

# cd /usr/ports/math/lapack
# make all install    (... or "pkg install lapack" ...)
# make deinstall      (... or "pkg delete lapack" ...)
===>  Deinstalling for math/lapack
===>   Deinstalling lapack-3.4.0_2
The following packages will be deinstalled:

        lapack-3.4.0_2

The deinstallation will free 28 MB
Deinstalling lapack-3.4.0_2...lapack-3.4.0_2 is required by:
qrupdate-1.1.1, deleting anyway
pkg: Cannot run script(DEINSTALL): Argument list too long
*** [deinstall] Error code 3
# 
>Fix:
The problem is that execve() returns E2BIG for arguments that exceed KERN_ARGMAX and this is a parameter that is compiled into the kernel. 
Rising the limit is no fix, since another port could easily exceed any new limit.

The best solution seems to be:
- instantiate the shell with popen instead of posix_spawn()
- do not use "-c" to pass the shell commands as arguments
- pipe the shell commands into the shell (this assumes, that the shell does not implement a similar limit on the length of a comamnd, but I have not verified that assumption)

If the shell implements a length limit on its commands:
- identify shell commands that may be split over multiple lines
- repeatedly call the command with a parameter list shorter than allowed by KERN_ARGMAX

>Release-Note:
>Audit-Trail:
>Unformatted: