Locking problems in pkg makes portmaster fail

[Stefan Esser]

Hi all,

I've been having problems with portmaster being aborted because of
failed attempts of pkg to obtain a lock. The same problem occurs
with pkg and pkg-devel (I installed pkg-devel to check whether it
was due to an already fixed problem in pkg-1.3.8).

Other possibly relevant machine details: -CURRENT/amd64 on ZFS.


A typical portmaster/pkg error message is:

--------------------------------------------------------------------
===>>> Creating a backup package for old version cuse4bsd-kmod-0.1.35
Creating package for cuse4bsd-kmod-0.1.35
pkg: Cannot get an advisory lock on a database, it is locked by another
process

===>  Installing for cuse4bsd-kmod-0.1.35
===>  Checking if cuse4bsd-kmod already installed
===>   cuse4bsd-kmod-0.1.35 is already installed
      You may wish to ``make deinstall'' and install this port again
      by ``make reinstall'' to upgrade it properly.
      If you really wish to overwrite the old port of cuse4bsd-kmod
      without deleting it first, set the variable "FORCE_PKG_REGISTER"
      in your environment or the "make install" command line.
*** Error code 1

Stop.
make[1]: stopped in /usr/svn/ports/head/multimedia/cuse4bsd-kmod
[...]
--------------------------------------------------------------------

In all these cases it was just portmaster that was using pkg, no other
process could have held a lock. (BTW: This was with "portmaster -dgw",
I have not checked whether other options hide this problem ...)


I have further debugged this problem and found, that the locking
problem occurs during this command from /usr/ports/mk/bsd.openssl.mk:

OPENSSL_SHLIBFILE!=     ${PKG_INFO} -ql ${OPENSSL_INSTALLED} | grep
"^`pkg query "%p" ${OPENSSL_INSTALLED}`/lib/libcrypto.so.[0-9]*$$"

(sorry about possible line-wrap - this should be a single line ...)

This command on its own can be executed without problems:

# pkg info -ql openssl | grep `pkg query %p openssl`/lib/libcrypto.so.*
/usr/local/lib/libcrypto.so.8

But when it is run in parallel to portmaster creating a package, that
process can not acquire the necessary lock and portmaster aborts.

This problem does only occur, if portmaster runs a background task
that parses bsd.openssl.mk (and it appears that every time this file
is read by "make", the pkg-grep-pkg command sequence is invoked).
(It seems to be due to a "make -V" in some other port's directory,
which has a ".include bsd.openssl.mk", which is executed by portmaster
in preparation of building the next port, but I have not identified
the exact line in portmaster that does that.)

I have no other work-around than to manually install the affected
ports.

Anyway: While it seems that this is a problem in portmaster, I still
think that multiple invocations of "pkg info" or "pkg query" at the
time a "pkg create" is running should not make the latter fail ...

Regards, STefan

PS: I'm supprised this problem has not been reported on the mail-lists,
    since it is unlikely to only affect me. I'll do a search in the PRs
    and create a new PR if I really cannot find an existing one ...