pkg: Cannot get a read lock on a database, it is locked by another process

[Rainer Hurling]

Am 28.07.2014 23:36, schrieb Kevin Oberman:
> On Mon, Jul 28, 2014 at 8:34 AM, Kris Moore <kris at pcbsd.org> wrote:
> 
>> On 07/27/2014 10:51, Stefan Esser wrote:
>>> The locking of the pkg database leads to soft failures, but I'm afraid,
>>> if these soft failures happen to coincide with certain administrative
>>> tasks, they can lead to unexpected results.
>>>
>>> One example is that portmaster fails to detect PKGNG (and then assumes
>>> to be working in a pre-PKGNG environment), if some pkg subcommand locks
>>> the database. To repeat:
>>>
>>> # pkg version -Bs &
>>> # pkg info
>>>
>>> As long as pkg version runs, pkg info fails to get the read lock (at
>>> least in the large majority of my tests). You can also prevent any pkg
>>> command from running, if you STOP (kill -STOP / ^Z) pkg version. Any
>>> other pkg command will fail, until "pkg version" has been "unstopped"
>>> and run to completion. This might even be a local DoS, if any command
>>> that read-locks the package DB for extended time can be executed by
>>> an unprivileged user.
>>>
>>> Similar error messages are reported by pkg_libcheck, which issues
>>> lots of pkg commands in parallel. (I have observed some 5 lock
>>> failures per 1000 installed packages on my system).
>>>
>>>
>>> I did not try to test all combinations of simultanous pkg commands
>>> and did not verify, whether e.g. "pkg upgrade" might be stopped
>>> half way through because of an error accessing the pkg database,
>>> but I have seen SQLITE error messages that indicated failed write
>>> operations (INSERT/UPDATE).
>>>
>>> Either the timeouts are too low, or the duration during which the
>>> database is locked by a single operation is too large (or there is
>>> no fairness and some processes never get access to the database?).
>>>
>>>
>>> I think this should be fixed, since pkg commands that lock the
>>> database can be run from CRON or by other means in the background
>>> and the operator who issues pkg commands in the foreground (or runs
>>> portmaster) receives spurious error messages (which might still be
>>> better than the batch jobs doing silly things, after they failed to
>>> obtain information from the pkg database ...)
>>>
>>> Regards, STefan
>>
>> +1 to this whole thing.
>>
>> I would prefer that pkg commands simply wait for the DB to become
>> unlocked again, instead of just failing outright.
>>
>> We have many scripts which monitor the system, check for updates,
>> display our GUI store-front, and its really annoying to have random bits
>> of it fail simply because of bad timing with another pkg process.
>>
>> --
>> Kris Moore
>> PC-BSD Software
>> iXsystems
> 
> 
> This is a real pain, but I only see it on one of my systems. That system is
> my only i386 system and also my only system running 9.2. Whether that has
> anything to do with it, I can't say, but it makes me very nervous. I just
> would like to know why only the one system has the problem.
> --
> R. Kevin Oberman, Network Engineer, Retired
> E-mail: rkoberman at gmail.com


I am running three boxes with recent HEAD. Only one of them was a new
installation, from the beginning on with pkgng (no pkg_ before). The
other two boxes were upgraded over years now, and so from old pkg_ to
new pkgng.

Locks, as described above, I only observe on the two upgraded boxes
(from older pkg_ to newer pkgng). So, for me, it looks more like a
remnant from the conversion towards the newer pkgng.

But of course, I may be completely wrong. So excuse me, if this is a
misleading contribution.

Rainer Hurling