merging pkg databases

[Sebastian Jäschke]

Am 07.11.2014 um 17:17 schrieb Matthew Seaman <matthew at freebsd.org>:
> 
> On 11/07/14 15:14, Sebastian Jäschke wrote:
>> For some reason we need to merge pkg(ng) databases together, so that
>> only the newer packages is in the resulting database, if both merged
>> databases have packages of the same type. And if two packages in the
>> databases have even the same version, always the same database is
>> overwritten. The reason is that we have a ro tree which is handled
>> differently than the rw pathes including /usr/local.
>> 
>> It seems to be not possible with the common tools for pkg, so our
>> company is willing to pay for a solution on this.
>> 
>> How should be ask whom for this? Is one of the followers of this fine
>> list a freelancer, willing and able to do this job? Should we ask the
>> foundation itself? We are quite new on a question like this, so
>> please advise!
> 
> When you say 'ro tree' and 'rw tree' are these just chunks of the usual
> filesystem layout with custom permissions?  New filesystem hierarchies
> you've invented (which implies custom package builds with variant
> $PREFIX / $LOCALBASE) or else some more complex system using overlays?

Within the ro tree (e.g. /usr/roports) there are only Packages build from the ports
to get the pathes work. All of these packages do NOT need anything below
/usr/local and are managed with this prefix.

PATH etc (e.g. LOCALBASE) in the machine is aware of this and tries to use
these ro files as long as they exist (e.g. libs) when one wants to build something
from source or ports.

PREFIX for the user is /usr/local. So it doesn’t matter if he uses pkg or ports,
he can write it.

Your idea below is quite nice, but as some pkgs are located under /usr/roports
and some under /usr/local, the resulting pkg database should be aware of the
real location of the files where the latest version of a package is installed.
Recreating the database again on a fresh machine is smart, but it would fail
about this.

On the other hand this structure is installed in a big buch of machines, all with
different installations under /usr/local - so it would be quite painfull to sort it
by hand for each single machine :(

Is there something else you can see? Or is it even you who could do a freelancing
job about this? :)

thanks in any case!

Sebastian

> If it's the first option, and maybe for the other two options, then a
> fairly quick solution is to dump out the lists of packages you want
> installed (i.e. the ones not set to autoremove)
> 
>   # pkg query -e '%a = 0' '%n-%v' > pkg-list1
> 
> Do that for both of your pkg databases, and then merge the two lists,
> removing any duplicates:
> 
>   # cat pkg-list1 pkg-list2 | sort -u > pkg-list
> 
> You'll need to check pkg-list manually for any duplicated packages with
> different versions, and choose which one (usually the most recent) you
> want installed.  Delete any unwanted duplicates from the list.
> 
> Then on a test box -- just a normally setup FreeBSD machine, with no
> special permissions settings --  install the edited list of packages:
> 
>   # xargs pkg install < pkg-list
> 
> Now compare the set of files installed on the test machine with what is
> on the original machine with the ro and rw filesystems.  Delete any
> packages on the original system not present on the new one, and install
> any that are missing -- you'll probably want to do some testing at this
> point to make sure you've got the right functionality still working. The
> idea is to end up with an identical package load on both systems.
> 
> Now you should be able to simply copy /var/db/local.sqlite from the test
> box to your original server.
> 
> Does that make sense to you?
> 
> 	Cheers,
> 
> 	Matthew
> 
>