Call for testers for yet another ports upgrade program, ports+

[Garrett Cooper]

Yoshihiro Ota wrote:
> To Whom Slowness of Portupgrade Concerns a Lot:
>
> As I got tired of long waiting of portupgrade trying to resolve dependencies, I came up with yet another tool for upgrading FreeBSD ports system.  Unlink other tools, it tries to maximize existing resource to maximize its performance.  This program attempts to wrap around with another 'make' and expand use of FreeBSD ports system.  The heart of ports+ is parsing INDEX and +CONTENTS files.  The rest is handed to GNU make.  I think it comes to a point where I seek wider audiences to test with it.
>
>
> Some short summaries of ports+
>   * Magnitude quicker dependency resolving compare to portupgrade.
>   * Dependencies are resolved only once for all upgrades where portupgrade does before each 'make deinstall and make install' of ports.
>   * Commands to execute for each ports are very flexible.
>   * Allow concurrent builds of ports with -j option. (yet experimental)
>
> Some dry runs comparisons with about 500 ports insalled on Mobile Intel(R) Celeron(R) CPU 2.40GHz.  Note that programs were run twice in a row
> after rebooting for each program.  That is to kill disk-cache effect from the other program.  The first run takes longer to read disk; however, the second run only accessed via file-cache.
>
> # time portupgrade -n -a -O
> 60.901u 36.208s 2:30.08 64.6%   147+13408k 4840+59io 77pf+0w
> # time portupgrade -n -a -O
> 83.581u 47.805s 2:51.34 76.6%   155+13197k 547+33io 0pf+0w
> # time gmake -n all
> 4.784u 1.982s 0:22.06 30.6%     258+2485k 1528+126io 9pf+0w
> # time gmake -n all
> 1.551u 0.205s 0:01.85 94.5%     179+8540k 0+0io 0pf+0w
>
> Three advises before trying:
> 1. Read the description in GNUmakefile and set PORTS+_IGNORE ports+.conf for ones you don't want to upgrade.
> 2. At least, backup /var/db/pkg so that you can try again even with other tools.
> 3. Do not 'make clean' so that you can try installing again.
>
> Ports+ requires:
> 1. GNU make, gmake.
> 2. GNU awk, gawk.
> 3. Up to date INDEX file.
>
> Known Issues:
> 1. Concurrent build with -j is still experimental although I always use it myself.  There is a race condition when more than two ports updating the same +CONTENTS files at the same time.  This could result +CONTENTS file out of sync.  However, you could try at your risk, too.
> 2. If OPTIONS changes dependences of a port, ports+ won't handle it at this moment.
> 3. I don't know if I have to maintain any other file other than +CONTENTS.
>
> For these who want to read the description in the archive without getting one, it is copied and available at:
> http://uyota.asablo.jp/blog/2007/07/26/1681765
>
> For these who want to give a shot wihtout reading further.  Try these commands below.  These two gmake examples are dry runs and won't harm your system.  You can try with 'gmake -n' and once you are confident, you can take -n option out.
>
> # fetch http://www.ne.jp/asahi/export/uyota/src/ports+-1.59.tar.bz2
> # tar xvf ports+-1.59.tar.bz2
> # cd ports+1.59
> # gmake test
> # gmake -n all
>
>
> Any feed backs are welcome including "it works".  I will applicate  bug reports, suggestions on anything or specially solving +CONTENTS file race condition on parallel builds.  I am indeed wondering if there is a way to queue or batch commands to run at even high load.
>
>
> Thanks,
> Hiro
Hiro,
You're correct when you say that reading INDEX* does take a long time
for portupgrade, but the problem is partly with how portupgrade formats
its version of the INDEX database in BDB format, as well as how it
doesn't buffer up proper information (pkg database information for
instance), and guesses at port dependencies (origins, deporigins, etc).
1. Can your solution be made bsdmake and bsd awk compatible?
2. Does your solution account for cases when you're trying to install
package a, which depends on package b, but because you built package b
while trying to build a, and are at an intermediate package c (between a
and b), the dependencies for a are only partially complete. Thus when
you try to install direct or indirect dependences, the install fails?
3. How is this different from pkg_version combined with similar scripts?
　ちゃんと 説明するのを頑張ってね、ほろさん ;)・・
-Garrett