portmaster: grep: Argument list too long

Mon Oct 19 06:04:43 UTC 2009

Kenyon Ralph wrote:
> On 2009-10-18T21:43:24-0700, Doug Barton <dougb at FreeBSD.org> wrote:
>> Kenyon Ralph wrote:
>>> I'm trying to replace perl 5.8 with perl 5.10, so I did this (as in the
>>> 20090328 /usr/ports/UPDATING entry):
>>>
>>> % sudo portmaster -v -t -D -o lang/perl5.10 lang/perl5.8
>>> % sudo portmaster -v -D -R -m 'FORCE_PKG_REGISTER=yes' -r perl-threaded-5.10.1
>>>
>>> (FORCE_PKG_REGISTER seems to be needed to keep libtool22 from erroring
>>> and stopping the portmaster run.)
>> I'm looking into this in more detail, but first I wonder why libtool
>> is being built at all. Did you follow the instructions in the 20090802
>> UPDATING entry to replace libtool15 with libtool22? You should not
>> need to use FORCE_PKG_REGISTER in this situation, that's a sign that
>> there is something wrong. I also suspect that it's masking a deeper
>> problem and is the cause of having things installed more than once in
>> the same portmaster run (which should never happen).
> 
> This system was first created around 2009-08-14, so it always had
> libtool22.

Ok

>> I'm also wondering if all your other ports are up to date (other than
>> switching perl versions). At least on my system perl is not required
>> by libtool so it doesn't make sense that it (or several of the other
>> things in your log) are being built at all.
> 
> Yes, before I started this perl upgrade, I had just csup'd /usr/ports
> and completed portmaster -a.

Ok

>> I'm also concerned that there is something wrong with your /var/db/pkg
>> directory, since grep can handle an argument list of many thousands of
>> items without any problems. Can you do this and let me know what it
>> says: ls -d /var/db/pkg/* | wc -l
> 
> After doing portmaster --check-depends which did a few +REQUIRED_BY
> updates:
> 
> % pkg_info | wc -l
>      845
> 
> % ls -d /var/db/pkg/* | wc -l
>      846
> 
> That is one more than portmaster -l:
> ===>>> 845 total installed ports

Ok, can you please figure out what the extra entry is?

>> Finally I would run 'portmaster --check-depends' just to see if there
>> is anything else strange going on. You will probably get some errors
>> about no installed version of perl5.8 which is normal.
> 
> Ah, I do remember seeing some warnings flying by saying that I should
> try --check-depends. As I said above, I executed it. Didn't see any
> errors about perl5.8.
> 
> I also ran portmaster --check-port-dbdir, deleted the directories that
> did not seem to be installed, and noticed that x11/kde4 and kdeutils4
> were not installed (even though the rest of kde4 seems to be installed).
> So I'm doing portmaster -v -t -D -R x11/kde4 right now, which is
> building several ports. I'll try resuming -r perl-threaded-5.10.1 after
> this.

Ok, when you get back to the perl upgrade please apply the attached
patch first. It is going to cause a lot of console output so you're
probably better off starting script first to capture it.


Doug

-- 

	Improve the effectiveness of your Internet presence with
	a domain name makeover!    http://SupersetSolutions.com/

-------------- next part --------------

--- portmaster	2009/09/15 07:50:11	2.54
+++ portmaster	2009/10/19 06:00:32
@@ -384,8 +384,15 @@
 
 iport_from_origin () {
 	local dir
+
+echo "Debug> About to do grep -l @comment ORIGIN:${1}$ $pdb/*/+CONTENTS"
+
 	dir=`grep -l "@comment ORIGIN:${1}$" $pdb/*/+CONTENTS`
 
+	if [ $? -ne 0 ]; then
+		fail "grep command failed"
+	fi
+
 	# It should not happen that more than one port meets this
 	# requirement, but it can if the pkg data is corrupted.
 	dir="${dir%%/+CONTENTS*}"
@@ -394,7 +401,13 @@
 
 origin_from_pdb () {
 	local o
+
+echo "Debug> About to do grep '@comment ORIGIN:' $pdb/$1/+CONTENTS 2>/dev/null"
+
 	o=`grep '@comment ORIGIN:' $pdb/$1/+CONTENTS 2>/dev/null`
+	if [ $? -ne 0 ]; then
+		fail "grep command failed"
+	fi
 	echo ${o#@comment ORIGIN:}
 }
 
@@ -417,6 +430,9 @@
 
 	# Always rely on the grep'ed dependencies instead of +REQUIRED_BY
 	grep_deps=`pm_mktemp grep-deps-${iport}`
+
+echo "Debug> About to do egrep -l "DEPORIGIN:($origin|$ro_opd)$" $pdb/*/+CONTENTS"
+
 	egrep -l "DEPORIGIN:($origin|$ro_opd)$" $pdb/*/+CONTENTS |
 	    cut -f 5 -d '/' | sort -u > $grep_deps
 
@@ -538,6 +554,9 @@
 	# To avoid having each word of the reason treated separately
 	IFS='
 '
+
+echo "Debug> About to do grep ^$sf| $pd/MOVED"
+
 	for l in `grep "^$sf|" $pd/MOVED`; do
 		case "$l" in
 		${sf}\|\|*) [ -n "$iport" ] || iport=`iport_from_origin $sf`
@@ -616,6 +635,9 @@
 		fi
 
 		if [ -s "$distinfo" ]; then
+
+echo "Debug> About to do grep '^MD5' $distinfo"
+
 			grep '^MD5' $distinfo | while read disc1 f disc2; do
 				f=${f#(} ; f=${f%)}
 				echo $f >> $DI_FILES
@@ -868,6 +890,9 @@
 	# Global: state
 	local state_set
 
+echo "Debug> About to do egrep -ql '^(FORBIDDEN|DEPRECATED|BROKEN|IGNORE)' Makefile"
+
+
 	if egrep -ql '^(FORBIDDEN|DEPRECATED|BROKEN|IGNORE)' Makefile; then
 		for state in FORBIDDEN DEPRECATED BROKEN IGNORE; do
 			state_set=`pm_make -V $state`
@@ -1059,6 +1084,8 @@
 		[ -s "$dist_list" ] || { unset dist_list ; return 0; }
 		[ -n "$DONT_SCRUB_DISTFILES" ] && return 0
 
+echo "Debug> About to do grep ^DISTFILE $dist_list"
+
 		# The grep is needed to allow for comments, etc.
 		for file in `grep ^DISTFILE $dist_list`; do
 			file=${file#DISTFILE:} ; file=${file%%:*}
@@ -1689,6 +1716,8 @@
 	# Global: MASTER_RB_LIST
 	local req_by
 
+echo "Debug> About to do grep -l DEPORIGIN:$portdir$ $pdb/*/+CONTENTS"
+
 	for req_by in `grep -l DEPORIGIN:$portdir$ $pdb/*/+CONTENTS | \
 	    cut -f 5 -d '/'`; do
 		MASTER_RB_LIST="${MASTER_RB_LIST}${req_by} "
