freebsd-update's install_verify routine excessive stating

[Bert JW Regeer]

Hackers,

Recently I decided it would be time to upgrade an older laptop that  
was still running 6.2-RELEASE to a more recent release 7.1-RELEASE (re- 
install not possible, laptop has no cd-rom drive, and does not boot  
from a USB one).

Everything went well, including the kernel update. It was not until  
after I rebooted and ran:

./freebsd-update.sh -f freebsd-update.conf install

That I started noticing something rather weird. It had been running  
for a good 30 minutes before I started wondering what was going on.  
top gave me nothing of value, other than that all of the time was  
spent running sh, and that it was spending all of its time in system,  
not user where I would have expected it. Thinking went wrong I stopped  
the process and started it again.

After a ktrace and kdump I found the culprit in install_verify in the  
freebsd-update utility, it would read one line, and then run stat on  
one of the many files that was listed in /usr/upgrade/files/.

install_verify () {
         # Generate a list of hashes
         cat $@ |
             cut -f 2,7 -d '|' |
             grep -E '^f' |
             cut -f 2 -d '|' |
             sort -u > filelist

         # Make sure all the hashes exist
         while read HASH; do
                 if ! [ -f files/${HASH}.gz ]; then
                         echo -n "Update files missing -- "
                         echo "this should never happen."
                         echo "Re-run '$0 fetch'."
                         return 1
                 fi
         done < filelist

         # Clean up
         rm filelist
}

running find /usr/upgrade/files/ | wc -l showed over 64000 files. So  
what was happening here is that freebsd-update.sh is doing due  
diligence in checking that all the files exist, however it uses the  
built in test utility to do so. While in normal situations this may  
not be that big of a deal since the changeset is likely to be small,  
running stat on 64000 individual files in a loop is rather slow.

In my case I have just removed the offending code and hoped all went  
well, however this is off course not an acceptable solution. I have  
not yet come up with an acceptable way to fix the offending code,  
hence my post to hackers. I am also not sure as to how I would file a  
pr bug report for this situation, any guidance would be greatly  
appreciated.

This email has become much longer than I had originally intended. I  
just wanted to get this out to see what you all had to say about this.

Cheers,
Bert JW Regeer