poudriere amassing fetch errors

[P S]

This doesn't sound like a poudriere-specific issue. There may be a
general fetching issue with bsd.port.mk, that I plan to investigate further.""


    I believe you will find the problem is in the code related to 'make checksum'. I discovered a while back(at least a year ago) that 'make checksum' will append the the full version of a file that was only partially downloaded instead of deleting it and starting from scratch. Which means 'make checksum' will fail no matter what because the file increases by one partial/full filesize + previous file and then 'make checksum' fails and pours-it-on(re-downloads) at each invocation. It's not the only issue with 'make checksum' but I can't remember the others at the moment. The manual fix is to go to the offending port directory and do a 'make distclean', then a 'make fetch', and finally a 'make checksum'. An alternative is to delete the offending distfile and do a 'make fetch ; make checksum' or just "make checksum"(not reliable) in the offending port directory.

   For me 'games/el-data' always fails. The el_linux_192.zip file makes it from a different site but the sound and music files never get fully downloaded(only port with an unstable link, sound and music files should be mirrored somewhere else). Of course it didn't stop them from showing 300MB sizes for 70MB files due to 'make checksum' failing and adding on. 'Biology/finchtv' also never passes checksum as of last week. The distfile was over 100MB(normally 6MB) when I discovered the checksum problem. 

   In my experience, 'make fetch' doesn't always check filesize just that the file is there so mis-sized or re-rolled files often fell through without being re-downloaded.  'Make checksum' would catch the filesize errors but doesn't 'delete and download' or append properly. I thought this was just my systems and connections but after seeing other people reporting similar effects I figured I better say something of what I've observed. I also haven't checked recently if the checksum problem still exists as I've just worked around the problem. I sent a PR back then but I never saw it show up anywhere.

   I have been downloading(using 'make fetch' and 'make checksum') all the 
distfiles(minus restricted, unavailable ) for the entire ports system 
whenever I update ports. I have most of the distfiles going back at 
least 5 years now. Currently, the distfile area of any ports system release is about 80GB in size(not including restricted/licensed limited or unavailable files, a couple GB extra maybe). I won't register or "sign in" for files so no restricted files(cad/systemc, biology/phred, biology/blast, java/jdk, etc) and very few manual retrievals(java/jre, astro/xephem). The distfile area as of 1 january 2012 is about 300GB having been updated every few weeks to months over the previous five years(starting before the modular Xorg release).

   Ports-related: mail/cvsmail downloads the same-named distfile as the mail/mailutils port but they come from different places, are different sizes, and overwrite each other when downloaded(verified last week). PR sent.