Enforcing "DIST_SUBDIR/DISTFILE" uniqueness

[Andrew Pantyukhin]

On 8/20/06, Doug Barton <dougb at freebsd.org> wrote:
> Andrew Pantyukhin wrote:
> > On 8/16/06, Andrew Pantyukhin <infofarmer at freebsd.org> wrote:
> >> I'd like to propose a policy to enforce a change in
> >> DIST_SUBDIR whenever a distfile is rerolled in-place, i.e.
> >> when checksum changes, but name stays unchanged.
> >>
> >> Moreover, effort should be made whenever possible to
> >> make the old file available for download from an
> >> alternative location.
> >>
> >> This policy will rid us of some fetch-related headaches.
> >> It also will make it possible to share distfiles between
> >> hosts with ports trees of different dates. Some rare issues
> >> might also be resolved as a result of this. For one, ftp
> >> mirrors could be configured to allow upload, but deny
> >> modification and/or deletion.
> >>
> >> One thing I would personally frown upon is using
> >> something like "fetch -o othername" to save a file with a
> >> different name. It looks all right, but it prevents us from
> >> looking for mirrors in an automated way when master
> >> sites go down.
> >
> > Well, if no one is really against,
>
> I am violently against this proposal, but I was really hoping
> that someone else would speak up first.

No need to be that violent, pal. Nothing's been set in stone yet
and the reason for me writing here is to discuss it, not fight
over it.

> > I'll start preparing statements
> > for documentation and thinking about a way to watch for
> > "violations". I also intend to go through CVS and find past
> > "offenders" to prod them about it.
> >
> > The recent openoffice update rerolled a file in-place, and while
> > it may seem irrelevant or even beneficial (erasing 286Mb of
> > the old file), the fact is that it prevents us from keeping distfile
> > history on unversioned file servers,
>
> IMO this represents a very small minority of FreeBSD users,
> and frankly I feel that it is incumbent on you to solve this problem
> for your circumstance.

The percentage of FreeBSD users who need 5-10 year old
sources in the CVS is very small, too. But we treasure our
src history and don't throw out any commits. Well, I happen
to treasure our ports history. I really want people to have a
chance, however slim, to be able to build ports using a very
old tree.

> > not to mention problems with fetch many of us experience.
>
> I recently added logic to portmaster to handle this kind of situation
> transparently for the user. A more general solution to this part of the
> problem could be had by improving the logic in the 'make checksum' target.

Great news, thanks!

> OTOH, your solution would break the logic that portmaster (and I believe
> portupgrade also) uses to detect and delete stale distfiles.

AFAICT portmaster's logic still misses the case when
DIST_SUBDIR has changed for whatever reason.

portupgrade --distclean will not be broken, it deals with
distfiles at the current DIST_SUBDIR

portsclean -D is actually broken now, and will be fixed if
my proposal is implemented. It doesn't erase an old file if
its path/name match those of a new file.