Re: do-fetch.mk never actually verifies the sha256 checksum

While working on alreay-landed review D50142 [1], I noticed that
checksum is checked not on `make fetch` but on `make extract`.

[1] https://reviews.freebsd.org/D50142

On Fri, 16 May 2025 10:02:27 -0500
"John Marino (FreeBSD)" <freebsd@marino.st> wrote:

> Yes, it's easy to reproduce with "make fetch".
> I guess one could debate if it's correct to pass the fetch stage with a bad
> download only to get caught later.
> To play devil's advocate, if there are multiple sites and/or site sources
> provided, sites that may have the correct checksum, it could never be
> successfully fetched (unless site sources are randomized, then you might
> get it by chance).
> So I think there's a reasonable argument to do the check during the fetch
> phase.
> 
> also, the hash is generated every time the port is built, so say a specific
> port is built 5 times with the same distfile.  That would be a total of 6
> hash generations, so that's not 100% increase, only 20%
> 
> John
> 
> On Fri, May 16, 2025 at 9:53 AM Ronald Klop <ronald-lists@klop.ws> wrote:
> 
> > I can't reproduce your issue. Made a deliberate typo in the checksum in a
> > distinfo file and get this error.
> >
> > ===>  Extracting for mongodb70-7.0.20
> > => SHA256 Checksum mismatch for mongodb-mongo-r7.0.20_GH0.tar.gz.
> > ...
> > ===>  Giving up on fetching files:  mongodb-mongo-r7.0.20_GH0.tar.gz
> > Make sure the Makefile and distinfo file
> > (/usr/ports/databases/mongodb70/distinfo)
> > are up to date.  If you are absolutely sure you want to override this
> > check, type "make NO_CHECKSUM=yes [other args]".
> > *** Error code 1
> >
> >
> > NB: I now realize the check happens on extract instead of fetch. I think
> > because people can get the files from other sources than only do-fetch. And
> > doing the check twice is a bit expensive for a checksum compared to
> > checking the file size. But I didn't design this.
> >
> > Regards,
> > Ronald.
> >
> >
> >
> > *Van:* "John Marino (FreeBSD)" <freebsd@marino.st>
> > *Datum:*vrijdag, 16 mei 2025 16:17
> > *Aan:*freebsd-ports@freebsd.org
> > *Onderwerp:*do-fetch.mk never actually verifies the sha256 checksum
> >
> > For each distribution file listed in a port's distinfo file, the file's
> > size and SHA256 hash is provided.  However, after a distribution file
> > candidate is downloaded, only the file's size is verified to match the
> > requirements.  The downloaded file is never hashed to verify it matches the
> > required checksum.
> >
> > basic logic per file:
> > 1. Verify an SHA256 list for the file is present in the distinfo file.
> > 2. Attempt fetch requiring file size listed in distinfo (size requirement
> > may be ignored)
> > 3. Upon successful download, verify downloaded file size matches
> > requirement.
> > 4. If file size matches => success (otherwise try backup sites or FAIL)
> >
> > I assume the original intent was to first check file size, and then
> > calculate the SHA256 sum of the downloaded file and compare that to the
> > distinfo requirements.
> >
> > So currently it's possible to successfully fetch a distribution file that
> > has the same size but a different checksum than the file specified in
> > distinfo.
> >
> > To interate -- the do-fetch.mk requires distinfo to provide an SHA256
> > checksum, but it doesn't do anything with it.


-- 
Tomoaki AOKI    <junchoon@dec.sakura.ne.jp>