[Bug 212065] fetch -r fails on a complete file, even with -S 12345 (OS version independent)

bugzilla-noreply at freebsd.org bugzilla-noreply at freebsd.org
Mon Aug 22 21:40:12 UTC 2016


https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=212065

            Bug ID: 212065
           Summary: fetch -r fails on a complete file, even with -S 12345
                    (OS version independent)
           Product: Base System
           Version: 10.3-RELEASE
          Hardware: Any
                OS: Any
            Status: New
          Severity: Affects Some People
          Priority: ---
         Component: bin
          Assignee: freebsd-bugs at FreeBSD.org
          Reporter: mandree at FreeBSD.org

When fetch(1) is used to resume (-r) on a completely downloaded file, it
erroneously fails with exit status 1. It should instead detect that condition
"file is complete and same size as on server" and exit with status 0 (=
success).

Adding to the -r option also a -S 12345 option, where -S is the same size as
the file locally has, and as the server thinks its file is, does NOT help. 
Using "-m" is not an option if the file date isn't reported by the server.

This bug has been reported on #bsdports (IRC) against 9.3 with portsnap, and
can be verified without portsnap on 10.3-RELEASE-p7. Commented verbose fetch
output on a hacked "portsnap fetch" downloading the initial snapshot as of
today.  First, see we have the file:

$ ls -l *.tgz
-rw-r--r--  1 root  wheel  74481909 Aug 22 23:03
80c30ef3c99969de68fad7d92a47c4e69cd6f1c8135504152d794c21305b1245.tgz

Next, ask server for the size:

$ fetch -s
http://ec2-eu-west-1.portsnap.freebsd.org/s/80c30ef3c99969de68fad7d92a47c4e69cd6f1c8135504152d794c21305b1245.tgz
; echo $?
74481909
0

So, sizes match. File is complete.
Now, try to resume the transfer, verbose mode:

$ fetch -rvv
http://ec2-eu-west-1.portsnap.freebsd.org/s/80c30ef3c99969de68fad7d92a47c4e69cd6f1c8135504152d794c21305b1245.tgz
; echo $?
[...]
---> ec2-eu-west-1.portsnap.freebsd.org:80
looking up ec2-eu-west-1.portsnap.freebsd.org
connecting to ec2-eu-west-1.portsnap.freebsd.org:80
requesting
http://ec2-eu-west-1.portsnap.freebsd.org/s/80c30ef3c99969de68fad7d92a47c4e69cd6f1c8135504152d794c21305b1245.tgz
>>> GET /s/80c30ef3c99969de68fad7d92a47c4e69cd6f1c8135504152d794c21305b1245.tgz HTTP/1.1
>>> Host: ec2-eu-west-1.portsnap.freebsd.org
>>> Accept: */*
>>> User-Agent: fetch libfetch/2.0
>>> Range: bytes=74481909-
>>> Connection: close
>>> 
<<< HTTP/1.1 416 Requested Range Not Satisfiable
<<< Content-Type: text/html
<<< Accept-Ranges: bytes
<<< Content-Length: 389
<<< Connection: close
content length: [389]
<<< Date: Mon, 22 Aug 2016 21:07:00 GMT
<<< Server: lighttpd/1.4.33
<<< 
fetch:
http://ec2-eu-west-1.portsnap.freebsd.org/s/80c30ef3c99969de68fad7d92a47c4e69cd6f1c8135504152d794c21305b1245.tgz:
Requested Range Not Satisfiable
1


OOPS! See that -S doesn't help, same size as above:


$ fetch -S 74481909 -r
http://ec2-eu-west-1.portsnap.freebsd.org/s/80c30ef3c99969de68fad7d92a47c4e69cd6f1c8135504152d794c21305b1245.tgz
; echo $?
fetch:
http://ec2-eu-west-1.portsnap.freebsd.org/s/80c30ef3c99969de68fad7d92a47c4e69cd6f1c8135504152d794c21305b1245.tgz:
Requested Range Not Satisfiable
1

NOTE: for proper robustness, fetch might want to issue a HEAD request to obtain
the file size unconditionally when resuming, so as to be able to tell the
condition "file shrunk on server" apart from "file has been completely
downloaded".


Other tools:
* curl 7.50.1 from ports fails in a similar way, and to add insult to injury,
wrongly concludes that the server did not support byte ranges when in fact it
does (but cannot start a download beyond the end of a file).
* wget 1.18 also sees the 416 code from the server, but concludes "The file is
already fully retrieved; nothing to do." and exits with status 0.
* lftp 4.7.3 with "get -c" does the right thing and exits silently if the file
is complete.

-- 
You are receiving this mail because:
You are the assignee for the bug.


More information about the freebsd-bugs mailing list