OT: wget bug

[Andrew Brampton]

2009/7/17 Joe R. Jah <jjah at cloud.ccsf.cc.ca.us>:
>
> Hello all,
>
> I want to wget a site at regular intervals and only get the updated pages,
> so I use the this wget command line:
>
> wget -b -m -nH http://host.domain/Directory/file.html
>
> It works fine on the first try, but it fails on subsequent tries with the
> following error message:
>
> --8<--
> Connecting to host.domain ... connected.
> HTTP request sent, awaiting response... 401 Unauthorized
> Authorization failed.
> --8<--

This to me seems like the remote server is replying with 401. Perhaps
wget is sending the If-Modified-Since HTTP header, and the remote
server does not support this. I would confirm this by running tcpdump
(or wireshark) to sniff the traffic and see what the remote server is
replying with.

If the remote server is truly returning 401, then you might either
need to use an alternative tool, or configure wget differently.

Hope this helps
Andrew