Download contents of http directory?

Sun Jan 11 08:36:40 PST 2004

On Mon, Jan 12, 2004 at 12:18:52AM +1030, Malcolm Kay wrote:
> On Sun, 11 Jan 2004 22:35, Matthew Seaman wrote:
> > On Sun, Jan 11, 2004 at 04:53:52PM +1030, Malcolm Kay wrote:
> > > This raises a question for which I don't know the answer:
> > >   How does one list an http directory that does allow it?
> >
> > Basically, you shouldn't.  If the web site administrator has set up
> > index.html files or otherwise prevented you from generating a
> > directory listing, it generally means that there's stuff in that
> > directory which you aren't meant to access.  It's impolite (at best)
> > to try and get round that, although the wise admin will take stronger
> > meansures to ensure that even if you can guess filenames, you still
> > can't download anything you shouldn't.
> >
> 
> Please reread my query -- I'm asking how to read the directory 
> when the administrator does allow it.

Oops.  Sorry about that.  Note to self: read what is written, not
anything else.

Just do a HTTP GET on the directory name. eg.

    % GET http://localhost/~matthew/
    <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
    <html>
     <head>
      <title>Index of /~matthew</title>
     </head>
     <body>
    <h1>Index of /~matthew</h1>
    <pre><img src="/icons/blank.gif" alt="Icon " /> <a href="?C=N;O=D">Name</a>                    <a href="?C=M;O=A">Last modified</a>      <a href="?C=S;O=A">Size</a>  <a href="?C=D;O=A">Description</a><hr /><img src="/icons/back.gif" alt="[DIR]" /> <a href="/">Parent Directory</a>                             -   
    <img src="/icons/folder.gif" alt="[DIR]" /> <a href="books/">books/</a>                  07-Aug-2003 20:12    -   
    <img src="/icons/folder.gif" alt="[DIR]" /> <a href="cryptosphere/">cryptosphere/</a>           02-Jun-2003 16:34    -   
    <img src="/icons/unknown.gif" alt="[   ]" /> <a href="foo.php">foo.php</a>                 08-Oct-2003 12:24  1.1K  
    <img src="/icons/text.gif" alt="[TXT]" /> <a href="mrtg-rrd.cgi">mrtg-rrd.cgi</a>            23-Jan-2003 13:30   24K  
    <img src="/icons/text.gif" alt="[TXT]" /> <a href="mrtg-rrd.cgi-1.20">mrtg-rrd.cgi-1.20</a>       31-Jan-2003 16:25   24K  
    <img src="/icons/text.gif" alt="[TXT]" /> <a href="mrtg-rrd.cgi.000">mrtg-rrd.cgi.000</a>        14-Jan-2003 12:55   24K  
    <img src="/icons/folder.gif" alt="[DIR]" /> <a href="mrtg/">mrtg/</a>                   18-Apr-2003 15:36    -   
    <img src="/icons/text.gif" alt="[TXT]" /> <a href="nwc.cgi">nwc.cgi</a>                 17-Nov-2002 08:04   12K  
    <img src="/icons/text.gif" alt="[TXT]" /> <a href="nwc.css">nwc.css</a>                 17-Nov-2002 08:04  2.0K  
    <img src="/icons/unknown.gif" alt="[   ]" /> <a href="nwc.php">nwc.php</a>                 17-Nov-2002 08:04   12K  
    <hr /></pre>
    <address>Apache/2.0.48 (Unix) PHP/4.3.4 Server at localhost Port 80</address>
    </body></html>

If you want that without all of the HTML fluff:

    % lynx -dump -nolist http://localhost/~matthew/

                                  Index of /~matthew

     Icon   Name                    Last modified      Size  Description
      __________________________________________________________________________
     [DIR]  Parent Directory                             -
     [DIR]  books/                  07-Aug-2003 20:12    -
     [DIR]  cryptosphere/           02-Jun-2003 16:34    -
     [   ]  foo.php                 08-Oct-2003 12:24  1.1K
     [TXT]  mrtg-rrd.cgi            23-Jan-2003 13:30   24K
     [TXT]  mrtg-rrd.cgi-1.20       31-Jan-2003 16:25   24K
     [TXT]  mrtg-rrd.cgi.000        14-Jan-2003 12:55   24K
     [DIR]  mrtg/                   18-Apr-2003 15:36    -
     [TXT]  nwc.cgi                 17-Nov-2002 08:04   12K
     [TXT]  nwc.css                 17-Nov-2002 08:04  2.0K
     [   ]  nwc.php                 17-Nov-2002 08:04   12K
      __________________________________________________________________________

        Apache/2.0.48 (Unix) PHP/4.3.4 Server at localhost Port 80

	Cheers,

	Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.                       26 The Paddocks
                                                      Savill Way
PGP: http://www.infracaninophile.co.uk/pgpkey         Marlow
Tel: +44 1628 476614                                  Bucks., SL7 1TH UK
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-questions/attachments/20040111/7fb88e26/attachment.bin