Free BSD in Googles's Index

Kris Kennaway kris at
Thu Apr 17 17:56:55 PDT 2003

On Thu, Apr 17, 2003 at 07:21:13PM -0400, Bill Moran wrote:
> Ashley Gillard wrote:
> >Dear Free BSD Representative:
> >
> >Our crawl team has discovered that Google is currently blocked from
> >crawling your site by the robot.txt that is on your site. We would like
> >to index your site at to provide you better
> >coverage and are interested in working with you to do this. 
> >
> >Google's mission is to deliver the best search experience on the
> >Internet by making the world's information universally accessible and
> >useful. To find out more about visit:
> >
> >
> >I look forward to hearing from you.
> First off, bento is a ports build cluster and it's probably impractical
> to have google index it, as the information probably changes before it
> can even reach your index.

Correct, it's not appropriate for google to index bento.  It contains
tens of gigabytes of log files.

> If you still think it would be practical to alter the robots.txt file,
> it's probably best to contact Satoshi Asami (asami at as he
> appears to maintain that site.

Nope, that's mine.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url :

More information about the freebsd-questions mailing list