Free BSD in Googles's Index
wmoran at potentialtech.com
Thu Apr 17 16:21:15 PDT 2003
Ashley Gillard wrote:
> Dear Free BSD Representative:
> Our crawl team has discovered that Google is currently blocked from
> crawling your site by the robot.txt that is on your site. We would like
> to index your site at http://bento.freebsd.org to provide you better
> coverage and are interested in working with you to do this.
> Google's mission is to deliver the best search experience on the
> Internet by making the world's information universally accessible and
> useful. To find out more about Google.com visit:
> I look forward to hearing from you.
First off, bento is a ports build cluster and it's probably impractical
to have google index it, as the information probably changes before it
can even reach your index.
If you still think it would be practical to alter the robots.txt file,
it's probably best to contact Satoshi Asami (asami at freebsd.org) as he
appears to maintain that site.
More information about the freebsd-questions