apache deining robots.txt 404

Bob bob at a1poweruser.com
Thu Jun 14 13:15:52 UTC 2007


I just moved FBSD 6.2 / apache-1.3.37.1 into production.
In the httpd-access,log I see all the search engines requests for robots.txt
getting a 404 code.

The previous production world was FBSD 6.0 /apache-1.3.33_2 and all the
search engines requests for robots.txt got a 200 code.

I use the same httpd-conf in both. So the only thing changed is the version
of FBSD and apache.
Has anybody else noticed this?
How can I allow robots.txt to be handled like before??



More information about the freebsd-questions mailing list