apache deining robots.txt 404

[Karol Kwiatkowski]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Bob wrote:
> I just moved FBSD 6.2 / apache-1.3.37.1 into production.
> In the httpd-access,log I see all the search engines requests for robots.txt
> getting a 404 code.
> 
> The previous production world was FBSD 6.0 /apache-1.3.33_2 and all the
> search engines requests for robots.txt got a 200 code.
> 
> I use the same httpd-conf in both. So the only thing changed is the version
> of FBSD and apache.
> Has anybody else noticed this?
> How can I allow robots.txt to be handled like before??

After [1]:

% 10.4.5 404 Not Found
%
% The server has not found anything matching the Request-URI [...]

It means you just don't have robots.txt in www root directory.

HTH,

Karol

[1] http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html

- --
Karol Kwiatkowski   <karol.kwiat at gmail dot com>
OpenPGP 0x06E09309
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (FreeBSD)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGcUb+ezeoPAwGIYsRCBSDAJ9eYxO81wc4a/QopZnexGXEklpL2QCfWixR
zb1Zh0Dlg533kpipNrgWswM=
=TmMc
-----END PGP SIGNATURE-----