svn commit: r41802 - head/en_US.ISO8859-1/htdocs

[Glen Barber]

On Fri, May 31, 2013 at 10:54:45PM +0000, Eitan Adler wrote:
> Author: eadler
> Date: Fri May 31 22:54:45 2013
> New Revision: 41802
> URL: http://svnweb.freebsd.org/changeset/doc/41802
> 
> Log:
>   There appear to be configuration errors causing an absurd amount of links to be generated.
>   For now, block access to search engines until the problem is properly resolved.
>   
>   While here, add $FreeBSD$
> 
> Modified:
>   head/en_US.ISO8859-1/htdocs/robots.txt
> 
> Modified: head/en_US.ISO8859-1/htdocs/robots.txt
> ==============================================================================
> --- head/en_US.ISO8859-1/htdocs/robots.txt	Fri May 31 19:10:06 2013	(r41801)
> +++ head/en_US.ISO8859-1/htdocs/robots.txt	Fri May 31 22:54:45 2013	(r41802)
> @@ -1,4 +1,8 @@
> +# $FreeBSD$
> +
>  User-agent: *
>  
>  Disallow: /cgi/
>  Disallow: /statistic
> +Disallow: /publish.html/hu-hu/

This is wrong.

http://www.freebsd.org/index.html/omgwtfbbqhax

> +Disallow: /cgi/

This already exists in the file...

Glen

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 488 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/svn-doc-all/attachments/20130531/bb27841a/attachment.sig>