git: c048664e41 - main - block GPTBot (OpenAI) from accessing cgi scripts

Go to: [ bottom of page ] [ top of archives ] [ this month ]

From: Wolfram Schneider <wosch_at_FreeBSD.org>
Date: Mon, 07 Aug 2023 13:52:01 UTC

The branch main has been updated by wosch:

URL: https://cgit.FreeBSD.org/doc/commit/?id=c048664e41ee40b03123d4370f89c7be217f9d39

commit c048664e41ee40b03123d4370f89c7be217f9d39
Author:     Wolfram Schneider <wosch@FreeBSD.org>
AuthorDate: 2023-08-07 13:51:53 +0000
Commit:     Wolfram Schneider <wosch@FreeBSD.org>
CommitDate: 2023-08-07 13:51:53 +0000

    block GPTBot (OpenAI) from accessing cgi scripts
---
 website/themes/beastie/layouts/robots.txt | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/website/themes/beastie/layouts/robots.txt b/website/themes/beastie/layouts/robots.txt
index b94bcd43cf..ac0950c19f 100644
--- a/website/themes/beastie/layouts/robots.txt
+++ b/website/themes/beastie/layouts/robots.txt
@@ -17,3 +17,7 @@ Disallow: /cgi/query-pr.cgi
 Disallow: /cgi/search.cgi
 Disallow: /cgi/url.cgi
 Disallow: /statistic
+
+User-agent: GPTBot
+Disallow: /cgi
+