From nobody Wed Aug 23 06:17:52 2023 X-Original-To: dev-commits-doc-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RVwxX4fxJz4r8lW for ; Wed, 23 Aug 2023 06:17:52 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RVwxX3zBdz4M5S; Wed, 23 Aug 2023 06:17:52 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692771472; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=LL8Uk/OBlrjRKy1ncBeSG5R3ejJ+Ia/l96eqkHe05/M=; b=v/Zgo3VU4PObJALD4t37vbE2+ZpGVTBA9sPNZtTTIOGsb9IRHUrJve/qpNEGRGBk1RPUJm nH7R76P7WZN1Dz/jTGlOVvui6Qaa8J0lXaEqQhySgtmcoSZMeOAy9GM4q8BTpwMCL0EfGX R6hdz/M/z3lw0h8/nhkLNxsRSyvbtUDFJgz3MMu4PDXGA1khWd2I4Tda+Tp0AzjR31zLRe adCE47k0DwIHJCEtANw/HY6xtLzrnqHR62xyCfSY/eeYK/73k7fvP9W4ZZ3f9ShEpCHYG9 wqNrt38FTWqVDthukHDu62QuIEITHEM/7D3MKysGfh6t1Mq17Sv9/PBN1tyROQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692771472; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=LL8Uk/OBlrjRKy1ncBeSG5R3ejJ+Ia/l96eqkHe05/M=; b=wHWadNQoZTlk4F/tFTZsL/Ala/oMWkdfYgzPffkp/emHfpKpWF/fcjq0/Q+lLqhGpIQw+F wYRRr3evnUUal36ppyAMLWFmptXFPkQ42pl50NBLU6sc6noiV6EYfSg4u0OJE0VYBMN+Vj EjQcnow2oq7K7z1bPU6GgdDyi7uaScxbfhE0wEiVHRYACAn0UyM02qKjZCjebki29W8N1P k1un4QBlkN/vpFg1wAN3GBUqevesT0l4hBb+/s4w2cOUsXi+imhhJXMPdWMaALlP+c0R7b PYzAmWntBZ65KlY3O9wH0yRZemyijSj7U3D3a7N7d11VmniHcuTiUwlHyeCUOQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1692771472; a=rsa-sha256; cv=none; b=S9lUkPz4bLWeqWQgprdXADMBOy8jP+kh7y8+f0ZTCf5L29a8KRq2OS7GghU8PEW5iJsIzR ddLiBI4NBwN1kGyTmdAeyel6IuxvnlMt6I48cgSXEDQJuIM4ugSBO3fHPXfIVZTaAZbYlc HCe8V8uSkXNkTx7A0JgFiE6XhNGtYvGB3RU17veMfjGw02Oufo9D+fD6s/c7OsGk1wAjon 0hiBrkquRg8wsd34Io0qU6C80w/W3e0LKOaRv9kaml91vVsl9//gTlDSoK99PLOFiQPMDF TJmaBLmPfpKQYBF8J5zg16+getKt2meTe2KBj6SczztNXUEHZO2EEFcOUXfwzg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4RVwxX2zjczZT2; Wed, 23 Aug 2023 06:17:52 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 37N6Hq3Z096861; Wed, 23 Aug 2023 06:17:52 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 37N6Hq2N096858; Wed, 23 Aug 2023 06:17:52 GMT (envelope-from git) Date: Wed, 23 Aug 2023 06:17:52 GMT Message-Id: <202308230617.37N6Hq2N096858@gitrepo.freebsd.org> To: doc-committers@FreeBSD.org, dev-commits-doc-all@FreeBSD.org From: Wolfram Schneider Subject: git: e09c06b9c2 - main - block CCBot (Common Crawl) from accessing cgi scripts List-Id: Commit messages for all branches of the doc repository List-Archive: https://lists.freebsd.org/archives/dev-commits-doc-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-doc-all@freebsd.org X-BeenThere: dev-commits-doc-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: wosch X-Git-Repository: doc X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: e09c06b9c2fa70e4884edae327d294c5efc3eb02 Auto-Submitted: auto-generated The branch main has been updated by wosch: URL: https://cgit.FreeBSD.org/doc/commit/?id=e09c06b9c2fa70e4884edae327d294c5efc3eb02 commit e09c06b9c2fa70e4884edae327d294c5efc3eb02 Author: Wolfram Schneider AuthorDate: 2023-08-23 06:17:36 +0000 Commit: Wolfram Schneider CommitDate: 2023-08-23 06:17:36 +0000 block CCBot (Common Crawl) from accessing cgi scripts --- website/themes/beastie/layouts/robots.txt | 3 +++ 1 file changed, 3 insertions(+) diff --git a/website/themes/beastie/layouts/robots.txt b/website/themes/beastie/layouts/robots.txt index 9bec7f76cf..1248457c75 100644 --- a/website/themes/beastie/layouts/robots.txt +++ b/website/themes/beastie/layouts/robots.txt @@ -14,3 +14,6 @@ Disallow: /statistic User-agent: GPTBot Disallow: /cgi +User-agent: CCBot +Disallow: /cgi +