[Bug 287944] [NEW PORT] www/iocaine: The deadliest poison known to AI

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=287944

--- Comment #6 from Sebastian Oswald <sko@rostwald.de> ---
(In reply to void from comment #4)

I don't know why it wasn't commited to the ports tree yet - I guess committers
are rather busy, so I didn't want to become too annoying.

That being said: Version 2.5.0 is outdated by now anyways. The new 3.x branch
got an overhaul of the configuration and scripting engine, so it is not a
drop-in upgrade.
Version 3.1 builds and runs successfully and I already began to update the
port, but I haven't come around to review and change the included config
files/templates and update the pkg-message to reflect all those changes.
TBH I also didn't put that very high on my priority list since this PR didn't
get any attention for almost half a year anyways...
If there is interest to commit this to ports, I'll update it to version 3.1 in
the next few days.


(In reply to Michael Osipov from comment #5)

Anubis is an (IMHO) annoying 'are you a robot' challange that gets in the way
of everyone and significantly slows down the browsing experience for humans. If
a browser is considered a bot, it is simply not served any content.

Iocaine specifically targets AI crawlers and sends them into an endless maze of
legitimately looking garbage. So it is basically a tarpit to poison AI models.
It can be either used with very crude and simple decision models, e.g. by
checking the http_user_agent string at the webserver/reverse proxy, or one can
leverage its scripting engine to make detection more advanced.
It *can* also use some form of proof-of-work or other captchas, but as the
documentation/description of the project also states: this form of decision
should always be the last resort as it is getting in the way of everyone and is
massively annoying to humans, especially if you are using a browser that by
default blocks cookies and javascript.

I personally use iocaine with the simple http_user_agent check (using the list
from www/github.com/ai-robots-txt/ai-robots.txt) at my webservers and reverse
proxies. Clients matching those strings can still retreive the robots.txt,
which contains a strict 'disallow' rule telling them to sod off - if they
ignore it and try to access anything else, they get redirected to iocaine and
are fed with gabrage.

From my logs it turns out they will happily eat several GB of crap until they
settle down and disappear completely after a while. Occasionally they reappear
but then cut out very quickly.
In raw numbers from my experience on several hosts: after ~2 months attacks
from the most aggressive crawlers like OpenAI/ChatGPT, meta and Google have
almost vanished completely. If they return, they usually stop after 2 requests
and at a "maze depth" of 1-2. Total traffic went down by >50% (up to 80% on
some hosts!) and I haven't seen a single occurence of a host being choked ever
since. Iocaine itself is extremely lightweigt, so apart from bandwidth (which
can be easily limited), it adds almost no cost and reduces load on backend
servers very significantly.

-- 
You are receiving this mail because:
You are the assignee for the bug.