Future of CTM
peter at wemm.org
Mon Aug 31 23:08:33 UTC 2015
I'm sure you're all aware of the 'Do you need CTM?' thread.
I'm torn about how much to say in public, but there are a couple of problems.
1) the deltas are on ftp.freebsd.org alongside the rest of the content. It
is very, very heavily indexed by search engines. I went looking for actual
users who fetched it from any of the ftp.freebsd.org servers. I could find
a couple of actual legitimate xEmpty and catchup downloads over the last two
years, but in general, only the search engines are downloading it. The
ratio of traffic from search engines to real users was somewhere in the
order of 1000:1 to 10000:1. The problem isn't so much the space, but rather
the number of files. Right now 50% of the *files* on the ftp server are CTM
which consumes about half of the load the crawlers cause us.
2) ctm_rmail is unauthenticated. If people are following the instructions
in the docs, it is trivial for hostile 3rd parties to remotely destroy
people using a mail feed. If you're not using undocumented pgp checking, I
(or any other person with a sense of mischief) could destroy your ctm pool.
*anyone* can do this and it is trivial. I am willing to demonstrate using
nothing that a non-freebsd.org person has access to if you don't believe me.
3) ctm has some "intersting" string handling. It predates the attention
that other tools that parse potentially hostile external data. I would bet
that there are exploitable buffer overruns in there. I suspect that it has
a variant of the bug that patch(1) had recently where you could trigger a
direct shell escape via the internal use of ed(1).
4) the deltas are fed to ftp.freebsd.org via unauthenticated rsync - a
hostile attacker can MITM that. 3rd party mirrors are unauthenticated and
won't re-check files with matching timestamps, so an injection of a hostile
delta won't be repaired if the size/timestamp match.
5) md5 can be brute forced with just minutes of cpu time these days. A
malicious delta could remain undetected unless there was an actual
6) I looked at mailing list subscribers. There are at least 6 people who
receive actual deltas via email, although its more likely that there are
Many of these problems cannot be fixed in a backwards compatible way. At
the very least, it needs:
* md5 replaced with sha256.
* an actual embedded crypto signature that can't be accidentally bypassed.
* the format changed so that new deltas can't be accidentally processed
without checks by old ctm.
* an audit/refresh of the string handling.
What's the best way to handle this? Fixing ctm in dozens of branches is
unlikely to be practical. I suggested that it should be moved a
branch-agnostic project and made available via ports so that it can be made
available to all branches.
I also want the deltas off ftp.freebsd.org as it causes half of the search
engine crawler load. I'm happy to have it hosted under another hostname
where web index crawlers can be blocked, but *not* ftp.freebsd.org.
If we continue mailing deltas via firstname.lastname@example.org before fixing the
signature/spoofing issues, then at the very least they need to wrapped in
pgp encoding to make sure that ctm_rmail cannot possibly decode them without
passing them through a gpg/pgp check/unwrap first.
However, I'd like to have an alternative email arrangement for that too.
Bots are very good at signing up for mailman mail lists - there's several
hundred bots who have signed up for ctm-* - its very easy to spot them, they
tend to send mail to gmail, in digest+html wrapping mode. They also
subscribe to both the regular and -fast lists at the same time.
So, who's going to fix ctm so it isn't suicidal to use it? To repeat:
- md5 -> sha256 or better.
- rsa2048 bit signature or better from a published signing key.
I can change the mailing list stuff to enforce gpg ascii armor encoding or
something like that in the meantime.
Peter Wemm - peter at wemm.org; peter at FreeBSD.org; peter at yahoo-inc.com; KI6FJV
UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246
More information about the ctm-users