Future of CTM

Peter Wemm peter at wemm.org
Mon Aug 31 23:08:33 UTC 2015

I'm sure you're all aware of the 'Do you need CTM?' thread.

I'm torn about how much to say in public, but there are a couple of problems.

1) the deltas are on ftp.freebsd.org alongside the rest of the content.  It 
is very, very heavily indexed by search engines.  I went looking for actual 
users who fetched it from any of the ftp.freebsd.org servers.  I could find 
a couple of actual legitimate xEmpty and catchup downloads over the last two 
years, but in general, only the search engines are downloading it.  The 
ratio of traffic from search engines to real users was somewhere in the 
order of 1000:1 to 10000:1.  The problem isn't so much the space, but rather 
the number of files.  Right now 50% of the *files* on the ftp server are CTM 
which consumes about half of the load the crawlers cause us.

2) ctm_rmail is unauthenticated.  If people are following the instructions 
in the docs, it is trivial for hostile 3rd parties to remotely destroy 
people using a mail feed.  If you're not using undocumented pgp checking, I 
(or any other person with a sense of mischief) could destroy your ctm pool.
*anyone* can do this and it is trivial.  I am willing to demonstrate using 
nothing that a non-freebsd.org person has access to if you don't believe me.

3) ctm has some "intersting" string handling.  It predates the attention 
that other tools that parse potentially hostile external data.  I would bet 
that there are exploitable buffer overruns in there.  I suspect that it has 
a variant of the bug that patch(1) had recently where you could trigger a 
direct shell escape via the internal use of ed(1).

4) the deltas are fed to ftp.freebsd.org via unauthenticated rsync - a 
hostile attacker can MITM that.  3rd party mirrors are unauthenticated and 
won't re-check files with matching timestamps, so an injection of a hostile 
delta won't be repaired if the size/timestamp match.

5) md5 can be brute forced with just minutes of cpu time these days.  A 
malicious delta could remain undetected unless there was an actual 
conflicting edit.

6) I looked at mailing list subscribers.  There are at least 6 people who 
receive actual deltas via email, although its more likely that there are 

Many of these problems cannot be fixed in a backwards compatible way.  At 
the very least, it needs:
* md5 replaced with sha256.
* an actual embedded crypto signature that can't be accidentally bypassed.
* the format changed so that new deltas can't be accidentally processed 
without checks by old ctm.
* an audit/refresh of the string handling.

What's the best way to handle this?  Fixing ctm in dozens of branches is 
unlikely to be practical.  I suggested that it should be moved a 
branch-agnostic project and made available via ports so that it can be made 
available to all branches.

I also want the deltas off ftp.freebsd.org as it causes half of the search 
engine crawler load.  I'm happy to have it hosted under another hostname 
where web index crawlers can be blocked, but *not* ftp.freebsd.org.

If we continue mailing deltas via ctm-*@freebsd.org before fixing the 
signature/spoofing issues, then at the very least they need to wrapped in 
pgp encoding to make sure that ctm_rmail cannot possibly decode them without 
passing them through a gpg/pgp check/unwrap first.

However, I'd like to have an alternative email arrangement for that too. 
Bots are very good at signing up for mailman mail lists - there's several 
hundred bots who have signed up for ctm-* - its very easy to spot them, they 
tend to send mail to gmail, in digest+html wrapping mode.  They also 
subscribe to both the regular and -fast lists at the same time.

So, who's going to fix ctm so it isn't suicidal to use it?  To repeat:
- md5 -> sha256 or better.
- rsa2048 bit signature or better from a published signing key.

I can change the mailing list stuff to enforce gpg ascii armor encoding or 
something like that in the meantime.

Peter Wemm - peter at wemm.org; peter at FreeBSD.org; peter at yahoo-inc.com; KI6FJV
UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246

More information about the ctm-users mailing list