Future of CTM

Tue Sep 1 23:25:45 UTC 2015

Peter Wemm wrote Mon, 31 Aug 2015 19:19:28 -0700

>  Content-Transfer-Encoding: quoted-printable
>  Content-Type: text/plain; charset="us-ascii"

I had to manually strip back to Ascii, Hence mangling of = below.

> On Tuesday, September 01, 2015 02:19:36 AM Julian H. Stacey wrote:
> > Peter Wemm wrote:
> > > I'm torn about how much to say in public, but there are a couple of=
> 
> > > problems.
> > ...
> > Thanks for the analysis Peter.
> >=20
> > Before we go deeper, might there by chance be a frustrated SOC
> > student whose project fizzled out & who might grasp CTM as a
> > replacement/ top up project ?  Or students coming to end of summer
> > project thinking "That was fun! What next to hack ?"
> >   http://lists.freebsd.org/pipermail/soc-status/2015-August/date.html=
> 
> > Perhaps not very likely but might be worth checking, as your post
> > nicely describes the remit.
> 
> I have been trying to find an example of somebody who is actually verif=
> ying=20
> signatures before piping the messages to to ctm_rmail.  Even the procma=
> ilrc=20
> files that you publish at http://www.berklix.com/jhs/txt/ctms.html don'=
> t do=20
> signature checking.   From your own pages:
> # JJLATER add a check for pgp signature, ref.
> #       http://www.freebsd.org/handbook/synching.html#CTM

Yes, it's been on my infinitely long To Do list a Long time. Blush ! ;-)

> I did find one person who gpg verified the files he downloaded from ftp=
>  and=20
> posted about a corrupted file:
> https://lists.freebsd.org/pipermail/ctm-users/2012-December/000376.html=
> 
> but even then it was a check to see if it was signed by *somebody*, rat=
> her=20
> than signed by the pgp key listed on the mailman info pages.   Even the=
> n, I'd=20
> bet he only did the gpg check as a diagnostic after the ctm run failed
> 
> I actually went looking for sample scripts for how to do this all safel=
> y and=20
> there was nothing obvious that turned up in likely searches.
> 
> There's some hints about how to do specific key verification here:=20
> http://stackoverflow.com/a/19016152
> but note the caveat about it needing to be a pubkey.gpg, not pubkey.asc=
> ..
> 
> I'd wager that few people (if anybody) are actually doing proper signin=
> g key=20

I wouldnt bet against that :-)

> verification of the email feed, and are therefore completely vulnerable=
>  to=20
> mischief.  Relying on the ctm-*@freebsd.org email list protection is *n=
> ot*=20
> sufficient for this, but I would rather not talk about specifics just y=
> et.
> 
> My biggest concern is that there is a vast quantity of published docume=
> ntation=20
> advising people to do dangerous things, with the "oh by the way, and yo=
> u=20
> probably should protect youself" aspect left as an exercise for the rea=
> der as=20
> an afterthought.

Yes. Keys were added later after CTM got going far as I recall.
Stephen added them on transmitter side.
I never got round to checking mine. Maybe nobody did.

CTM has been rather an under cared for orphan, its a nice tool with
some advantages,(push rather than pull technology) (but as you note,
some problems too). Though thanks to Stephen for keeping it runnning !

Getting the handbook changed to better document CTM was difficult,
I seem to recall giving up long ago, I just published my own stuff
since, nice you found 
	http://ctm.berklix.org	 Or
	http://www.berklix.com/~jhs/txt/ctms.html
ideally it would all merged to freebsd.org, along with Stephens latest scripts.
Problem:
  Nobody `owns' freebsd.org handbook ctm pages, so I 'spose the doc
  team don't know who is authoritative. I recall I've submitted
  stuff presumably with send-pr way back, but I don't now recall.
  Long ago I sent (unsolicited by Stephen) request to doc@ to appoint
  & accept Stephen as authoritative for the CTM handbook section,
  as he drives the deltas.  I'm pretty sure I would have also told
  doc@ that as phk@ was original author, if they have any doubts
  accepting Stephen as authoritative, just ask phk@ to confirm.

  If commiters want a backup or addition to Stephen to authorise changes to
  http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/ctm.html doc@
  might do worse than me, as I provide backup on http://ctm.freebsd.org
  & been using CTM ages, or they could appoint phk@ as original
  owner, or you, whoever has a stake in & knowledge of CTM.

> We can't retroactively recall all the bad advice so the only real optio=
> n is to=20
> break the old dangerous ways and give corrected instructions on how to =
> do it=20
> safely.  Make it so that you *need* the script that verifies signatures=
>  before=20
> decoding it and sending the delta to ctm_rmail.  It should be a choice =
> to opt-
> out of being safe, not something you have to research and implement you=
> rself=20
> to opt-in.

OK, but best we first appoint someone or group to own
	http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/ctm.html
	&
	/usr/src/usr.sbin/*ctm*
To be authoritative for commiters to accept & commit diffs to ensure
manuals & handbook are consistent with existing code & Stephen's
latest config etc.  Then write more code. Then maybe timetable a warning
a change of defaults to use key signing by default.

> That's what lead to my current thinking.  Would this effort be well spe=
> nt?  I'm=20
> not convinced that it is, but I wouldn't stop somebody from doing the r=
> efresh=20
> work.

I could do it, I'd prefer someone else did it, I'm not clear the current users
need enhanced security, or they would have asked for it, self inclded.

> I'm wondering whether to ask Stephen to switch away from detached signa=
> tures=20
> to help force the issue.

The whole thing runs on a shoe string, wobbling the
string would be bad, consolidation of manuals & examples is needed
first, then code update with optional inbuilt sig, then a timetable
to warn when/ if it will be used by default.

Don't worry about other people's security though, if they don't
want it! Just warn them is sufficient.  Security V. convenience is
users decision, not yours, mine, or ours on ctm-users@ or @freebsd.org

> ie: replace the "ctm-*.nnnn.xz" + "ctm-
> *.nnnn.xz.sig" files with "ctm-*.nnnn.xz.gpg" so that gpg is needed to =
> decode=20
> it and at least have the signature status presented right there at deco=
> de=20
> time.  Likewise for the email deltas, sign and encode the deltas rather=
>  than=20
> clearsign - that forces it to be run through gpg in front of ctm_rmail.=
>   A=20
> script to check that its signed by the *right* keys would need to be wr=
> itten=20
> and published for that to be worth anything though.  (Processing a ctm =
> email=20
> packet with a valid signature by evilguy at terrorist.org is no safer than=
> =20
> accepting unsigned things)

You'r presumably right, but every time I read encryption stuff, I need to 
read more manuals :-)

> However, at the very least, I still want to move the ctm files from=20
> ftp://ftp.freebsd.org to something like ftp://ctm.freebsd.org because o=
> f the=20
> crawler issue.

Have you tried, & are http crawlers ignoring 
	<html>
	  <head>
	    <meta name="ROBOTS" content="NOFOLLOW">
	  </head>
	  <body>
robots.txt
http://ftp.gnu.org/old-gnu/Manuals/wget-1.8.1/html_node/wget_41.html

When deltas by mail fail, I ftp ftp2.de.freebsd.org 
Will moving it mean all ftp.freebsd.org mirrors no longer carry it ?
I guess there's few people ftp'ing so we dont need many mirrors.
Need it somewhere though, preferably with a mirror
Stephen recently wrote he regularly mirrors to
ftp://ctm.berklix.org but I'm not sure what & where to, I dont see much there
& I suspect what's there is partly my old manually placed stuff.

Cheers,
Julian
--
Julian Stacey, BSD Linux Unix C Sys Eng Consultant Munich http://berklix.com
 Reply after previous text, like a play - Not before, which looses context.
 Indent previous text with "> "         Insert new lines before 80 chars.
 Send plain text, Not quoted-printable, Not HTML, Not ms.doc, Not base64.
 Subsidise contraception V. Global warming, pollution, famine, migration.