MITM attacks against portsnap and freebsd-update

[Matthew Rezny]

On Friday 11 April 2014 14:20:18 Anton Afanasyev wrote:
> On Fri, Apr 11, 2014 at 11:04 AM, Matthew Rezny <matthew at reztek.cz> wrote:
> > The biggest effort would be adding rsync to base, but being that we have
> > svn(lite) in base it should not be a big deal to add rsync.
> 
> I may be too naive and/or just not understand things as well as those who
> do move code into base, so excuse my ignorance, but why was svnlite moved
> into base, and why even consider moving rsync into base?
> Sure, it is nice if the base includes everything needed to allow
> development of it; it is also a must to be able to update and build your
> ports. But why include tools that do this, rather than a bootstrap for
> installing those tools?

There is a strong feeling that the base OS should be 'complete', which 
includes the ability to update and rebuild the entire base OS (world). It was 
not always possible to fetch updates with just base components, but since csup 
had been in the base for some years, many users had come to expect that 
ability. When CVS was retired, this expectation generated lengthy discussion 
on the MLs. Before the inclusion of svnlite in 10, someone put a good deal of 
effort into a svnup tool that worked more similarly to csup. While I applaud 
its author for the effort, I found it to be unreliable; it crashed several 
times before getting a full checkout, then could never update the tree without 
crashing, and of course it still had the same issue of clobbering local 
changes just as c(v)sup had done. Shortly after I setup a local svn mirror 
with the intent to use rsync for local distribution within my LAN, I got wind 
of svnlite coming in 10 and so I jumped straight to that. Checkout from local 
mirror is MUCH faster than from the official repos (with so many small files, the 
latency across the WAN results in more time spent waiting between files than 
actually transferring file contents), so it was still a worthwhile exercise.

> For developing and updating base, why not include a script that fetches a
> (sufficiently fresh) snapshot of the ports tree and let the user decide
> whether they want to use svn or any other port to update their sources? If
> it is deemed too large a download (a valid concern) - download only svn and
> its dependencies, possibly even to a ports tree rooted in a location
> different from /usr/ports, and build svn from that.
> For keeping ports up to date, why not include a script that fetches a
> (sufficiently fresh) copy of the ports tree and tell the user that the
> preferred method to update is rsync; heck, create a port that uses rsync to
> do what Matthew described above, and /offer/ to install it for the the user
> from the tree that was just downloaded.
> 
> Something along the lines of the above would completely remove the need to
> keep unrelated code in base - and the need to keep it updated - , while
> still allowing the end user to keep base and ports up to date.
> 
> 
> Anton

You have made a most excellent suggestion that had not crossed my mind. Due to 
the aforementioned discussions, I had it stuck in my head that the tools must 
be in base. However, pkg(ng) has set precedence with the bootstrap in base 
which installs the pkg package. Since I always install a ports tree and update 
it before any ports, and I don't use binary packages (yet, local poudriere is 
still on my TODO list), I have been getting pkg via the install of portmaster 
(the first port I install) and thus have never used the bootstrapper.

The bootstrap idea is great as it allows more flexibility and rapid development 
(which is exactly the reason pkg is done the way it is). Svnlite will still be 
around for those that must use only tools in base. Since this is a proposed 
portsnap replacement, that would also be capable of doing the base src as a 
bonus, there is really no reason it can't be a tool that lives in ports with a 
handy bootstrap in base. The bootstrap could simply be a sh script which uses 
fetch to fetch the tar.xz files via ftp/http(s), verify hashes, decompress into 
an appropriate staging area (/var/portsync), verify hashes of the tar files, 
then extract to /usr/src and/or /usr/ports. There could then be a port, say 
port-mgmt/portsync, which implements the update functionality. Installation of 
this port could pull in rsync as a dep. Since it is in ports, the full 
portsync could be written in Python (that would be my choice as my sh-fu is 
weak) without waiting for Python to maybe if ever make it's way into base. The 
portsync.sh in base could automatically invoke portsync.py if present.

I think I just added one more thing to my lengthy TODO list... DOH!

(from later in the thread)

> >  I know .. Git-lovers are upset..
> 
> Was this addressed to me, or just a general rant/reference that I did not
> understand?

I don't think that was directed at anyone in particular, but there were 
numerous requests to forget SVN and use git, apparently not realizing that SVN 
had been used for src for years and CVS was merely a (hackish) export from 
there. Git's decentralized nature does not fit well with the project's needs. 
It just does not work well for maintaining a single, central authoritative 
repo that is the one truth. Those who wish to use git for their local hacking 
can do so as git allows checkout from SVN to initialize a local repo, and they 
can generate diffs to submit back to our central repo. Also, it has been my 
experience on projects with many fewer developers, that not only is git a 
nightmare of merge races, but its inability to track changes on a per file 
basis within a changeset puts it on par with CVS (if not worse, in that 
sometimes its diff output outright lies about the progeny of a change).