Upcoming Releases Schedule...

[Robert Watson]

On Fri, 19 Sep 2008, Jo Rhett wrote:

> Look, I understand what you're saying here.  And I don't discredit or 
> disagree that it shouldn't be handled this way.  But what you have addressed 
> is a stepwise integration policy of a developer, and does not address how to 
> get a business to commit those resources.
>
> Why?  Because you are asking for the business to commit the resources 
> without a goal.  No, I'm not saying that FreeBSD has to guarantee anything. 
> We both know that guarantees on open source projects aren't worth much.

This whole discussion is about open source guarantees -- after some extensive 
discussions and careful thinking about available resources, the FreeBSD 
Project declared a set of guarantees about what FreeBSD versions we would 
continue to support over time.  We tried to be realistic about the level of 
staffing available, the nature of the vulnerabilities that would arise, and 
the degree to which conservative highly incremental changes could be used to 
support a branch long after release.  The problem is that the 18 months for 
all releases + extra time for extended support releases is still a short 
period of time.  The tension here is between making promises we can definitely 
keep and starting to make promises that we can't.  We'd like to err on the 
former, rather than latter, side.

> What I'm saying is that your scoped outline has no goal.  Nothing to even 
> reach for.  Except maybe perhaps a commit bit for me.  If I had wanted a 
> commit bit, I'm sure I would have one by now.  I'm not particularly worried 
> about that.  I don't even really care to have one (though I would if that 
> was necessary).
>
> But that's the highest goal of your outlined scope.  A commit bit.

You already identified the end goal: extend support lifetimes.  You placed 
constraint on how that could be accomplished: you were going to do the work. 
What I've done is identified our normal model for getting from the current 
starting point (a guy on a mailing list who, while enthusiastic, hasn't shown 
us the beef) to the proposed outcome (a guy on the security-officer team, 
commit access required to participate in the support process, etc).  Here are 
the subgoals I broke it out into:

- To really contribute to the discussion about support lifetimes and
   contribute during non-disclosure periods, you need to be on the security
   officer team and a FreeBSD committer.  The former to you have access to the
   required information and discussion, the latter so that you can act on it.

- To be on the security officer team, you need to be a FreeBSD committer in
   good standing who has shown both the technical acumen and long-term
   commitment to the project required to work effectively on that team.
   Because sensitive information is involved, and because we give the security
   officer team special privileges in the project to accomplish its task, we
   select proposed members very carefully.

- To be a FreeBSD committer, you need to have show a signficant long-term
   commitment to the project, the ability to identify and work with a mentor,
   and build the confidence of the core team that you are a candidate in whom
   we can place significant trust (direct write access to the source code of a
   system deploy on literally millions of devices).

- To show a significant long-term commitment to the project, you need to
   identify past work and context for your potential future contributions and
   find a potential mentor (committer) with whom you have an existing
   professional relationship.  Common examples are things like "has worked with
   you over an extended period to get your work merged," or "someone who has
   been watching your work over time and concluded that they are in a good
   position to go through the mentoring process with you."

> If you've seen the appropriate Southpark episode: "Step 3: Profit!"  "Dude, 
> what's step 2?"

Let's make sure we understand each other clearly: the reason you're getting 
replies using words like "demand" is that it is easy to read your e-mails in 
exactly the above light.  It is not a stretch to interpret them as reading, 
"Put me on the security officer team without having gone through any of the 
normal processes used to vet a new member".  We can talk about changing the 
process, but I think you can't contribute to that conversation constructively 
without understanding the process.  Simply demanding change (and that's how it 
reads) shows a lack of respect for how we got to where we are, and puts people 
people on the defensive.  Or offensive, in some cases. :-)

> There's *nothing* wrong with what you have said.  What you have said makes 
> perfect sense from an integration perspective.  But I don't think it 
> addresses the issues at hand -- businesses need to have explicit goals and 
> at least a haphazard guess at the requirements to reach those goals before 
> they'll commit resources.

If you approach a software company and say "Look, we like your product, but it 
would really help us if you supported each minor release for 24 instead of 18 
months, and the way you're going to do this is put our employee on your 
security response team", I think you'd see a lot of raised eyebrows there as 
well.  To convince the company to do this, unless you're really shoving a 
*lot* of money at them, you'll need to build credibility as a consumer, work 
with them to identify appropriate resources, and go through their vetting 
process.

When it comes to commercial OS products, like Windows and Mac OS X, there is 
often a strict requirement to live on the most recent minor release in order 
to continue to receive support.  For example, you won't make a lot of headway 
turning up at Apple and demanding security updates for Mac OS X 10.5.0 a year 
after it has been released.  The answer will be "Great, update 10.5.3" (or 
something along those lines) -- not only will it fix the security issues, but 
it will support the hardware we now sell.  In that sense, we're actually quite 
different: rather than saying "Sorry, 6.2 is vulnerable, please upgrade to 
6.3", we say "You can live on 6.2 for up to 18 months and receive *only* 
security and critical errata patches".

Don't get me wrong: I would love to see us support all releases for 24 months 
(or even more) after they ship.  I think our users would appreciate that also. 
But it's not as simple as turning up and saying "My company wants me to do 
it".  Our community can change over time, but we do this slowly and carefully 
because we want to maintain the properties that produced the community that 
attracted its current members.  That is to say: a community of developers in 
which a high level of trust can be placed implicitly in its members because 
they have gone through a significant vetting process that not only considers 
their technical background, but also their ability to work within the 
community socially, and their commitment to the project they are joining.

There are some edge cases in the model that we might appeal to -- for example, 
in the presence of a highly competent mentor, we might allow the sponsoring 
for a commit bit of an employee of a company who will only work in a narrow 
area affecting their product, and see more than the usual supervision.  This 
is sometimes the case for committers from hardware companies so that they can 
maintain device drivers they have been writing and submitting for some time, 
in which case the proposal can bank on the credibility of the company's 
contributions despite a weakness in the credibility of the individual.  This 
requires a highly engaged mentor and some very careful thinking about scope.

> I don't see these problems as being in conflict.
>
> In fact, I would personally suggest that most of the resources you need to 
> improve your releases don't need commit bits.  I personally have no 
> objection at all to running all patches through another set (or two, or 
> three) of eyeballs.  It's a damn good practice ;-)  It's unlikely to slow me 
> down one bit.
>
> ^^^ you may know more about the resource limitations and why commit bits are 
> necessary to relieve your resource strain.  In that situation, please 
> educate me.  Or everyone.  In particular, I'll be happy to buy you 
> coffee/beer/poison of choice at your leisure if that would make this easier.

Commit bits serve two functions:

- They are credential based on a reasonable vetting process, used both as an
   authenticator and for the purposes of authorization and policy.

- They are tool required to do work -- commit to appropriate branches, etc.

I think it's the former category in which it plays the strongest role here: 
reviewing patches can be done without a commit bit, as you say, but the commit 
bit reflects a level of trust in a contributor and is a good tool for this. 
The strategy I laid out doesn't have to culminate in a commit bit, but it does 
have to culiminate in something functionally congruent: you need to build your 
credibility as a technical contributor to the project before we hook you up to 
confidential vulnerability mailing lists, give you early access to patches, 
and so on.  Whether you build credibility in the most direct way for this 
"job" by essentially doing it independently until we break down and say 
"despite being a bit pushy ont he mailing lists, he's really done this well 
and would make a good candidate to join secteam so that we can make this 
unofficial thing official".  (See also: freebsd-update)

The "tool" element isn't unimportant either: preparing patches for commit 
benefits significantly from doing it in the context of revision control -- we 
expect members of the security team to take ownership of vulnerabilities, 
driving them through analysis, correction, QA, merge, and writing and release 
of the security advisory.  This allows individual issues to move independently 
through the process, and avoids the synchronization issues associated with 
pipelined handling (i.e., lots of explicit handoffs through stages owned by 
different members).  It is fundamentally collaborative work, but experience 
suggests that having owners for issues makes the system function better.

Robert N M Watson
Computer Laboratory
University of Cambridge