Upcoming Releases Schedule...

Sat Sep 20 21:01:46 UTC 2008

On 2008.09.19 21:30:11 -0700, Jo Rhett wrote:
> On Sep 19, 2008, at 8:57 PM, David N wrote:
> > How long are you expecting support for a RELENG to last, 1, 2, 3
> > years? 5 years? (comparison, Ubuntu LTS is 3 years, security updates)
> 
> 2 years for each supported branch would be excellent, although I'm  
> open to alternatives.  Right now 6.4 will EoL before 6.3 will :-(

Eh, where did you get that information?  AFAIK the EoL date of 6.4 has
not yet been decided (and I should know though of course I could have
missed it).  The EoL date is normally decided right around the release
time.  In the past it was done post-release but lately it has been
done before the release to include info in the release announcement.

If, when we get closer to the actual release, still is the plan for
6.4 to be the last RELENG_6 release I'm almost certain 6.4 will be a
Extended Support release.

On 2008.09.19 22:43:56 -0700, Jo Rhett wrote:
> On Sep 19, 2008, at 9:41 PM, Gary Palmer wrote:
> > On Fri, Sep 19, 2008 at 07:38:32PM -0700, Jo Rhett wrote:
> >> Without input from the current release team extending the support
> >> schedule is not possible.
> >
> > Inquiry - is release team the constraint?
> 
> I don't know.  I asked why not, and was told the release team said
> this was all they could do with the resources they have.  No further
> information has been provided.

Initially, the description is from my "world view".  It's entirely
possible I miss some parts, have forgotten, or remember incorrectly.

</disclaimer>

OK, before being able to say what is required for support, we need to
define "support".

Currently the entities which has a defined support for releases is the
FreeBSD Security Team (secteam) which supports the base system for
RELENG_* as defined out the security website [1], and the FreeBSD
Ports Management Team which has defined "support" for the FreeBSD
Ports Collection.

The security team will, for the defined periods, do it's utmost to
make sure security issues identifies in the supported branches are
fixed.  When it has been published how long a release is supported,
that period may at times be extended, but it's never shortened.  It
should be mentioned in this regard that after a release is out the
door and hasn't blown up requiring a point release to fix it (think
4.6.2 / 5.2.1) the security team owns the release branch, so to speak.

The Ports Collection has a slightly more vague definition of what is
"supported" [2], but it is still documented.  Ports normally "support"
the releases secteam does, but they could support less if they want
to.  For the Ports Collection there is also two parts to that
"support" (and that's the reason I write "support" with quotes).  One
part is the ports infrastructure (bsd.ports.mk and more) which portmgr
and others do a lot to make sure works on the supported releasess.
The other part is the ~19000 individual ports.  The individual ports,
to a degree, supports what the maintainer of that port has an interest
in supporting.  While there are of course rules and guidelines for
what a port must support, if a maintainer doesn't want to spend time
supporting it there aren't that much anyone else can do about it (if
somebody else cares enough they can submit patches, but e.g. for X000
broken ports that's a lot of work).

The Release Engineering team implicitly (as in that I don't think they
ever defined this per policy but it's what effectively being done)
support whatever secteam supports, wrt. for Errata Notices.  As the
security team "own" the release branches (RELENG_X_Y) secteam is also
involved at least partly in each Errata Notice which goes out.

Now, to define what the above support requires.

For the ports collection (if we ignore the part with individual ports
maintainers) requires quite a lot of time per release to support
(especially for older releases), as all infrastructure changes has to
be tested on all supported versions, and for each supported release
and architecture there need to be hardware to build packages.  I'm not
going to go more into this as I'm not involved with this so I don't
know the details on than that it's non-trivial both wrt. time and
hardware.

For what secteam support of the base system costs, it's mainly time
for the members of the security team which is the cost.  The more
branches, the more time is required.  This is not a linear cost and
has multiple parts to it:

- The more branches are supported, the more likely it is we need to
  deal with a security issue for third party software.  Both because
  software is added/removed in newer branches so we might only support
  a given program in old branches (e.g. GNU tar, GNU gzip, GNU cpio
  and more hare not in newer FreeBSD versions).  It's also possible
  that an older release will have a vulnerable version, but newer
  FreeBSD versions are not vulnerable due to newer version of the
  third party software.

  It also happens that an FreeBSD specific issue has silently /
  unknowingly to the security impact been fixed in newer FreeBSD
  versions, but that is very rare.

- The more branches are supported, the more versions of both third
  party code and FreeBSD code need to be supported and the more likely
  it is that the software differs meaning that we need to adopt the
  fix to the branch.  The real painful case for this was
  FreeBSD-SA-07:01.jail which AFAIR needed 6 different patches.  This
  is one of the largest time cost with support many branches as this
  is by no means a linear cost.  The older a branch is, the more
  likely it is that the code is much different than newer FreeBSD
  versions.

  This also the reason secteam was very happy when we could
  discontinue FreeBSD 4 support as it was significantly different from
  FreeBSD 5+.  In that respect supporting FreeBSD 5 in the end was
  much cheaper than supporting FreeBSD 4 in the end.  Of course this
  is less likely to be a problem in the future like it was with
  FreeBSD 4, but still - FreeBSD 5 and FreeBSD 8 are rather different
  and would not be fun to support both.

- Even if we don't need different patches for older releases we still
  need to look at them for each advisory and potentially do separate
  testing for each release.

- For some issues which are e.g. in the platform (as in i386, amd64,
  sparc64 etc.) dependent FreeBSD code or otherwise platform dependent
  code the complexity and time is a multiple of numbers of supported
  platforms and supported branches.  There aren't many issues of this
  type but they could happen.

- The actual advisory handling takes longer time the more releases is
  involved as the advisory need to contain info per release and
  patches need to be applied per branch both to the subversion tree
  and freebsd-update.  This is not a big part, but it still takes time
  when you have 8 supported branches like we did at one point.

There is also a cost in hardware for supported branches though this is
less of an issue.  Hardware is needed for testing / developing patches
and for freebsd-update builds.  Since we now have VMware and qemu
hardware for reference systems are a smaller problem now.  The time
freebsd-update builds takes is an issue, but not a big one as we could
"just" get more hardware if needed.

The more releases are supported, the more disk-space is also needed
for freebsd-update mirrors.  Again, far from an unsolvable problem by
any means, but also a factor - included for (slightly more)
completeness.

This is the "costs" I could come up with now - I'm sure there are more
which I have forgotten.

While I'm not going more into the general discussion of how long to
support branches, I will note that as rwatson has said - adding more
people to secteam is not as simple as it sounds (though we are in the
process of expanding right now).

Other is the thread have mentioned external people could support the
older branches.  This is partially correct, but only if the support is
distributed externally as well.  Even after a branch is not supported
anymore the FreeBSD Security Team holds a lock on the branch and only
in special cases allow commits to those branches.  The reason for this
is that some people might still just pull the latest version for
RELENG_X_Y for whatever reason and they should not "suddenly" get less
verified code.  Newer patches also wouldn't make it to freebsd-update
as that is managed by secteam.

We have had one case where a committer was interested in supporting an
older release and back-ported patches from security advisories for a
while.  The patches for the older releases were then reviewed in each
case by the security team before commit, but that only lasted for a
while and was a couple of years ago AFAIR.  In theory this could
happen again if the Security Officer at the time is OK with it - I
haven't talked with Colin about this in a while, so I can't recall is
position.  There would still need to be committer which is the
interface to secteam and do the commits.  Most issues (though of
course not all) which gets advisories are not public at the time of
the advisory, so a fix to older branches would be likely be delayed
some compared to initial disclosure.

I hope this makes it a bit clearer what the cost of supporting old
releases is (and even then I haven't gone much into the important part
of also having ports support).

[1] http://security.FreeBSD.org/
[2] http://www.freebsd.org/ports/

PS. I hope there aren't too many spelling and grammar errors in this
mail, but just reading it over one time was more than enough :-).

-- 
Simon L. Nielsen
Hat: FreeBSD Deputy Security Officer