Where is FreeBSD going?

[jsd at jdyson.com]

Wes Peters said:
[Charset iso-8859-1 unsupported, filtering to ASCII...]
> On Monday 05 January 2004 11:14 am, Brett Glass wrote:
> > I'd like to see a more open and inclusive form of governance for
> > FreeBSD. The current system of governance has, as its underlying
> > assumption, that the most prolific coders make the best leaders.
> > In my personal experience, this isn't a valid assumption. System
> > administrators and end users have a big stake in FreeBSD, and are
> > just as likely (perhaps more likely) to be good leaders for the
> > project.
> 
> The current system of governance is open and inclusive of those who have 
> demonstrated the talent, ability, and willingness to be contributors to 
> FreeBSD.  The current core team is made up of a mix of big-time coders 
> like Peter and Warner, and small-time coders like myself (now slightly 
> below middle of the pack on commits) and a variety of other skills. 
>
...
> 
> Somebody whose viewpoint doesn't extend beyond the virtual memory system, 
> for instance, may be critical to the success of a kernel, but that 
> doesn't necessarily make them the best person to steer a complex product 
> that brings 10,000 applications along with it.
>
It is INTERESTING to comment on someone whose viewpoint doesn't extend
beyond the VM system, because out of Greenman, me and even Matt Dillon,
(and the extremely respected alc), I don't know of any people
with a myopic VM viewpoint.  An example of that might be Matts ability
and succes dealing with the VERY IMPORTANT NFS issues, or perhaps my implementation
of the vfs_bio merged cache, minimal-copy pipe code, kernel memory management
improvements (which aren't really VM per se), early playing with the ATA
driver, SIGNIFICANT filesystem work (e.g. the vastly improved LFS didnt'
get installed because of softupdates making it redundant), careful rework of
certain portions of low level code, and it is definitely ludicrous
to claim that Greenman was VM myopic.

The biggest problem that I currently see on the technical side has
NOTHING to do with the individual competencies, but the SMP locking
complexity issues that I had predicted would happen.  By looking at the
locking from the VFS, VM, IPC and hardware standpoints (I admittedly
wasn't and STILL AM NOT competent on networking issues), it is
very very clear that restructuring the system to support more
coherent and orderly locking would make the system INFINITELY more
maintainable.  It might even be worthwhile to start a rearchitecting
now, recognizing that there were important things learned during the
current exercise.

The VFS and VM systems have numerous interdependencies, due to the very
desired specified coherency, desired modularity and natural control/data
flow.  EVEN THOUGH it is very possible to make superficial modifications to
the traditional structure in order to support adequate SMP locking, the
design will likely become unmaintainable for future improvements or
restructuring, the structure will be susceptable to bit rot.

The VFS, VM and scheduling mechanisms could have (with nominal effort)
been upgraded to use more of a realtime kernel structure (while retaining
the timesharing behavior when desirable.)  Using tsleep
or its derivatives for process blocking with control/data stack context
being intermingled with sundry data structure (and subsystem) locks
make for a design that will sustain a high priesthood for years.
(A wonderful side-effect of breaking the tsleep/stack marriage, is
that VFS layering can be much easier decoupled from the VM and VFS
interaction and coherency issues.)  This should also have positive
consequences WRT network stack state...

Perhaps a good first step would have been to progressively remove the
dependency upon stack context during thread/process blocking.  This
has several interesting positive side effects...  However, this definitely
breaks from the sleep/wakeup paradigm.

There are numerous ways to break the dependency on the stack context, and
I am partial to using continuations along with a few other possible
paradigms.  I haven't looked into these issues for years, but there might
be some schemes that are even more effective or architecturally 'clean.'

All this said, I still think that FreeBSD is the best choice for a general
purpose OS.  It is good that the system still works smoothly under load (something
that I tirelessly strived for), making sure that during heavy loading conditions
that system latencies (while waiting on various internal resources) are
minimized.  FreeBSD is proof that system caching doesnt' have to be continually
manually tuned for any normal configuration...

John