FreeBSD Kernel Internals Documentation

Polytropon freebsd at
Fri Dec 30 06:40:02 UTC 2011

On Thu, 29 Dec 2011 22:43:16 -0500, David Jackson wrote:
> However, My finding is that due to poor documentation, [...]

That kind of statement doesn't fit well to FreeBSD
which is known for its excellent documentation, often
considered superior to other open source projects.

> [...] the FreeBSD kernel
> is nearly impenetrable to an outsider. I have been able to find no
> comprehensive documentation of kernel internals. I have found it nearly
> impossible, due to lack of comprehensive documentation, much of any of the
> kernel internals. What I see is an internal cliche of developers who are
> aware of its myraid of undocumented esoteric secrets, and very little to
> actually help anyone else to understand it.

The source code itself is very verbose and well documented.
Unlike other operating systems, FreeBSD provides man pages
that can be accessed offline and provide information about
many kernel aspects, such as interfaces and device drivers.

Searching ""
or something of that kind will also bring further information
and and resources.

> Any good, well designed software projects will have comprehensive
> documentation of the source code, this includes code comments, information
> on what every piece of code does, how the entire system fits together, and
> descriptions of every variable and function. Any well run project would
> insist that code contributors upload full and comprehensive documentation
> of how their source code is written, how it works, etc.

I think this is what you'll find in the FreeBSD kernel
sources. I had a random peek at some files in the
/usr/src/sys subtree (where the kernel sources reside)
and found that it fits your claim.

> Documentation is vital and good practice because it saves time, it prevents
> people new to the project having to waste immense amounts of time trying to
> figure out a vast and cryptic puzzle. Without good documentation software
> can be nearly useless, unmaintainable and difficult for an outsider to
> learn, to the point where it may actually take less time to just throw it
> out and start from scratch.

I couldn't agree more. As a developer, I've worked with
many operating systems, and finally I found FreeBSD which
provides the documentation in a way I feel comfortable
with, unlike its predecessors in my "IT career". :-)

However, there are differences in how you judge documentation
to be _good_. Talk to a mainframer, and he will tell you a
different story. Then talk to a "Windows" person and explain
what documentation is, and he'll tell you that you don't
need it. :-)

> These are reasons that FreeBSD needs better documentation, documentation of
> how the entire system fits together, what lines of code do, the purpose of
> variables and functions, etc, in descriptive English. This is key to
> developing maintainable software.

As I said, that _is_ what you already find in the source
tree. Allow me to give you one example (out of many):

File randomly picked: /usr/src/sys/nfs/nfs_common.c

 * Help break down an mbuf chain by setting the first siz bytes contiguous
 * pointed to by returned val.
 * This is used by the macros nfsm_dissect for tough
 * cases. (The macros use the vars. dpos and dpos2)
void *
nfsm_disct(struct mbuf **mdp, caddr_t *dposp, int siz, int left, int how)

and so on. Descriptive english text, usable variable
and type names.

Or how about /usr/sys/boot/i386/btx/btxldr/btxldr.S

 * BTX program loader for ELF clients.
start:          cld                             # String ops inc
                testl $OPT_SET(RBX_MUTE), 4(%esp) # Check first argument
                setnz muted                     #  for RBX_MUTE, set flag
                movl $m_logo,%esi               # Identify
                call putstr                     #  ourselves
                movzwl BDA_MEM,%eax             # Get base memory
                shll $0xa,%eax                  #  in bytes
                movl %eax,%ebp                  # Base of user stack
                movl $m_mem,%esi                # Display
                call hexout                     #  amount of
                call putstr                     #  base memory
                lgdt gdtdesc                    # Load new GDT

If that is insufficient, maybe you can give an
example for comparison of how you imagine the
documentation should be provided.

> I saw where someone automatically generated "documentation" with Doxygen.
> This is nearly useless, because all it shows is a huge list of functions
> and variables but does not include any text on what they do. At best,
> Doxygen can only provide a template for documentation that can be filled in
> with descriptive English information on what everything does.

If properly used Doxygen can help, but it's not the
entire solution, I fully agree. Some people need to
learn the hard way, just like me when I visited my
_own_ source code a decade later and found something
like kk[w.q].x = fbx(x,x0+a0+1+aa[q].t,dt,h,dh-1+3);
which is totally ugly. :-)

> One idea might be to have an official wiki that contains the template
> generated by Doxygen which can then be filled in. When changes to the
> source code is made, it is good practice for the commiter of such changes
> to document their code as it is submitted.

Also Wikis aren't for everything. I don't know how
_you_ as a developer think about it, but _I_ like
to have the documentation available _in_ the code
I consult, offline, without the need to use a web
browser, everything based on _files_ so I can use
the system's powerful tools to work with them (e.
g. search for information, filter, create lists
and so on).

FreeBSD has an official Wiki, and there are several
others. Again a question arises: Would you like the
documentation to be spread across arbitrary Wikis,
user home pages, web forums and blogs?

> This allows others who come along who need to maintain the code to more
> easily understand what the code does.

History teaches that this is achieved by using the
code itself. :-)

> Another idea which would also improve the useability of FreeBSD would be to
> have a wiki which would be updated by kernel contributors whenever they add
> support for a certain piece of hardware. This would make finding hardware
> compatability information easier from one central, up to date and current
> source of information.

Those informations will typically be added to the UPDATING
information, as well as to the release notes of an upcoming
RELEASE. Deltas in the src/ subtree also indicate when
something new is present. As FreeBSD is a _quality_ operating
system, developers pay attention to write the documentation
along with releasing new drivers, so you aren't stuck with
an ugly-hacked kernel module full of lines that look like
the fbx() example I gave above. :-)

> These documentaiton ideas, for commiters to document their code when they
> upload it, and document their hardware support additions, are just good
> software practices that should be highly recommended and encouraged

I fully agree with that statement. If you could, for example,
point to documentation you judge _good_, or point at some
FreeBSD documentation that you think needs heavy improvement,
it would help to back your argumentation. Otherwise, it can
be seen as "already done".

Magdeburg, Germany
Happy FreeBSD user since 4.0
Andra moi ennepe, Mousa, ...

More information about the freebsd-questions mailing list