[FreeBSD-Announce] January-February 2004 Status Report

Scott Long scottl at freebsd.org
Wed Mar 17 10:00:44 PST 2004

January-February 2004 Status Report


   2004 started with another exciting two months for the project. FreeBSD
   5.2 was released in early January and then quickly followed in
   February with the 5.2.1 bug-fix release. Looking forward, we are
   expecting a late-April release date for FreeBSD 4.10, and mid-summer
   date for FreeBSD 5.3. And don't forget to support the FreeBSD vendors
   and developers by buying a copy of the latest CD or DVD sets.


   Scott Long

     * Bluetooth stack for FreeBSD (Netgraph implementation) 
     * Automatic sizing of TCP send buffers
     * Compile FreeBSD with Intels C compiler (icc)
     * Disk and device I/O
     * FreeBSD GNOME Project Report
     * FreeBSD Package Grid
     * FreeBSD ports monitoring system
     * FreeBSD/arm Status Report
     * FreeBSD/ia64
     * FreeSBIE
     * kgi4BSD
     * libarchive/bsdtar
     * Move ARP out of routing table
     * NanoBSD
     * Network interface naming changes
     * Network Stack Locking
     * Porting OpenBSD's pf
     * PowerPC Port
     * SGI XFS port for FreeBSD
     * Testbed for testing and qualification of TCP performance
     * The FreeBSD Dutch Documentation Project.
     * The FreeBSD Simplified Chinese Project
     * Verify source reachability option for ipfw2
     * vinum + GEOM
     * Weekly cvs-src summaries

Bluetooth stack for FreeBSD (Netgraph implementation)

   Contact: Maksim Yevmenkin <m_evmenkin at yahoo.com>

   Not much to report. Bluetooth Service Discovery Procotol daemon sdpd
   was integrated with existing Bluetooth utilities. From now on users
   should not use GNU sdpd (Linux BlueZ port).

   Bluetooth HID profile implementation is almost complete. Thanks to
   Matt Peterson < matt at peterson dot org > for giving me Bluetooth
   keyboard and mouse for development.

Automatic sizing of TCP send buffers

   Contact: Andre Oppermann <andre at FreeBSD.org>

   The current TCP send and receive buffers are static and set to a
   conservative value to preserve kernel memory. This is sub-optimal for
   connections with a high bandwidth*delay product because the size of
   the TCP send buffer determines how big the send window can get. For
   high bandwidth trans-continental links this seriously limits the
   maximum transfer speed per TCP connection. For example a 170ms RTT and
   a 32kB send buffer limit the speed to approximately 1.5Mbit per second
   even thought you might have a 10Mbit pipe.

   This project makes the TCP send buffer to automatically adapt to the
   optimal buffer size for maximal link usage. In the case above this
   would be a buffer of approximately 220kB. The main challenge is to
   have a stable and reliable measurement of the link parameters and
   manage the kernel memory properly and in a fair way. We don't want to
   have a few connections to monopolize all available socket buffer space
   and many edge cases have to be considered. The first implementation
   will be tuned conservatively but even that will provide significantly
   better performance than the static buffers currently. Work on this
   project is already in progress.

Compile FreeBSD with Intels C compiler (icc)

   URL: http://www.Leidinger.net/FreeBSD/

   Contact: Alexander Leidinger <netchild at FreeBSD.org>

   If nothing bad happened, the icc patches got committed around the date
   of the deadline for submissions of this report. Please search the
   archives of -current and/or cvs-all for more information.

   The next steps in this project are to
     * fix the kernel to also run without problems when compiled with icc
     * fix the kernel if some problems surface after more people give it
       a try
     * get some ports to compile with icc

Disk and device I/O

   Contact: Poul-Henning Kamp <phk at FreeBSD.org>

   In the overall area of disk and device I/O, a significant milestone
   was reached with the implementation of proper reference counting on
   dev_t. We are now able to properly allocate and free dev_t. Cloning
   device drivers also had the job made easier for them with the addition
   of the unit number management routines.

   It is not quite decided which will be the next step in the quest for a
   truly SMPng I/O subsystem, but a leading candidate is to implement the
   device-access vnode bypass to get more concurrency in the system:
   Instead of taking the tour through the vnodes for each i/o operation
   on a device we will go directly from the file descriptor layer to
   DEVFS/SPECFS. In addition to Giant-less disk I/O, this should enable
   us to pull the entire tty subsystem and the PTY driver out from under
   Giant and we expect that to improve the "snappiness" of the system

FreeBSD GNOME Project Report

   URL: http://www.FreeBSD.org/gnome/

   Contact: FreeBSD GNOME Team <gnome at FreeBSD.org>

   It has been a year since our last status report, but we haven't slowed
   down. Since the last report, Alexander Nedotsukov (bland) and Pav
   Lucistnik (pav) have joined the FreeBSD GNOME team. GNOME 2.4 was
   released back in September 2003, followed by 2.4.1 and 2.4.2. We are
   actively working on getting GNOME 2.6.0 out the door at the end of
   March. GNOME 2.6 Beta releases can be obtained via the project URL

   To help make GNOME 2.6.0 our best release to date, we have created a
   script to automate the upgrade from GNOME 2.4. We also have a new
   GNOME package build server that builds and serves i386 packages for
   all supported FreeBSD releases. We plan on having the GNOME 2.6.0
   packages available the moment 2.6.0 hits the ports tree.

   Included in the release of GNOME 2.6 is GTK+ 2.4, the next installment
   in the GTK+ 2 series. Because GTK+ 2 has become very stable over the
   past few years, the FreeBSD GNOME Team is pushing for GTK+ 2 support
   to be included by default in all applications that support it. This
   has already been done with Mozilla, Firefox, and Thunderbird. A
   complete GNOME Desktop and application environment can already be
   built using only GTK+ 2. The ultimate goal is to phase GTK+ 1 out of
   the ports tree.

FreeBSD Package Grid

   Contact: Kris Kennaway <kris at FreeBSD.org>

   Distributed package builds are currently done using a set of
   home-grown shell scripts for managing, scheduling and dispatching of
   package builds on the client machines. This has been sufficient for
   our needs in the past, but has a number of significant shortcomings
   that limit future growth. I am rewriting the package build scripts to
   work on top of Sun GridEngine (ports/sysutils/sge), as a client
   application of a "FreeBSD package grid". Some of the design goals for
   the new system are:
     * Better robustness against machine failure, and more efficient
       scheduling of build jobs
     * Support for remote build machines, to make better use of machine
       resources and clusters that are not on the same LAN as the build
     * Ability for other committers to submit port build jobs to the
       system, for testing of changes, new ports, etc.

FreeBSD ports monitoring system

   URL: http://portsmon.firepipe.net/index.html

   Contact: Mark Linimon <linimon_at_lonesome_dot_com>

   Thanks to the loan of a box by Will Andrews, the system has been moved
   into production. The previous installation at lonesome.com now refers
   you to the new system. As part of the installation, a preliminary FAQ
   was added.

   The database is updated once per hour.

   New reports available include ones about ports marked DEPRECATED,
   since that function has now been incorporated into bsd.port.mk. (The
   author hopes that this will allow the port deprecation process to be
   much more visible to the general FreeBSD user community.) In addition,
   a report for ports marked FORBIDDEN was added (the code was
   essentially the same).

   The next topic of interest is to try to identify ports which are slave
   ports because the status of these ports is not currently being updated
   automatically. This problem also affects FreshPorts. PR ports/63683 is
   an attempt to address this problem. Also, preliminary work has been
   done on creating some graphs and charts for various statistics, and in
   creating a tool to browse port dependencies for the entire ports tree.

   Some general observations about the trends in ports PRs can be made:
     * In the past 6 months, the amount of time to get ports PRs
       committed has dropped dramatically. (This is especially true of
       PRs for new ports.)
     * The queue of PRs for existing ports that are unmaintained has
       similarly been trimmed. Both of these two items are due in large
       part to a few very active committers (how do they ever get their
       "real" work done?) Thanks, guys, you know who you are.
     * There is still a fairly high number of PRs (~400/~750) which apply
       to existing ports, and have been assigned to a FreeBSD committer.
       This represents around 370 individual ports. We seem to have a
       much harder time getting these numbers to go down; basically, we
       just hold our own most weeks. This is somewhat disappointing.
     * The number of ports marked BROKEN has jumped dramatically,
       currently standing at over 250 (for i386-current). This represents
       less a sudden problem as it does Kris' effort to bring existing
       brokenness to people's attention -- thus, a much larger percentage
       of ports with build errors are now labeled as BROKEN.
     * Approximately two-thirds of the port build errors are still due to
       compilation problems, primarily from the gcc3.3 import. Another
       10% fail to install correctly. The reasons for the others are more

FreeBSD/arm Status Report

   Contact: Olivier Houchard <cognet at FreeBSD.org>

   Development goes reasonably fast, right now it boots single user. It
   is still very simics-centric, and it deserves a huge cleanup and a few
   bug fixes, but there's already a decent amount of code to work with,
   mostly taken from NetBSD. I now plan to work on real hardware support
   (as soon as I can get some), to get the missing userland bits (mainly
   rtld and the pthread libs) so that I can build a full world.


   URL: http://www.FreeBSD.org/platforms/ia64/index.html

   Contact: Marcel Moolenaar <marcel at FreeBSD.org>

   Work on the PMAP overhaul has been put into gear. A lot of issues will
   be addressed, including support for sparse physical memory and of
   course SMP. Performance will be addressed to the extend possible, but
   functionality has priority. The redesign will lay the foundation for
   NUMA support where possible. An example of this is limiting TLB
   shootdowns to processors that actually have or had TLBs belonging to
   the PMAP loaded. Of course, without NUMA hardware the implementation
   of NUMA support is quite limited.


   URL: http://www.freesbie.org
   URL: mailto:freesbie at gufi.org
   URL: http://www.freesbie.org/?section=mirror-en

   Contact: FreeSBIE Staff <staff at FreeSBIE.org>

   The FreeSBIE Project aims to develop a set of scripts that allow
   anyone to create their own FreeBSD Bootable Cdrom, with their own set
   of installed packages. The Project releases an ISO builded with
   FreeSBIE scripts, to show what they can do. On Sunday 29 February
   2004, FreeSBIE 1.0 was released and it had a great success, as there
   were post on Slashdot.org, OSnews, DaemonNews and BSDForums. Thanks to
   the huge amount of feedback they got, FreeSBIE Developers are now
   developing new features such as support for archs different from i386.
   Website redesign is on the way too.


   URL: http://www.FreeBSD.org/~nsouch/kgi4BSD

   Contact: Nicholas Souchu <nsouch at FreeBSD.org>

   Move to Perforce is done. I spent some time on building a common
   compilation tree with Linux: until now drivers were build in a FreeBSD
   makefile tree, not compatible with Linux.

   The next priorities are ANSI support and keymaps in the KGC Kernel
   Graphic Console system.


   URL: http://people.freebsd.org/~kientzle/

   Contact: Tim Kientzle <kientzle at FreeBSD.org>

   libarchive, with complete documentation, has been committed to
   -CURRENT. bsdtar should follow soon. For a few months, gtar and bsdtar
   will both be available in the base system. Once bsdtar is in the tree,
   I hope to resume work on libpkg and my pkg_add rewrite.

   Note that bsdtar is not an exact replacement for gtar: it does some
   things better (reads/writes standard formats, archive ACLs and file
   flags, detects format and compression automatically), some things
   worse (does not handle multi-volume archives or sparse files) and a
   few things just different (writes POSIX-format archives by default,
   not GNU-format). The command lines are sufficiently similar that most
   users should have no problems with the transition. However, people who
   rely on peculiar options or capabilities of gtar may have to look to

Move ARP out of routing table

   Contact: Andre Oppermann <andre at FreeBSD.org>

   The ARP IP address to MAC address mapping does not belong into the
   routing table (FIB) as it is currently done. This will move it to its
   own hash based structure which will be instantiated per each 802.1
   broadcast domain. With this change it is possible to have more than
   one interface in the same IP subnet and layer 2 broadcast domain. The
   ARP handling and the routing table will be quite a bit simplified
   afterwards. As an additional benefit full MAC address based accosting
   will be provided. Work on this project is already in progress.


   Contact: Poul-Henning Kamp <phk at FreeBSD.org>

   NanoBSD, src/tools/tools/nanobsd, is a tool for stuffing FreeBSD onto
   small disk media (like CompactFlash) for embedded applications. The
   disk image is built with three partitions, two for software images and
   one for configuration files. Having two software partitions means that
   new software can be uploaded to the non-active partition while running
   off the active partition.

   The first really public version has been committed and many
   suggestions and offers of patches have started pouring in.

Network interface naming changes

   Contact: Brooks Davis <brooks at FreeBSD.org>

   The first actual feature related to the if_xname conversion was
   committed in early February. Network interfaces can now be renamed
   with "ifconfig <if> name <newname>".

   Work is slowly progressing on a new network interface cloning API to
   enable interesting cloners like auto-configurating vlans. This work is
   taking place in the perforce repository under:

Network Stack Locking

   Contact: Sam Leffler <sam at FreeBSD.org>
   Contact: Robert Watson <rwatson at FreeBSD.org>

   This project is aimed at converting the FreeBSD network stack from
   running under the single Giant kernel lock to permitting it to run in
   a fully parallel manner on multiple CPUs (i.e., a fully threaded
   network stack). This will improve performance/latency through
   reentrancy and preemption on single-processor machines, and also on
   multi-processor machines by permitting real parallelism in the
   processing of network traffic. As of FreeBSD 5.2, it was possible to
   run low level network functions, as well as the IP filtering and
   forwarding plane, without the Giant lock, as well as "process to
   completion" in the interrupt handler.

   Work continues to improve the maturity and completeness of the locking
   (and performance) of the network stack for 5.3. The network stack
   locking development branch has been updated cothe latest CVS HEAD,
   tracking a variety of FreeBSD changes, including tracking and driving
   changes in the interface and device cloning APIs, push-down and fixes
   to locking in the Berkeley Packet Filter, consistency improvements in
   allocation flags for network objects, diagnosis of excessive
   acquisition of Giant in various system callouts and timeouts, removal
   of Giant from several system callouts, "const"-ification of a number
   of global variables in the network stack (IPv4, IPv6, elsewhere) as
   part of ananalysis of locking requirements, fine-grain locking of a
   number of pseudo-interfaces (disc, loopback, faith, stf, gif, tap,
   tun), IP encapsulation and tunneling, initial review and locking of
   parts of PPP and SLIP, experimentation with PCB assertions on IPv6,
   additional socket locking assertions, graphing of the FreeBSD sockets
   layer to support locking analysis, merging of theMT_TAG to m_tag
   conversion to improve the ability to queue packets, moving of the
   debug.mpsafenet tunable to controlling Giant over the forwarding plane
   to Giant over the entire stack("dual-mode" to support non-MPSAFE
   protocols), adaption of existing network lock assertions to also
   assert Giant when running non-MPSAFE, analysis of high cost of
   select() locking, improved locking and synchronization annotations,
   TCP callouts run MPSAFE, logtimeout() runs MPSAFE, uma_timeout() runs
   MPSAFE, callout sampling instrumentation, loadav() runs MPSAFE,
   AppleTalk locking begun: AARP locked down and DDP analysis, rawcb list
   locked, locking analysis of mrouter and IP ID code, IGMP locked, IPv6
   analysis begun, IPX/SPX analysis begun, PPP timeouts converted to
   callouts, Netgraph analysis begun. Many of these changes have not yet
   been merged to the main FreeBSDtree, but this is a work in progress.

   In related work on Pipe IPC (not quite network stack locking),
   substantial time was invested in diagnosing an increase in the cost of
   pipe allocation since FreeBSD 4.x, as well as coalescing the several
   allocations needed to create a pipe, as well as moving to slab
   allocation so as to amortize the cost of pipe initialization. Future
   work here will include caching the VM structures supporting pipe

   Recent contributors include Robert Watson, Sam Leffler, MaxLaier,
   Maurycy Pawlowski-Wieronski, Brooks Davis, and many others who are
   omitted here only by accident.

Porting OpenBSD's pf

   URL: http://pf4freebsd.love2party.net/
   URL: http://www.benzedrine.cx/pf.html
   URL: http://openbsd.org/faq/pf/index.html
   URL: http://www.rofug.ro/projects/freebsd-altq/

   Contact: Max Laier <max at love2party.net>
   Contact: Pyun YongHyeon <yongari at kt-is.co.kr>

   The sources were imported from OpenBSD 3.4R and patched with diffs
   obtained from the port. Since March the 8th it is linked to the build
   and install. There is some more work to be done in order make pf a
   home inside the tree, but the biggest hunk of work was lifted during
   the past two month.

   OpenBSD 3.5 is scheduled for early May, so we might see an update
   before 5.3R. Work towards integration of the - often requested - ALTQ
   framework is in progress also, though it is not yet clear how well it
   goes along with the ongoing work towards a giant free net stack.

PowerPC Port

   Contact: Peter Grehan <grehan at FreeBSD.org>

   After a slow time at the end of last year due to a disk crash, the
   project is moving along rapidly. The loader is fully functional with
   Forth support. Syscons has been integrated. New Powerbook models are
   supported. Work is starting on a G5 port.

   There's still lots to do, so as usual volunteers are most welcome.

SGI XFS port for FreeBSD

   Contact: Alexander Kabaev <kan at FreeBSD.org>
   Contact: Russell Cattelan <cattelan at thebarn.com>

   Not much has changed since last report was submitted. The read-onle
   access XFS volumes is quite stable now. The work is underway to
   rewrite xfs_buf layer to minimize local changes intrusiveness. Initial
   attempt to make XFS code to compile and run on amd64 is in progress

   We really need a care-taker for our userland tools.

Testbed for testing and qualification of TCP performance

   Contact: Andre Oppermann <andre at FreeBSD.org>

   The TCP performance test and qualification testbed is an automated
   environment that simulates various common and uncommon end-to-end
   network and link characteristics such as delay, bandwidth limitations,
   congestion, packet drops, packet corruption and out of order arrival.
   The testbed automatically steps through all link types and tests
   various TCP optimizations and parameter adjustments. In the end all
   data is graphically arranged and compared against standard behaviour
   and each other to judge the positive or negative effects of the
   modifications. Work on this project has just started and is based on
   FreeBSDs dummynet.

The FreeBSD Dutch Documentation Project.

   Contact: Remko Lodder <remko at elvandar.org>

   The Dutch Documentation Project is a ongoing project in translating
   the handbook and other documentation to the dutch language. Currently
   there is 1 active person (me) translating the documentation. I am
   currently working on the handbook/basics section. But i can use some
   more hands, please drop me an email if you wish to help out so that
   the dutch translation will speed up and be ready in some time. Contact
   remko at elvandar.org for information.

The FreeBSD Simplified Chinese Project

   URL: http://www.freebsd.org.cn
   URL: http://www.freebsd.org.cn/snap/zh_CN/
   URL: http://www.freebsd.org.cn/snap/doc/zh_CN.GB2312/books/handbook/

   Contact: Dong LI <ld at FreeBSD.org.cn>
   Contact: Xin LI <delphij at frontfree.net>

   The project is a joint effort of volunteers, which focus in the
   internationalization and localization of the FreeBSD Operating System
   and applications running on FreeBSD. All of the work resulted in this
   project will be contributed back to the FreeBSD project.

   Thanks to many volunteers' help, by this time of writing, we have
   finished more than 60% of the translation of the FreeBSD Handbook. We
   plan to submit a preliminary translation of the FreeBSD website as
   well as the FreeBSD Handbook when most part of them were finished,
   which is expected to happen in a couple of months. The snapshot of the
   documentation translation effort could be accessed through the URL
   listed above.

   The project also supported individual efforts on porting applications
   (especially software that supports Simplified and/or Traditional
   Chinese) to FreeBSD. We are also doing some research on making FreeBSD
   kernel and base system more i18n-aware.

Verify source reachability option for ipfw2

   URL: http://www.nrg4u.com/freebsd/ipfw_versrcreach.diff

   Contact: Andre Oppermann <andre at FreeBSD.org>

   The verify source reachability option for ipfw2 checks if the source
   IP address of a packet entering the machine is reachable at all. Thus
   if we can't send a packet back because we don't have a route back we
   don't have to forward it because two way communication isn't possible
   anyway. It is more than likely that such a packet is spoofed. This
   option is almost the same as what is known on Cisco IOS as "ip verify
   unicast source reachable-via [any|ifn]". Using this option only makes
   sense when you don't have a default route which naturally always
   matches. So this is useful for machines acting as routers with a
   default-free view of the entire Internet as common when running a BGP
   daemon (Zebra/Quagga or OpenBSD bgpd).

   One useful way of enabling it globally on a router looks like this:
   ipfw add xxxx deny ip from any to any not versrcreach or for an
   individual interface only: ipfw add xxxx deny ip from any to any not
   versrcreach recv fxp0

vinum + GEOM

   URL: http://mailbox.univie.ac.at/~le/geom_vinum.tar.gz

   Contact: Lukas Ertl <le at FreeBSD.org>

   The "geomification" of vinum has made some progress. I now have all
   basic setups working (concatenated plexes, striped plexes, RAID5
   plexes, and RAID1), but I still have to implement correct error
   handling and status change handling.

   Still missing is a userland tool, so currently you still have to use
   "old-style" vinum to configure your setup.

Weekly cvs-src summaries

   URL: http://excel.xl0.org/FreeBSD/
   URL: http://mocart.pinco.pl/FreeBSD/

   Contact: Mark Johnston <mark at xl0.org>

   I have been producing weekly summaries of commits and the surrounding
   discussions as reported on the cvs-src mailing list. These summaries
   are posted to -current on Sunday evenings and archived on the Web. The
   reception has been overwhelmingly good. As of the end of February,
   Polish translations are being produced by Lukasz Dudek and Szymon
   Roczniak; they are also planning to translate the older summaries.

More information about the freebsd-announce mailing list