cvs-src summary for June 7-14
mjohnston at skyweb.ca
Tue Jun 15 03:38:31 GMT 2004
Here's the summary - hope you like locking. I am sorely tempted to go out and
write some lockless, race-condition-filled code, just for spite.
On a somewhat related note, I'm considering moving the discussion summary to
the format that (IIRC) is used by the Linux kernel traffic newsletter, mostly
direct quotes with some explanation in between. It feels a bit stupid
paraphrasing everyone's perfectly good comments (and coming up with 50 ways
to say "replied") for consistency's sake. If this bothers you, g'ahead and
let me know.
FreeBSD cvs-src summary for 07/06/04 to 14/06/04
This is a regular weekly summary of FreeBSD's cutting-edge development.
It is intended to help the FreeBSD community keep up with the fast-paced
work going on in FreeBSD-CURRENT by distilling the deluge of data from
the CVS mailing list into a (hopefully) easy-to-read newsletter. This
newsletter is marked up in reStructuredText_, so any odd punctuation
that you see is likely intended for the reST parser.
.. _reStructuredText: http://docutils.sourceforge.net/rst.html
You can get old summaries, and an HTML version of this one, at
http://www.xl0.org/FreeBSD/. Please send any comments to Mark Johnston
(mark at xl0.org).
For Lukasz Dudek and Szymon Roczniak's Polish translations of these
summaries, which may lag the English ones slightly, please see
ipfw gains lookup table support
Ruslan Ermilov (ru) added support to ipfw for lookup tables, which allows
a table of IP addresses or subnets to be created and efficiently used in
rules. The code was originally contributed by Vsevolod Lobko, and refined
ALTQ framework imported
Max Laier (mlaier) imported the ALTQ queueing framework from KAME. ALTQ
provides advanced QoS and queueing for traffic shaping. It was not
initially enabled, but a later commit linked it to the build; you will
need to recompile network modules and userland code that uses the ifnet
structure. As usual, a full buildworld/installworld is safest. The
queueing, and individual network modules, have not yet changed.
GEOM-supporting vinum added
Lukas Ertl (le) committed a first version of Vinum, the FreeBSD volume
manager, updated to use GEOM, the 5.x disk framework. This will allow
Vinum to be used in 5.x. He also added gvinum, the userland tool to
manage the GEOMified vinum.
esp driver added
Scott Long (scottl) imported the sbus version of the NetBSD esp driver,
which will handle the SCSI card in Sun Ultra 1e and 2 machines. Once the
PCI version is ported, the driver can also replace the current amd driver,
which handles amd53c974-based SCSI cards.
IP over Firewire support added
Doug Rabson (dfr) added a new driver to support IP over Firewire. The
new driver conforms to `RFC 2734`_ and `RFC 3146`_, and is intended to
eventually supersede the fwe driver.
.. _`RFC 2734`: ftp://ftp.rfc-editor.org/in-notes/rfc2734.txt
.. _`RFC 3146`: ftp://ftp.rfc-editor.org/in-notes/rfc3146.txt
DRM code updated
Eric Anholt (anholt) updated the DRM (Direct Rendering Manager) code from
DRI (Direct Rendering Infrastructure) CVS. The update consists mostly of
new PCI IDs and a new packet for the Radeon card.
sha1 and rmd160 checksum utilities added
Oliver Eikemeier (eik) added sha1 and rmd160 checksum-calculating tools,
which are similar to the existing md5 tool and used in the same way.
Locking added to UNIX domain sockets, among other things
Robert Watson (rwatson) added locking to UNIX domain sockets. This code
was submitted by Sam Leffler (sam), and was originally written for BSD/OS,
with Robert doing several tweaks to it. He also made small locking
commits to a number of other subsystems, mostly networking-related.
CVS updated to 1.11.17
Peter Wemm (peter) updated the system CVS to version 1.11.17. This
replaces the old version 1.11.15. 1.11.17 is a security and bugfix
release, although the security updates were previously applied.
Poul-Henning Kamp (phk) eliminated COMPAT_SUNOS, compatibility code for
SunOS 4.x, which came from BSD4.4-Lite1's sparc32 port.
struct tty now reference counted
Poul-Henning Kamp (phk) committed reference counting for the tty code;
however, drivers are not yet using the new calls.
A few words on locking
Poul-Henning Kamp (phk) fixed a race condition in the process management
code, prompting a huge (but productive) thread about locking. I have
summarized most of it, but left out some subthreads that were essentially
duplicates of others.
Initially, Poul-Henning posted with a description of why the old locking
was wrong - it locked an object, decremented its reference count, then
unlocked it and took the reference count again to decide whether to
destroy it. If another thread decremented the reference count and
destroyed the object in the meantime, the first thread will try to take
the reference count of the dead object and crash. His suggestion was to
save the reference count and use that when deciding whether to destroy the
object, rather than trying to take it again.
Nate Lawson (njl) noted that if another thread increments the reference
count after the unlock but before the destruction, you could destroy an
object that was still in use. He suggested that you could hold the lock
until the object is freed. Brian Feldman (green) suggested locking the
object as well as the list of objects of that type, so once the reference
count reaches zero, it can't be found to be incremented again. Darren
Reed (darrenr) agreed, and suggested an alternate way, using separate read
and write locking to improve performance.
Lukas Ertl (le) had a similar followup to Nate's, asking whether it's
possible for another thread to increment the reference count after it's
found to be zero. Garrett Wollman (wollman) answered that that would only
be possible if not all the references were counted. Ruslan Ermilov (ru)
asked him to clarify whether he meant that a change from 0 to 1 wasn't
possible; Julian Elischer (julian) responded, answering that once the
object is removed from the list, and only one reference (the one in your
own code) remains, nobody else can find the object to increment it.
Bosko Milekic (bosko) responded to a request by M. Warner Losh (warner)
for clarification, explaining how the race condition works. He
also posted code for preventing a similar problem when using an
atomically-updated reference counter, instead of one protected by a lock.
Warner suggested that a locking section in the handbook would be a good
place for the techniques discussed. Nate Lawson followed up that Bosko's
new code could have a race if the reference count increment path isn't
done properly. Bosko responded that it could not, since the reference
count is done atomically and it can't go up once it reaches zero, since
the code that set it to 0 had the last reference.
Pawel Jakub Dawidek (pjd) followed up to Bosko's post about atomic
reference counts, wondering whether it's limited to values of 0 and 1.
Bosko replied, saying that it can count any number of references. Pawel
asked whether atomic operations are also atomic on MP machines; Warner
answered that they are, which makes them very bad for performance.
Poul-Henning also replied, agreeing that atomic operations are not cheap
and mutexes should be generally used instead, since they are easier to
understand and validate with WITNESS. Julian also warned that the atomic
method could result in a page fault.
Julian suggested that a more formal way of handling reference counts is
needed; Poul-Henning replied that it is often bad for performance and code
clarity. Julian argued that most people found it worth pursuing anyway;
Poul-Henning encouraged him to go ahead with it. Julian offered to take
discussion of such a formal system to -arch, if anyone was interested.
Warner responded to Julian's post, saying that documentation of how to
properly do reference counting is needed. Poul-Henning suggested that
people should be familiar with locking from computer science courses;
Warner suggested documentation would still help, and Poul-Henning agreed,
saying it was on his list of articles to write in the future.
Mike Silbersack (silby) chimed in, titling the thread "Locking the
Bikeshed". I decided to keep my title, though, simply out of fear of
wearing out my k key.
Important bug fixes
3ware 6xxx RAID bug fixed
Vinod Kashyap (vkashyap) patched a bug in the twe driver for 3ware RAID
controllers that was causing 6xxx series controllers to resubmit the same
request repeatedly when the command queue was full. He also merged the
fix to 4.x.
Other bug fixes
Christian S. J. Peron (csjp) patched the ipfw code so that PCBs will be
looked up only once per ipfw activation. This should significantly speed
up UID/GID-based firewall rules. He provided a `benchmark graph`_ showing
.. _`benchmark graph`: http://people.freebsd.org/~csjp/ip_fw_pcb.png
David Schultz fixed an error in the code for rounding floating-point
numbers to integers that resulted in incorrect values for some inputs.
Ian Dowse (iedowse) fixed a bug in the USB code that was causing a
"restarting port 1" error followed by "giving up port 1" on a VIA USB 1
controller. This fix was also MFC'ed.
Ruslan Ermilov (ru) fixed a crash in make on 4.x when a .DEFAULT target in
a Makefile has no commands.
More information about the freebsd-current