Plans about GIANT-LOCK, SMP performance ...

[Robert Watson]

On Sat, 8 Jan 2005 dkouroun at cc.uoi.gr wrote:

> Does anybody know what are the plans, for locking mechanism like
> GIANT-LOCK in future 5.3 Releases? I have heard from many people that
> this GIANT-Locking mechanism is not the best thing to have! Some said
> that it may be replaced. Some also said that it won't.
> 
> 1:) What are the plans about it? 
> 
> 2:) How does this affect performance? 
> 
> 3:) Which is version is best for an Opteron QUAD? 
>     4.10 Release or > 5.X Release ??? 

One of the single largest projects associated with the FreeBSD 5.x release
line has been the elimination of the Giant lock from many of the kernel
code paths.  FreeBSD 5.0 - 5.2 largely involved introducing the
infrastructure necessary to remove Giant from major subsystems, and while
Giant was removed in some of these releases from a number of important IPC
paths, etc, it was still present over the majority of device drivers, the
network subsystem, and the storage subsystem.  In FreeBSD 5.3, the network
subsystem runs without the Giant lock in most cases (some legacy
components, such as support for IPX/SPX require Giant -- most of these
cases will be eliminated in 5.4).  The result is substantial increases in
effective parallelism on SMP on multi-processor systems.  This work
continues in both the 6.x and 5.x branches, with many additional
improvements expected over the next few months (and releases).  In
particular, now that the basic SMPng infrastructure works, we're moving
into the optimization phase in which additional locking overhead
associated with finer graind lock is profiled, analyzed, and optimized.

Performance measurements generally suggest that work loads capable of
exploiting parallelism in the network stack may see substantial
improvements on SMP over previous releases, including things like
databases, large scale web servers, etc.  Other workloads may see less
improvement, or in some cases, a performance reduction with respect to
4.x.  In particular, the change in kernel architecture to remove the Giant
lock has introduced additional locking operations that are not free -- on
UP systems, this effect may be more noticeable, since there are no
parallelism gains to be had from finer grained locking (although the
preemption benefits can be observable).  As I mention above, this is the
subject of active work.  One of the substantial goals for the 6.x line,
with a likely merge to 5.x as the work matures, is the removal of Giant
from large parts of VFS.  There are current patches to do this in many
useful cases for the UFS/UFS2 file system, available for testing on the
freebsd-current mailing list.  These are highly experimental, but continue
the SMPng work in important ways.  You can start to look for the benefits
of this work in mainstream FreeBSD releases in 6-8 months, I think.

For the 64-bit AMD processor line, FreeBSD 5.3/amd64 is generally
recommended, as it is able to take advantage of the 64-bit feature set,
including large address spaces, which can be very important for database
applications, large scale image processing, computation, etc.  One caution
I might have is the following: some workload do not benefit from 64-bit
operating system code and applications, as they are not able to use the
additional memory effectively.  When this is the case, the additional
address space (etc) introduce an overhead relative to 32-bit operation, so
it may be that these applications see a performance hit when not running
the 32-bit version of FreeBSD for the platform (the i386 release).

Robert N M Watson