Heavy I/O blocks FreeBSD box for several seconds

[O. Hartmann]

When performing an update on the ports tree via "portsnap fetch update" 
or when checking out (or) large Subversion repositories or when copying 
large data files (~ 50 to 250 GB in size, results from numerical 
modelings) or when compiling world, FreeBD 9.0 and FreeBSD 8.2-STABLE 
tend to "freeze" for several seconds or drop overall performance 
dramatically for seconds. On boxes with only console- or terminal access 
(no GUI) a running 'vi' gets stuck for seconds while one of the 
processes producing heavy I/O is running, or the output of a 'cat' of a 
large file stops for several seconds.

Using X11, this phenomenon gets even worse and the 'freezing' tends to 
persist sometimes for more than 10 or 15 seconds.

The boxes in question are all 64Bit, do have 2 to 8 CPUs/cores (no SMT) 
and not less than 8 GB of RAM (the 8 core box is a dual socket Dell 
server with two 4-core C2D-type XEON CPUs and 16 GB of RAM). All these 
boxes uses ZFS filesystems for data along with UFS2 for the OS.

On a notebook with a relative modern Core-5 dual core CPU and 4 GB RAM 
(running FreeBSD 9.0-CURRENT/amd64), not ZFS, all UFS, with a 500GB 
harddrive at 5400 upm (Dell Latitude E6510), this phenomenon is even 
worse. Heavy disk I/O blocks the GUI for nearly half a minute, even when 
no X11 is activated, the console gets stuck for a while. First I thought 
this could be a problem with the "slow" harddrive, but I tried also a 
Linux Ubuntu 11.04 on the box and forcing heavy I/O by copying the large 
files in question from one location on the disk to another is performed 
even faster and without any freezing of a console or GUI (used EXT4 
filesystem).
I'm curious about this behavior. I use FreeBSD as my favourite HPC 
platform as far as this is possible with FreeBSD, but I realized this 
bottleneck when it comes to heavy I/O and I'd like to know whether this 
is only a "superficial" phenomenon (like caused by the outdated X11 
version FreeBSD use or a low priorized tty handling, means only the 
observer "realize" a performance drop).
I've got not the time to investigate this deeper (I'd like to perform 
some benchmarks on the notebook if it is available again, but my 
knowledge on Linux/Ubuntu is very limited and the how-to setting up 
FreeBSD and Linux to match each other on the same hardware could be tricky).

My kernel setups on FreeBSD are mostly the GENERIC kernel with extracted 
drivers I do not use (like some USB devices, SAS/SCSI adaptors etc. we 
do not use, et cetera), nothing special. Either way I follow the tips 
presented in "tuning(7)" or not, the phenomenon is present.

Thanks,
Oliver