blogbench and write-open serialization

[Ivan Voras]

During the BSDCan & DevSummit I got interested in finding out why
blogbench is so slow on FreeBSD. After talking to jhb, it looked like
one of the reasons might be that opening files with O_RDWR or O_WRONLY
(anything opening the file for writing) is serialized.

To check this, I've written a small test program, which I've run on
CentOS 6.3 and FreeBSD 10-HEAD on the same hardware. Here are the results:

https://wiki.freebsd.org/Benchmarking/OpenCloseBenchmark

Conclusions:

* Linux opens and closes files much faster than FreeBSD
* Linux does not serialize write-open operations, while FreeBSD does
* Even with O_RDONLY, FreeBSD is much slower in opening (and closing) files.

I'd welcome a review of these results and comments.