Poor read() performance, and I can't profile it
Kirk Strauser
kirk at strauser.com
Thu Jun 5 20:26:38 UTC 2008
I'm running a command (dumprecspg from my XBaseToPg project) on a FreeBSD 7
server. I've noticed that throughput on that program is a lot lower than I
would have expected, and further investigation found it spending most of its
time in the kernel, presumably in read() [1].
I was testing the same software on my desktop PC when I noticed that it ran
*much* faster, and found that it was spending only about 1% as much time in
the kernel on Linux as it was on FreeBSD.
I ran a quick-and-dirty comparison of the same software on two different
machines, the FreeBSD server being by far the more powerful of the two. I
ran the same command on both machines from various filesystems (to rule out
differences in drive performance), and posted the output of zsh's "time"
command for the fastest run in each setting. The results are below.
Any ideas what could be causing this horrible performance? I'm willing to
try just about anything.
FreeBSD on a Dell Poweredge 1600SC server:
7-STABLE from 2008-03-09
2x 2.4GHz P4 Xeon
3GB RAM
Changes to /etc/make.conf:
CPUTYPE?=pentium4
Kernel config:
include GENERIC
ident JAIL1
options PMAP_SHPGPERPROC=301
nooption SCHED_4BSD
option SCHED_ULE
root : Fujitsu 36GB, 10k RPM
Best time: 6.37s user 9.68s system 99% cpu 16.068 total
/tmp : tmpfs
Best time: 6.29s user 10.88s system 99% cpu 17.194 total
/fast : 4 Seagate Cheetah 36GB, 15k RPM SCSI320 drives in RAID-0 with
gstripe, 128KB stripe size with kern.geom.stripe.fast enabled and
stripe.fast_failed=0
Best time: 6.60s user 9.46s system 99% cpu 16.088 total
Conclusion: Since gstat showed all drives as idle through most of all the
tests, it looks like the rest is running entirely from
buffers.
Linux on a Dell Dimension 4600 desktop:
Ubuntu 8.04
2.4GHz P4
1GB RAM
root: WD 250GB SATA
Best time: 7.60s user 0.92s system 97% cpu 8.722 total
Conclusion: I don't know if there's an equivalent to gstat in Linux, but
the system overhead is about one-hundredth as much as in
FreeBSD.
[1] I can't run gprof on FreeBSD because if I build the binary with -pg,
then it segfaults on startup:
$ gdb /tmp/xbase/bin/dumprecspg /tmp/dumprecspg.core
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd"...
Core was generated by `dumprecspg'.
Program terminated with signal 11, Segmentation fault.
Reading symbols from /tmp/xbase/lib/libxbase64.so.1.0...done.
Loaded symbols for /tmp/xbase/lib/libxbase64.so.1.0
Reading symbols from /lib/libgcc_s.so.1...done.
Loaded symbols for /lib/libgcc_s.so.1
Reading symbols from /lib/libc.so.7...done.
Loaded symbols for /lib/libc.so.7
Reading symbols from /libexec/ld-elf.so.1...done.
Loaded symbols for /libexec/ld-elf.so.1
#0 0x0807110c in main (ac=Cannot access memory at address 0x18
) at dumprecspg.cpp:63
63 int main(int ac,char** av)
(gdb)
--
Kirk Strauser
More information about the freebsd-questions
mailing list