Some initial postmark numbers from a dual-PIII+ATA, 4.x and 6.x

[Robert Watson]

I pulled down the postmark benchmark, and gave it a spin on a dual PIII
800mhz box w/256mb of RAM from the netperf cluster.  I ran all tests
against a UFS1 partition shared by the 4.x and 6.x install on the box, so
that the file system would be on the same spot on the disk.  I used the
following postmark parameters: 

	size 300 100000
	transactions 10000

I have the numbers below, but here are the conclusions: on this hardware,
using a single ATA disk, there was no real measurable performance
difference between UP/SMP, and 4.x/6.x -- 6.x came out slightly ahead on
t/s, but not hugely so.  I take this to mean that the hardware was
basically I/O bound on file system meta-data operations. 

One observation which is worthy of caution.  When re-running the same
measurement several times on the same configuration, I saw that Postmark
is quite sensitive to short runs, in that it appears to use integer
division by the measured number of seconds (in integer representation) of
a test.  This means that the transaction rates, typically associated with
long runs of transactions, tend to reasonably reflect mean rates, but that
the "alone" measurements, associated with the setup and tear-down of the
benchmark, are very sensitive to minor timing differences.  As such, I
would recommend against using these figures in evaluating the performance
of a configuration.  I have included only full transaction run
measurements below.  Just as an example: I saw the "create alone" rate
bounce between 166 and 250 per/sec depending on whether the creation of
500 objects took two or three seconds.

Another thing to keep in mind when comparing this to some of the other
numbers that have been posted: I compared specifically using UFS1 and soft
updates.  I didn't frob settings such as async.

One final note -- because the system is I/O-bound, the effective CPU
consumption appears to have little impact on the result.  The 6.x
performance appears marginally better here, but there is more CPU usage. 
While I didn't attempt to measure precise CPU usage as yet, my impression
is that the 6.x system time was at least 1/3 again the CPU consumption on
4.x, so if this benchmark were running using this CPU/etc, but the I/O
throughput were higher, presumably the increase CPU consumption would play
a larger role.  I'll attempt to measure a little better the CPU use over
the weekend.  On the other hand, this box is hardly a speed demon... 

Numbers below.

Robert N M Watson



UP=Uniprocessor
SMP=Multiprocessor
MV=MPSAFE VFS

tran sec	Length of transaction run in seconds
tran/s		Number of transactions/sec over total run
creat/s		Number of create transactions/sec ("mixed")
read/s		Number of read transactions/sec ("mixed")
app/s		Number of append transactions/sec ("mixed")
del/s		Number of delete transactions/sec ("mixed")
rd/s		Number of read transaction/sec ("mixed")
wr/s		Number of write transactions/sec ("mixed")

           tran sec  tran/s  creat/s  read/s  app/s  del/s  rd/s  wr/s
4.x UP     76        131     66       65      66     65     4.00  4.45
           75        133     67       66      67     66     4.00  4.45

4.x SMP    74        135     68       66      68     67     4.11  4.56
           76        131     66       65      66     65     3.95  4.39

6.x UP     73        136     68       67      69     68     4.17  4.62
           71        140     70       69      70     69     4.22  4.69

6.x UP MV  71        140     70       69      70     69     4.22  4.69
           71        140     70       69      70     69     4.22  4.69

6.x SMP    72        138     69       68      70     68     4.17  4.62
           72        138     69       68      70     68     4.22  4.69

6.x SMP MV 72        138     69       68      70     68     4.17  4.62
           74        135     68       66      68     67     4.11  4.56