Yet another INDEX builder.

[Vasil Dimov]

On Sat, Sep 30, 2006 at 02:46:36PM -0400, Kris Kennaway wrote:
[...]
> 
> The only situation where you might reasonably expect to see something
> approaching a factor of 2 speed improvement is if your ports tree is
> on a striped or mirrored device on separate controllers, so 2 reads
> can be completely decoupled from each other in hardware.  I've not
> tried this though.
> 

Here are some timings from my machine:

/usr/ports# diskinfo -t /dev/stripe/gs0
/dev/stripe/gs0
	512         	# sectorsize
	12582649856 	# mediasize in bytes (12G)
	24575488    	# mediasize in sectors

Seek times:
	Full stroke:	  250 iter in   1.341719 sec =    5.367 msec
	Half stroke:	  250 iter in   1.648136 sec =    6.593 msec
	Quarter stroke:	  500 iter in   2.668755 sec =    5.338 msec
	Short forward:	  400 iter in   2.317754 sec =    5.794 msec
	Short backward:	  400 iter in   1.124325 sec =    2.811 msec
	Seq outer:	 2048 iter in   0.310119 sec =    0.151 msec
	Seq inner:	 2048 iter in   0.337578 sec =    0.165 msec
Transfer rates:
	outside:       102400 kbytes in   0.989050 sec =   103534 kbytes/sec
	middle:        102400 kbytes in   0.996347 sec =   102775 kbytes/sec
	inside:        102400 kbytes in   1.029199 sec =    99495 kbytes/sec

/usr/ports# dmesg |grep ^CPU
CPU: AMD Athlon(tm) 64 Processor 3500+ (2210.07-MHz K8-class CPU)

During all tests gstat(8) showed that the stripe was quite idling:

 L(q)  ops/s    r/s   kBps   ms/r    w/s   kBps   ms/w   %busy Name
    0     22     22     44    7.6      0      0    0.0   16.6| stripe/gs0
 L(q)  ops/s    r/s   kBps   ms/r    w/s   kBps   ms/w   %busy Name
    0     24     24     48    1.8      0      0    0.0    4.3| stripe/gs0
 L(q)  ops/s    r/s   kBps   ms/r    w/s   kBps   ms/w   %busy Name
    0     30     28     56    1.4      2    255    0.8    4.0| stripe/gs0
 L(q)  ops/s    r/s   kBps   ms/r    w/s   kBps   ms/w   %busy Name
    0      2      2      4    5.9      0      0    0.0    1.2| stripe/gs0

/usr/ports# time make index INDEX_JOBS=1
real	19m29.649s
user	12m9.367s
sys	5m13.881s

/usr/ports# time make index INDEX_JOBS=2
real	18m38.334s
user	12m54.316s
sys	5m4.062s

/usr/ports# time make index INDEX_JOBS=3
real	18m0.804s
user	12m29.657s
sys	5m16.442s

/usr/ports# time make index INDEX_JOBS=4
real	18m33.941s
user	12m56.167s
sys	5m12.183s

/usr/ports# time make index INDEX_JOBS=10
real	20m30.924s
user	13m56.722s
sys	5m29.590s

/tmp$ time ./build_index.py
......archivers......
The port             archivers/fileroller is obsolete
The port             archivers/gshar+gunshar is obsolete
[... a lot more obsolete ports ...]
The port             x11-wm/wmx is obsolete
The port             x11-wm/xwmm is obsolete
......dns......
......polish......
......arabic......
......accessibility......
......net-mgmt......
......x11-themes......
......net-im......
......net-p2p......
The tree (15813ports) takes  1606.92809916 seconds to explore.
Now computing the recursively extended dependencies.
Took  1.09403419495 seconds.
Now converting to packages and sorting.
Last phase takes  1.55835795403 seconds. 
Total time spent:  1611.23766518 seconds.

real	26m51.585s
user	15m14.370s
sys	10m49.645s

-- 
Vasil Dimov
gro.DSBeerF at dv
%
Peace cannot be kept by force; it
can only be achieved by understanding.
                -- A. Einstein
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 155 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-ports/attachments/20061002/04dfbf55/attachment.pgp