dump cache performance

[Jean-Francois Dockes]

Hello,

I took a look at the "cache management for the dump command" project on the
freebsd.org project ideas page.
http://www.freebsd.org/projects/ideas/ideas.html#p-extenddump

It appears that modifying dump to use a shared cache in a very simple way
(move the control structures to the shared segment and perform simple locking)
yields substantial speed increases.

A patch implementing this is attached. 

Some numbers follow. The test system is an intel core i5 750, with recent
SATA disks. The tests all record an improvement with the shared cache, but
the values vary widely from 7% to 236%. It would be interesting to have
more tests on different configurations.

Would someone be interested in reviewing the patch and/or perform
more tests ?

Regards,
J.F. Dockes


Some tests results
===================

The command used in all cases is  "dump -0aC XX -f /dev/null filesystem"

The current dump actually uses 5 times the value of the -C option for
cache. The patched version uses a single shared memory segment. So "olddump
-C 10" and "newdump -C 50" are equivalent in terms of cache memory usage.

---------------
Tests performed on a small slice (3.7GB/4GB). The filesystem is quite full, and
has been pushed beyond full then partially pruned a few times to simulate
one which would actually had a life. It contains a mix of /home/ user
files (avg size 68 kB). Tests were run both in single disk and mirror mode.

Mirrored slice
    Split cache -C 10: 18 MB/s
    Shared cache -C 10 : 42 MB/s (+133%)

Same slice, without the mirroring
    Split cache -C 10: 11 MB/s
    Shared cache -C 50: 37 MB/s  (+236%)

--------
Tests on /var (500 MB / 5 GB). Mirrored slice
    Split cache -C 10: 15 MB/s
    Shared cache -C 50: 28 MB/s (+86%)

-----------
Tests on a bigger slice (24 GB / 43 GB) with mostly big files. Single disk
    Split cache -C 10: 15 MB/s
    Shared cache -C 50: 35 MB/s (+133%)

-----------
Tests on /usr (464 GB / 595 GB), mirrored
    Split cache, -C 50: 57 MB/s
    Shared cache, -C 250: 63 MB/s (+10%)
Level 1 tests (5GB dump)
    Split cache, -C 50: 38 MB/s
    Shared cache, -C 250: 41 MB/s (+7%)

-------------- next part --------------
A non-text attachment was scrubbed...
Name: dump-shared-cache.diff
Type: text/x-patch
Size: 3230 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20101024/3fd5dea8/dump-shared-cache.bin
-------------- next part --------------
diff
-------------- next part --------------
A non-text attachment was scrubbed...
Name: dump-shared-cache.diff
Type: text/x-patch
Size: 3230 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20101024/3fd5dea8/dump-shared-cache-0001.bin