Increasing GELI performance

[Pawel Jakub Dawidek]

On Fri, Jul 27, 2007 at 10:00:35PM +0100, Dominic Bishop wrote:
> I just tried your suggestion of geli on the raw device and it is no better
> at all:
> 
> dd if=/dev/da0.eli of=/dev/null bs=1m count=1000
> 1000+0 records in
> 1000+0 records out
> 1048576000 bytes transferred in 29.739186 secs (35259069 bytes/sec)
> 
> dd if=/dev/zero of=/dev/da0.eli bs=1m count=1000
> 1000+0 records in
> 1000+0 records out
> 1048576000 bytes transferred in 23.501061 secs (44618241 bytes/sec)
> 
> Using top -S with 1s refresh to list the geli processes whilst doing this it
> seems only one of them is doing anything at any given time, the others are
> sitting in a state of "geli:w", I assume that is a truncation of something,
> maybe geli:wait at a guess.

No matter how many cores/cpus you have if you run single-threaded
application. What you do exactly is:
1. Send read of 128kB.
2. One of geli threads picks it up, decrypts and sends it back.
3. Send next read of 128kB.
4. One of geli threads picks it up, decrypts and sends it back.
...

All threads will be used when there are more threads accessing provider.

> For comparison, here are the same tests done on the unencrypted raw device:
> 
> dd if=/dev/da0 of=/dev/null bs=1m count=1000
> 1000+0 records in
> 1000+0 records out
> 1048576000 bytes transferred in 5.802704 secs (180704717 bytes/sec)
> 
> dd if=/dev/zero of=/dev/da0 bs=1m count=1000
> 1000+0 records in
> 1000+0 records out
> 1048576000 bytes transferred in 4.026869 secs (260394859 bytes/sec)

You can see how much CPU power your one core has by doing something like
this:

	# kldload geom_zero
	# geli onetime -s 4096 gzero
	# sysctl kern.geom.zero.clear=0
	# dd if=/dev/gzero.eli of=/dev/null bs=1m count=4096

/dev/gzero is similar to /dev/zero, but it is a GEOM provider.
This experiment will show you how much encryption throughput you can get
from one geli thread, without disk overhead.

-- 
Pawel Jakub Dawidek                       http://www.wheel.pl
pjd at FreeBSD.org                           http://www.FreeBSD.org
FreeBSD committer                         Am I Evil? Yes, I Am!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-geom/attachments/20070730/da31f9fd/attachment.pgp