RELENG_7 heavy disk = system crawls

[Dieter]

> > hardware crypto accellerators
> 
> The soekris ones work and are cheap. I thougt I saw posts that show
> openssl -speed on today's fast cpu's being faster than the accel
> cards. Disk crypto is symmetric, not initial pki session setup.

The accel cards probably don't get updated as often as cpus do.
Then there is the question of how many cards would you need and
having enough free slots to plug them into.  And the cards aren't
going to help with zfs.

Does anyone make a disk controller with crypto built in?  Then it would
scale with the number of drives.  And not need extra slots.

> > throw lots of cpu (e.g. phenom2 x4) at it
> 
> That would only help it get done busying out the system sooner, not
> balance things out while actively under load. Which as a file server,
> it always is.
> 
> Well, ok, it will help after the system is able to do spindle ->
> geli -> fs -> process at the max sustained read/write speed of the
> spindles. Which is about 56MiB/sec reading in this case. Which is
> over 10x faster than I'm getting now. Which means I'd need maybe
> 10 x 1.8GHz worth of cpu before I have any free cycles to devote
> to the user interface :)

Well, you could look into mainboards with 2 or 4 CPU sockets.  Tyan
probably has something.  

> Maybe I'm just clueless this month and the list is too busy to beat
> me about the head with it.

Anyone running geli+zfs is sitting there waiting for the user
interface to update their screen.  :-)

>>> The data disks are hanging off a dumb ata133 pdc20269 card

Ok, here is a REALLY UGLY way to slow the disks down:

for disk in ad2 ad4 ...
do
	atacontrol mode disk UDMA66
done

Not a good solution.  If one transfer is eating your user interface,
with 8 drives you'd have to set the speed very very low.  You really
want a way to set the priority.

BTW, if your data is so important that encryption is required,
"checksum all the way back to the uberblock", etc. etc. you
might want to upgrade the disks from pata to at least sata.
Pata doesn't do error detection on the control info, so
it could happily write your bits to the wrong sector.  Sata
checks the control info as well as the data.

Current 7200 rpm sata speeds:

dd reading the bare drive, no filesystem, at fast end of the drive:

                        extended device statistics  
device     r/s   w/s    kr/s    kw/s wait svc_t  %b  
ad18     1871.1   0.0 118303.1     0.0    2   0.8  98 

dd reading a file from FFS:
28125720000 bytes transferred in 264.921699 secs (106166162 bytes/sec)

Or you could go with the green disks that spin slower.