Terabyte harddisks, GELI, AMD64, Samba and Zen...

[Ivan Voras]

Solon Luigi Lutz wrote:

>               -------Sequential Output-------- ---Sequential Input-- --Random--
>               -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
> Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
>          1000 37922 31.5 48829 12.0 23827  5.0 30054 36.0 43666  6.0 1120.0  2.5

Interesting - your CPU doesn't look overwhelmed much.

> But I have not been able to get more than 17 MB/s when using Samba to
> transfer data - FTP maxes out around 27 MB/s. I also tried that on
> i386 32-bit and found it to be 8 MB/s and 17 MB/s - not good, but
> nothing to worry.
> 
> What made me feel really uncomfortable was the fact, that just some
> minutes ago some 3000 1GB files suddenly disappeared while working
> in a directory. They where gone, but the filesystem did not report
> some additional 3TB to be free and after unmounting and remounting the
> filesystem the files were back where the belonged...


> This just happened some minutes later again, now with only 2500 files
> dis- and -reappearing again.

This can mean either file system corruption (which fsck fixed on boot?), 
a bug (read cache bug, where the memory representation of the directory 
doesn't agree with on-disk state) or a hardware memory error. Of these, 
hardware errors are easiest to check in your case. Download a memtest86 
boot CD ISO, burn it and let it run for a few hours. Next, you can try a 
"full" fsck, which would probably a few last days on such a big array 
(big arrays are inconvenient to have without journaling). If both fail, 
we may look for a bug somewhere.

> Questions until now:
> 
> 1. 10TB as a single volume, too big for good? (fsck time: 30 min with softupdates)

Yes, too big. Softupdates doesn't even do a full fsck - if you tried a 
full fsck it will require about a dozen GB of memory (or memory+swap) 
and take a really long time. If you're not scared of it, you should run 
7-current and re-create the file system with gjournal, or even ZFS.

> 2. GELI unstable on big disks and/or AMD64?

You're the first to complain :)

> 3. Why is Samba so slow?

Search Google... Samba is notoriously slow on FreeBSD, but there are few 
ways to tune it which will help.

> 4. Does the crypto-framwork gain speed advantages from dual-core CPUs?

No, and the same goes for most GEOM classes.

> 5. Will the GPT-stuff change over the next releases in a way I need to
> do DUMP/RESTORE?

I don't think so, except if someone discovers an incompatibility in the 
way FreeBSD handles GPT wrt other OSs. Shouldn't happen.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 249 bytes
Desc: OpenPGP digital signature
Url : http://lists.freebsd.org/pipermail/freebsd-questions/attachments/20070411/5a4c31ed/signature.pgp