Summary: 12TB GEOM stripe, newfs, then fsck: cannot alloc 768053748 bytes for blockmap

[Frantisek Rysanek]

On 9 Jun 2005 at 9:46, Alex Zbyslaw wrote:
> >A local friend has suggested to increase the block size
> >
> It ought also make sense if you are serving up *large* files (didn't you 
> say video/audio?).
...
> I don't have anything on your scale (23Gb 
> of document database pales into insignificance against 12Tb :-) ) 
> 
I myself have never met that sort of data volume, either.
I actually just work for a reseller / assembly shop.
We needed to test these three units before shipping
them. FreeBSD 5 is the only free UNIX known to me that 
handles >2TB "out of the box" on a 32bit x86 PC.
(I didn't bother to install a 64bit OS on the dual
Nocona box used for testing.)

While I was at it, I thought I could give GEOM a try.
Never used it before.
Took me 5 minutes to find the example in the manpage.
Then it took two commands in the shell and the block device 
was up. Unbelievable. Me being a FreeBSD illiterate.

I seem to recall that the boxes will be used for medium-term 
video archival at their final destination, separately.

Thanks again for your help :-)

Frank Rysanek


P.S.: It makes you wonder. A disk's transfer rate grows 
roughly with the square root of the disk's capacity
(== storage density), if other conditions are unchanged (RPM, 
number of heads etc.) It used to take three minutes to read my 
first 100MB hard drive. It takes 40 minutes to over 1h to read 
the current desktop drives - alone, at their respective 
maximum sustained rate.
The RAID controllers have a lower total throughput than the 
sum of their drives - maybe 150 MBps per unit of 16 drives.
Using three separate SCSI busses for the three RAID units, 
maybe I could crank them up to 400 MBps of sustained transfer 
rate. That's about 8 hours just to read the whole 12TB thing.
Think of sipping a swimming pool with a straw.