fsck memory usage

[Julian Elischer]

On Wed, 30 Jul 2003, Julian Elischer wrote:

> 
> 
> On Wed, 30 Jul 2003, Brooks Davis wrote:
> 
> > Does anyone have a formula to give fsck memory requirements based on
> > disk size and newfs arguments.  I'm going to be building some large
> > (3.3TB) file systems and I'd like to figure out if I'm going to need
> > to go 64-bit to have sufficent address space to fsck them.  The mean
> > filesize will be pretty big so I will be able to set bytes/i-node
> > fairly high.

> 
> I needed 700MB of memory in teh fsck process to be able to fsck a 1 TB
> filesystem.
> (actually that's what I needed for pass 1.. I ended up 
> splitting it into 2 smaller ones after fsck's
> first 2 hours only got to pass 2..)
> 

I came to the conclusion that the maximum you could fsck on a 32 bit
machine was somewhere around 3TB (maybe a bit less) if you had 4GB of
ram in the machine and a 2GB virtual space for the process

I think it should be possible to produce a 'streaming' fsck
that uses files of orderred metadata (i.e. sorted records)
to check much larger filesystems, but you'd need a filesystem set asside
specifically to hold the workfiles as they'd be bretty damned big..

Think of the old tape based mechanisms that used tbe done when tapes
were bigger than ram.. e.g. merge sorts and similar..
files would be made containing all referenced inode and blocks and
directorie entries etc.
and then they would eb sorted into approriate orders (possible several)
and a 'merge/select' pass woudl be made on them to generate
output files containing lists of 'good' and 'bad' references etc.