fsck memory usage

[Terry Lambert]

Brooks Davis wrote:
> Does anyone have a formula to give fsck memory requirements based on
> disk size and newfs arguments.  I'm going to be building some large
> (3.3TB) file systems and I'd like to figure out if I'm going to need
> to go 64-bit to have sufficent address space to fsck them.  The mean
> filesize will be pretty big so I will be able to set bytes/i-node
> fairly high.

The amount of memory really depends on which passes you end up
having to run, since it has to "remember" state information.

By far, the thing that takes the most time is the cylinder
group bitmaps, which it has to assume are dirty, and then
correlate all the blocks in every file on the disk vs. the
bitmaps to make sure that a particular bit which is set is OK
to unset.

This is also the pass that takes the most memory, in my own
experience, unless you have a single very very large file that
lost its directory entry but has a referenced inode, and you're
moving it to lost+found.

About 6 months ago I suggested a change to the on disk layout
and the addition of a soft dependency, so you you could (1) know
which CG bitmaps are dirty and (2) put in a concurrency stall
point to force CG bitmaps out to disk, once they hit the high
water mark.

You would still have to pass through the entire directory stucture
once in order to clean the CG bitmaps, but with a smaller list of
bitmaps, there would be a heck of a lot less memory (and swap)
that was consumed in the process.

For this to work, you have to have a "dirty CG bitmap list"; my
initial feeling was that you could put a magic number in inode 1
(used to hold whiteout information; formerly used to hold the
bad block list) and steal a fixed size block from the front of
it, and still maintain your whiteouts.

In any case, the answer to "how much memory" really depends on
(1) how many CG's you have, (2) the size of a single bitmap, (3)
the largest file you have on your disk at one time, and (4) the
amount of "fill" that your directories have.

Basically, any one of these could, if it were "just right", push
the number higher than anoy of the other pushes it.  So basically
it's a "highest tide" function, if you want to calculate how much
it's going to take for a given instance, and it's "really, really
a lot", if you don't limit the size of a single file on the volume,
and you just want to calculate out the max it will take.

Any way you look at it, though, the answer will also vary with
wersion of fsck (there's been some internal reorganization at
various points in the past), so you're not going to be able to
make up a number that applies to all BSD's.

-- Terry