defragmentation in FreeBSD 4.11

[Lowell Gilbert]

Freminlins <freminlins at gmail.com> writes:

> Gayn Winters wrote:
>  
> > What I get from reading this article is that if the use of the file
> > system is to store lots of small files, then use a small block size.  Am
> > I missing something?
> 
> No and yes! There is a minimum block and fragment size. In this case
> there were not enough contiguous fragments to enable an 8K file to be
> created. Without checking I believe Solaris uses 8K blocks.

That's correct, and also note that Solaris does not offer the
flexibility in block sizes that is available from FreeBSD.

And it is correct that a small block size will vastly reduce the
ability to hit this fully-fragmented condition.  At a cost of higher
overhead, of course.

> > Also, in most situations, buying a big enough disk is far better than
> > worrying about what happens when a not-big-enough disk starts to get
> > full.
> 
> Indeed. But... in the case I linked to there was apparently plenty of
> free space, just not enough free contiguous space. The author also
> implies that a bigger disk would not solve the problem:
> 
>   "it creates and deletes tons of small files and thus the fragmentation 
>   over a period of time."

Where "plenty of free space" is a large absolute number, but still a
small fraction of the disk.  Full fragmentation of the empty space
becomes exponentially less likely as the amount of free space
increases.

Note that when deciding where to allocate a fragment, there are two
possible policies the filesystem can follow: either a space-efficient
method that is more likely to require moving data when a file expands,
or a time-efficient method that will create new fragments more often.
FreeBSD can automatically shift between the two as the filesystem 
moves over or under a threshold "percent full" value.  Solaris, I
believe, needs to be explicitly changed between the two.

> As mentioned though I have never seen this myself despite running very
> busy mail and web servers with what must be billions of files being
> created/deleted in that time.
> 
> It certainly grabbed my attention so I thought it may be of interest to others.

It's an academic curiosity, but not an issue to take into account in
planning your data center.  Planning your filesystem layout, however,
may well be.

Be well.