UFS2 snapshots on large filesystems

[Eric Anderson]

Scott Long wrote:
> user wrote:
> 
>> Hello,
>>
>> Considering a PC server running FreeBSD with 4 400 GB hard drives 
>> attached
>> to a hardware raid controller doing raid-5.
>>
>> So this will present itself to the OS as a 1.2TB filesystem.
>>
>> Any comments on taking one or multiple snapshots of a filesystem of this
>> size ?
>>
>> Given current disk capacities, I would not exactly consider this 1.2TB
>> filesystem a "large" one ... any comments on say ... a 6-8 TB filesystem
>> and making one or more snapshots of it ?
>>
>> Assume they are marginally busy - perhaps a 5-10% data turnover per 
>> day...
>>
>> Thanks.
>>
> 
> The UFS snapshot code was written at a time when disks were typically 
> around 4-9GB in size, not 400GB in size =-)  Unfortunately, the amount
> of time it takes to do the initial snapshot bookkeeping scales linearly
> with the size of the drive, and many people have reported that it takes
> considerable amount of time (anywhere from several minutes to several 
> dozen minutes) on large drives/arrays like you describe.  So, you should
> test and plan accordingly if you are interested in using them.

I have several 2TB filesystems, which I do snapshots on.  I can report 
that it indeed takes a long time to run, but works nevertheless.  One 
thing to keep in mind though - fsck'ing a 2TB filesystem can take 2GB of 
memory (depends on how many files you have, and a few other factors).  I 
have 4GB of memory in this box, and what I've seen so far is between 
1.5GB and 2.5GB of memory required for fsck to run smoothly on a single 
partition.  You definitely also want to turn off background fsck, unless 
you have extreme amounts of time on your hands.  An fsck of a 70% full 
2TB filesystem with a ton of files on it takes many hours, so I often 
mount the filesystems unclean (FreeBSD lets me do this) with rw, and 
continue my work until I can unmount, fsck, and remount the fs.

This brings me to UFS Journaling - Scott, how's it coming along?  I know 
you've been busy with the 6.0-RELEASE (great work by the way!!), but I'm 
itching for this.  Is there anything I can do to help?

Eric



-- 
------------------------------------------------------------------------
Eric Anderson        Sr. Systems Administrator        Centaur Technology
Anything that works is better than anything that doesn't.
------------------------------------------------------------------------