comments on newfs raw disk ? Safe ? (7 terabyte array)
Eric Anderson
anderson at freebsd.org
Sat Feb 10 07:12:16 UTC 2007
On 02/09/07 18:05, Antony Mawer wrote:
> On 9/02/2007 3:38 AM, Eric Anderson wrote:
>>> Only "collateral" problems because of the partition size: a regular
>>> (non-softupdates) fsck will take a LONG time to finish and eat a LOT of
>>> memory while it's doing its stuff. You'll need a lot of swap space (1GB
>>> per TB? someone had empirical numbers on this, I'm sure) if you think
>>> you'll need to fsck it entirely. Creating snapshots will also take a
>>> long time on it, and you probably want to search the lists for
>>> recommendations about creating snapshots in a second level directory in
>>> order not to block the root directory. Related to this is
>>> background-fsck which works by creating snapshots, so you'll probably
>>> want to disable it.
>> I have 5 10Tb file systems (and some 2Tb ones, but who cares about those
>> tiny things? :)), and I can tell you that an empty huge file system is
>> pretty easily fsck-able, but a full one will kill you. It greatly
>> depends on how many files (inodes) you have used on the file system. If
>> you have a massive amount of small files, you'll be eating up a ton of
>> memory. My 'rule of thumb' for my data (which averages to about
>> 16k/file) is 1G of memory for each 1Tb of disk space used. So, on a
>> 10Tb file system, if I ever want the fsck to complete, I need an AMD64
>> box with *at least* 10G of memory, plus a lot of time. A *lot* of time.
>> By 'a lot', I mean anywhere from a day, to several days.
>
> Has anyone looked at the changes in DragonFly that were made in the 1.8
> release? I noticed the other day, reading the release notes
> (http://www.dragonflybsd.org/community/release1_8.shtml) the point:
>
> "Greatly reduce the memory allocated by fsck when fscking filesytems
> with a huge number of directories (primarily mirors with lots of
> hardlinked files). Otherwise fsck can run out of memory on such
> filesystems."
>
> Whether or not this helps in the general case, or only the scenario
> described, I do not know... but it would be interesting for someone with
> enough filesystem-foo to have a look at!
>
> --Antony
I'll check that out - didn't know about it, thanks!
Eric
More information about the freebsd-fs
mailing list