Large filesystems with massive inodes on BSD5.2

[Doug White]

On Wed, 7 Jan 2004, Tom Arnold wrote:

> I'm building a fileserver thats going to house a large number of small
> files ( no, not an innd server but same idea sorta ).
> I've got a 3.3TB RAID hanging off a FreeBSD 5.2-RC box ( cvsup'd to RELENG_5_2
> monday night ).
> I've broken it up into 2 1TB filesystems newfs'd "stock" and the remainder
> a 1TB partition and a 300gig partition.

wow. :)

> Now the fun part.
> newfs'd the final two partitions -b 4096 -f 512 -i 512 as they are going to
> have many many small files on them.  Needless to say if the server reboots
> unexpectedly fsck takes a long long to run.  So this is why for a production
> server I'm playing with 5.2

Unless you're going to fill the filesystem with 512 byte files, you
probably have the sizes set too low.  You're probably OK with the default
of 4096 bytes/inode.

> Newfs'd UFS2 softupdates enabled.  Crash the box to force a dirty reboot.
> Boots finally after about 20 minutes of playing with snapshots.  About
> 4 hours later I get this lovely error :
>
> Jan  7 02:58:12 tarnold2 fsck: /dev/da1s1e: CANNOT CREATE SNAPSHOT /export/database/.snap/fsck_snapshot: File
> too large
> Jan  7 02:58:12 tarnold2 fsck:
> Jan  7 02:58:12 tarnold2 fsck: /dev/da1s1e: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
> Jan  7 20:21:24 tarnold2 su: tarnold to root on /dev/ttyp0
>
> da1s1e is the largest or the two filesystems with huge numbers of inodes.

Heh. Thats funny.  I'd like to see Kirk's reaction to that.

> Am I running into some kind of filesystem limits by having 2billion inodes
> on it?  Am I even asking on the correct mailing list or should I ask this
> on -Current?

This is the right spot.  Interesting that fsck fell over creating the
snapshot due to E2BIG.

-- 
Doug White                    |  FreeBSD: The Power to Serve
dwhite at gumbysoft.com          |  www.FreeBSD.org