Default inode number too low in FFS nowadays?

[Borja Marcos]

On Nov 2, 2011, at 2:13 PM, Jeremy Chadwick wrote:

> On Wed, Nov 02, 2011 at 12:57:33PM +0100, Borja Marcos wrote:
>> Today I?ve come across an issue long ago forgotten :) Running out of i-nodes.
> 
> The last time I dealt with this was about 3 months ago, on a Netapp
> filer; hardly forgotten.  :-)
> 
>> The condition was triggered on a 8 GB /usr filesystem which includes the ports tree, on which I have compiled several ports and their dependencies.  I know, maybe not the best practices, it's a machine being used to test a couple of thingies, but I wonder how many newbies can run into such a problem. I guess many.
>> 
>> Filesystem     1K-blocks    Used        Avail           Capacity iused     ifree      %iused  Mounted on
>> /dev/ad1s1e   8245660     5308960 2277048    70%        507099  61219   89%       /usr
>> 
>> Now i-nodes aren't completely full, as I have deleted /usr/obj (turns out I had made a make world as well).
>> 
>> Looking at the number of i-nodes per /usr subdirectory, I have noticed that, wow! /usr/ports consumes A LOT of them. 
>> 
>> freebsd9-borja#	find . -print | wc -l
>>  405481
> 
> "find -x /usr/ports | wc -l" on my system returns 146420.

My find was on /usr/ports after having compiled the mentioned ports. It's using so many inodes after compiling the mentioned ports.

> Your 8GB filesystem is only ~21% the capacity of mine.  So if we do some
> simple math, your /usr should really be showing (assuming yours is as
> "bare bones" as mine) around 26% inodes used.  Yet what you have is 89%.
> 
> You probably just have a lot of other software on /usr that "abuses"
> lots of individual files.  You list off what ports you built from
> source, which means you have lots of "build cruft" laying around en
> masse (work/ directories).

Yes, of course, I am aware of that. But a mother-of-all-stuff-and-not-too-large /usr filesystem can create problems if you build plenty of ports.

The issue is: given that tasks such as port building now consume inordinate amounts of inodes, is the current default value reasonable? This is an issue on which I can imagine a newbie having problems, hence my suggestion to increase the default number.

> Really?  That isn't what the default installer does (re: single
> filesystem), and hasn't since...  well, as long as I can remember
> (2.2.8).

Sorry, I'm wrong on that.

> I wouldn't recommend a /usr/ports filesystem.  It solves nothing.
> Imagine: /usr/ports can now run out of inodes while /usr won't.  What
> would this address?  It would decrease impact to applications creating
> new files on /usr.  How often do you have that happening on your system?
> For me it's extremely rare.  For comparison, /var gets a lot more
> activity.

Yes, but there are ports that use /usr as far as I know. Anyway, it would be an issue if the user is just building ports.

> Adding a /usr/ports filesystem also provides zero scalability.  Meaning:
> say someone decides "8GB for /usr/ports is enough" during the installer
> and so on.  Then down the road, we run out of space, or inodes, or
> whatever else.  Oops, what now?  Symlinks all over the place?  Yeah, not
> going to happen.

Good point. 

> We were already in this boat regarding the root filesystem size, which
> for quite some time defaulted to 512MBytes, which was not enough.
> Today, the default size is 1GBytes, which is enough for a kernel built
> with debugging symbols and so on, but tends to reach ~55-58% used.
> 
> There's a reason /usr on FreeBSD defaults to "all the remaining space on
> the disk" if you pick the defaults/auto.  Surprise.  :-)
> 
> The summarised version is:
> 
> 1. You have control over this yourself: newfs(8) -i flag.  You can even
> input this flag during sysinstall when building a new system.
> 
> 2. When you built the system, you chose to assume /usr should be 8GBytes.
> Your usage habits may have changed, software gotten larger, etc..  It
> may be time for you to re-evaluate how you create your filesystems and
> what sizes you choose.  You chose something that didn't scale.  :-)

I'm not complaining about the issue, I know well I have to clean up, etc. Been using FreeBSD since 2.0.5 ;)
But I'm wondering if it could be an issue to others.

And the /usr was small because I need a lot of room for /var. 

> 4. For a limited-space (not just size but inodes!) filesystem, consider
> using packages (pkg_add -r) for anything that does not require
> customisation during the build/configure phase.  For example, for quite
> some time now on my systems, I've always installed the package versions
> of perl, python27, cmake, gettext, pcre, etc. solely because they are
> "key softwares" that I hate building from source.

Unfortunately there are several key ports that *do* need compile time customization. Moreover, I could be wrong, as I've been compiling ports for many, many years, but do we have a mechanism to document which compile-time options were used to build a package? I'm not aware of it.

Just a suggestion anyway. The calculation for the default number of inodes was right many years ago, in my opinion the number is too low nowadays.

And yes, I know this isn't an issue with ZFS, been using it for a long time as well. But FFS is still very widely used.




Borja.