Very large directory

[Phillip Salzman]

Thanks Peter (along with everyone else who's responded.)

I've received a couple perl scripts from a few different people.  I can't
use any of them until this evening due to the current load of the machines
though.  Last night I ended up doing a strings on the directory, and taking
its output into a for loop removing the files.

--
Phillip Salzman

> -----Original Message-----
> From: Peter Jeremy [mailto:PeterJeremy at optushome.com.au] 
> Sent: January 20, 2005 3:46 AM
> To: Phillip Salzman
> Cc: stable at freebsd.org
> Subject: Re: Very large directory
> 
> 
> On Wed, 2005-Jan-19 21:30:53 -0600, Phillip Salzman wrote:
> >They've been running for a little while now - and recently we've 
> >noticed a lot of disk space disappearing.  Shortly after 
> that, a simple 
> >du into our /var/spool returned a not so nice error:
> >
> >	du: fts_read: Cannot allocate memory
> >
> >No matter what command I run on that directory, I just don't seem to 
> >have enough available resources  to show the files let alone delete 
> >them (echo *, ls, find, rm -rf, etc.)
> 
> I suspect you will need to write something that uses 
> dirent(3) to scan the offending directory and delete (or 
> whatever) the files one by one.
> 
> Skeleton code (in perl) would look like:
> 
> chdir $some_dir or die "Can't cd $some_dir: $!";
> opendir(DIR, ".") or die "Can't opendir: $!";
> while (my $file = readdir(DIR)) {
> 	next if ($file eq '.' || $file eq '..');
> 	next if (&this_file_is_still_needed($file));
> 	unlink $file or warn "Unable to delete $file: $!";
> }
> closedir DIR;
> 
> If you've reached the point where you can't actually read the 
> entire directory into user memory, expect the cleanup to take 
> quite a while.
> 
> Once you've finished the cleanup, you should confirm that the 
> directory has shrunk to a sensible size.  If not, you need to 
> re-create the directory and move the remaining files into the 
> new directory.
> 
> -- 
> Peter Jeremy
>