filesystem: 12h to delete 32GB of data (4 million files)

Bill Moran wmoran at potentialtech.com
Wed May 6 15:10:25 UTC 2009


In response to Olivier Mueller <om-lists-bsd at omx.ch>:
> 
> Yes, it is one of the best options. My initial goal was to delete all
> files older than N days by cron  (find | xargs | rm, etc.), but if each
> cronjob takes 2 hours (and takes so much cpu time), it's probably not
> the best way.  
> 
> I'll make some more tests on an test-server later this week and speak
> with the devs. Thanks again for your very constructive feedback! 

Based on your comments here, it really sounds like your devs need to
implement some sort of cache cleaning algo into their code.  If it's
just deleting the oldest files, then you could probably run it far
more frequently if you simply created a new cache directory each
hour, and deleted the previous one.

Honestly, I'm really confused -- if you can just throw away the cache
each night, then why are you caching to begin with?  If you just need
temp files, why doesn't the app clean up its temp files when it's
done with them?

If you have access to the developers, I think you'll be able to come
up with a much better solution by working with them.

-- 
Bill Moran
http://www.potentialtech.com
http://people.collaborativefusion.com/~wmoran/


More information about the freebsd-performance mailing list