find -exec surprisingly slow

[Matthew Seaman]

On Sun, Aug 15, 2004 at 08:31:43AM +0930, Paul A. Hoadley wrote:
> Hello,
> 
> I'm in the process of cleaning a Maildir full of spam.  It has
> somewhere in the vicinity of 400K files in it.  I started running
> this yesterday:
> 
> find . -atime +1 -exec mv {} /home/paulh/tmp/spam/sne/ \;
> 
> It's been running for well over 12 hours.  It certainly is
> working---the spams are slowly moving to their new home---but it is
> taking a long time.  It's a very modest system, running 4.8-R on a
> P2-350.  I assume this is all overhead for spawning a shell and
> running mv 400K times.  Is there a better way to move all files based
> on some characteristic of their date stamp?  Maybe separating the find
> and the move, piping it through xargs?  It's mostly done now, but I
> will know better for next time.

Yup.  Invoking mv 40,000 times is not particularly efficient.
Something like this would have been better:

    find . -atime +1 -print0 | xargs -0 -J % mv % /home/paulh/tmp/spam/sne/

xargs defaults to taking up to 5,000 arguments from it's stdin to
generate the mv commands (or up to ARG_MAX - 4096 = 61440 bytes), so
that would have done the job with only 8 or so invocations of mv.

	Cheers,

	Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.                       26 The Paddocks
                                                      Savill Way
PGP: http://www.infracaninophile.co.uk/pgpkey         Marlow
Tel: +44 1628 476614                                  Bucks., SL7 1TH UK
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-questions/attachments/20040815/14578f17/attachment-0001.bin