Re: Tool to compare directories and delete duplicate files from one directory

On 5/15/23 23:26, Sysadmin Lists wrote:
>> ----------------------------------------
>> From: David Christensen <dpchrist@holgerdanske.com>
>> Date: May 15, 2023, 1:43:38 AM
>> To: <questions@freebsd.org>
>> Subject: Re: Tool to compare directories and delete duplicate files from one directory
>>
>>
>> I looks like your script only finds duplicates when the subpath is
>> identical (?):
>>
> Yeah. Wasn't that the original problem description? I went off the example
> given by Paul earlier in this thread, and it looked like only files with
> matching subpaths were being considered (because the OP accidentally rsync'd
> files from a source to a bunch of destination dirs).
>

Glad to see this thread has turned into an interesting discussion....


Just as the OP :-) I will clarify....

There was no accidental rsync in place.


Due to lack of storage my files where basically all over the place on 
different zpools. The problem is that most of those were on iscsi drives 
(all running Freebsd), so I needed to get them in a single place. Of 
course as the files where all over things became a mess.

I bought a few new drives and created a new zpool just for this case. So 
virtually I had to sync the multiple directories to a single 
destination. *but* of course I didn't use the --remove-source-files 
option as I didn't want things to be destructive.


But then I needed the extra space too and that's where this post came from.


Regards,


Kaya