What's a good program for recursively managing duplicates?

MeadowP

n00b
Joined
Oct 25, 2006
Messages
24
If I have a huge folder named "Folder" with multiple duplicate files (within the main folder and in sub-directories) and I want to make a new copy of "Folder" without any duplicates whatsoever, how can I do it?

The only reason I need a program for it is because there is no way I can keep track of so many files in my head (100,000 is a low estimate).

I only need the basics, ie: if they have the same filename, check the filesize. if they have the same filesize, they are duplicates. if not, rename one of them when you move them to the new "folder"

Should this be moved to the Programming forum?
 
Is there the chance of finding a duplicate of a file that is in the main folder in a sub-folder?
ie:
  • Folder
    • Sub Folder
      • File
  • File
wher File's are duplicates.

If this type of scenario is possible, it could still be done, but each file would have to be checked against every file that has already been copied. Also you could create a sort of index file and use md5 or another hash to be absolutely sure you have a duplicate file although that complicates the script somewhat ;)

Only other thing to consider would be the platform this is on which will definately effect how it is scripted.
 
Back
Top