[solved] Software or script to find duplicates and replace by symlinks

postcd

Weaksauce
Joined
Nov 24, 2016
Messages
96
Hello, i am on Windows 10.

i have two HDDs and they have some common (duplicate) files between them (usually movies or videos).
I am already using CClearner to find duplicates and remove them, but in this case, i do not want just remove duplicates, but i need to replace duplicates by a symbolic link to a original file.
Original file would be one that belong to certain HDD or folder(and its subfolders) or the file that is oldest.

I googled and found some software, but none seem to be working the way, it allows quick completion (maybe under 1 hour) of this task,i need to repeat this task maybe once per month. The amount of files checked would be large, (maybe 200,000, unless script/app is made to work only with certain file types). And comparison if files are same would have to not be done based on file name, but maybe on some hash of the file or size in bytes and extension.
 
Hello, i am on Windows 10.

i have two HDDs and they have some common (duplicate) files between them (usually movies or videos).
I am already using CClearner to find duplicates and remove them, but in this case, i do not want just remove duplicates, but i need to replace duplicates by a symbolic link to a original file.
Original file would be one that belong to certain HDD or folder(and its subfolders) or the file that is oldest.

I googled and found some software, but none seem to be working the way, it allows quick completion (maybe under 1 hour) of this task,i need to repeat this task maybe once per month. The amount of files checked would be large, (maybe 200,000, unless script/app is made to work only with certain file types). And comparison if files are same would have to not be done based on file name, but maybe on some hash of the file or size in bytes and extension.

What you're describing sounds like a job for a database instead of symlinking physical files.
 
Thanks DGMurdock, here i am reporting the results:


in short the Duplicate commander worked good, except my app does not work with resulting .symlink file. It does not follow windows links (.symlink files) and hardlinks are not possible because these are cross HDD files. Second app that also worked is NoClone (google this hash to DL: E7466F6855BAE8801469AA689F00638F956EA3A0) and other apps not worked as i describe further:

fequal - no this software does not work as i described. I already tried it, but i tried once again and it do not find any identic files, though they have same size in bytes and CSDiff tool shows them as identic by content comparison. Also "“there is no option to deal with duplicate files automatically”"
doubles - it do not support linking, just deleting and shows dup files within one directory which is unwanted
Duplicate Commander - it is harder to learn it, but it works, except the resulting links are not treated as media files by my app that uses them, the app somehow not follow the link, though in WIndows, the type is ".symlink" and it works to open such symlink file inside media player.
jdupes - nice tool, but does not support links across different drives, only hard links across one drive

Is there any other solution on how to create "link" in Windows in an automated way, except .symlink files ?
 
Last edited:
Back
Top