MOTUnation.com

I have many drives holding many years of work, and some of my files are archived 3 or more times in multiple folders on multiple drives, so I'd like to archive each file once on one "Master" drive, then clone that drive as an offsite backup. When that's done, I'll erase the original drives with all the extra backups.

I've tried a number of apps that supposedly find duplicates across multiple drives and allow me to delete any extraneous copies, but I've never had enough confidence in the apps to go through with the delete process, so I'm curious if anyone has really found a good, safe way to do this? If anyone has any thoughts, or truly reliable (Mac only) app suggestions before I resort to doing it all manually, I'd definitely appreciate it!

I've done something similar using the Finder copy function. At least until recently, when the Finder sees a duplicate on the source of the copy, it tells you that, and whether it is newer or older than the version on the destination, and lets you decide whether to skip copying that files, or to overwrite the file on the destination. Presumably, you would choose to overwrite if the version on the source is newer than the version already on the destination.

If you follow that process to copy all the files you have to the root folder of the new destination disk, eventually, you will have the latest version of all your unique files on the new disk, and all the duplicates and old versions on the old disks, which you can then erase after you clone the new disk.

I think you have to do this in Snow Leopard. Somewhere along the way, Apple changed the Finder copy process so you can't make decisions to overwrite or skip on a file by file basis. Lion might work. It would be easy to set up a couple of folders of test files and see what works.

Thanks bayswater for the suggestion. I forgot that the older OS X's did that. That was so much better than the way it currently works in Mavericks.

The other thing that makes this challenging is that many duplicates are buried several folders deep on the drives, so I can't just drag files between drives and have the OS see there's a duplicate conflict (or at least I don't know a way to make the Finder look into all folders on the drive during the file dragging process). That was the appeal of trying a "Duplicate Finder" app, but as I said, the ones I've found make me nervous, so perhaps the manual approach you suggested is still the best.

to find and remove duplicate files you may use DuplicateFilesDeleter

I use a utility called "Chipmunk" which can search drives or folders for duplicates. It scans every file and even identical files with different names are identified.

It shows the results in a folder tree and a list. You can trash the duplicates manually from the list, or you can select folders and tell the program to trash any duplicates that are either outside the selected folder, or inside the selected folder.

The other thing is that with DP projects, if you have multiple versions it might keep some older versions (because they are different). So you may end up with older projects with a few random files in the audio folder.

Honestly, for us folks working with audio, and samples, and recordings, and mp3's, etc. etc.; this is a nicely designed tool.

http://furry-rodents.com/index.html

stubbsonic wrote:I use a utility called "Chipmunk" which can search drives or folders for duplicates. It scans every file and even identical files with different names are
...
http://furry-rodents.com/index.html

I'm going to check that out. It would be nice to free up some drive space.

The one thing I will add is to do this when you have time on your hands. It's easy to be too aggressive during the process. I've been known to accidentally trash important files.

I wouldn't empty trash till after a reboot.

If you find that you made mistakes during the process, you can always do a complete restore from Time Machine and try again.

Hey thanks everyone! I'd given up on this since my original post, thinking I was the only one in the universe who ever had the problem. I'll definitely check out Chipmunk and DuplicateFilesDeleter... and proceed carefully.

DuplicateFilesDeleter did a great job with me and my employee. It works very well and removes duplicate files quickly and very best result oriented solution.