Merging Folders

[bayswater]

If you work on DP projects using more than one computer, you'll have a means of keeping the two computers in sync. If it's a DP project folder, just copying the newer version over the older version will do, as long as one is consistently newer than the other. But if you have to update larger more complicated folders, or you have done different work on different computers, what to do?

I've come across two solutions with a search. First, with Finder, Alt-Dragging one of the folders over the other will merge contents, and if there are duplicate files, will keep the most recent version in the destination folder. But it only checks whether there are duplicates with different dates in the highest level of the directory. So if you have a DP project on two computers, edits on files not at the highest level will not necessarily be checked to identify the most recent version. Finder might replace a new file with an older version, or not copy over the file at all.

Supposedly the Terminal "Ditto" command deals with this by doing a recursive search through all the directory levels. But having tested this, and despite claims that it always keeps the most recent version of a file, I find that this is not necessarily so. In my tests, it kept the version in the originating folder and replaced the version in the destination folder, even if the destination had a new version. This might be something that changed between OS C 10.13 and OS X 10.14.

So neither of these solutions will work reliably across all the situations where you work on a project on two computers.

I've searched the App Store for a utility without finding anything that deals specifically with this.

Does anyone have a simple reliable process?

[monkey man]

Hey Brother Stoivo,

In CCC, if you select "Copy Some Files" instead of "Copy All Files", "... only items that have been modified since the last backup task will be copied."

Could this be a solution for you?

[bayswater]

Hey Brother Stoivo,

In CCC, if you select "Copy Some Files" instead of "Copy All Files", "... only items that have been modified since the last backup task will be copied."

Could this be a solution for you?

Thanks but no, probably not. The same file might be modified too and might be more recent. The more recent file would be overwritten by the older version. It needs to be something that compares mod dates in source and destination and keeps the more recent of the two. As i mentioned the finder will do this but only at the top level of the folder hierarchy.

[HCMarkus]

A "simple reliable process" would seemingly be to write to a shared project drive at the end of every session and always start work from that drive. Could be networked storage. Or even sneaker net.

Then always be sure to back up that drive!

[bayswater]

A "simple reliable process" would seemingly be to write to a shared project drive at the end of every session and always start work from that drive. Could be networked storage. Or even sneaker net.

Then always be sure to back up that drive!

I do that now. My routine is to move a project to the internal drive, and then move it back when I'm done. But now and then, there are more complicated pairs of folders where several updates have been made on each over a longer period, perhaps by different people. Of course it can be done but it would be nice if there was a reliable but mindless way to it. And there must be one.

[monkey man]

Hey Brother Stoivo,

In CCC, if you select "Copy Some Files" instead of "Copy All Files", "... only items that have been modified since the last backup task will be copied."

Could this be a solution for you?

Thanks but no, probably not. The same file might be modified too and might be more recent. The more recent file would be overwritten by the older version. It needs to be something that compares mod dates in source and destination and keeps the more recent of the two. As i mentioned the finder will do this but only at the top level of the folder hierarchy.

So when CCC says, "... only items that have been modified since the last backup task will be copied", what does it mean then brother?

If it meant "new files", it'd have said so, but it specifically said, "modified".

[bayswater]

So when CCC says, "... only items that have been modified since the last backup task will be copied", what does it mean then brother?

If it meant "new files", it'd have said so, but it specifically said, "modified".

Consider this. You have a project folder for MM_Kazahkstan_Marching_Band_Hits. You copy it for a road trip and modify one of the files. You send that file to your assistant producer back in Melbourne who puts it the studio copy of the folder, and makes further edits. You get home, and CC your travel copy over to the studio copy. Which of the two modified files does it keep? Does it keep the most recent (the studio copy) or the one from the source (the travel copy). It looks like it will replace the most recent copy made in the studio with the older copy you did on the road trip.

The Finder keeps the most recent, but only if its in the root level of the folder. Ditto keeps the source copy.

[bayswater]

I tried some of the sync utilities on the App Store, but they were a bit vague on how they identify files have changed, what they do with old files that are removed, and keep some of the useful features for a paid version.

But FreeFileSync seems to work. https://freefilesync.org The interface reminds me of OS 9, but it's pretty easy to use. Drag and drop the folders into the left and right side of the window, and click on Synchronize Two Way. It consistently copied newer versions to both folders, copied files that were missing from either folder, moved old deleted versions to a selected folder, and worked on multiple levels of the folder hierarchy. It triggers on file size and date.

The downside is it only does pairs of folders so if there are three versions of a folder, it has to be run three times. There are other that sync multiple folders but I can figure these out.

Looks like it is a wrapper for rsync. There were other similar apps but these appear to only copy over portions of the files that are different rather than the whole file. This would be an advantage in situations with thousands of very large files, but seems a bit risky to me.

[monkey man]

Thank you for the explanamation Brother Stoivo.

Putting it in MonkeyLabs Industries™ terms such as the 3rd-World March-By Series™ really helped the MonkeyLabs Debunkulating Reconfabulator™ make sense of it.

[HCMarkus]

Thank you for the explanamation Brother Stoivo.

Putting it in MonkeyLabs Industries™ terms such as the 3rd-World March-By Series™ really helped the MonkeyLabs Debunkulating Reconfabulator™ make sense of it.

I would think it more comprehendible if the Reconfabulator made scents of it, Monkey.

[mothra]

So when CCC says, "... only items that have been modified since the last backup task will be copied", what does it mean then brother?

If it meant "new files", it'd have said so, but it specifically said, "modified".

Consider this. You have a project folder for MM_Kazahkstan_Marching_Band_Hits. You copy it for a road trip and modify one of the files. You send that file to your assistant producer back in Melbourne who puts it the studio copy of the folder, and makes further edits. You get home, and CC your travel copy over to the studio copy. Which of the two modified files does it keep? Does it keep the most recent (the studio copy) or the one from the source (the travel copy). It looks like it will replace the most recent copy made in the studio with the older copy you did on the road trip.

The Finder keeps the most recent, but only if its in the root level of the folder. Ditto keeps the source copy.

That sounds like you really should just pick a central location somewhere on a network and keep one copy to work on there. You're making things harder for yourself this way, especially if folks back at the studio are off modifying files you're still working on. You're never going to find a backup solution that covers that.

[mikehalloran]

NFS can handle that. Just ask anyone — except those trying to implement it for widespread use.

https://en.wikipedia.org/wiki/Network_File_System

The big problem with NFS is that it requires blazing internet speed unless in a closed network (US Navy Mark VI gun) or the task is relatively simple such as CATV servers and cable boxes in your home. Let's just say that I helped make both happen and we'll leave it at that.

[monkey man]

... and from Mike's highfalutin comments to the bottom of the barrel:

Thank you for the explanamation Brother Stoivo.

Putting it in MonkeyLabs Industries™ terms such as the 3rd-World March-By Series™ really helped the MonkeyLabs Debunkulating Reconfabulator™ make sense of it.

I would think it more comprehendible if the Reconfabulator made scents of it, Monkey.

You're right-on-the-money, Brother Sir Markus; scents smell like profit to me.

What can I say, the development of MLI products has been a labour of love.

[HCMarkus]

... and from Mike's highfalutin comments to the bottom of the barrel:

Thank you for the explanamation Brother Stoivo.

Putting it in MonkeyLabs Industries™ terms such as the 3rd-World March-By Series™ really helped the MonkeyLabs Debunkulating Reconfabulator™ make sense of it.

I would think it more comprehendible if the Reconfabulator made scents of it, Monkey.

You're right-on-the-money, Brother Sir Markus; scents smell like profit to me.

What can I say, the development of MLI products has been a labour of love.

Here's to the discovery of MLI by a loaded venture capitalist

[bayswater]

That sounds like you really should just pick a central location somewhere on a network and keep one copy to work on there. You're making things harder for yourself this way, especially if folks back at the studio are off modifying files you're still working on. You're never going to find a backup solution that covers that.

I was using a central server but it became frustrating dealing with network drops. FreeFileSync seems to do everything I need. It can store setups that sync pairs of folders in different locations. I tired it out over the weekend on large test folders and it worked well. I can set it up as a simple event no more complicated than copying a project back to a central server, and can probably automate it. There is also a version that works in the background and keeps an eye out for directory changes and then does syncs at prescribed times.

At this point i have the test folder on three different macs and a network server and all three macs have a TM copy so while there might be small variations at any one time things look pretty safe. It also keeps a copy of any files overwritten in the sync process so you can always go backwards.