Internal Storage, Dropbox and iCloud (and Time Machine, OneDrive and Backblaze)

[mikehalloran]

I've promised a few times that I'd start a new topic on this subject.

…
Any good reason not to keep this same setup with the new Studio, or do folks consider it robust enough to run work projects / samples etc off say the large internal 8TB in the new studio? My inclination is just to go with a 1TB internal and keep my outboard enclosure for all my work and samples.

There's the R/W speed issue. Just about everything is faster than your MP 6.1 — the 2 Lane bus was the bottleneck and TB 1/2 to SSDs was no faster than USB 3 to the same drives.

An NVMe 3 x4 blade over TB 3/4 is about 6x that speed. Only the M2 Mini with one internal SSD is slower. It has the slowest internal R/W of any current Mac.

A Studio with one NVMe 4 x6 SSD is about 10x faster than what you had before. Those with two are about 18x faster. All smaller capacity Macs use one and the larger capacity use two in a modified RAID array that's not quite 2x faster. Apple doesn't publish which has what but there have been enough teardowns — still, Apple changes this now and then.

Any NVMe blade over TB 3/4 is the same 6x — in this case, the 4 Lane over TB3 is your bottleneck. Multiple SSDs will be slower but with greater capacity.

In real world applications, audio hardly notices and video/AV does.

But there's more:

If you share files with others, how much internal storage you have might become very important.

It used to be that you stored your files on the Mac or an external drive and could sync them to Dropbox copies. Beginning with Monterey 12.3, that's reversed.
https://www.macrumors.com/2023/01/17/dr ... -by-april/
Sharing over iCloud and Dropbox now requires the files to be in the Cloud. If you have enough internal storage, you can sync these files to your Mac — but not to on an external drive. If you don't have enough storage, the files stay in the cloud and will not store copies on your Mac.

It gets worse — Apple app files (Logic, Final Cut etc.) can no longer be shared over Dropbox — iCloud only.

What does this mean? Well, for starters, Dropbox files residing only in the cloud will not be backed up to Time Machine or *Backblaze — only sync'd copies found on your internal drive will. Apple has been doing this with iCloud quite awhile but no one really noticed until the changes to Dropbox.

People on many forums are going absolutely bonkers over this.

*Backblaze with a Business Enterprise account can include external drives and NAS but personal accounts cannot.

[mikehalloran]

I see no reason not to keep VI libraries on an external. With the exception of SampleTank, I've never observed a difference between keeping them on a SATA III SSDs over Thunderbolt and on the internal even though the internal's R/W speed (twin NVMe 3 x4 SSDs on my iMP) is 13x faster. SATA III is the speed bottleneck and it's about the same speed over USB 3. Thunderbolt never ever goes to sleep on me, something I can't say about storage over USB-anything.

SampleTank often needs to be reloaded when calling it for the first time in a new project. Having it on the internal drive makes reloading the libraries quite a bit faster. Once loaded, I notice no difference in streaming or bounce to disk, however.

I sometimes open up projects stored on my archive drive (another SATA III SSD in the TB dock) to tweak it for a different market or perform a requested edit or remix. BTD is noticeably slower if I don't bring it back to the internal first.

[Chris T]

Ok thanks Mike.

The conventional wisdom used to be keep the program drive separate to your work and samples drives. Is this still the case?

I ask because the hefty potential of the Mac Studio's 8TB internal drive is a tempting prospect, especially if it's so much faster than any other external drive option.

Does the spec of the new M2 machines now no longer require us to separate work drive from program drive etc, or should we whenever possible still stick to the old advice re. separate, dedicated drive allocations?...

[HCMarkus]

I didn't spring for 8TB of (very fast and very expensive) internal storage when I purchased my Mac Studio; I went for 2TB. As such, I've been streaming VI Samples from External NVMe drives in TB enclosures. Has worked great for me.

I run my projects from the Internal SSD, but offload old and dormant projects to External storage.

The DropBox situation is not good. But I'm pretty well stuck with it due to circumstances beyond my control.

[Chris T]

Thanks HCMarkus. I'm curious:

- which NVMe TB enclosure do you use?

- Does running your DP sessions from the new Mac Studio's Internal SSD impair performance in any way (e.g. due to resource sharing with apps / OS)? In the past, back in the "spinning disk era" we were always advised to have a separate 'work' drive, but I'm not sure whether that is now a moot issue....

- I don't use Dropbox for backup - I use Backblaze instead. I guess I'll be ok in this regard whichever size drive I get?..

Cheers

[HCMarkus]

Thanks HCMarkus. I'm curious:

1. which NVMe TB enclosure do you use?

2. Does running your DP sessions from the new Mac Studio's Internal SSD impair performance in any way (e.g. due to resource sharing with apps / OS)? In the past, back in the "spinning disk era" we were always advised to have a separate 'work' drive, but I'm not sure whether that is now a moot issue....

3. I don't use Dropbox for backup - I use Backblaze instead. I guess I'll be ok in this regard whichever size drive I get?..

Cheers

1. I'm running two OWC Envoy Express Thunderbolt Enclosures each housing a Samsung 970 Evo Plus 2TB NVMe SSD. They cost $90 and max out at 1550MB/s, as they only connect via two TB lanes. There are now available other, faster enclosures that use four lane TB connections, but I'm ok with the performance I'm getting on sample loads. From my reading, and as Mike suggests, a faster connection won't necessarily speed up VI Sample loads. Lots of folks are happily using SATA SSDs for sample storage.

https://www.amazon.com/dp/B08LYQPRPJ?re ... =owc+envoy

2. No impairment whatsoever. Issue is non-existent in this age of fast SSD storage.

3. BB will back up all attached drives, no matter what size. I use BB for my cloud backup. I use DropBox for sharing files with clients and between my Macs.

[mikehalloran]

…
The conventional wisdom used to be keep the program drive separate to your work and samples drives. Is this still the case?

Not since the Mac G4 with its blazing bus speeds up to Ultra ATA/66 and Ultra ATA/100. That's 66MB/s and 100MB/s bus speed.

I ask because the hefty potential of the Mac Studio's 8TB internal drive is a tempting prospect, especially if it's so much faster than any other external drive option.

Well but… speed is only part of it. Your current externals are fast enough unless rendering large AV. Heck, so are mine unless doing BTD, then the pokey 560MB/s does make a difference.

The real issues are file sharing and backup. If you are sharing video and want to work on or back up those files locally, then your system's internal storage needs to be big enough. Time Machine will backup external drives but it cannot backup files residing in the cloud. When your Mac has enough internal storage, you can elect to have copies of those files stored on your drive where you work on them locally, even offline if needed and they will sync to the cloud when your internet is robust enough (this just happens—you've no control over when/how). You'd think living in the Silicon Valley that I have excellent internet speed—not true at all (we're still over copper).

Does the spec of the new M2 machines now no longer require us to separate work drive from program drive etc, or should we whenever possible still stick to the old advice re. separate, dedicated drive allocations?...

That became obsolete with the SATA bus, introduced with the G5. Apple's white papers made it quite clear that internal data transfer on a drive was much faster—they no longer recommended partitions unless you needed them to have multiple MacOS on your drive. With SSDs, there are even better reasons not to do this—data allocation is completely different and running it unnecessarily through the bus is a useless bottleneck.

With APFS, it gets weird. You no longer partition. Instead, you create Volumes where all share the combined space. The disk utility companies will tell you in the very fine print that most of their advertised features don't work with APFS because Apple has never released the white papers that let them know exactly what's going on. I read this notice on the TechTool website just yesterday when wondering if I should upgrade to TTP17. I passed.

In the real world, audio is so slow that streaming your VIs from an external shouldn't make a difference. However, when the Mac Pro 7.1 was introduced, one of the promises was that running a separate hardware server for VEP was never going to be necessary again.

[Chris T]

I'm finally setting up my new Mac 2 Studio Ultra.

I got the 8TB internal drive (despite te hefty $!) with this thinking that, when I travel, I'd have a "studio in a box" and not have to also lug around my heavy OWC TB enclosure (which holds my SSD drives of VI Sample libraries).

Unimaginatively, my SSD sample drives are called "Samples 1" "Samples 2" and "Samples 3".

If instead I were to put the entire contents of these 3 drives (totalling around 5TB) on my 8TB internal, my concerns would be:

1. Too much for the single internal to handle on the Mac Studio: since it would be simultaneously dealing with my DP project, VI samples AND video. Could this be a problem (especially on big sessions)?

2. Opening up past projects, it would need to "find" all the samples. Does anyone know, if I were to simply make 3 new folders on the root of the MAC Studio drive called "Samples 1" "Samples 2" and "Samples 3", would DP/Vi's find them automatically, or would I need to manually direct every VI when it first opens to my newly located "sample drives" (= folders on my internal HD). Any thoughts?

Thanks,
Chris

[mikehalloran]

Could this be a problem

No. SSDs and NAND don’t work that way.

Your internal NAND is a pair of 4TB NVMe 4 x6 chips in a “modified RAID 0" array at about 8000 Mbps. Until TB5/USB 4.2 happens , there is no way to configure any external storage capable of handling that much data anywhere near that speed. It is not a linear 25% increase over the 4 Lane bottleneck present in TB3/4, NVMe 3 x4 or 4 x4.

Apple has a default location for your libraries. It is Users/Shared (assuming all Users have the same privileges). Library installers such as Native Access will use that path unless you give them different destinations. You can copy and paste your VIs to that directory but you want to do it in a way that lets DP find them.

[Chris T]

Apple has a default location for your libraries. It is Users/Shared (assuming all Users have the same privileges). Library installers such as Native Access will use that path unless you give them different destinations. You can copy and paste your VIs to that directory but you want to do it in a way that lets DP find them.

Thanks Mike. So it sounds like the internal HD will handle anything I throw at it. My question now is, since I have a lot of VI libraries (over 5TB and always rising!), and I'd rather not move my libraries (for several reasons), would running my libraries, (as I do currently), from my external OWC SSD slow my system down in any noticeable way?

My plan is to just copy my VI / Kontakt sample libraries to my Studio internal drive if/when I need to travel, to avoid also having to take the OWC around.

[HCMarkus]

Chris, I doubt you will see a huge difference between internal and external drives for loading VI samples and am almost certain the slower external drive will not limit polyphony during playback.

That said, why not experiment just a bit? Try copying a few sample sets to your internal, then redirect the appropriate VI back and forth and see what results you get.

[Chris T]

Thanks HC. Yes I'll see how it goes, and it's good that I have the options!

Cheers

[mikehalloran]

Exactly. I am in the process of moving many of my libraries over but I’m not in a hurry. Not everything will be moved internally, however, as I only have 8TB and I need to reserve 2-3TB for file sharing over iCloud and Dropbox.

Because SampleTank often needs to be rebuilt, there’s a notable difference between having it on the external and internal—not streaming, just rebuilding.

Next week, I’ll be revisiting a project with a number of NI libraries. I’ll play with it as-is including BTD, then move them to the internal and compare.