Two Streaming FW VI Drives: Possible?

[Tobor]

Hi,

I was about to copy my maxed out 500g VI FW drive onto my new 1TB FW drive, when the thought occurred to me whether it was possible or desirable to use both drives to stream VIs simultaneously. There is also a FW audio drive and FW interface on the bus.

Currently I have been unable to stream Ivory and Omnisphere live at the same time without glitches, for example. Perhaps if I had my Ivory samples on one drive and my Steam engine on another, it could work? Is there any reason why this wouldn't be desirable, or is it even possible?

Tobor

[dosuna11]

I do it.

[FMiguelez]

.

If they share the same bus, it would be essentially like having one FW disk with more space, would it not?

I'd only do it if they had separate buses.

[Tobor]

.

If they share the same bus, it would be essentially like having one FW disk with more space, would it not?

I'd only do it if they had separate buses.

It seems to me like the reason Ivory and Omnisphere can't work together is that each sample set is huge (relatively) and the hard drive has to work extra hard to access a lot of samples over a large area. If each drive only had to access half of those samples, in theory it should make playback run, well, twice as smoothly. I don't generally need to run a lot of simultaneous VIs, but when you're using ones with big footprints (Trilian to come, for example) it would be nice to get your rhythm section running in real time for a while without having to commit to audio.

So dosuna11, did you notice a dramatic change when you employed this method?

Correct me if I'm wrong, but it appears that the firewire bus is only passing MIDI information and/or audio, while the hard drives have to scurry around like crazy to access the samples. Less work on the hard drives: more efficient performance.

Also if you put all your VI 'eggs' in one basket (hard drive), discarding the empty baskets when you upgrade, there's some waste involved. I was going to shift my old VI drive to upgrade my audio/video drive, but now am rethinking that strategy. Two VI drives in theory should be twice as productive as one.

Tobor

[FMiguelez]

Two VI drives in theory should be twice as productive as one.

Agreed. Especially if they do not share the same data bus, or if it at least is wide enough to let the data packets flow freely.

At this time, I'm working out how I will install and handle all my VSL SP. And all the other VIs.
I must try to install unlikely-to-play-together instruments in the same HDs.

Also, I suppose one needs to make sure there are no bottle necks at any point in the process chain. The trick is to know where these might be.

Would it be accurate to state that the data streams will only be as fast as the bottleneck permits? That the fastest data streams will be equal to whatever speed the bottlenecks allow?

[Tobor]

Two VI drives in theory should be twice as productive as one.

Agreed. Especially if they do not share the same data bus, or if it at least is wide enough to let the data packets flow freely.

At this time, I'm working out how I will install and handle all my VSL SP. And all the other VIs.
I must try to install unlikely-to-play-together instruments in the same HDs.

Also, I suppose one needs to make sure there are no bottle necks at any point in the process chain. The trick is to know where these might be.

Would it be accurate to state that the data streams will only be as fast as the bottleneck permits? That the fastest data streams will be equal to whatever speed the bottlenecks allow?

Yes, that makes perfect sense.

But where are those streams heaviest? Let's take Ivory, for example. Does the FW data stream or hard drive do most of the work? For example, I'll play three chords at different velocity levels. Now, does the FW data stream consist of just the MIDI info (notes, velocities), which would be fairly light duty, and the audio data (audio streaming back from the hard drive), or does it also contain all the little info about what samples the hard drive has to find to play? Or does the hard drive interpret the MIDI notes, decide what samples should be playing, then find and play those samples? Under that scenario, it seems like the hard drive would have to do most of the heavy lifting.

Tobor

[FMiguelez]

.

I would love to listen to those answers given by an expert on the subject, which obviously I'm not

If you want to hear my guess, I'd say that the Ivory "app" recognizes the velocity, note number, etc. given by the DAW, and then it transmits some sort of query to the FW disk according to what the app got. I suppose the message is sent and it seeks the correct samples based on that.

What comes out of the disk is the resulting data stream, which would be big.
I understand that seek times, spin velocities and read times are key for the HDs. But also, all this info flows into the computer and its data buses. At this point I don't know how much gets stored into RAM or virtual memory, and how much it keeps streaming from the disk.
There are a few points where bottlenecks can happen. So if you have a very fast HD (15K rmp), but you get a bottleneck somewhere, then you wouldn't really be taking advantage of that disk's speed.

ALL THIS IS GUESSING ON MY PART. Hopefully an educated guess

There are members here who are expert programmers and know computers' guts intimately. I'd LOVE to hear from them regarding this.
Feel free to correct me, so I can learn somethin'.

[Shooshie]

This has always been my basic hard-drive method, at least since it began to matter. I separate by busses when possible, and by drive. I distribute VI libraries to drives that can access independently. Those VI's that share the same drive are the ones that are not likely to be competing with each other. Some are on the SATA bus, others on the Firewire bus. USB 2.0 is not fast enough for these sorts of applications, IMO, but I'm certain someone is using it and probably succeeding.

I currently have 5 SATA drives in my Mac Pro, four Firewire-800 drives, and a bunch of older Firewire 400 drives. Two of my SATA drives are 10,000 rpm WD Raptors, for extra streaming speed.

My DP files, with all their audio soundbites, reside on a drive that is separate from all the VI libraries. It's not possible to create a perfect system in which every VI has its own drive and every drive has its own bus (unless you want to network lots of computers, of course), but if you sit down with paper and pen, then examine how you use your VI's, you will quickly see a best-compromise scenario emerge in which many VI's can occupy the same drive because they don't often compete with each other. Then you can keep the heavy-duty libraries separate.

I am told that USB 2.0 (or all USB) has trouble duplexing, so it sends and receives alternately. This has the effect of slowing the drives down by nearly half their rated speed. VI's need to be on drives/busses that can send and receive full-duplex, for reasons which should be pretty obvious. The more VI's you're streaming, the more important that is. So, don't use USB 2.0 for streaming drives. USB 3.0 may change all that, but we'll have to see about that. And if it does, will it be the same for USB 3.0 PCIe cards? Or will it be necessary to have it "built-in?" Apple may not be giving us that anytime soon, what with their new Light Peak on the horizon. Which is faster? Much remains to be seen. I'd not be buying a new computer right now, though. Maybe this time next year, if one can wait. (Fernando, pay no attention to that. Buy your computer!) Soon, streaming an entire orchestra will be no problem with 64-bit memory access and blindingly-fast I/O of the new drive technologies coming through the pipeline.


Shooshie

[FMiguelez]

I'd not be buying a new computer right now, though. Maybe this time next year, if one can wait. (Fernando, pay no attention to that. Buy your computer!)

I know. At this point, almost anything is better than what I have now

I've always wondered something. In cases where, like with VSL stuff, where you basically have all your samples loaded into RAM, how important is to have fast disks? Like the ones you mentioned you have (the 10k rpm ones). You are not really streaming in this case, correct?

I mean, once everything is loaded in RAM, then the 7.2K rpm HDs are ok, right?
Now, LOADING the samples into RAM may be slower, but once everything is there it wouldn't make a difference, would it? (I'm talking about VSL here, not other engines where you must stream from disk in real time).

Or am I miss understanding something?

[Shooshie]

VSL is way too big to load in RAM. Most VI's are large enough that you really have to stream them. They only load the first second or so of each sample, then the rest of it is seamlessly spliced streaming to the end of the part that's in RAM.

However, with VSL you can have it analyze the samples that are being used in your song, and it will then unload all the rest. So the ones that are actually loaded are the ones that play notes in that particular song. If there are not too many active notes, then THOSE might be contained in RAM alone. Not sure about that.

Shooshie

[Tobor]

This has always been my basic hard-drive method, at least since it began to matter. I separate by busses when possible, and by drive. I distribute VI libraries to drives that can access independently. Those VI's that share the same drive are the ones that are not likely to be competing with each other. Some are on the SATA bus, others on the Firewire bus. USB 2.0 is not fast enough for these sorts of applications, IMO, but I'm certain someone is using it and probably succeeding.

I currently have 5 SATA drives in my Mac Pro, four Firewire-800 drives, and a bunch of older Firewire 400 drives. Two of my SATA drives are 10,000 rpm WD Raptors, for extra streaming speed.

My DP files, with all their audio soundbites, reside on a drive that is separate from all the VI libraries. It's not possible to create a perfect system in which every VI has its own drive and every drive has its own bus (unless you want to network lots of computers, of course), but if you sit down with paper and pen, then examine how you use your VI's, you will quickly see a best-compromise scenario emerge in which many VI's can occupy the same drive because they don't often compete with each other. Then you can keep the heavy-duty libraries separate.

Shooshie

The Shoosh-mon, coming to the rescue again. Thanks! I guess I never thought of piling all my FW drives into one chain, but now it makes sense to me. I might even be able to work out a system where I can use aliases to vary up the drives used, with Ivory's samples for example on every drive but by changing folder names and using aliases I can point the MIDI to the right drive.

I installed the Scarbee Pre-Bass today on my main VI drive. Ivory was playing beautifully until I loaded the Scarbee into Kontakt and then it started fritzing. As an aside that Scarbee bass is very nice. I'd love to hear them together and maybe the only way to do that is to separate them!

Anyway thanks guys for all the great feedback. I guess there's only one way to find out, so it's time to plug and play. I have yet another FW drive that I had stationed near my laptop but since I hardly use it I'll add that to the chain as well and see what happens.

Tobor

[dosuna11]

I daisy chain my drives. My problem is lack of ram and I am again an abandoned soul by Apple.

[bjornln]

There is a clear advantage to have your samples spread out over more than one drive.
If you have your data spread out over two drives, you have two heads that can read (relatively) small files. In this case the bandwidth should be less of a problem.


/B

[Tobor]

I chained my FW drives together today and can already see the positive results. Ivory and Omnisphere playing together glitch-free. Yay!!

Now I can use my third drive for rhythm section/Komplete 5 and the 4th for audio. Each VI seems to have their own method for 'pointing' to their data. It's relatively easy to switch the active Omni/Steam and Ivory folders back and forth between drives (data duplicated on each), now must figure out how to do the same with BFD, for example. Different projects might suggest different optimal drive layouts. Fortunately the VIs seem to update their changing drive positions globally, otherwise it would be a nightmare trying to retrace tracks for each project.

This could help me wrangle a few more months on this computer before the inevitable upgrade.

Thanks for all the great input!

Tobor