DP 9 and 10 Cannot read heavy automation well, Proven!

[Michael Canavan]

It is NOT clinical proof, but my conclusion does make a lot of sense. It most certainly shows there is a long term inconsistency between DP versions. But I would be thrilled to find out that I am wrong, and that I am stupid and should have just pushed some dumb button to make DP work perfectly. Something like "set event chasing" right? I know this program like the back of my hand, I have tried hard to deal with various automations issues, including this one, for MANY years. For sure I would love to be done with this issue. I have presented the best methodology I can to hopefully put some eyes on this in a new unit of time. I cannot stay back in Sierra forever.

Correct me if I'm wrong here, but wasn't it DP9 that brought in NextGen PreGen etc.?
If you get the time it might be interesting to attempt to disable it as much as possible. I recall problems with VEP and NextGen when it first came out, it's possible there are leftover bugs etc.

Another thought, this:

However once things are bounced to audio, the MIDI is basically shut off because I make new blank tracks on each bounced MIDI track, "hiding" the former MIDI filled track as a sub-take.

Out of curiosity have you attempted to automate from a duplicated Chunk with only the plug ins etc. that you have not flattened to audio in it? I tend to do a MIDI and instrument Chunk then print all tracks as audio, then duplicate the Chunk, getting rid of all unused tracks. It might help, have you looked at what kind RAM and CPU spiking is happening when you have automation issues? I get the feeling that the more complex demands of NextGen might be possibly at fault here.

I sent a copy of my post to tech support.I think there are many new people working at MOTU in the last couple yrs. Maybe they will look under the hood at DPs automation architecture and fix something.

It's important to badger them if you love DP. I'm a big fan, and I want DP to work, so I badger tech support on bugs.

Maybe also Get rid of the dumb ramping when one pastes sections of a song around. Its really hard to mix and automate in DP, I must say. Try past a chorus into a heavily automated file sometime. After you have done so, look under the hood and watch 250 automation points slowly drift from bar 50 to bar 100....with your reverb slowly getting wetter, your compression ration slowly strangling your rhythm guitar as it increases or what have you.....Yikes.

I get why it works like that, but it should be an option that you set, or better yet, the way it works now should be a right click option on an automation line, and it should default to a jump from one state to the next. Basically it's more like a sound design tool or only good for filters as is stands.

[James Steele]

Here we go again. For the sake of everybody reading this, it might be good to let people know that as far as I recall you record snapshot of every single parameter of every single plug-in whether or not that parameter changes. Sort of a "brute force" approach, as I recall. This instead of determining which parameters of the plug in are going to change and automating just those. I mean, I hate to say it, but you're sort of the extreme case here. Maybe you need to get it overwith and switch to ProTools. Not sure if MOTU is going to be able to track down the problem because of a workflow that you employ that I would suspect is used be less than one percent of DP users out there? Also, I don't the the "ProTools is meant for mixing" sentiment you told us you hear is necessarily because these people were employing the same "brute force" automation approach you use in DP and became frustrated, but that's just sort of become the conventional wisdom. There are many people making excellent mixes in DP every day, but there's no disputing that ProTools has established themselves at the big name, and if you don't want to look dumb or have people look at you funny, you just repeat what everybody else repeats.

But really man... there comes a point where if DP is just holding you back so much... move on. Life is short. Make you're music. We'll throw a little party for you in the break room and wish you well. I know Doris will be bummed as she's been harboring a crush for years, but she'll get over it. I think she's sweet on MLC as well.

[monkey man]

That bloody Gorilla gets all the girls.

[MIDI Life Crisis]

I am not sure whether you are joking or being antagonistic?

Is there a difference? lol I am essentially trying to point out that perhaps: 1) maybe you're saturating so much automation into your mixes that it simply isn't practical to expect ANY DAW to keep up; or 2) something(s) that you are using are creating the bottleneck.

It's kind of like driving a car. It can hold just so much weight, but we can certainly put more weight on it than it can handle, and it might be just fine. Then add a UHaul to it and load that up. Again, it will move, but it's gonna put a strain on the engine. Eventually you can put just so much stuff in and wham, you're in the hole for $1400 for a new clutch (I'll only drive sticks).

And I do tend to lose my patience, for which I do not apologize (nothing personal, it's bus-i-ness...) when I hear a musician or recording engineer talk about excessive practices and then bitch that they don't work as expected. As a professional composer using DP, my highest track count in my most massive project was for a feature length film. In one DP project I had my score, sound design, all dialogue, SFX, sweetening, ADR and all other aspects of the film, including 5.1 surround and a TON of automation, and had about 50 0r 60 tracks at most. That was on a G5 cheese grater and DP never even blinked. So when I hear someone doing an orchestral score and needing 100s if not 1000s of tracks for every change from pizz to arco, et al, it tells me that the person doing so is not working efficiently. That goes against the grain of what making art is about. For me it's not about taking an idea and piling as much stuff as possible on top of it, or controlling every tiny little detail. Quite the opposite. It's about distilling the concept to its essential character and then using only as many notes as I require to present that concept as simply as possible. That's the way composers have been doing it for centuries. It just 'effing works.

What computers have created is a breed of musicians who rely so heavily on technology that the technology itself starts to dictate (and in the current case, limit) what the practitioner can create. If a phrase just isn't right, I don't look to add more stuff to fix, I look to what needs to be removed to make it clearer. What seems to be the case for some is that the technology is guiding the creative process instead of the artist, who is only there to sever the technology. I hate that. It makes me mad as hell.

If I seem antagonistic, it's because I see the same kind of process playing out in these threads. Every o/p response seems to pile on more details that are intended support the claim that it's definitely MOTUs fault and that the people taking the time and care to respond simply don't understand, so more details are added until the other contributing members are just tired of the circular arguments.

If that's personal, I'm sorry. I truly am. I just call 'em as I see 'em.

[MIDI Life Crisis]

That bloody Gorilla gets all the girls.

Smile when you say that... oh wait, you ARE smiling... lol

[monkey man]

You handsome fella, you.

[toodamnhip]

Here we go again. For the sake of everybody reading this, it might be good to let people know that as far as I recall you record snapshot of every single parameter of every single plug-in whether or not that parameter changes. Sort of a "brute force" approach, as I recall. This instead of determining which parameters of the plug in are going to change and automating just those. I mean, I hate to say it, but you're sort of the extreme case here. Maybe you need to get it overwith and switch to ProTools. Not sure if MOTU is going to be able to track down the problem because of a workflow that you employ that I would suspect is used be less than one percent of DP users out there? Also, I don't the the "ProTools is meant for mixing" sentiment you told us you hear is necessarily because these people were employing the same "brute force" automation approach you use in DP and became frustrated, but that's just sort of become the conventional wisdom. There are many people making excellent mixes in DP every day, but there's no disputing that ProTools has established themselves at the big name, and if you don't want to look dumb or have people look at you funny, you just repeat what everybody else repeats.

But really man... there comes a point where if DP is just holding you back so much... move on. Life is short. Make you're music. We'll throw a little party for you in the break room and wish you well. I know Doris will be bummed as she's been harboring a crush for years, but she'll get over it. I think she's sweet on MLC as well.

Here we go again.

It's not really "again". It's after a whole gamut of system tests, on a new comp, through all permutations. I would think this info would be valuable.

I recall you record snapshot of every single parameter of every single plug-in whether or not that parameter changes. Sort of a "brute force" approach

I don't automate crazy plugs like Ozone. I will add something VERY important here. It is DP's fault that I have to create automation anchors before every major section of a song because of its terrible automation Ramping design. If I don't create an anchor of automation before a chorus, subsequent pastes result in terrible automation drifting. As far as "brute force", I could automate ONE parameter and it would still ramp, then what? You try it James. Automate 1 parameter, then past a chorus where the parameter is a different level. Watch the ramp for 24 bars. I think the only reason I get flack here is because MOTU doesn't fix things so it seems like I go on incessantly. How about we require DP to actually read its automation correctly instead of making me a one off outlier? Or we have MOTU design DP so that one can paste sections around without ruining earlier or later data, something that has nothing to do with how I work in particular. By the way, James, blaming my work habits at all for this would be technically inaccurate . It is ALL Dp. I will tell you why. Amidst ALL of these tests, me and my technician have been monitoring CPU usage very closely. Why? because there is also a slow graphics issue which is a known problem with the folks at DP, with graphics especially slow in DP 8. And, when watching CPU resources etc, you find out that DP chokes because of its own software inadequacies. There is tons of headroom on my comp. Tons of unused RAM, yet DP chokes graphically, and in other ways (they are supposedly fixing that very graphics issue soon). So I am not taxing the computer at all in many cases. Its just plain inefficiency or Dp getting in its own way, with the computer sitting at 50% or less at times. And please remember the fact that automation did NOT fail in 8.07 and earlier versions. So? Should a song play properly in an older version and choke more in a newer version with the CPU having plenty of headroom left? That's me being difficult as an outlier? If DP wasn't meant to read the automation I write, it would never have worked at all. Yet it read automation properly for MANY years, and then suddenly does not. We can argue about MOTUs priorities to bringing in youngsters who like to loop. Sure, maybe automation isn't a priority compared to keeping up with looping rappers etc. But to blame me for my point of view and not blame MOTU for having automation go backwards in capability? Really?

As far as Pro tools, my and my partner seriously considered it before recently adding a B room. But I have too many songs that need stems, stingers and alt mixes for our publishers so we are already down the rabbit hole. Perhaps Pro Tools in the future, for sure. But to me, since 8.07 was working, I assumed it wouldn't go backwards....So we installed a 2nd Dp system. Isn't that supposed to be a good thing?

You know man, I like DP other than these dumb issues. I really don't want to go back to Pro Tools. But I might for sure. But if I were to go to Pro Tools and have everyone here happy about it, I;d ask why everyone was cool with DP losing the functionality it formerly had. Makes no sense.

As far as people maybe excellent mixes every day on DP, well, depends on who you ask. A lot of the film guys make stems and have others mix. Or the stems are mock ups that get replaced so that they aren't making world class mixes themselves. A lot of music guys also stem out to Pro Tools. The difference between 80% mixed and 100% master is a chasm. I don't see DP as a recording standard anywhere in the mixing world, and I think these inefficiencies are why, and they hurt all of us here. If that's ok with people I sure don't understand it. Because DP could beat Pro Tools in so many areas. But I would disagree that a lot of people REALLY mix hard in DP. I do because I am meticulous and my own mixing is too wrapped around the production to separate easily.
So to summarize- Get rid of the terribly designed ramping, and have DP properly read automation. I don't think requesting these two things makes me an outlier at all. And maybe, if DP can read automation to the level a given computer has headroom for, AND learn to play well with 3rd party products, it could one day become a standard in the studio world instead of losing market share year by year. Hate to be mean, but I see DP becoming obscure more than becoming ubiquitous.

[toodamnhip]

I am essentially trying to point out that perhaps: 1) maybe you're saturating so much automation into your mixes that it simply isn't practical to expect ANY DAW to keep up; or 2) something(s) that you are using are creating the bottleneck.

And I do tend to lose my patience, for which I do not apologize (nothing personal, it's bus-i-ness...) when I hear a musician or recording engineer talk about excessive practices and then bitch that they don't work as expected. As a professional composer using DP, my highest track count in my most massive project was for a feature length film. In one DP project I had my score, sound design, all dialogue, SFX, sweetening, ADR and all other aspects of the film, including 5.1 surround and a TON of automation, and had about 50 0r 60 tracks at most.
What computers have created is a breed of musicians who rely so heavily on technology that the technology itself starts to dictate (and in the current case, limit) what the practitioner can create. If a phrase just isn't right, I don't look to add more stuff to fix, I look to what needs to be removed to make it clearer. What seems to be the case for some is that the technology is guiding the creative process instead of the artist, who is only there to sever the technology. I hate that. It makes me mad as hell.

If I seem antagonistic, it's because I see the same kind of process playing out in these threads. Every o/p response seems to pile on more details that are intended support the claim that it's definitely MOTUs fault and that the people taking the time and care to respond simply don't understand, so more details are added until the other contributing members are just tired of the circular arguments.

If that's personal, I'm sorry. I truly am. I just call 'em as I see 'em.

You assume a lot my brother, I am not sure where to start. For one, just to make clear that I am not some VI abusing hack that just puts 200 of everything into DP and then complains that 20 Abbey Road reverbs won't automate..lol. , I started out and still am a chordal Jazz guitarist who was mentored by Joe Pass. Before I had my 1st computer , I was playing Giant Steps in 12 different keys, along with gigging in Salsa bands on guitar synth, arranging horn charts, and playing Led Zeppelin on the side. I have played world premier operas as a guitarist and toured with the likes of many legends and icons. Yet honestly, I am no more special than any of the other amazing players and composers who frequent this forum. I just automate more I guess, and do my own detailed mixes. Not really boasting just trying to get across the premise that I know what I'm doing. I don't think I can be characterized as anything close to what you seemed to assume in your response. But I do feel you on the next gen of paint by numbers players out there.

And honestly, I didn't say track count was the issue. I said automation amount was the issue. Please examine and read exactly what I wrote. A perfect duplication of what I wrote and my procedures would be helpful. You've been around this site a long time, you're one of the veterans, I like most of what you say. No antagonism here. But let's be accurate and watch for assumptions.

Now, one last problem that might relate to the issues I am having...somewhat. Though please keep in mind, DP 8.07 and earlier versions worked....I do use Busses. I create a guitar bus, horn section bus, strings bus, drum bus, etc. Now, my earlier tests this year concluded this other sad fact. I could get 60 Abbey Road reverbs to work, (as a test only), if they were all on audio tracks. And only 12 to work when they were on Busses. But running sections into busses IS a standard way of mixing. I am not an outlier in this. So know that DP does NOT handle CPU efficiency well at all when placing any type of plug in on a bus. I believe DP is actually 1/5th as efficient running any processing on a bus compared to an audio track. So that may aggravate things in my mixes too. But it is NOT Un standard to bus things and apply some processing on them. And again, it all read back fine before version 9 and above. Someone else may have hit the nail on the head when they asked about Next Gen pre gen. THAT is what changed the most in 9 and above.
Well, I've beaten this horse dead.

I just want Dp to read the automation it used to read before. Lets look at the logic of that instead of going after the messenger. I am a fan in so many ways. Know that.

[toodamnhip]

Correct me if I'm wrong here, but wasn't it DP9 that brought in NextGen PreGen etc.?
If you get the time it might be interesting to attempt to disable it as much as possible. I recall problems with VEP and NextGen when it first came out, it's possible there are leftover bugs etc.

Yes, it is next gen that I suspect creates the change in performance capabilities, and it cannot be shut off anymore unfortunately. Many have asked that it could be shut off. I am not alone in this.

Out of curiosity have you attempted to automate from a duplicated Chunk with only the plug ins etc. that you have not flattened to audio in it? I tend to do a MIDI and instrument Chunk then print all tracks as audio, then duplicate the Chunk, getting rid of all unused tracks. It might help, have you looked at what kind RAM and CPU spiking is happening when you have automation issues? I get the feeling that the more complex demands of NextGen might be possibly at fault here.

Yes, what I do is I make a duplicate chunk with NO VI remnants before I mix, and mix on that VI-free chunk.

It's important to badger them if you love DP. I'm a big fan, and I want DP to work, so I badger tech support on bugs.

If I only badger the guys at MOTU, I can't piss James off. That's no fun lol...(JK James)

I get why it works like that, but it should be an option that you set, or better yet, the way it works now should be a right click option on an automation line, and it should default to a jump from one state to the next. Basically it's more like a sound design tool or only good for filters as is stands.

Sure, make automation ramping an option for all those who want a reverb send to drift for 24 bars, I have no problem with that.

[Phil O]

However once things are bounced to audio, the MIDI is basically shut off because I make new blank tracks on each bounced MIDI track, "hiding" the former MIDI filled track as a sub-take.

Once you've bounced to audio, disable the VIs (I like to use the TO window for this). This reduces the CPU load considerably. That little enable button is your friend here.

Phil

[toodamnhip]

However once things are bounced to audio, the MIDI is basically shut off because I make new blank tracks on each bounced MIDI track, "hiding" the former MIDI filled track as a sub-take.

Once you've bounced to audio, disable the VIs (I like to use the TO window for this). This reduces the CPU load considerably. That little enable button is your friend here.

Phil

Of course. That and also eventually deleting all remnants of VIs so as to mix on a purely audio-based file. Be advised I have made tests in the past that show that even when the VIs are supposedly "disabled", they can eat resources. Not saying this is a DP issue, may be dependent on a particular VI. But that's why I mix from chunks that have not even a VI remnant.

[Michael Canavan]

Now, one last problem that might relate to the issues I am having...somewhat. Though please keep in mind, DP 8.07 and earlier versions worked....I do use Busses. I create a guitar bus, horn section bus, strings bus, drum bus, etc. Now, my earlier tests this year concluded this other sad fact. I could get 60 Abbey Road reverbs to work, (as a test only), if they were all on audio tracks. And only 12 to work when they were on Busses. But running sections into busses IS a standard way of mixing. I am not an outlier in this. So know that DP does NOT handle CPU efficiency well at all when placing any type of plug in on a bus. I believe DP is actually 1/5th as efficient running any processing on a bus compared to an audio track. So that may aggravate things in my mixes too. But it is NOT Un standard to bus things and apply some processing on them. And again, it all read back fine before version 9 and above. Someone else may have hit the nail on the head when they asked about Next Gen pre gen. THAT is what changed the most in 9 and above.

So my guess is DP8 doesn't get much more than 12 busses either. Busses run in Real Time (RT) and if everything is working correctly regular tracks run in PreGen (PG) which will allow for more plug ins than 8. I did testing and DP comfortably does as well as the two leanest OSX DAWs, Logic and reap.., whereas before it struggled behind them.

Yes, it is next gen that I suspect creates the change in performance capabilities, and it cannot be shut off anymore unfortunately. Many have asked that it could be shut off. I am not alone in this.

There are ways of forcing a track in RT. I'm not in front of DP right now, but opening the plug in GUI of a track can force the track into Real Time. I'm taking a guess here, but your odd automation issues seem like a PreGen issue where it's not re-rendering the track to PG fast enough, and automation is getting hosed.

Not really boasting just trying to get across the premise that I know what I'm doing. I don't think I can be characterized as anything close to what you seemed to assume in your response. But I do feel you on the next gen of paint by numbers players out there.

I'm not on board with this. It's a slightly off topic discussion, but I love all kinds of music, not just music that's informed by classical and jazz. More than 90% of the music that will be made in any DAW is going to be pop music of some sort, whatever label you want to call it, Rock, EDM, Blues, Folk, Funk etc. etc. it's all pretty much the same in terms of composition. There aren't very many, as in less than a handful of even subforms dedicated to DP. I don't think it does anyone any favors or makes them look good at all to attempt to bash on musicians who use a DAW like an instrument and heavily edit etc. It's always been a good argument IMO that engineers and producers can and do impart into the creative process. What makes Dark Side of the Moon a gold standard in terms of recording isn't just the playing, in fact we're not really talking about the playing when we talk about the mix on that record. There are people who do great things with a home studio, average instrument talent and exceptional DAW skills. In fact most people who buy DAWs are buying them for home studios, are people with not a lot of free time from their day job, who want to write a song or two.
I look at the KVR forums and it's almost all bedroom producers, Ableton is almost all techies who work in IT or programming for day jobs. It's obvious there's an interest at MOTU in getting some of those users with the addition of Clips etc. I just think we do MOTU a disservice if Motunation becomes too elitist. It's a private forum, but it's in the unique position of being the only solely dedicated to MOTU and DP forum out there.

[bayswater]

I'm not that surprised that DP would bend under the weight of huge amounts of automation data. It's a lot of irrelevant calculations that have to be done on what could be thousands of parameters hundreds of times a second. A software designer might anticipate this, and skip all the calculations on flat line automation curves, but they probably wouldn't think of that.

But I was surprised to hear that bussing to effects is less efficient than leaving effects in the tracks. I was wondering if for some odd reason, DP was applying the effects to each of the incoming busses and then summing them, rather than applying the effect to the summed signal.

I have a test project with hundreds of tracks so I tried a reduced version to see if I could reproduce this.
This has 200 audio tracks each playing a different audio file, each with one instance of a 4 band EQ. These are bussed to 18 Aux channels, each with a compressor, then to the master out with a leveler. Buffer is set to max (2048). This is a 2018 I7 Mini with 6 cores.

With this I get CPU at 132% showing in Activity Monitor. DP Effect Performance (coalesced) shows 34.5% for the EQs, 2.4% for dynamics and 0.9% for the leveler.

Copying the EQ to each of the Aux channels I get 130% for CPU, 36% for EQ, 3% for dynamics and 1% for the leveller.

Removing the EQ from the tracks results in 106% CPU, 0.20% for EQ, 2.8% for dynamics, and .85% for the leveller

Setting aside normal variation in CPU load -- it jumps around a bit during playback, there is nothing here to suggest that bussing was less efficient in the test project. Going from about 200 instances of EQ to about 20 reduced the resources used by EQ by a factor of more than 10, more or less what you'd expect.

[Michael Canavan]

I'm not that surprised that DP would bend under the weight of huge amounts of automation data. It's a lot of irrelevant calculations that have to be done on what could be thousands of parameters hundreds of times a second. A software designer might anticipate this, and skip all the calculations on flat line automation curves, but they probably wouldn't think of that.

Again, TDC has flatly stated that earlier versions of DP v8, works just fine with automation. I'm at this point confused as to how everyone keeps missing this? DP8 does the automation just fine, 9 and 10 are not working as expected. There's no reason here to question the working method of TDC, if DP had never been able to do what he wants it to, then yes, your speculation would make sense.

But I was surprised to hear that bussing to effects is less efficient than leaving effects in the tracks. I was wondering if for some odd reason, DP was applying the effects to each of the incoming busses and then summing them, rather than applying the effect to the summed signal.

I have a test project with hundreds of tracks so I tried a reduced version to see if I could reproduce this.
This has 200 audio tracks each playing a different audio file, each with one instance of a 4 band EQ. These are bussed to 18 Aux channels, each with a compressor, then to the master out with a leveler. Buffer is set to max (2048). This is a 2018 I7 Mini with 6 cores.

With this I get CPU at 132% showing in Activity Monitor. DP Effect Performance (coalesced) shows 34.5% for the EQs, 2.4% for dynamics and 0.9% for the leveler.

Copying the EQ to each of the Aux channels I get 130% for CPU, 36% for EQ, 3% for dynamics and 1% for the leveller.

Removing the EQ from the tracks results in 106% CPU, 0.20% for EQ, 2.8% for dynamics, and .85% for the leveller

Setting aside normal variation in CPU load -- it jumps around a bit during playback, there is nothing here to suggest that bussing was less efficient in the test project. Going from about 200 instances of EQ to about 20 reduced the resources used by EQ by a factor of more than 10, more or less what you'd expect.

You're still not completely getting it, which is of course no big deal.
Any track bussed to an Aux track is going to render in Real Time, as you tested, it doesn't really matter where on the bus the plug in is, it's going to run in RT. Multi out plug ins, tracks bussed to other tracks etc. all render in RT..
Where NextGen PreGen (PG) works and where you will see drastic improvement is on any track not routed to an Aux track, not record armed, and not with multi audio outputs like a Kontakt Multi etc. You will not notice any significant difference on tracks that are Real Time, like you tested.

Take that same 200 tracks and route them straight to the master out, and you will see a significant drop in CPU consumption.

[bayswater]

Again, TDC has flatly stated that earlier versions of DP v8, works just fine with automation. I'm at this point confused as to how everyone keeps missing this? DP8 does the automation just fine, 9 and 10 are not working as expected. There's no reason here to question the working method of TDC, if DP had never been able to do what he wants it to, then yes, your speculation would make sense.

If you assume everything in the DP 8 design was carefully carried over to DP 9 and 10, and that the designers are aware of the effect of changes, then fine. I'm not saying the workflow is not valid, just that we should not assume it is consciously supported, even it if worked in the past.

Where NextGen PreGen (PG) works and where you will see drastic improvement is on any track not routed to an Aux track, not record armed, and not with multi audio outputs like a Kontakt Multi etc. You will not notice any significant difference on tracks that are Real Time, like you tested.

And yet I didn't. The difference in CPU was just what you'd expect from the number of instance being run regardless of whether they were pre-gen or RT. My point was simply that the test didn't show anything to support the contention that there is something wrong with DP when it is used in a common arrangement, routing tracks to busses for purposes of applying effects. Using 20 times as many instances used 20 times a much CPU. If pre-gen was a big deal, wouldn't you expect the 200 tracks to be more efficient than the 20 busses?