My biggest DP issue: MIDI automation

[Gabe S.]

btw, I don't think the solution is that difficult in terms of software engineering. Given all the complex things DP can do, I think it would be a relatively simple solution. They need to make a rule for DP that when you start automating a MIDI parameter, DP looks to see what that first automated value is and then it places a copy of that value at the beginning of the session. If you're in the middle of a song, and there was a prior controller point, then DP places the copy of your controller point one tick later than the last point. Boom. Done. How hard is that? They could even make it a Preference option in case people don't want it to work that way: "Copy first MIDI automated controller value to beginning of session or after most recent value."

C'mon…..DP can track entire Undo histories WITH BRANCHES up to a gazillion steps away……I think it can make a copy of your first controller point and place it one tick later than a previous controller point, or at the beginning of the session.

That's what we humans are doing manually! I'm quite sure the software is capable of it.

If they implemented that, literally every single word I have typed about this problem would become obsolete. And I can't think of a problem that it would cause. Can anyone come up with a reason that it SHOULDN'T work this way??

-g

[stubbsonic]

Perhaps you missed my post in this thread, but I feel like the easier solution is a chasing setting.

It would work easily with two separate chasing functions. Chase backward works as it currently does, chasing notes and controllers as selected.

A special Chase Forward option could be added for each controller that would say, in essence, "Look backward and, finding nothing, look forward as well, set playback to the first controller value found." This would replicated the audio automation process (apart from the vectors).

IMHO, this is an elegant solution for this particular issue, and would not require any special hoop-jumping or any changes in the track. Further more if I change my mind about later levels, I don't need any special function to change the level at the beginning of the sequence.

Sorry to repeat myself. And maybe I've overlooked some reason this wouldn't work.

[reedster]

To make the current behavior more tolerable, I turn off chasing except for notes - otherwise after rewind on playback, CC are set to default values - full up for Vol (yikes!), midpoint for Pan and full off for CC#1.

At least that way DP behaves more like synths in that it remembers the last value and stays there until a new value arrives.

[bayswater]

Hmm... Not for me. I just tried it, and it works.

I forgot to go into Overwrite mode. Duh.

[frankf]

Gabe, I watched your first video and I'd like to know the following:
-The first time the loop plays, at what volume to you want it to play? Since there's no CC7 data before the first note, DP will play back at the last seen CC7 event or the DP default, which is 127. I think what you're asking is for DP to correct for the fact that you did not place an CC7 point before the phrase: to look ahead, find the next CC7 or any CC and apply that before the first note of your MIDI phrase and NOT chase.
-So you would have to define the phrase that you want to apply this to if there are any prior phrases, no?
-When looping as per your example, would DP stutter having to do this in real time?

I'm of the school, actually what Shooshie said, that you should place a snapshot either at the beginning of the track or prior to the first MIDI note in a track. I would not do what you show in your demo, insert CC7 ramp in the middle of a phrase and expect it to play at the same volume when looped OR when chasing. there is simply no data to chase.

Also note that MIDI and Audio automation are different beasts in DP:
MIDI: 127 discreet changes in CC7 and CC10
Audio: Continuous sample accurate (or ramp) automation

But back to what you asked (complained) about. MIDI CC7 and Audio Volume track automation do behave differently. It's easy to see if you put an empty audio and a MIDI track side by side in the SE with the MIDI track in lines mode and take a snapshot a few bars in. You will see the Volume line going all the way back to the beginning of the track, even though DP does not place a break point there. (DP does behave like a break point has been inserted. Try adding a breakpoint between the beginning of the track and the previously entered breakpoint) The MIDI line stops at the location of the snapshot. Because there is no CC7 automation point before the snapshot location, DP sets CC7 for the track to its default: CC7=127. I agree it would be better if MIDI automation tracked back from the first CC7 data point in a track and placed an identically valued point at the start of the track ala volume automation. However, this has been the way MIDI automation has worked forever, at least in DP, and I consider it good practice to set MIDI CC automation points before any notes play. I put them in bar 0 myself. I don't consider this a work around, instead a work flow.

Another way to implement is to change DP's default CC7 value to a user defined value via preference.

[Shooshie]

Hi Shooshie.

Thank you for your input. (I have tried Automation snapshots in the past.) Your workaround is another legitimate workaround like dix's, but again it comes with the price of slowing you down later in the mix because you froze the mix at the start, meaning when you start enabling automation playback, the Track Mutes will be frozen into the Enabled position, so if you want to briefly Mute a track, it won't be brief. You'll have to go to that track, go to the beginning of the session, delete that track Mute automation so you can mute the track while automation playback is enabled. Also, if you want to change the pan, you'll have to go delete that too.

If I was going to try your method, I would adapt your recipe a couple ways:

-First, when you said create a "New Mix". I would hesitate doing that if you have any plugins assigned already. They'll be erased if you choose New Mix. I would do Duplicate Mix if I was to choose anything in that menu to preserve all the plugins.

-Second, on the Automation Setup Page, in the Enable Automation window, under MIDI, I would UNcheck both Pan and Track Mute, or at the very least Track Mute. Then carry on with the rest of your recipe. The less parameters that you write, the less you tie your hands. In my experience, losing an instrument's pan is not mission critical. But losing volume stinks. My preference would be to uncheck pan and Track Mutes. Then after the user does your recipe, then go back into Automation Setup page and the re-enable those parameters. It's still a bunch of steps and things to remember, but this will keep you from losing those parameters.


Ok, but let's dig a little further: Given how DP acts with MIDI automation, I have another problem scenario for you that your Automation Snapshot workaround will not help you with. And again, it just makes no sense that it works this way:

I made another video! I know! So exciting!……. https://www.youtube.com/watch?v=ogmwaPssGCM

In this video, I'm working with Mod Controller (#1) automation. We are now in the middle of a theoretical mix. No Snapshots can help us here. I made a patch in a VI where the Mod Wheel controls a filter cutoff. I then want to automate that filter cutoff. This patch is an arpeggiated bass line.

The video starts, and the MIDI note has already been recorded to trigger the arpeggiator. So, when I hit play, I'm recording the mod wheel automation on the track live. And you can see it being recorded. The filter that's being automated starts mostly closed and gives the arpeggiated bass line a nice mellow feel. As I keep opening up the filter with the Mod Wheel, the sound keeps getting more lively. Great. Sounds good. That's how I want it to sound.

But, what will happen when it loops back to the beginning to play the sequence again? Remember, the sound started out nice and mellow and then got more lively as it went with the filter opening. The next time I hear it, I want to hear it the same way----mellow start and then get more lively. Do you think it will start out mellow?

The answer is NO! The first automation point came after the note was played. It starts out JAMMING----filter OPEN, and then when it hits the first automation point, the filter does an immediate shut down and THEN it gets mellow. This is because DP doesn't remember where the Mod wheel was before that first point of automation because it didn't send any information back to the beginning of the session when I started automating. Again, this is a mix KILLER. Now, I have to go back and move controller automation points around to make sure I put a Mod Wheel automation point BEFORE the note plays to shut the filter down. Here we go, chasing down more workarounds.

Now, if I was automating a Filter plugin on an audio track, I wouldn't have to do that! The audio track automation would have remembered where the filter started and would have played back perfectly the second time around.

So, I keep asking myself, how many things do I need to workaround when it comes to DP and MIDI automation. It just doesn't work the way it's supposed to. MIDI Automation should be one of the most BASIC things DP does for goodness sake! And again, all of these things----volume, mutes, pans, Mod Wheel----all of these things are VITAL to the mix. If any of them get hosed, your intended mix changes! And it's so easy to mess up given how it works at the moment.

I'm glad MOTU keeps working hard on adding new features to DP. But I think they really need to spend some time correcting how MIDI automation is implemented.

(I've been sending these writes-ups to Dave and Matt too. But I want people to chime here and give their experiences. This is what can help make it a more relevant topic for MOTU. Thanks to you guys that have responded already.)

Thanks.
-gabe

Gabe, you're preaching to the choir. I'd also make the same amendments to my work-around list; I was just in a hurry last night, and did it off the top of my head. I always Duplicate the mix to create a new mix. Actually, as far as faders are concerned, you don't have to do that anymore. MOTU fixed that about 5 years ago. But for plugins, of course, that's necessary.

As for the rest of what you said, I completely agree. It needs to be fixed. In fact, ALL of DP needs an update so that ALL MIDI works like Audio, at least where it's appropriate. I understand why it does not. I just think it's time to go beyond what were previously considered completely different beasts and make them look alike.

Remember when Adobe Illustrator and Adobe Photoshop were completely different beasts? The one was a vector app, and the other was bit-mapped, and never the twain shall meet. ... Until... one day Adobe made them work very much alike. Oh, they're still different, because they exist for different purposes, but each has enough of the other to feel like familiar territory. Well, when MOTU made DP, they combined MIDI and Audio; two completely different beasts. At first, it was such a joy to have them together that we overlooked things like the low-res controllers of MIDI vs. the smooth ramps in Audio. And the way automation works in general. And assigning Input/Outputs. And Mutes. And so on, and so forth. I think MOTU could now address those issues and make the two "sides" of DP behave more alike, sometimes even combine them in the same command. They've attempted to do that with tempo. If they could go all the way now, and make it all transparent to users, then we'd have something truly amazing.

I'm used to it. I deal with it. I make my way around it with the tools provided, but I think it's time to change it. Until then, I'll keep making workarounds. The object is to get our work done as quickly and easily as possible. The tools provided make a lot of things possible in what seem like arcane routes from A to B, but they work. We must not let the lack of specific features overcome our determination to find ways to get around those lacks. But I'm 100% in agreement that it's time to change. MOTU has the programming talent to do this. It needs to be done.

Shooshie

[Gabe S.]

Gabe, I watched your first video and I'd like to know the following:
-The first time the loop plays, at what volume to you want it to play? Since there's no CC7 data before the first note, DP will play back at the last seen CC7 event or the DP default, which is 127. I think what you're asking is for DP to correct for the fact that you did not place an CC7 point before the phrase: to look ahead, find the next CC7 or any CC and apply that before the first note of your MIDI phrase and NOT chase.

Hi frankf. Thanks for your notes.

I just want DP to play back at the level the fader was sitting at. In my first video, the fader was sitting at 14. DP could learn this from the first CC7 automation point because the first automation point was 14. It's not like the fader could have been sitting at 98 and then somehow with automation on, I snuck the fader past the automation system and magically appeared at 14. The fader was sitting at 14, and the first automation point indicates that. So, DP knows what level the fader was at with the first touch on the fader.

Later, I just want it to play at 14 when it rolls over that same area again. When I automated, I made a very subtle move up to around 20 or so. I certainly have no interest in a default of 127 for the volume level on that track all of sudden out of the blue.

And you said I want "DP to correct for the fact that did not place an CC7 point before the phrase." You make it sound like I made a mistake for not automating sooner. Well, I don't have to place that "pre-placed" point on an audio track, and I don't have to automate sooner on an audio track. Why do I have to do it on a MIDI track? To say "well, they're different" is not a valid argument anymore. Could you imagine having to also place all these silly "pre-placed" points on audio channels for automation? I think we'd all go find another platform to work on, so why do we have to deal with this when working with MIDI?

When my volume is at 14, a volume 127 is from left field and has no business being the volume of the first part of that track. So, yes, what I want DP to do is look at the first point of automation on this current pass, see what the value is, and then look backwards on that track to see if there is an automation point already. If there isn't, then put a point of the same value as the first CC7 value at the beginning of the session while I'm recording the automation. That's how audio automation works. I'll say it again-----I don't have to place an audio automation point at the beginning of the session.

Stubbsonic mentioned something about a "Chase Forward" feature. Hey, if that works, great! But chasing backwards can only work if there is a point to chase. DP would have to start inserting a point earlier so there is a point to chase. Someone else mentioned turning off chase to avoid some of these problems. I really don't like that solution. Again, these are all poor workarounds for a glaring problem…...

-So you would have to define the phrase that you want to apply this to if there are any prior phrases, no?

To answer this, let's consider an example based around my first video: an RMX MIDI groove. Let's actually assume that this was not the first groove on this track----like we are now in the middle of the song, and that there was no automation up to this point. If the fader has been at 14 this whole time, then THAT'S the volume I want the track to be at until this automation move. DP should read the first automation point, which while be level 14 and look backwards and look to see if there's any prior CC7. In this case, there isn't any, so it should place a level 14 CC7 controller at the beginning of the song. And this would be correct. The other grooves have all been playing at 14. That's where the fader has been sitting the whole time. I just wanted to bump up the volume a little bit in this current part of the song. I DEFINITELY DO NOT want the fader to jump to 127 for all the groovers prior to this point! That's what it will do now. If there was CC7 automation on the track prior to this point, then DP is a clever program. I'm sure it's possible to make it search for the most recent CC7 on that track and if it finds it, then it can place a CC7 of 14 one tick later than the last automation point.

-When looping as per your example, would DP stutter having to do this in real time?

Do you know how many calculations DP makes in realtime? I have 14 Altiverbs running live in realtime in my trailer template. And that's just the beginning. I have 5 Kontakts, Omnisphere, RMX, endless DSP plugins, audio playing, over 100 audio/aux channels. Placing a MIDI control is about the simplest thing in the world for a modern day computer.

I'm of the school, actually what Shooshie said, that you should place a snapshot either at the beginning of the track or prior to the first MIDI note in a track. I would not do what you show in your demo, insert CC7 ramp in the middle of a phrase and expect it to play at the same volume when looped OR when chasing. there is simply no data to chase.

Why not? That's how audio automation works!

Also note that MIDI and Audio automation are different beasts in DP:
MIDI: 127 discreet changes in CC7 and CC10
Audio: Continuous sample accurate (or ramp) automation

I hear you, but this is 2015. Surely it is possible to make this more user friendly. They are implementing live Spectrograms on audio tracks in DP9 for goodness sake! What kind of processing does THAT take?

But back to what you asked (complained) about. MIDI CC7 and Audio Volume track automation do behave differently. It's easy to see if you put an empty audio and a MIDI track side by side in the SE with the MIDI track in lines mode and take a snapshot a few bars in. You will see the Volume line going all the way back to the beginning of the track, even though DP does not place a break point there. (DP does behave like a break point has been inserted. Try adding a breakpoint between the beginning of the track and the previously entered breakpoint) The MIDI line stops at the location of the snapshot. Because there is no CC7 automation point before the snapshot location, DP sets CC7 for the track to its default: CC7=127. I agree it would be better if MIDI automation tracked back from the first CC7 data point in a track and placed an identically valued point at the start of the track ala volume automation. However, this has been the way MIDI automation has worked forever, at least in DP, and I consider it good practice to set MIDI CC automation points before any notes play. I put them in bar 0 myself. I don't consider this a work around, instead a work flow.

Another way to implement is to change DP's default CC7 value to a user defined value via preference.

Of course, just because this is how it's implemented at the moment doesn't mean it's the right way to do it. For MOTU to say "this is how we've always done it and that's how we'll always do it" would be silly. They've changed all kinds of things in the program. You may not view your method to be a workaround, but it is. I do the same thing as you! We are both putting CC points at the session start because our mixes mix won't work properly if we don't!---well, at least on MIDI tracks……we don't have to do it on audio track because they're special…..haha ARGH! And no one has jumped on here and said, "You're wrong Gabe, I love it like this"…….bascially every person who has bothered to respond says their way of dealing with it is a workaround……that's all I've been trying to do is start a dialog on this. And that's how you get developers to put things on their list of thing-to-do. I have a long history of beta-testing for music companies---Waves, Spectrasonics, (even Opcode back in the day), etc. (MOTU was never interested in me beta-testing for them. That's fine.) I'm used to that dialog with developers. And DP is not a hobby for me. I use it all the time and this particular topic has been a sore spot for me for a long time. I've never taken the time to sit and write about it like this, and make videos and respond to people and blah blah blah. But now I am taking the time and I want to see if we can put this on MOTU's radar as something to deal with.

So, again, I appreciate everyone's time.

Cheers.
-gabe

[frankf]

Gabe,
You don't have to defend yourself. You were pretty adamant in your OP about how you feel about this feature and if you read all of my post you will see that I agree with adding it. That said, because of the reasons I gave, I'd still stuck with the workflow I describe as it served me well

[frankf]

If I didn't mention it earlier I use this workflow of inserting a snapshot at the head of each track whether the track is a MIDI track, audio track or aux track

[Gabe S.]

If I didn't mention it earlier I use this workflow of inserting a snapshot at the head of each track whether the track is a MIDI track, audio track or aux track

hehe well, I hope a day comes when we don't have to take those steps anymore. I don't do it on audio tracks since it's not necessary, but I have been working similarly to you with the MIDI tracks.

I may experiment a bit more with a global snapshot for volume just as a safety/something-to-revert-to, but my mixes don't really have a starting point in that traditional sense of the word. With a large template, I start with what's there and then adapt it to the music I'm creating. So for me, it's a constant flow process of taking a well established starting point and shaping it to a specific creation. And of course, I'm always adding stuff to it as well. So I want it to be as flexible as possible.

Cheers.
-gabe

[Michael Canavan]

Just to be fair, all other DAWs I've used are also 'dumb' about MIDI automation, in that previous to the first writing of automation it sets a default of some sort, so you often have to write fades before MIDI notes start so things don't go haywire etc.

I've often thought that maybe glitches in the usability of MIDI in DAWs has been one of those "OSC is right around the corner." things, as in developers have thought that a new standard was imminent. That's obviously not the case.

[cleamon]

This is the reason I always route my MIDI track's VI output through an AUX and automate it there. Works EXACTLY like audio then, plus I can use audio plugins. Of course, my projects are small and contain relatively few tracks (less than 30).
Just saying.

[Shooshie]

Years ago, I drove cars with standard transmissions. I had my reasons, including more control over the car and cheaper repairs, though clutch replacements sometimes brought parity to the cost aspect. People seemed to gravitate to automatic transmissions, and some accused me of being old fashioned, not appreciating a good thing, and I responded with indignation that they could not understand my reasons for wanting precision control over my driving.

Time passed, I got older, my foot got tired, bumper-to-bumper traffic got unbearable, and I finally swore I'd never have another standard transmission. And it's been great.

When someone calls something I'm doing in DP a "workaround," I feel my inner wolf's fur raising along my backbone. Sometimes my fangs appear. I learned how to do things in DP and to have extreme control over what I do, while working at deadline speed. When working with clients, I usually have what they want before they finish their first sentence. Then I have to wait for them to quit talking so I can demonstrate it for them. One reason I'm able to do that is that I developed methods for those things. "Workarounds" as some people call them.

In conversations like this, it's that accusation of "workarounds" that tends to make me combative. Frank's right: these things are done this way, because that's how they are done. And yet my foot grows tired; the bumper-to-bumper traffic makes me want easier methods, not just faster ones. So, I have to also agree with Gabe. Yes, we do things this way, but yes, there COULD be an easier way if someone would just program it.

I never complained about this particular thing, because it takes me less than 5 seconds to FIX it for any number of tracks. I don't care if there are 300 tracks. It still takes just about 5 seconds or less to deal with it for the whole project. Is that a workaround? By comparison, it takes forever to name tracks. Shouldn't there be some feature to make that faster? (Yes, there ought to be) In fact, there are many things that ping my radar a lot more attentively than MIDI automation. But I do agree with Gabe that it could be done better.

What I don't like is calling current methodology "workarounds" in a disparaging way. Half of what I do could be called that. Heck, there are things I don't even bother to share, because I know people will slam them as workarounds, and say they are too complicated. And I feel that's a pity. I spend a good deal of time here trying to teach techniques, both basic and advanced, that will help people to realize what they can do with DP. If you're using this software, you've got tremendous power in your hands. Power to do amazing things with music. When newbies complain that DP just doesn't seem up to the task of a real DAW like Logic, I patiently try to explain to them what they have in their hands if they just learned a thing or two. But when someone like Gabe complains, you have to listen. Gabe is one of us. He's an old-timer telling us that an automatic transmission is a good idea, while a lot of us are just very, very accustomed to shifting gears manually. So yes, I get it. Let's hope for the best with the upcoming DP9. (and with automation lanes already announced at NAMM, who knows what other changes may be in the pipeline?)

But just for the sake of keeping the peace, guys, let's try our best not to disparage our old ways as "workarounds." You can call it what you like, but when you disparage it, you're disrespecting those of us who made a career out of those things, and who have helped pave the way for others. Yes, I'm ready for the automatic transmission, but I'll never forget speeding across the country, or just over the hills from Fairfax to Bolinas with a standard, downshifting on the curves, powering up the hills, a beautiful woman in the seat beside me...

Shooshie

[MIDI Life Crisis]

Manual transmissions can increase you gas mileage dramatically- if you understand how to use it effectively. Same thing with any tool, including DP. I've used MIDI consistently and intensely since 1987 and don't see the issue being presented here as significant. Chase controllers. Done.

[Gabe S.]

Shooshie, that was an awesome piece of text! Your thoughts are always appreciated. I will try not to use the "W" word as much (see below).

hehe I've been waiting for MLC to weigh in. I'm bummed that MLC is not on our side…..ah well….

With the analogy of cars, you and MLC are bringing up my other passion-----cars. My hobby is driving at the racetrack.

Anyway, this car analogy plays well into our story. Lots of guys LOVE manual transmissions. They feel connected to the soul of the car. (In our analogy, a manual transmission is like MIDI automation. It works but has some potential pitfalls.) On the streets, (analogy: casual DP user/less complex mixes), the manual transmission is ok. Not a lot of crazy shifting going on at redline. Rare chance of a money-shift (blowing up your engine---analogy: ruining a mix). But on the racetrack, you live at redline, things move very fast, lots of decisions are made quickly, and the chance of a money-shift under this intense use is extremely high (analogy: pro user of DP mixing tons of tracks all day long under deadline pressure).

Now, there's an automatic transmission that's totally smooth (analogy: audio automation) and it's impossible to blow up an engine. But for a number of reasons it's not ideal on the track (analogy: technical differences between audio and MIDI tracks).

So, BMW decided to evolve their track-oriented cars (the M3 line) and they started offering a paddle shifting dual-clutch transmission. The car is technically a manual because it has two clutches and no automatic gearbox, and the user is in charge of shifting the gears, but there's no clutch pedal because the computer handles all the shift-revving and clutch tasks. The paddle shifters are LIGHTNING fast----much faster shifting than a manual could ever be. It's impossible to blow up the engine because the computer will ignore your shift request if you try to do an unsafe shift for the engine. That's what I drive on the track (BMW 2009 M3). It's a fantastic driving experience. I would not drive any other transmission on track. Fun, fast, simple, connected and no blown up engines.

Every person in the world who knows about transmissions will tell you the dual-clutch paddle shift is the fastest shifting that you can do. The point is the world evolved to this. BMW (and other high-end manufacturers) didn't throw their hands in the air and say "well, that's just how it's done". And yes, there's still tons of manuals driving quite nicely out there and they're definitely having fun. But I prefer the evolved paddle shifters. I know I can't blow up my engine and it's faster. Just like what I've been asking for in this thread----faster working process and no ruined mixes. (The current Porsche 911 GT3, which is considered one of the most track-capable cars out there, is only offered in paddle-shift now. You can't even get a manual version of it. The world is evolving.)

Phew---ok I'm done with the analogy.

In my years of beta-testing, I have made MANY MANY requests. Poor Eric Persing at Spectrasonics has had to sift through my barrage of dumb requests over the years…..hahaha!! (10 years ago, I was the beta-test moderator for Stylus RMX and wrote that 70,000-word user guide/manual for RMX.) Many requests he rejected and said they weren't what he wanted for the product. But other requests would generate a discussion and occasionally, a request would see the light of day. There are many very good testers involved with all of our favorite companies, and they all make suggestions that I think are useful and benefit the end user. They offer an outside, rubber-meets-the-road perspective of these products. My goal in all of this would be to challenge MOTU to evolve the MIDI automation so that it's more intuitive and faster to use. If Michael Canavan is right and all the MIDI programs work this current way, then MOTU would have an edge on providing a better MIDI workflow than everyone else. I think that's a great way to stand out from competing DAWs.

And cleamon's idea of routing everything through Auxes is a legit---uh-----alternative method of performing automation tasks. (I won't say the "W" word!) But I can't do that because I have too many MIDI tracks…..

How cool would this be?: A preference check-box option that says: "Use continuous automation for MIDI." Oooh that would be cool.

Thanks for the conversation guys.

Cheers.
-gabe

PS. Here's video of me driving at Buttonwillow Raceway. All that fancy rev matching you hear on downshifts is being handled by the computer. Awesome! Of course, some will criticize and say I'm just along for the ride since the computer is handling those tasks……ah well…. https://www.youtube.com/watch?v=9RhEXjUdcPU