How is this kind of thing done?

[FMiguelez]

I stumbled upon this YouTube video, and I found it quite amusing because it's music (production technique) AND science, two of my passions in life

I've never done anything like that, and I'm not sure how I would approach it for doing it myself.

Is that something as simple as pitch shifting, together with some kind of vocoder, perhaps?

The rhythm of the original speech seems to dictate the rhythm of the "melodies" because you can see it syncs with the scientists and actor of the video as they spoke.
[Edit: Well, this to an extent because upon closer inspection, I just noticed the video was edited to fit the melody (with Brian Cox, for instance) ].

I've never bothered seeing how DP interprets "pitch" for normal spoken sentences. Could it be done there? I think it would give too many artifacts. Perhaps a better pitch shifter?
What kind tools would you suggest?

Some scientists are made to "sing" much better than others. I can definitely hear the artifacts in Richard Feynman, but Brian Cox and Michio Kaku sound absolutely brilliant

https://www.youtube.com/watch?v=DZGINaR ... 64C7551022





Four forces of nature...

[FMiguelez]

Hmmmm....

On second thought, slicing on MachFive, some of its effects and IRCAM would probably be my weapon of choice.

Any better ideas?

[Shooshie]

Seeing how it's full of digital artifacts, and the digital artifacts are what give that kind of thing its character, then it doesn't matter that you will get those artifacts by transposing the audio with either Antares Autotune, Melodyne or DP Pitch.

I haven't used Autotune or Melodyne, but I've used DP Pitch. You might try it with the latter, simply because it's a free tool at your disposal. If you start seeing promise, but it's not quite there in DP, then go to one of the specialty tools. My guess would be that Melodyne might do the best job, because the guy is such a fanatic (in a good way) and has taken it so far. But getting back to DP Pitch...

Within the pitch window, I have found that the little "MIDI Piano Roll" controls are bursting with errors. You can cut through most of those errors by cutting pitch segments so that the bad portions can be deleted, then stretching good ones to the lengths you need. (or pasting them in) You can transpose by quite a lot, but you will get extreme artifacts in the form of the sound itself, so it's not good for transposing an opera singer to a distant key, but for making Cher sing in tune, or making physicists sing at all, it's passable. Just keep simplifying the fragments and single notes struggling to be two notes by cutting and deleting until you have just simple pitches, well connected.

When you're having trouble editing them, zoom in. That shows you a lot more detail, and you can deal with it accordingly.

As for finding the music in speech, I've been doing that for decades. Good poetry exploits the music of the language, and it can be done with mastery, or it can be done in sing-song nursery rhyme speech. Listen to yourself as you talk. Every mood of every expression has a pitched component. For me, ending a phrase with a question mark usually causes an octave portamento. I've measured most of my speech pitches and noted their changes with moods, audience and time.

Once, when creating a five-piano show with Yamaha Disklaviers, I was asked to "make them talk." The star of the show would have a dialog with pianos. It just happened that I already knew these intervalic relationships, and it wasn't hard to hear those I didn't already know, so it only took me an evening or two to create dialog in the pianos. It just had to be quick to make it work. The result was "almost wonderful." People would tell me after each performance what they thought the pianos had said, and it was always pretty much on target. They got the gist of it, anyway, if not the exact words. I said "almost," because the manager kept changing what they wanted the pianos to say, and in the end some of the phrases were quick hack jobs. With a little more time, I could have made them all sound more natural and inspired, but when you're the only music director a show has, and everyone needs you for everything from dawn to midnight, you don't always get to polish everything. Still, it worked, and the people got it. That was great fun!

So, turning actual speech into music is taking it in the other direction. Instead of being "quick" so that the musical intervals sounded like speech, I think I'd be doing the opposite: slowing down the musical intervals of speech. Not much, just enough that they can be heard and harmonized.

When writing poetry, I'll walk around repeating phrases endlessly until I get it to roll off the tongue with a certain musical property. I know what I want to say and the images I want to generate, but rather than just say it as is common in so much modern poetry, I also want it to sound like it sings without effort. I don't want it to sound like I spent an hour on a single line, trying different words and phrases until I got it to sing. It can't be a tongue twister; it has to sound natural, but sound as though by some freak coincidence, these spontaneous words also just happened to sing. Pitch, rhythm, internal rhymes, alliteration, consonance, assonance... all the tools of the poet come into play with the addition of my lifelong obsession with the written or improvised musical notes.

What I'm saying is that to make THEIR speech into music, you will do the same thing, but kind of in reverse. You say their words as they say them, over and over with a little emphasis on the musical intervals, and gradually you hear a musical line in the phrase. Then you harmonize it and pick out its rhythmic components and write to it.

Is that making sense to you? I don't think you'll have any trouble if you try it. Just remember that you've got to stay loose — keep your mind relaxed and open — and let the phrases sing to you. Then you have to work with each one of them until you hear what it's trying to sing. It may take 3 seconds or 3 hours for a single phrase to come to you musically. Just don't settle for anything less than what inspires you to hear it played back. Oh... and it's got to gel together musically as a whole, not just a collection of phrases.

Once you find what you're looking for in a phrase, use DP's Pitch tools to emphasize the musical intervals, and even to change them to fit what you're hearing. Do time stretches, cuts and pastes, repetitions, maybe even retrograde, to get it rhythmically interesting. I'm betting that you can do a FAR better job than the one we just heard without leaving Digital Performer.

Sounds groovy...
Shoosh

[BKK-OZ]

Another example (vocals start about 30 secs in): https://m.youtube.com/watch?v=nXJ5_m_lmiM

Methinks we are talking about a vocoder, ala the TC Helicon, using a MIDI controller to drive pitch, with something like DP's beat alignment to get the vox to sit where you want it.

[Babz]

It is done using Auto-tune via its "Target Notes via MIDI" option. You basically play melody notes on a MIDI keyboard along with the speech. Then go back and clean things up with the mouse. Or you could do it note-by-note with just a mouse, but I would think that would get quite tedious. You could do it with other pitch tracking plugins besides actual Auto-tune, if they allow you to input with MIDI notes. AFAIK, DP's pitch features do not.

This technique has been around for a long time. You see it all over YouTube, usually with "Autotune Remix" or "Songify" as part of the title. Some of them are hilarious and very clever.

I was interested in this technique awhile back and posted here asking if anyone knew how it was done. Given how widespread these kind of videos are, I was surprised no one here had really done it.

I researched auto-tune remixing fairly extensively at the time (google "how to do auto-tune remix videos" or some such), but other things got in the way, and I don't own Auto-tune, and never got around to buying it, and ... So I haven't tried it, but remain interested, and would like to try it someday.

See this thread:

http://www.motunation.com/forum/viewtop ... tune+video

The real master of this is a guy named John Boswell (who works under various handles, usually "Melodysheep"). He is the one who did all the PBS stuff. (The auto-tuned Julia Child and Bob Ross videos are AWESOME!) I found a few interviews online with Boswell, but none where he really goes into detail about exactly how he works. I would love to find a step-by-step tutorial video. You might want to try contacting him. If you find anything, please post.

Or, if you own Auto-tune, please give it a try and report back how it goes.

Best,
Babz

[Babz]

Oh, I should also mention that before you do the melody via MIDI note input, you usually will need to chop up, stretch/compress the speech to fit the rhythm you want. You basically compose a suitable backing track based around the speaking material you have to work with, chop it up rhythmically, and then input the melody you want. Then you cut the video to your music track. An involved process for sure.

Best,
Babz

[Babz]

The Charles Ramsey video was one of the most viral (famous, viewed, whatever we call it nowadays):

https://www.youtube.com/watch?v=_nHBdLjUdQE

[Shooshie]

I want to know why we can copy pitch "notes" and paste them in a MIDI track to create MIDI from audio, but we can't take MIDI and paste it into the Pitch layer. Seems like MOTU would have made that to go either direction. That would effectively work like Autotune's "Target Notes via MIDI." (but does not)

I wonder if they did that as a concession to Antares and/or Melodyne. I had really hoped it worked that way. I'm still happy that we've got it, and that it even will create MIDI. Just thinking of that extra step they didn't do and complaining out loud.

Shoosh

[Babz]

It seems to me that DP's pitch features are more oriented toward correcting subtle pitch problems than at special effects. I wish that I could just draw in pitch data (to do tape stop effects, etc.), but it won't let me do that on polyphonic audio or on anything where it doesn't detect pitch. You can only manipulate pitch, if it detects pitch first. Wish they would change this.

People have also often wished for a simple varispeed effect. I don't recall the explanation for why this has been a problem for DP. I have heard it said that things are different in Pro Tools, for example, but I don't recall the specifics.

I wish they would port the pitch capabilities of Mach V over to DP.

Babz

[cuttime]

And of course, there is this, if you want to be on the cutting edge of tomorrow's dated effects!

[James Steele]

And of course, there is this, if you want to be on the cutting edge of tomorrow's dated effects!

Well dress me in pigtails and a Hello Kitty backpack! Heck... that would be fun to have just to bust out for laughs!

[Phil O]

Well dress me in pigtails and a Hello Kitty backpack!

Now there's an image I don't want in my head!

[FMiguelez]

Thank you all so much for the answers and thoughts

Right now I'm watching a few tutorials on how to do it with Autotune.
I've never felt the need for AT. DP's pitch has been more than enough for my tuning needs, and since I'm not really into that kind of music, AT always seems to slip down my to-buy list.

But I LOVE smart studio trickery and techniques, so that might change soon, just for the fun of it.

Babz, I don't know who Charles Ramsey is, but that video was more impressive than the one I posted (for the technique).

Thanks!

[Shooshie]

The Charles Ramsey video was one of the most viral (famous, viewed, whatever we call it nowadays):

https://www.youtube.com/watch?v=_nHBdLjUdQE

I had not seen that. It was fantastic!

[BKK-OZ]

Right now I'm watching a few tutorials on how to do it with Autotune.

Any in particular you would recommend?