Mastering software

[Shooshie]

I spent all night with both apps and hear a difference. Voodoo? To me, its there. Don't claim to understand it and don't think null is or has ever been the final answer...

A positive null test proves that the files are identical. If you get 100% cancellation when running to files 180° out of phase, then the files are identical and any difference you hear is the result of variances in human perception, which can certainly be enhanced by staying up all night listening for differences.

If you want to know if there is any physical difference between two audio files, a null test is the final answer.

Ahh, but the Null Test can't tell you which file has greater airspace in the presence, or fatter bass, or more edginess in the solo. Sure, the files are identical, but one has the edgy, fatter, airier presence!

Hey, I know this, because i've been told by people who know people who know a great engineer who did the squawk-squawk track on that Beatles album with the band on the album cover; you know the one. And he said the null test can only go so far; and while it may show what's scientifically identical, everyone knows that even identical twins can have real different personalities.

And while we're at it, there's this piano player who can play one single note that will make you cry, because it's so expressive. And it's crazy to think that a Disklavier can reproduce a note like that, because it's using magnets and rods, and nothing can replace the human finger with the gift of touch. Yep, that guy's finger can play a single note and get a standing ovation for it, because no audience ever heard any note so expressive on a piano.

And I've got this compressor that can make a digital track sound exactly like TUBES, and it gives you all this edgier, fatter, airier presence that no null test can detect, because its dials go to eleven.

And Kennedy was shot by a UFO, and Julia Child's recipes were so good, because they used puppies, and oh man, I'm just getting started!

Except that I'll stop here. Not to worry; I'm just repeating things I've heard when I've tried to convince someone that the null test proves beyond any modicum of doubt that two audio tracks are I-Freaking-Dentical.

Identical, yes, but remember what I said about identical twins. One's often the good one and the other one is the bad one, so even a null test can't tell you which track is the good one and which is the bad one... it takes that piano player with the expressive finger to tell you that. He came from a UFO that shot Kennedy, and he eats puppies, because he's a friend of Julia Childs, and he's got this compressor that's part reverb, and it puts fat airiness back into the presence and makes it edgy, and...

Ok, ok, ok... have your null test! Did I tell you that mine goes to eleven... oh never mind.



Shooshie

[Dan Worley]

Identical, yes, but remember what I said about identical twins. One's often the good one and the other one is the bad one, so even a null test can't tell you which track is the good one and which is the bad one... it takes that piano player with the expressive finger to tell you that. He came from a UFO that shot Kennedy, and he eats puppies, because he's a friend of Julia Childs, and he's got this compressor that's part reverb, and it puts fat airiness back into the presence and makes it edgy, and...

Gosh, I didn't believe any of this until you pulled it all together for us at the end. Now it all makes perfect sense. So sad about Kennedy and the little puppies, though.

But didn't you mean phat airness, or am I confused by this point?

[Shooshie]

But didn't you mean phat airness, or am I confused by this point?

Darn spell checker. Can't write something cool anymore.

Shoosh

[Timeline]

I spent all night with both apps and hear a difference. Voodoo? To me, its there. Don't claim to understand it and don't think null is or has ever been the final answer...

A positive null test proves that the files are identical. If you get 100% cancellation when running to files 180° out of phase, then the files are identical and any difference you hear is the result of variances in human perception, which can certainly be enhanced by staying up all night listening for differences.

If you want to know if there is any physical difference between two audio files, a null test is the final answer.

And it's crazy to think that a Disklavier can reproduce a note like that, because it's using magnets and rods, and nothing can replace the human finger with the gift of touch. Yep, that guy's finger can play a single note and get a standing ovation for it, because no audience ever heard any note so expressive on a piano.

So is that like the finger of Eric Von Zipper in Beach Blanket Bingo?

[Timeline]

So Shooshie. Do you have a high bandwidth Spectrum analyzer to look at frequencies just above human hearing when a complete digital null to the ear exists? Are you saying that harmonics just above the hearing range have no effect on the sound? If so your a bit closed minded to the fact that we might be perceiving pressure from sound that can effect presence and tone from these ranges and unless we measure properly, we don't really know, at least above 44.1.

[Shooshie]

So Shooshie. Do you have a high bandwidth Spectrum analyzer to look at frequencies just above human hearing when a complete digital null to the ear exists? Are you saying that harmonics just above the hearing range have no effect on the sound? If so your a bit closed minded to the fact that we might be perceiving pressure from sound that can effect presence and tone from these ranges and unless we measure properly, we don't really know, at least above 44.1.

I use meters for testing a null effect. Meters can go on up a good way. Sometimes my tests are at 88.2K, which of course takes it way on up out of the hearing range. I don't do this often, but when I do, I try to be thorough.

[bayswater]

So Shooshie. Do you have a high bandwidth Spectrum analyzer to look at frequencies just above human hearing when a complete digital null to the ear exists? Are you saying that harmonics just above the hearing range have no effect on the sound? If so your a bit closed minded to the fact that we might be perceiving pressure from sound that can effect presence and tone from these ranges and unless we measure properly, we don't really know, at least above 44.1.

The null test deals with all frequencies. If the file nulls out, the audio is identical at all frequencies, not just those in the normal range of audibility. It is irrelevant whether anyone can hear them, sense them or is affected by them in any way. You don't need meters to detect this. A nulled out audio file consists of only zeros. That's how you know the files are identical.

[Timeline]

So Shooshie. Do you have a high bandwidth Spectrum analyzer to look at frequencies just above human hearing when a complete digital null to the ear exists? Are you saying that harmonics just above the hearing range have no effect on the sound? If so your a bit closed minded to the fact that we might be perceiving pressure from sound that can effect presence and tone from these ranges and unless we measure properly, we don't really know, at least above 44.1.

The null test deals with all frequencies. If the file nulls out, the audio is identical at all frequencies, not just those in the normal range of audibility. It is irrelevant whether anyone can hear them, sense them or is affected by them in any way. You don't need meters to detect this. A nulled out audio file consists of only zeros. That's how you know the files are identical.

Not necessarily at higher frequency, I'm told where phase becomes more unstable but yes Null counts as the basis of these tests and has been the method since HP sold studios the first intermodulation distortion analyzers back in the '60s. I still have one sitting around in my garage somewhere. It was a very interesting piece of test gear in its day and automatically nulled from its inputs to find distortion harmonics not part of the A channel feed.

This is where it all started and I would never say that Null is useless. I'm saying other factors MIGHT be involved, especially in digital .

Pretty absolute on that opinion Bayw. Yes the Null deals with higher frequencies but if phase can vary even slightly as you go up into higher than hearing frequencies, maybe its not the end all test and possibly there is more to learn here and I intend to keep an open mind on the subject until i read an AES or tech paper from someone i respect who can settle this.

If the point is to just post here to hijack or pick out someone posting by sarcastic slams who is simply keeping an open mind on this subject then I have allot less respect SHOOSHIE than ever before for your position. I laughed as well but at your post and agree it was well written but is this really fair to point at me rather than the test concept itself?

I should expect such on this off topic channel i guess but surprised coming from a moderator.

[bayswater]

Pretty absolute on that opinion Bayw. Yes the Null deals with higher frequencies but if phase can vary even slightly as you go up into higher than hearing frequencies, maybe its not the end all test and possibly there is more to learn here and I intend to keep an open mind on the subject until i read an AES or tech paper from someone i respect who can settle this.

Timeline, this is simple math, it's not complicated, and not a matter of opinion.

Take a simple example. I have a digital audio file. It has 7 samples. The values are {5, -10, 14, 23, -36, 4, -8}. I load that sample into an audio app (e.g. Logic), and bounce it back to disk. Looking at the new audio file I see {5, -10, 14, 23, -36, 4, -7}. The last value changed. So Logic in this example, did something to the audio.

In real situations, I wouldn't want to actually look at all the values in an audio file, so I would take the output of Logic and invert it and get {-5, 10, -14, -23, 36, -4, 7}. Then, I'd add this to the original sample and get {0, 0, 0, 0, 0, 0, -1}. They are not all zeros, so the null test failed. The files are different, and they will probably sound different.

But if the bounced back audio was {5, -10, 14, 23, -36, 4, -8}, then the inverted file is {-5, 10, -14, -23, 36, -4, 8}, and the null test gives {0, 0, 0, 0, 0, 0, 0}. The files are therefore identical. If played on the same system they will sound identical. Playing the first one, then the second one would be the same as playing the first one twice, or the second one twice.

Can the argument above be extended to any digital audio, including all its characteristics such as frequency, phase, offset, etc? Yes, because the process of digitizing renders any audio and all of its characteristics down to a string of numbers like the examples above.

[Timeline]

Well Baywater, here is an opinion from a plugin tech you might be interested in regarding plugins as I'm asking him about Null in regards to their 0 phase HD eq at PSP I'm considering buying because their plugs work well in DSP-Q. In this statement null is not enough:

Re: Null tests
In my opinion many internet sources are really not right about using Null test about the proper solution. For instance it is often claimed that all digital filters are the same because a cheap EQ can almost cancel with a 500$ EQ. They always forget to mention that the test is done under several reestrictions. It is done using a peak filter with average gain and Q and of course in the middle of the frequency band. The truth is that digital equalizers meet the strongest issues in the lowest and highest octaves. It is also worth mention that things get even more complicated with shelving filters which are rareely working the same - especially when the plugin mimics any analog equalizer.

On the other hand there are penty of situation when a plugin doesn't pass the Null test with its counterpart which doesn't actually say anything about its quality. In most of real situations a Null test simply doesn't make sense. It can be used to compare a reference converter with a lower quality converter or other equipment which are really comparable but in most cases it doesn't work for plugins. At PSP we mostly care about the sound and mostly we are using selected measurement adequate to what we want to measure. Using a Null test solely would not work for us.
For instance measuring harmonics can be done without a Null test and in most cases I do care more how the compressor really behaves and sounds then just its THD.

I hope you found this helpful.

[bayswater]

Well Baywater, here is an opinion from a plugin tech you might be interested in regarding plugins as I'm asking him about Null in regards to their 0 phase HD eq at PSP I'm considering buying because their plugs work well in DSP-Q. In this statement null is not enough:

Re: Null tests
In my opinion many internet sources are really not right about using Null test about the proper solution. For instance it is often claimed that all digital filters are the same because a cheap EQ can almost cancel with a 500$ EQ. They always forget to mention that the test is done under several reestrictions. It is done using a peak filter with average gain and Q and of course in the middle of the frequency band. The truth is that digital equalizers meet the strongest issues in the lowest and highest octaves. It is also worth mention that things get even more complicated with shelving filters which are rareely working the same - especially when the plugin mimics any analog equalizer.

On the other hand there are penty of situation when a plugin doesn't pass the Null test with its counterpart which doesn't actually say anything about its quality. In most of real situations a Null test simply doesn't make sense. It can be used to compare a reference converter with a lower quality converter or other equipment which are really comparable but in most cases it doesn't work for plugins. At PSP we mostly care about the sound and mostly we are using selected measurement adequate to what we want to measure. Using a Null test solely would not work for us.
For instance measuring harmonics can be done without a Null test and in most cases I do care more how the compressor really behaves and sounds then just its THD.

I hope you found this helpful.

What he says is that if the null test FAILS but almost passes, there can still be an audible difference between the two pieces of audio in the test. That I agree with.

When I did the null test differences between Logic and DP, I found the null test sometimes showed differences around -130 db. The reaction at this forum was that this was enough to account for the blind listening tests reported by Radiogal that concluded DP sounds better than Logic. While I doubt anyone could really hear the -130db signal in isolation, it is credible that when it is present in the overall signal, people can detect its presence.

If your sources had said "For instance it is often claimed that all digital filters are the same because a cheap EQ can completely cancel with a 500$ EQ.", then I'd still have to conclude they will sound the same because they are identical.

[Timeline]

I would agree with that BW. Interesting on your Logic test too. Thanks for posting. Regards, GB

[Shooshie]

Gary, I wasn't ridiculing you, personally. I was joking about the nature of this discussion, period. When you start talking about null tests and digital audio, that's when a lot of misinformation starts flying, and it's based on one fundamental principle: digital audio quantifies analog audio.

What that means is there are no gray areas in digital audio. The mystery of analog cannot be described perfectly in numbers, so digital audio compromises and samples it 44,100 times per second. Those numbers may not describe the analog completely, but they are ABSOLUTE in their description of digital audio. There are no mysterious numbers in digital audio. Every number is the product of a measurement and some mathematical changes as the audio is mixed.

That's a lot of numbers, and to the human mind it's too many to comprehend. That's why we hear it as music, and to our ears it can be just as mysterious as analog, but to the computer there's nothing mysterious about it. The computer is describing an absolute answer to each of a lot of arithmetical problems, very fast.

Because the answer is absolute, there are no mysterious upper partials or strange interactions taking place in the audio. To our ears, it may seem that way, but to the computer, it's going to do it the same way every time.

It's possible that a plug in may have a randomizer designed into it, and that may produce unpredictable results. In reality its very predictable, but a digital randomizer uses extraneous variables to produce what seem like random results. Even so, when we do null tests, we are not using randomizers. To do so would give the test no meaning. So, when we do null tests, we are simply comparing one set of numbers with another.

Once those numbers are spelled out, in the final two-track, they may as well be set in stone. Those ARE your tracks. They will not change from playback to playback. They do not wear out, as needles and grooves do. If they do degrade, the machine will no longer play them. But again, I digress; we're not talking about how they wear. We're talking about the original set of numbers produced in a 2-track.

So, take two sets of numbers. If they are identical — and that means each number matches the corresponding number in the other set — then there are no differences anywhere in their sound; not in the audible range, not in the inaudible range. If they add up to zero, they add up to zero. Zero vibrations make zero overtones. Zero partials. Nothing.

We, as humans, forget that we're listening to numbers. That's a good thing. If we were aware of those numbers at all times, we'd go crazy. It sure wouldn't be music. So, as we listen, it's easy to be lulled back into that analog dream where no two things are ever quite the same. Ever. In those old analog recordings, played back on analog gear, even fluctuations in your household voltage could produce differences, though probably not enough to be audible except in extreme circumstances.

A whole industry capitalized on the nebulous quality of analog, selling us cables, bricks, rocks, connectors, reflectors, and all sorts of things that were reputed to imbue the audio with magical qualities. Believing in magical qualities is enough to make you think you hear them. In some cases, the difference was real, measurable. In other cases, well, probably not.

But when you're dealing with digital audio at the lowest level: the two-track output of a master, there is never any magical quality, never any unpredictable result, never any change. That two track is the same every time.

Once you play it through an amplifier and bring it back into the analog world, THAT sound is subject to analog magic. But remember that the digital two-track is not responsible for those changes. Any changes that happen at that point are caused by extraneous factors like power surges, amplifier components, speaker cones, room acoustics, moving your listening location even a few inches. But the nulled two-track is identical to its null-tested twin. It cannot and is not any different from the other one. If it were different, it would not have passed the null test.

Some people may not be as diligent in obtaining null results. If there is any difference in the loudness between the two tracks, then they are not the same. Sometimes I get null results (by ear) if I mess with the loudness of one phase-reversed track. That means that one of them came out with similar wave shapes, but different amplitudes. That's not null, but it's LIKE null if they two cancel each other out when you find the right volume. That's a case of identical files, one of which had x amount added to every number.

Anyway, we can discuss it until we're blue in the face, but in the end a two files that pass a null test (producing a string of zeroes) cannot be different. There is no magic that can make that not true.

But people too often forget what they're dealing with and start to believe otherwise. And as I said earlier, once you believe in magical or unexplainable events, you start hearing or seeing them. It's human nature. Happens to me as much as to you; I just have to remember what I'm dealing with and put things back in context.

Shooshie

[FMiguelez]

But people too often forget what they're dealing with and start to believe otherwise. And as I said earlier, once you believe in magical or unexplainable events, you start hearing or seeing them. It's human nature. Happens to me as much as to you; I just have to remember what I'm dealing with and put things back in context.

I couldn't agree more with that. And it applies to sooooo many other things as well.

[Timeline]

A return email from Bill Whittlock on Null. An engineer i highly regard:Hi Gary,

It’s good to hear from you … and you should feel free to ask about anything – no “sacred cows” here! The late Deane Jensen had some strong opinions about Hafler’s “Null Test”, which raised so much hope and remains a controversy subject among audiophiles.

Deane and I discussed the Null Test at great length and we agreed that the test’s huge flaw is that it confuses distortion with simple time delay. Everyone agrees that a simple time delay, that might result from moving one’s head an inch farther from a violin player, for example, has no effect whatsoever on the waveform of the musical signal or the sound of the instrument. Digital workstations (any conversion from A to D or vice-versa) involves time delays or “latency”, making it even more important to distinguish between distortion and delay! Deane and I always agreed that “fidelity” in the context of sound reproduction means waveform fidelity or accuracy. We who believe this try desperately to preserve this waveform because it contains the timbre of natural sounds. The following is a quote from Deane, circa 1988 (Deane and Gary had not only just written the AES paper but Deane had earlier completed design of the Boulder power amplifiers):

"NULL TEST” vs. "SPECTRAL CONTAMINATION”
Others have used the so-called “NULL TEST", subtracting the
input signal from the output signal in an attempt to evaluate the
accuracy of power amplifiers. The simplicity of the “NULL TEST" is
seductive, and has an intuitive appeal, but is nonetheless
fundamentally flawed. The fundamental flaw of the "NULL TEST" is
that the resulting input/output difference signal contains both linear
errors (frequency response magnitude and delay) and non-linear
distortion which cannot be separated. As a result, an ideal, perfect,
and distortion-less amplifier, which has only a benign "frequency
independent group delay” (all frequency components delayed equally)
would produce a poor null which could be misinterpreted as
indicating poor amplifier performance.

Spectral Contamination is a new method for measuring distortion proposed by Deane in a 1988 AES paper by himself and Gary Sokolich (copy attached). I believe such a test might finally be the key that links what we hear to what we can measure. It was quite an accomplishment for Deane to put together the equipment required for such tests back in 1988. Much more suitable hardware is now available. I’m having ongoing discussions with the folks at Audio Precision about revisiting the entire topic and, if I can find the time, do some comparative testing and write another paper.

In a nutshell, the spectral contamination test applies a complex waveform containing a broad spectrum of tones. The spectrum is carefully chosen so that complex cross-modulation and inter-modulation products will fall into “empty” spaces in the input spectrum. The spectrum that comes out of the device under test is then examined for new spectral content in those empty spaces. The test reveals the ugly, non-harmonically-related “grunge” that’s produced by amplifiers, especially when the input spectrum extends into the ultra-sonic range. Some folks at Peavey later tested speaker drivers using the technique – and published some interesting results. As I said, I think it is a very revealing test that may prove to be the “Holy Grail” proving that what audiophiles hear often is real … in the form of a “grunge”, “veil”, “background clutter” or similar term. It’s my belief that a totally silent background, both during quiet passages (i.e., suppression of hum and buzz) and loud passages (i.e., lack of irritating dissonance), is essential for that much sought-after “suspension of disbelief” among listeners.

All that being said, I still have great respect for David Hafler – he was a brilliant circuit designer. I still own several pieces of Hafler gear … it’s only the Null Test that I have issues with!

Best regards,
Bill Whitlock
Technology Manager
Jensen Transformers, Inc.
AES Life Fellow – IEEE Life Senior
(from my home office in Oxnard, CA)


----- Original Message -----
From: gary@eirec.com
To: techsupport@jensen-transformers.com
Sent: Friday, September 19, 2014 9:22 AM
Subject: hello, question about Null Tests and their end all be all relevense

I have the utmost respect for your engineering opinions and hope this is ok to ask you about.

Do you find these types of measurement tests tell you everything you need to know about differences in the sound or tone of two say identical devices, analog or digital or even digital software too in regards to their digital audio engine designs?

How alternatively would differing DAW’s be tested to find differences we think we can hear and would differences we cannot hear above our hearing range change our perception of how we think they sound?

Regards,

GB