Trouble-Shooting: clicks, pops, squeals, dropouts.

[TheRealRoach]

Table of Contents
[Intro] - Brief overview of general pro audio and windows, and symptoms of audio problems
[1.*] - Recommended Tools
[2.*] - Testing DPC Latency
[3.*] - Disabling and uninstalling unnecessary background applications
[4.*] - Disabling background Windows services
[5.*] - Operating System-specific links
[6.*] - Other possible causes, also known as, "what a freakin crap-shoot..."
[7.*] - The benefits of a full reformat and install

Intro
Hey everyone,

The nightmare problems of clicks, pops, squeals and dropouts have long plagued many MOTU/Windows XP users. But, this past year (summer 2008 - summer 2009) has seen a growing number of users on XP 64 bit, Vista 32 and 64, and Win7, who still experience the same problems, but not necessarily related to the specific "FIXES" that I posted in the past. So I'm hoping to provide a general trouble-shooting guide that will at least be able to pinpoint where the problem lies no matter what your hardware configuration or operating system version.

First off, it is important to understand how Windows systems work as far as pro audio is concerned. The windows operating systems are not real-time machines. They process packets of information and when that is done, it moves on to the next packet.

Consider this scenario:
There are two buildings: a composer's house in suburbs and a concert hall with orchestra ready to play in the downtown core. There is a courier with his car waiting in the driveway at the composer's house. When the composer finishes writing enough music to fix into a box, the courier drives the box downtown and distributes it to the orchestra who then start playing the music. Meanwhile, the composer - back in the 'burbs - is still racing to write enough music to fill the next box. Once that box is filled, another courier brings it down, but this time there is more traffic on the highway and it takes longer to deliver the information because other cars are also trying to get downtown. The orchestra has, unfortunately, reached the end of the music from the first delivery, and must stop until the next box arrives... an "audio interruption", if you will. This is how Windows machines deal with "real-time" media, or in other words, media that should play uninterrupted.

- The size of the "box" is the directly related to your audio card's buffer size. The larger the box/buffer size, the larger the amount of information is delivered in every trip, thereby allowing a more forgivable time-frame to deliver the next box. In other words, it doesn't matter if there is a huge traffic jam when delivering the next box since you delivered such a large amount of information from the previous trip. The downside is that any changes in the information heading "downtown" will take longer to apply... a latency. When one makes the box smaller, it thereby reduces the amount of time it takes from the point at which the composer writes a note, and the time at which the orchestra plays that note. But, this requires that the courier is allowed to get downtown reliably fast. There are a few ways to do this:
- increase the speed limit of the highway (increase your processor/system speed).
- increase the number of lanes on the highway (add multiple processors/cores).
- reduce the overall amount of traffic (get rid of all processes, programs, drivers, that aren't vital to your audio production needs, the BEST solution).

Ok, so end of scenario. The principles illustrated are universal to any system that needs to deliver near-real-time media playback/recording. All interruptions in playback - resulting in clicks, pops, squeals and dropouts - are caused by an interruption of the data being recorded into the system and/or being played back out of the system. These interruptions in data-flow are measured in micro-seconds and are called Deferred Procedure Calls, or DPC.

[It should be noted that it is assumed that your system is configured to MOTU's minimum system requirements, including the recommended Texas Instruments FW chipset]

Trouble-shooting clicks, pops, squeals, dropouts (referred to as "interruptions" from this point on):
- [1.1] download DPC Latency Checker: http://www.thesycon.de/deu/latency_check.shtml
- [1.2] download Process Explorer: http://technet.microsoft.com/en-us/sysi ... 96653.aspx
- [2.1] Open any session in your DAW that generates the problematic playback issues and press play. Run DPC Latency Checker. You will notice that the audio interruptions correspond with a spike in the DPC Latency Checkers graph. The DPC readout values indicate how long information is waiting. A short time of <50 microseconds is recommended, and any "spikes" of >150 microseconds can cause issues. You might see something like this:
http://img231.imageshack.us/img231/1552 ... ampiz5.png Extremely large spikes. Or maybe just this:
http://www.roachmouthrecords.com/Cubase4_9400GT_00.JPG But, the point is that your readout should have no absolute maximums above 150 microseconds.

- [2.2] If it seems that audio interruptions correspond with spikes on the DPC Checker, then the cause is a hardware driver, software driver, or program running in background that is intermittently hogging the system bus, and thereby interrupting the data flow.
- [2.3] What we know at the point is that the MOTU hardware or system hardwareis likely NOT defective... it is likely software based.
- [2.4] Close DPC latency checker.
- [3.1] Tracking it down: Keep your session and DAW open, but with transport stopped. Open Process Explorer.
- [3.2] You will see a detailed read-out of all system processes and an explanation of where they reside and what they control. (You may have noticed that opening task manager will sometimes show half-a-dozen or more "svchost.exe" processes... well this will tell you what each one is actually doing).
- [3.3] Keep an eye on the readout and watch for any processes that are regularly requiring processor time.
- [3.4] One by one - starting with the hogs - start disabling processes that are not vital system or pro audio processes. Things like Antivirus, Automatic Updaters (Adobe and lots of other programs install these), network controllers (especially wireless network cards), hardware monitors (printer cartridge ink monitors). An ideal system should have less than approximately 22 processes running, even with your DAW up and running.
- [3.5] Once you have disabled the obvious processor-hogs, close Process Explorer, and your DAW. Then reopen your DAW and try running your session. (You CANNOT play audio with Process Explorer open since it itself will cause pops and clicks as it monitors the system.)
- [3.6] Hopefully you have eliminated your interruptions by this point... but if you haven't repeat from step [3.1] and continue taking out processes.
- [4.1] If you have eliminated as much as seems possible but still have interruptions then it is time to look at system services.
- [4.2] Close your DAW software and head over to Black Viper's side on custom system services configurations: http://www.blackviper.com/ and click on the link to your OS. For example: Windows Vista Service Pack 2 Service Configuration.
- [4.3] Services are bits of software that come pre-installed and configured as part of Microsoft Windows. They could be the cause of interruptions even if users have just freshly installed and reformatted their system. It is recommended to start with the "SAFE" configuration, although "Power User" and "Bare Bones" are available, although I don't recommend Bare Bones.
- [5.1] Run your DAW and hopefully you have eliminated any interruptions.
- [5.2] If you still are having issues that it may be driver related at which point you will need investigate more specific options.
- [5.3] If you are an XP user, then you should read and try what is mentioned here: http://www.motunation.com/forum/viewtop ... 14&t=31518
- [5.4] If you are an XP, Vista or 7 user, you should read and try what is mentioned here: http://www.motunation.com/forum/viewtop ... 14&t=34558

- [6.1] Other possibilities unrelated to audio that can cause interruptions:
- [6.2.1] Video cards. If you have made it to this step and are still having problems, then uninstall your video card driver (please make sure you have the appropriate installation files to re-install them afterward). You will be prompted to reboot your system and the system will run in very low resolution and you will prompted that the system "Found New Hardware...". Do not install any video card drivers, but rather run your DAW and see if you still have interruptions.
- [6.2.2] Many have recommended to get the simplest, most basic video card that you can find for your system. Many have recommended ATI-based cards in the past as nVidia cards' drivers have been notorious for causing DPC spikes. For example, I have an nvidia/geforce 9400GT video card that would always cause serious interruptions only when I had the Waves SSL plugin's GUI open. The cause was a problem in OpenGL graphics acceleration for which the Waves plugins' graphics are programmed with. I tried dozens of new and old drivers, but finally a silver-bullet fix appeared with Geforce driver 186.18. I know... it's ridiculously specific.
- [6.3] Network. I recommend that you disable all LAN/WLAN network hardware from your system bios, along with any onboard FW and sound hardware.
- [6.4] USB devices. I've read reports of simple human interface devices such as your keyboard or mouse causing interruptions. It's worth trying an older PS/2 style mouse and keyboard (these were the older style connectors pre-USB. They are circular connectors with a half-dozen or so pins inside).
- [6.4] Another possibility is your motherboard's BIOS. You should exercise extreme caution when flashing any sort of bios as accidentally installing an incompatible bios version could turn your computer into expensive paperweight. Do a great deal of research on brand-specifics forums.
- [6.5] Some UAD (universal audio) cards create interruptions very similar in symptom to the video card problems, again the root problem being an intermittent hogging of the system bus. UAD driver upgrades/downgrades have solved issues in the past for users.

- [7.1] And finally, one of the last resorts - and most highly recommended actions - would be to fully reformat your system and re-install your operating system from scratch, paying special attention to perform steps in [4.*]. Reboot. Install your MOTU drivers, Reboot. Install your DAW, Reboot. Run and test your session.
- [7.2] The benefit: Reformatting does take a while to do when you factor in the time it takes to reinstall the OS and any necessary programs, but it is the only way to definitely remove any and all left-over remnants of old drivers and programs that may be interfering. As soon as your operating system is up and running create a drive image backup using a program like Paragon, or Norton Ghost. This way you can reliably return to your clean, fresh install in a matter of minutes instead of hours. And hopefully, when you have your audio system fully functioning with drivers and DAW you will create a second drive image backup.
- [7.3] The point is to Install the absolute minimum drivers, software and hardware required to get your DAW to function. If you are still experiencing interruptions that it must be something with the actual hardware in your system (again I would check video card first, along with disabling all unecessary onboard hardware from the bios), the hardwares' drivers (read up on your particular motherboard chipset drivers, and video card drivers), the MOTU drivers or any bugs/glitches that occur in the DAW software itself.

[DJ Synchro]

Great guide!

I personally followed your steps when I had serious performance issues on my previous system. In the end I couldn't find a solution no matter what I tried. It finally came down to a faulty motherboard (In device manager one of the PCI Standard PCI-to-PCI device had a problem although there wasn't any question mark shown before the device. In the resources tab the device wasn't any resources because it has a problem)

So looking carefully in Device Manager could also show the nature of the problems.
Now, I bought a new system with an Intel i7 CPU and a Asus P6T SE Mainboard and everything runs smooth like a baby's butt

[TheRealRoach]

*bump* added some more points.