The following post comes from my colleague Steve Ball, Senior Program Manager for Sound in Windows Vista, and continues his team's on-going series on how Windows Vista treats various forms of audio.
-----
Part I: Why does my Windows sound sometimes "glitch?"
Windows is a rich and complex OS designed for multi-tasking users whose tasks must share access to scarce system hardware and resources. Unfortunately, despite multiple decades of incredible advances in PC and CPU architectures, there are non-trivial, complex interactions between applications, processes, and devices in even the most advanced personal computers that make a supposedly "easy" task -- like playing back music without occasional glitches -- much more difficult than it may seem at first glance.
Another way of thinking about this: it seems odd that a modern >$2000 PC may sometimes have trouble seamlessly playing back music when $20 CD players can effortlessly playback music without glitches.
So why do many $2000 PCs occasionally glitch while playing back music? The quick answer is this: Windows is not a single-function device like a CD player.
A slightly longer answer goes like this: even an average Windows machine today is commonly used simultaneously as a media player, word processor, presentation projector, spreadsheet number cruncher, authoring tool, photo editor, media server, video recorder, music composition tool, communications device, search engine, virus detector, data compressor and decompressor, and backup manager. And these are only a few of the possible tasks and processes that are run at the same time on the hundreds of millions of Windows machines that are in use today. Each of these tasks or processes, in isolation, would hardly tax the resources of modern PC hardware. But in our multi-tasking world, unavoidable resource conflicts do sometimes occur between the huge and diverse ecosystem of Windows hardware that enables these tasks. Even on the most expensive, brand-new machine, occasional glitches can occur if and when the system attempts to divide its finite resources among these multiple, diverse, independent, power-hungry activities.
What is a glitch?
A glitch is a perceivable error, gap or pop in the sound caused by discontinuities in the audio signal during playback or recording which result from processing or timing problems. Glitches during music playback can sound like a loud "pop" or like a brief slice of silence randomly inserted where your music should have been. Some customers have also described what "glitching" in their own words as:
- audio stops a little bit
- breaks up
- choppy
- clicking
- corruption
- crackle/crackling/crackly
- interruption
- jitters
- jumpy
- skipping/skip/skips
For the purpose of this discussion, let's lump all of these descriptions together under one general class of problems and call these "glitching." While a glitch that happens during music playback can be annoying and unsettling, a glitch that occurs while you are recording or communicating with someone can result in frustrating and unacceptable data loss.
What causes my Windows sound and music to glitch?
Digital media processing is time-sensitive. Playback requires specific work to be performed by a given deadline -- otherwise presentation or data loss can occur. A "glitch" occurs when a deadline for time-sensitive processing is missed or when time-sensitive data is lost.
For example, in Windows Vista, playing back music involves "work" that must be done at least every 10 milliseconds so that there can be a continuous stream of music out to your speakers. The "simple" task of playing back music consists of the following steps, all of which must be completed before a strict deadline:
- a small chunk of data from a music file needs to be read from a disc (CD or hard drive)
- this data needs to be "decompressed" or "decoded" (usually in system memory) so it can be streamed out to your speakers in a format that your sound hardware understands
- the decompressed sound data needs to be copied from system memory to your sound hardware memory
- the data in your sound hardware needs to be sent to your speakers at the appropriate time
- repeat steps 1-4 flawlessly every 10 milliseconds (ms)
In this example, if any of these steps aren't completed on time, then the user could hear a glitch in the music playback.
Elliot Omiya, Architect on the Sound dev team, puts this 10ms cycle into perspective: "it's just slightly longer than the time it takes a nerve impulse to travel from the end of your finger to your brain (~8ms), known as NCV (nerve conduction velocity). Because synapses are like network switches, there is switching time involved before the nerve impulse gets to the brain, i.e., switching time adds to latency."
There is some good news in this story: Windows developers have made significant progress over the years in reducing glitching across key multimedia scenarios. For example, music playback on an otherwise "lightly loaded" system can be generally as smooth as that $20 CD player.
But because of the multi-tasking nature of Windows and the vast array of new and legacy hardware in the ~1B PCs that are used to playback music today, this allegedly simple process is made more complex by the resource sharing that occurs between applications and hardware. For example, it is not uncommon for certain older devices driver to occasionally "lock out" the CPU for 10-50ms, thereby causing obvious audio glitches. This is just one example of the kinds of complex hardware, driver, and OS interactions that can cause glitches.
In summary, some of the common sources of glitches today include:
- CPU starvation
- GPU starvation
- Resource contention from devices and drivers (sometimes called "IO contention")
- Network devices
- And, of course ... bugs in applications, OS, drivers and/or hardware
My colleague on the Windows Sound team, Larry Osterman, also pointed out to me recently that humans are actually "hard-wired" to be disturbed by audio glitches. In an exchange about this topic, Larry observed that audio glitches are more obvious than video glitches because the ear's tuned to notice high frequency transients -- his visceral example of this idea is an image of a stick snapping in the woods behind you as an audio event that wakes you up before a bear wanders into your path.
In my second post on this topic, I'll go a bit deeper in sharing details of work we’ve done in Windows Vista to address some of the known sources of potential sound glitches, including some additional background about a recent discovery of an apparent connection between multimedia playback and network throughput.
I wish to acknowledge the contributions and suggestions from my colleagues Hakon Strande, Richard Fricks, Alex Ferreira, Lan Ye, Larry Osterman and Elliot Omiya for this series of posts.