Audio compression may not be so bad

We all listen to compressed music on our computers and portable players and we all know that compression is supposed to hurt sound quality. But does it hurt it enough to affect our listening pleasure?

I tested the effect of compression with a jury of music professionals and the results surprised me. Although the type of compression had a definite effect-my judges preferred .wma to .m4a and .mp3--the level of compression had little effect on how they rated sound quality. Most importantly, none of the compressed music sounded really bad to their well-trained ears.

The most common consumer audio formats--.mp3, .wma, and .m4a--all use lossy compression that throws away musical nuances to cut download times and fit more songs on our players. For instance, if you ripped Dire Straits' album Brothers in Arms to your hard drive as uncompressed .wav files, they would take up 556MB of space. You couldn't fit 15 albums of equal length on an 8GB iPod Touch. But as .wma files, compressed to 160kbps, the whole album is less than 64MB, leaving room for 127 similarly-squeezed albums.

Many people say they can hear the difference between music compressed at different levels, but their experiences could be shaped by their expectations. They know they're listening to a highly compressed song, so they blame the compression for every real or imagined flaw.

To get a less biased judgment, I set up blind audio quality tests. A jury of musicians rated various compressed samples on how closely they resembled the original.

Sound Methodology

You should consider my results to be anecdotal, not definitive. I only used two recordings and four jurors--hardly a scientific sample size. Besides, this type of test is inherently subjective; people respond to what they like.

I ripped two 30-second clips from CDs for tests. One came from the first movement of Tchaikovsky's sixth symphony, performed by the London Symphony Orchestra, with Igor Markevitch conducting. The other came from Joan Osborne's cover of Bob Dylan's "Man in the Long Black Coat."

I saved seven versions of each clip. To test music formats, I saved the clips with 128kbps compression in the three most popular formats: .m4a (which uses aac compression and is what you get from iTunes), .wma (Microsoft's format), and the old standard, generic .mp3.

To find out how compression levels (as opposed to formats) affected audio, I also saved .mp3s at 192, 256, and 320kbps. The larger number means less compression and, in theory, better sound. I also saved the clips as uncompressed .wav files.

[Want to try a portion of this test? We've set up a page with versions of both clips at different compression rates. (You'll need to have Quicktime installed.) You can listen and tell us what you think the quality is of each clip. SPOILER ALERT! Hold down the Control key as you click the link, then wait a couple of minutes before you go the new tab. The less compressed the file, the longer it takes to load, so if you see the page load, you'll get an idea of which files are high quality. Click here for the test.]

I played the uncompressed .wav file, then a compressed version, and asked the jurors to grade the second sample compared to the original. For one test, I played the uncompressed original twice, without the jurors knowing they were listening to the same file twice to see if they could detect the high quality .wav version.

I played the files using my Lenovo X60 laptop, connected to my home theater sound system with a Yamaha RX-V465 receiver and satellite speakers and a subwoofer from Cambridge Soundworks.

I instructed the jurors to grade the audio from 5 ("sounds identical to the uncompressed original") through 1 ("unacceptable"). Decimal numbers were allowed, and comments encouraged.

Page Break

Signals and Noise

When I looked at those numbers and comments, I found that, for this jury at least, the type of compression mattered more than the level of compression, and that neither mattered all that much.

Basically, the scores were all good. The lowest score anyone gave anything was 3, defined in the jurors' instructions as "Definitely inferior, but you could still enjoy the music." Most scores were 4 ("Slightly inferior, but not enough to affect your listening pleasure") or above.

But Microsoft's .wma compression proved slightly superior. Compressed to the same level (128kbps), .wma files received generally higher scores on average than either .m4as or .mp3s--and that held true on both clips.

In fact, jurist Fran Avni (a singer, songwriter, performer, and music producer with more than 30 years experience) felt that .wma compression actually made both clips sound better than the uncompressed orginals. While she didn't agree, Madeline Prager pointed out that the compression masked musician errors. Madeline has played First Viola in several orchestras, and now teaches at the San Francisco Conservatory of Music. (Full disclosure: She's also my wife.)

Not that .wma's advantages were overwhelming. The format's scores averaged 4.51. The .m4a files averaged 4.26; the .mp3s, 4.01.

No clear pattern appeared when the jurors compared compression levels. I wasn't surprised that 192kbps .mp3 compression scored better than the same format at 128kbps. But I was taken aback that its average score of 4.19 beat out the scores for 256kbps and 320kbps .mp3s, and even the uncompressed .wav file.

The judges particularly liked the192kbps .mp3 version of the Tchaikovsky clip, which earned an average rating of 4.28 (still lower than the 128kbps .wma and .m4a clips). Steve Carter, a pianist and recording engineer who's played with Pete Escovedo, Ray Obiedo, and Taj Mahal, described it as "pretty close, maybe less smooth" than the uncompressed original. Steve rated it 4.8--the highest score he gave anything.

But the Joan Osborne clip compressed to192kbps .mp3 didn't do anywhere near as well with the judges. Its average score of 4.10 still did better than the 256kbps version's 3.63, but here the 320kbps and uncompressed versions beat it out--as you would expect--with 4.50 average scores.

The big surprise was that the uncompressed .wav versions of the two songs didn't get the highest scores.

The Tchaikovsky clip got an average rating of 3.80. The uncompressed Osborne version did better, with an average score of 4.5. But only one judge, Dan Howard, a musician, singer, and music teacher who has worked as a radio DJ and concert producer, correctly judged it "As good as [the] original."

How did so many trained ears fail to recognize the higher quality of less compressed tracks? While the result may seem like a fluke, it's not out of line with other, larger studies of music compression.

In 2001, the PC World Test Center conducted extensive audio compression tests, using four music samples, 30 judges, and a more controlled audio environment. While that study didn't test whether jurors could distinguish an uncompressed clip, it did find that listeners could detect very little difference between a clip at 128 kbps and one at 256 kbps.

And tests by Stanford University Music Professor Jonathan Berger indicate that young people are actually coming to prefer the sound of compressed music. Berger sees an increasing preference for MP3s and believes that the students he tested like the "sizzle" or metallic sound that the format imparts.

The bottom line from my testing seems to be this: My jury of professionals generally preferred WMA compression, but weren't able to detect much difference in files with bitrates higher than 192 kbps. In other words, even though they call it lossy compression, you may not be losing as much as you think.