Ultrathin showdown: Apple MacBook Air vs. Lenovo ThinkPad X300 vs. Toshiba Portege R500

When it comes to laptops, ultrathin is in -- particularly since the launch of Apple's MacBook Air earlier this year. As might be expected, though, the Air isn't the only game in town -- skinny laptops are available from a variety of other vendors.

However, stylish doesn't always mean functional. You needn't look further than the latest style in women's shoes to know that what looks good isn't necessarily comfortable. Is the MacBook Air with its ultrasvelte shape actually as comfortable to use as larger, more traditionally shaped laptops? And, come to that, what about its competitors? How easy are they to use?

Note that we're not talking about the low-cost ultraportable laptops like the Asus Eee and Everex CloudBook. These laptops are larger and more full-featured. For instance, that new breed of ultraportables has, at most, 10-in. displays, while the smallest display in this group was 12.1 inches. However, they also contain a number of compromises (such as tweaked keyboards or less ports) that could affect the user experience.

To find out how these thin notebooks really rate, we asked the usability experts at Perceptive Sciences, a US-based user experience testing firm, to run the Air and two Windows-based ultrathin laptops -- the Lenovo ThinkPad X300 and the Toshiba Portege R500-S5002-- through a gamut of hands-on tests with 20 independent users.

So how does the much-ballyhooed MacBook Air stack up against its competitors? Here's what we learned.

Air and its competitors

Before we get to the results, a few words are in order about the laptops themselves and how we tested them.

Apple's MacBook Air has garnered a lot of publicity since its introduction in January, largely because of how thin it is -- at its thickest point it's a scant .76 inches thick. Weighing in at three pounds, the Air might be easy to carry, but it is comparatively powerful, with a 13.3-in. display and an Intel 1.6-MHz Core 2 Duo processor and an 80GB hard drive.

Although they haven't received the level of attention that the MacBook has, the skinny offerings from Toshiba and Lenovo have been well reviewed as well. The Lenovo ThinkPad X300, which was introduced in February 2008, is slightly lighter but also slightly thicker than Apple's offering and has the same size display as the Air. The review unit had a 1.2-GHz Core 2 Duo processor and, uniquely for this group, a 64GB solid-state drive.

Toshiba's Portege R500-S5002 has a particular emphasis on lightness. At 2.4 pounds, the company has been claiming it is the world's lightest full laptop. The trade-off is in screen size; at 12.1 inches, the R500 has the smallest display in this group. Like the Lenovo, this laptop came with a 1.2-GHz Core 2 Duo processor. It also had a standard 120GB hard drive. (We were told after testing began that the Portege R500-S5002 has been discontinued; however, a Toshiba representative assured us that the size of the current model, the Portege R500-S5006V, which comes with a 160GB hard drive and a 1.33-GHz processor, is the same.)

Page Break

How we tested

Our tests were not the performance-related analyses typically found in product reviews. We weren't interested in aspects such as processor speed or battery life.

Rather, Tom Thornton, senior research scientist at Perceptive Sciences, and his team focused on the usability of the hardware. Perceptive Sciences developed nine tasks for these ultrathin laptops and then examined how quickly and easily the testers completed those tasks. A total of 20 people were recruited to participate in the tests -- 11 men and nine women. Of those, half were students and half were business users who travel anywhere from once a month to once a week.

Usability testing is part science, part art. That's why the results are a combination of the objective -- the time it took to complete specific tasks and the success rate at completing those tasks -- and the subjective impressions of the testers.

For each given task, each participant tested two of the devices while observed by a Perceptive Sciences staffer. Half of the time, they tested one of the two laptops first; the other half of the time they started with the other laptop. The purpose of this approach was to negate any advantage or disadvantage of the order in which the devices were tested. After the tests were done, the researchers interviewed each of the testers to get their more subjective reactions.

The testers were mixed in terms of which operating systems they used in their everyday life: Eleven were PC-only users, seven used both PCs and Macs, and two were Mac-only users. None of the users were familiar with the specific laptops they were testing.

In fact, the tests were designed to keep the operating system out of the mix as much as possible, although Thornton acknowledged that it was impossible to avoid that issue entirely. Mac users were bound to feel more comfortable with the Mac OS, and PC users would tend to be more comfortable with Windows. The MacBook Air came with the latest version of the Mac OS X, dubbed Leopard, while both the Lenovo X300 and the Toshiba Portege R500 were equipped with Windows Vista Business.

What we tested

Even though we couldn't try out every sort of task that people would use these notebooks for, Perceptive Sciences came up with a representative mix for their testers to try.

1. Take the computer out of the box, plug it in and turn it on.

There were no clear-cut winners in this out-of-the-box test -- and each laptop had a glitch. Thornton said that the ThinkPad X300's battery wasn't initially connected to the laptop, which took testers additional time to correct. Some testers were confused about whether to use the extra power cord extension that was included in the Air's box, which also took a bit of time to sort out. The Toshiba Portege R500 had the longest boot time, and "people were frustrated by that," Thornton said.

2. Connect to the wireless network.

The MacBook Air and Lenovo performed similarly, with testers taking, on average, about 2 minutes 30 seconds to connect. The Toshiba took, on average, about 3 minutes 45 seconds.

"People had more errors [with the Toshiba] as far as going to the wrong place [in the interface] to connect," said Emily Swinkels, research scientist at Perceptive Sciences. "Also, the Toshiba has a wireless switch on the side. One user accidentally turned that off."

Page Break

3. Access a pre-established Yahoo Mail account and send an e-mail.

One focus of this test was to evaluate the keyboard and trackpad, areas in which the MacBook Air had a clear advantage.

"It took about a minute longer to type an e-mail on the Lenovo and the Toshiba than on the Mac," Thornton said. "People had more trouble typing and had more errors, particularly on the Toshiba. People commented they didn't understand key placement [on the two PC notebooks]; there were a lot of keys where they didn't expect them. The Mac keyboard is more like a full-size keyboard."

Swinkels noted errors caused by the trackpads on both the Lenovo and Toshiba. "There are three buttons on the top of the trackpads that [testers] were touching accidentally," she noted. Thornton added that key placement and trackpad buttons were more a consistent complaint for the Toshiba than the Lenovo.

4. Connect a USB flash drive, create a folder and copy an MP3 file from the flash drive to the folder.

Overall, the Lenovo ThinkPad X300 bested its competitors in terms of time to complete this task, but Thornton noted that the results of this particular test were directly related to how familiar each tester was with either PC laptops or Mac laptops.

"The Lenovo was much faster; it took about 2 minutes to complete the task," Swinkels said. "The Mac and Toshiba took closer to 3 minutes."

One significant failing of the MacBook Air was that not all flash drives fit the device well because of the device's ultrathin nature. "We wound up having to use a small, thin flash drive for the Mac," Thornton said. In addition, most users had to lift the Air off the table and twist it a bit to get the drive connected.

The time required to create a folder and copy a file correlated with how accustomed the testers were to the Mac or Windows operating systems.

5. Plug in headphones, listen to an MP3 file and adjust the volume without using the trackpad.

The goal of this test was to use the hardware, not the operating system, for tasks related to listening to music. The overall winner was the MacBook Air, although users found the headphone jack was more clearly marked and easier to find on the ThinkPad X300. Testers of all three laptops had a hard time finding the volume controls, although once found, users found the Air's controls easiest to use and most precise.

6. Eject the USB flash drive and delete the folder.

The three laptops were roughly even on this task, with one operating system-related exception. The MacBook Air provided a message when a user pulled out the flash drive, but Windows laptops didn't. As a result, some people stuck the drive back into the USB slot of the Lenovo and Toshiba laptops, which added to the time required to complete the task.

Page Break

7. Open an e-mail and click on a link to watch a movie preview, adjusting the display as necessary.

This task tested display quality; here, the MacBook Air edged out the Portege R500 and the ThinkPad X300. Thornton said that's because testers believed the Air had a brighter display. The Lenovo's display quality lagged behind the Toshiba's on this test, according to the testers.

8. Connect the computer to a projector and project the laptop display to a screen.

The Toshiba Portege R500 was the winner in this test, with both the fastest completion times and greatest ease of use. Users particularly liked that it and the Air started projecting images immediately after the projector was plugged in.

By contrast, testers found the correct plug-in port on the Lenovo ThinkPad X300 more quickly than on the other two laptops but because the Lenovo didn't automatically start the display after the plug was inserted, users lost time finding the correct key combination to start the projection. "Almost everybody failed or had errors doing this task on the Lenovo," Swinkels said.

9. Shut down the computer.

The MacBook Air was the clear winner in this seemingly simple test. Shutting down the Mac required about 10 seconds, on average, which was four times faster than both the Lenovo and Toshiba. One common error on the Windows-based laptops was that testers pressed the power button, which puts the computer to sleep but doesn't turn it off. They then had to start again and figure out how to fully turn off the laptop.

The envelope, please

In the end, the testers found the Apple MacBook Air to be overall the most usable of the three ultrathin laptops, although it didn't excel in all the tests and categories. The Lenovo ThinkPad X300 was deemed the second most usable laptop and the Toshiba Portege R500 came in third.

"The MacBook Air has the clear edge over both the Lenovo and the Toshiba in terms of usability metrics," said Thornton.

The overall results were extrapolated from three factors. The first was the time it took for the testers to execute each of the nine tasks. "On average, where there were reliable differences in time, participants completed those tasks about a full minute faster using the Mac Air," Thornton said.

The second factor was the completion rate for each task for each laptop. Most users completed most tasks, but sometimes there was a difficulty and the task was not completed.

The third factor was the subjective comments of the testers.

Although the MacBook Air came out on top overall, the results varied from test to test, according to Thornton. For instance, the Air only had a slight advantage over the Toshiba Portege R500 when it came to testers' subjective opinions of portability, although the Air had a wider advantage over the Lenovo ThinkPad X300. And it was a dead heat between the MacBook Air and the Lenovo in terms of which had the most functions and features, with the Toshiba lagging behind.

The Lenovo was highly rated for its durability while testers felt the MacBook Air was fragile, Thornton reported. The Toshiba was considered by testers to be the most portable and compact, and it was virtually a dead heat in terms of which of the three laptops was best for business use in the testers' estimation.

The Air was, by far, considered the most fun. And testers made the MacBook Air the clear winner when asked which laptop they'd purchase. Testers also considered the Air to be the best value, according to Thornton.

So although the other laptops also had their strengths, the bottom line is a clear win in usability and user perception for the MacBook Air.