Credit: Adam Patrick Murray / IDG
There’s no one or right or true way to review anything, smartphones included.
A good review can be conversational, comparative, critique, any combination of the three or something else entirely. That being said, one thing that every good review has in common is a sense of intent. They don’t necessarily have to agree with you but, at the end of the day, they should be able to tell how you feel about the product.
Whether you’re recommending, warning or deciphering the person who is going to get the most value out of a particular piece of tech, you always want to make your argument in such a way that it leaves the reader in a place that’s far from ambiguous.
With those ideas in mind, here’s a look at some of the questions and things that I consider when I write reviews for products like the Galaxy Z Flip or the Huawei P40 Pro.
How much does design matter?
Credit: Fergus Halliday | IDG If you’re spending this much money on a smartphone, you want it to look good and feel as good as it looks. Design might seem more subjective than other aspects of the smartphone experience but the reality is that it's all subjective. Nobody operates from a purely objective or neutral point of view and, when all is said and done, a review is just an opinion.
It’s an informed one but an opinion nonetheless. The physical design of a phone like the Google Pixel 4 or iPhone 11 Pro matters because it sets the foundation for how you feel about everything else. It’s literally the first thing you encounter when you unbox a smartphone and if your gut instinct is that you don’t like it, it’s worth thinking about why.
What features matter?
Credit: Oppo When it comes to smartphones, not every feature matters equally.
Some people care a lot more about the inclusion of a headphone jack than they do wireless charging or water resistance. It’s important for a review to not just list those features but to think about what they represent.
When I review a smartphone, I try to look at the context to get a sense of where it sits in terms of features. What does it have that it’s predecessor lacked? Does that change in the status quo come with a higher price or does it end up adding more bang for your buck to the thing?
What about competitors? Is the Samsung Galaxy A70’s in-screen fingerprint sensor that unique or can it be found in other smartphones that occupy a similar price point? When you buy a smartphone, you’re inevitably making a choice of one brand over another and while finding the differences those brands can sometimes feel like you’re splitting hairs, it’s a necessary sin on the road to finding the best outcome.
Where are the trade-offs or caveats?
Credit: Fergus Halliday | IDG Nothing is perfect.
There’s always something that could be better about any given smartphone. No brand can have it all and, after spending a week or two using any smartphone as your daily driver, you begin to get a sense of where the priorities are based on where shortcuts have been taken.
Maybe they cared more about battery life than camera quality. Maybe they cared more about software security than hardware design or feel-factor. Regardless, decisions are always made to prioritise some aspects of the smartphone experience over others and, when you’re reviewing a smartphone, identifying those is an important part of the process.
How do we measure performance?
Credit: Fergus Halliday | IDG There are two aspects to how we approach writing about smartphone performance.
The first comes in the form of benchmarks. These provide a relative measurement of how powerful the processor and GPU inside a smartphone is and allow you to quantify and represent the difference in processor power offered by different devices. It allows us to know, for sure, that the iPhone’s A12 processor is capable of outperforming Samsung’s latest Exynos chipset for example.
However, benchmarks often only really reflect how good different devices are at passing benchmarks.
The other half of the process here is to try and convey what the regular experience of using the device is actually like.
There are a handful of possible outcomes here. Either the device is too slow for daily usage to be comfortable or the opposite. It’s important to document whether specific tasks like camera processing or gaming yield slower performance but the end-goal of the performance section of the review really is to answer the only question that truly matters.
Is the processor in this fast enough that you never really have to think about whether the processor in this phone is fast enough?
What’s the narrative around this product? Does the reality live up to the hype?
Sometimes, there’s a wider narrative or questions around the technology or innovation being introduced by a new smartphone. See for example, Samsung’s Galaxy Fold or Galaxy Z Flip.
Credit: Fergus Halliday | IDG If that’s the case, then scrutinising whether the reality of using said product gets anywhere near the rosy scenarios painted by the marketing and ad campaigns is something that’s very important to touch on.
When reviewing smartphones and other tech that hinges on a core innovation, you have to think about a) whether that innovation provides meaningful improvements to regular usage and b) whether the inclusion of that innovation justifies the premium price that’s usually involved.
How we measure camera quality
Credit: Fergus Halliday | IDG As smartphone photography has become the locus of attention and innovation in the smartphone world, it’s become trickier and trickier to write about the hypercompact cameras involved.
Part of the problem here is that not everyone is equipped to apply the consistent and scientific rigour of camera sensor comparison websites like DxOMark. Another other issue is that everyone uses their smartphone camera differently and some shoppers simply don’t care at all.
My approach here has to try and avoid overthinking it. I use the camera on any given smartphone the same way I do other smartphones and try to spot the differences. Is the autofocus inconsistent? Does the shutter lag in low-light scenarios? How does the zoom compare to similarly-priced smartphones?
Reviewing a smartphone camera involves identifying and documenting strengths, weaknesses and providing plenty of camera samples to back those claims up. Oftentimes, the story isn’t about whether or not a smartphone has a good camera but about what kind of camera that is.
How do we measure battery life?
As with camera quality or performance, there’s no perfect way to measure battery life.
Still, anecdotal experience is usually a good place to start. How frequently do you have to charge up? As with performance, it’s often the case that a device either has enough battery life that you forget to worry about battery life or it doesn’t. You want to make a determination about which of those two extremes the device falls closer towards.
On top of this, I also measure how long it takes a smartphone to run down its battery from 100% to 0% by streaming video content over Wi-Fi since most people today tend to stream content to their phones rather than store files locally. The results of this test, plus other internal battery performance metrics automatically tracked by Android and iOS, helps ground your opinion and give a quantitative sense of what daily screen time looks like.
How do scores work?
Credit: Fergus Halliday | IDG When I review smartphones and other tech, I use a five-star system as a shorthand for the quality of the final product. The goal here isn’t to split hairs as much as it is to provide at-a-glance shorthand for how positive or negative I am.
1-Star represents a product that does not achieve the thing it sets out to achieve. Either it doesn’t work or it's not fit for purpose.
2-Stars represents a product that sort of achieves the thing it sets out to achieve. It ‘works’ but it rarely excels or goes beyond basic functionality.
3-Stars represents a product that definitely meets your expectations for what it should do and then some. It’s not quite exceptional and it doesn’t get everything right but it does sit above the average.
4-Stars is reserved for products that are genuinely great and easy to recommend. They don’t just get the job done. They do it in style and make it look easy. A 4 Star smartphone is a smartphone that gets you excited.
5-Stars represents perfection but nothing is that good. Even the thing you think is the best is not that good. That being said, this end of the sliding scale is useful as a tool for measuring the relative quality of smartphones that go beyond a 4-Star rating.
