Over the span of a couple of weeks, we saw three phones released, and with every release, the manufacturer touted the device's incredible "best ever" DXO Mark Mobile performance rating:
- Samsung released the Galaxy Note 8 with a DXO Camera score of 94
- Apple released the iPhone 8 Plus with a DXO Camera score of 94
- Google released the Pixel 2 / Pixel 2 XL with a DXO Camera score of 98
Manufacturers love touting these scores to "prove" that they have designed the finest camera a distinguished tech user could ask for. For all intents and purposes, technology should get better and this means every new phone released (at the high end) should have better overall performance than its predecessor. Why would you buy an inferior phone?
While most blogs blindly write headlines repeating this single "representative" number, very few actually take the time to read the full DXO reviews and explain the details to their readers.
It's complicated
The first thing to keep in mind that blending complex factors into a single easy to digest number is complicated and sometimes may mislead some readers. While most blogs only show the single number, DXO actually provides a generous amount of valuable information for the curious reader.
The DXO tests include a slew of carefully controlled tests and other real world tests that are more subjective.
If we pick on today's "highest ranking" phone, the Google Pixel 2, here is how the rating of 98 is made up:
DXO provides detailed test results and write-ups for each of these categories. While most blogs will tout that the Pixel 2 has a rating of 98 (the best ever rating for a smartphone), they rarely provide the makeup of that number.
And the make-up of that number is critical to your buying decision. If you will use the camera primarily for video, you may notice it scored 96. You can also check out how DXO made up that score by evaluating what is important to you about video (which attributes are more important to you).
- Exposure and contrast
- color
- Autofocus
- Texture
- Noise
- Artifacts
- Stabilization
Remeber that the video rating fo 96 is not a straight average but rather a "black box" formulae closely guarded by DXO.
Is DXO Mark Trustworthy?
The next question is "can you trust the DXO testing methodology"?
Having reviewed the public information made available by DXO, I say yes. They have a well-documented methodology that is as good as it is going to get. I trust their rating but use the detailed review information to make up my mind, not the single number most blogs publicise.
It is also important to keep in mind that DXO is a for-profit consulting company that manufacturers hire. DXO works with manufacturers to tune their imaging systems and get the best possible performance out of the equipment and software. DXO also sells image quality testing solutions.
I do not believe this consulting arm influences the device ratings in any way but it is still an important fact to keep in mind.
DXO Optics Pro
DXO Optics makes very good photo improvement software because of all this camera/lens knowledge they have accumulated. They know the shortcomings of each of the camera/lens combos and can this build specific correction profiles.
I own their software and paid for it myself.
90% of all the questions I receive these days is about comparing the iPhone to the Google Pixel2. In addition to all the information I have already written and the info provided above, there is one more piece of knowledge you should consider.
The Google Camera app on the Pixel 2 does not natively support RAW (the iPhone 5s or newer) does. This means DXO Optics Pro has corrective filters for all these iPhone RAW images, but does not for the Google Pixel2. This could be a major deciding factor for more astute or demanding mobile photographer.
Conclusion
I know most users simply don't care about the details. They want one easy to read headline that justifies their belief (Google is better / iPhone is better). My ask is that you, my more knowledgeable readers, take the time to look at the data that makes up the numbers.
It's a worthwhile investment of your time.
