Samsung is being called out by the highly respected and thorough Ars Technica for apparently “artificially” boosting the Galaxy Note 3’s performance specifically when it comes to benchmark testing. The blog found that while under normal testing the Note 3 vastly outperformed the LG G2, which has the same processor, after stripping away some fancy benchmark-specific code, the phone scored about the same as its LG competitor.
Ars has a very good, very long explanation of how they arrived at their findings, and the end result is an artificial benchmark bump of between 20 and 50 percent in all areas depending on which benchmarking tools you use, including industry standards like Antutu and Geekbench. It’s a good read if you’re interested in that sort of thing, but the upshot is, you probably aren’t. Which is why Samsung has even more egg on its face.
Artificially enhancing performance benchmarks for a smartphone these days is like artificially enhancing the smoothness of your elbows via plastic surgery: it may mean that overall, you technically present a more attractive package on the surface, but no one’s really going to know or care that you’ve had any work done.
Apple’s iPhone 5s reportedly benchmarks up in the same ranks as some fairly recent Mac computers, for instance, but that’s not something your average iPhone 5s buyer is likely to know. Also, it doesn’t mean anything; benchmark scores doesn’t mean one device will be able to handle the same tasks as the other, like running a professional video editing software suite for example.
Long ago, Apple realized that a specs race wasn’t the same as the race for market dominance. Actual buyers cared about the phone experience, not abstract numbers which may or may not be borne out by really using software and apps. It’s true that Apple still talks about performance when it touts new devices – but it does so relatively, explaining only how much faster or more efficient something is compared to previous generations. That frames the discussion in terms that everyday users can understand, making it genuinely useful information.
The end result is that Samsung looks like it’s grasping when it takes an abstract (essentially meaningless, for all intents and purposes) number and artificially builds that up to win praise from some whitecoats who test these things for a living. It seems to be doing this as a matter of course now, as Ars says it’s seeing similar behaviour in testing the new Galaxy Note 10 Android tablet from Samsung as well. And, in the end, its unadjusted numbers were actually faster than competitors like the G2 anyway; if for some reason as an OEM you’re still concerned with winning a specs race on paper at this point (which you shouldn’t be), you don’t need to win by a wide margin, especially at the risk of looking foolish.
Unadjusted numbers would’ve won faint praise from the crowd that likes them, and gone unnoticed by most. Artificially altered ones attract a whole lot of negative attention and result in a net bad look for Samsung. The Note 3, like most high-end Samsung hardware, is probably a great phone, but now it’s embroiled in a doping scandal, over a number nobody really cares about.