uTest, a company known for providing a variety of testing solutions for desktop, web and mobile, is launching a new solution designed to grade mobile apps’ performance under real-world conditions, and then compare the app’s rating with that of its competition. The solution, for obvious reasons (i.e., desperate need) is arriving first on Android, with an iOS version to follow soon. The app testing process takes just a few minutes, the company claims, and will then return a report grading the app on a scale of 1 to 100.
In addition to the score, the report also details any issues discovered during the app download, installation and basic usage. To provide more context, the AppGrader report, as it’s called, also compares the app’s grade to those of the most popular applications in the Android Market Google Play store that are found in the same app store category.
The system isn’t designed to replace the testing and QA work developers already do, of course, but is meant to function as more of a final step that can give more insight on how the app will run when actually put into the hands of users. Explains uTest CMO Matt Johnson, this “in-the-wild testing provides ‘last mile’ assurance that the apps work on real devices, under real-world conditions, in a wide variety of locations,” he says. However, there’s no reason why developers couldn’t continue to run AppGrader after the app’s launch, if need be, or as they continue to push out minor updates and tweaks to the app in question.
To use the service, developers just upload the Android APK file to get started, and AppGrader will send out an email notification within a few minutes after the testing is complete. For apps that crash, developers will also be given the device-specific crash log for additional diagnostic details.
For now, the service tests the apps on top Android devices, like the Samsung Galaxy Nexus/Samsung Galaxy S II, Google’s Nexus S, LG Nitro HD, Samsung Galaxy Tab, HTC Thunderbolt, Sony Ericson Xperia, Motorola Droid X2 and the T-Mobile My Touch. Apps are also tested on U.S. carriers AT&T, Verizon, and Sprint.
Although all mobile developers could benefit from more testing tools, there’s more of need to address the Android developer base first. On Android, developers don’t just have to deal with an incredible number of device types in the wild, they’re also constantly challenged by OS fragmentation, too. According to Google’s own statistics, only 4.9% of users are running the latest version of Android (Ice Cream Sandwich), 3.3% are stuck on the version just prior (Honeycomb) while 64.4% are on Gingerbread, which was first released back in December 2010. The remaining 27.4% are running versions that are even older, if you can believe it.
To put this in perspective, iOS users update to the latest version remarkably fast. (One report shows 38% hit iOS 5 within 5 days of its release, for example). It’s not entirely fair to make judgements about the users on either platform, however – iOS users have access to upgrades, while Android users, either due to carrier or OEM restrictions, often do not. But it does showcase the greater challenges that Android developers have to deal with when it comes to building apps for a number of handsets and software versions.
uTest’s AppGrader is available now, from here. The service is free, as the company expects it might entice users to try out the company’s other mobile testing products.