Software testing marketplace uTest today announced the results of its so-called “TV Networks Bug Battle” competition. More than 500 software professionals from 30 countries around the world participated in the quarterly competition, reporting a total of 908 technical, functional and GUI bugs in the web and mobile apps of NBC, CBS, Fox and ABC.
Testers were challenged to search the sites for bugs – performing a combination of exploratory, functional and usability testing. At its conclusion, participants filled out a detailed survey in which they ranked each site based on video quality, ease of use, community features and actual TV content/shows. After carefully reviewing each bug and survey response, uTest awarded roughly $4,000 in prize money based on the quality of bugs and feedback.
Top findings:
– Nearly 50% of survey respondents chose video quality as the attribute most important to them when evaluating an online TV network. NBC.com scored highest in video quality.
– Ease of use was deemed most important by 33% of the participants, followed by TV content & shows (12%) and community features (5%). CBS.com scored highest in ease of use.
– 70% of respondents watch at least one show online each week, with more than one quarter watching four or more. 7% watch seven or more programs.
– More than 10% of the total reported bugs were found on mobile devices.
– None of the TV network support mobile video watching as they rely on Flash (tested on Blackberry, iPhone, Android and other mobile platforms).
– Cross-site scripting (XSS) vulnerabilities of varying degrees of severity were reported on three out of the four sites.
TV Network comparison (top-two box score of testers who rated each site as “excellent” or “good”):
Here’s the full report: