Software testing marketplace uTest today announced the results of its so-called “TV Networks Bug Battle” competition. More than 500 software professionals from 30 countries around the world participated in the quarterly competition, reporting a total of 908 technical, functional and GUI bugs in the web and mobile apps of NBC, CBS, Fox and ABC.
Testers were challenged to search the sites for bugs – performing a combination of exploratory, functional and usability testing. At its conclusion, participants filled out a detailed survey in which they ranked each site based on video quality, ease of use, community features and actual TV content/shows. After carefully reviewing each bug and survey response, uTest awarded roughly $4,000 in prize money based on the quality of bugs and feedback.
Top findings:
- Nearly 50% of survey respondents chose video quality as the attribute most important to them when evaluating an online TV network. NBC.com scored highest in video quality.
- Ease of use was deemed most important by 33% of the participants, followed by TV content & shows (12%) and community features (5%). CBS.com scored highest in ease of use.
- 70% of respondents watch at least one show online each week, with more than one quarter watching four or more. 7% watch seven or more programs.
- More than 10% of the total reported bugs were found on mobile devices.
- None of the TV network support mobile video watching as they rely on Flash (tested on Blackberry, iPhone, Android and other mobile platforms).
- Cross-site scripting (XSS) vulnerabilities of varying degrees of severity were reported on three out of the four sites.
TV Network comparison (top-two box score of testers who rated each site as “excellent” or “good”):
Here’s the full report:
uTest provides in-the-wild testing services for web, desktop and mobile applications. By leveraging a community of more than 60,000+ professional software testers from 190 countries, uTest helps companies test their products under real-world conditions. Thousands of companies– from startups to global enterprises such as Google, Microsoft, HBO, Zynga and USA Today – turn to uTest to complement the in-the-lab testing they do, and to help them launch better web and mobile apps. uTest’s services span the entire...
Austin, TX
Seattle, WA
San Diego, CA
Menlo Park, CA
Boston, MA
Disrupt Europe: Berlin Hackathon
Berlin, Germany
