uTest, which provides software developers with a variety of testing services for their Web, desktop and mobile applications, has raised $17 million in Series D funding in a round led by QuestMark Partners. The new capital will be used to launch new tools to aid developers, and to open more offices in the United States – namely Seattle, Chicago, Los Angeles and Dallas (for starters).

QuestMark co-invested with earlier uTest backers Scale Venture Partners, Longworth Venture Partners, Egan-Managed Capital and Mesco, at double the valuation of the company’s $13 million Series C round. The startup has secured more than $37 million since its inception in 2007.

uTest says part of the fresh funding will be used for mergers and acquisitions.

The company basically provides “in-the-wild” testing services that span the entire development lifecycle for any software application through a series of crowd-sourced testings, including functional, security, localization and usability testing.

uTest says it currently has a community of 45,000+ testers readily available from 180 countries to put Web, mobile and desktop apps to the test on “real devices under real-world conditions”.

Customer includes well-known global brands like Google, Microsoft, Groupon, AOL and the BBC.

Looks like uTest’s got its work cut out for it, too:

UTest allows companies to outsource their QA testing to ‘the cloud.’ The company provides on-demand testing services through its community of 30,000+ professional testers from more than 160 countries around the world.

Customers specify their testing requirements and uTest handpicks the testing team that matches their needs best, based upon locations, languages, OS, browsers, mobile devices and/or carriers. Testersa are paid depending on the number and type of bugs they find, and the marketprice for bug finds fluctuates depending on the number of bugs left to find, the demand for testers, and other criteria.

The new funding will be used towards expanding uTest’s tester-base, moving into service categories, and expanding the company’s recently launched usability testing and load testing services.

Testers were challenged to search the sites for bugs – performing a combination of exploratory, functional and usability testing. At its conclusion, participants filled out a detailed survey in which they ranked each site based on video quality, ease of use, community features and actual TV content/shows. After carefully reviewing each bug and survey response, uTest awarded roughly $4,000 in prize money based on the quality of bugs and feedback.

Top findings:

- Nearly 50% of survey respondents chose video quality as the attribute most important to them when evaluating an online TV network. NBC.com scored highest in video quality.
- Ease of use was deemed most important by 33% of the participants, followed by TV content & shows (12%) and community features (5%). CBS.com scored highest in ease of use.
- 70% of respondents watch at least one show online each week, with more than one quarter watching four or more. 7% watch seven or more programs.
- More than 10% of the total reported bugs were found on mobile devices.
- None of the TV network support mobile video watching as they rely on Flash (tested on Blackberry, iPhone, Android and other mobile platforms).
- Cross-site scripting (XSS) vulnerabilities of varying degrees of severity were reported on three out of the four sites.

TV Network comparison (top-two box score of testers who rated each site as “excellent” or “good”):

Here’s the full report:

uTest, a startup that allows companies to outsource their QA testing to ‘the cloud’ has just concluded its latest quarterly bug battle, during which it put some of the world’s largest social networks to the test. Hundreds of participants (many of which have been involved in product testing for over a year) did their best to uncover flaws across Facebook, MySpace, and LinkedIn, with $3,000 up for grabs for the testers who identified the most crucial bugs.

Below are the final results for the number of bugs found. It’s worth noting that this data is prone to bias and may well overstate the number of ‘showstopping bugs’ (testers probably had a strong incentive to rate their bugs as ‘showstopper’ so as to have a better chance at the prize). But it also meshes fairly well with anecdotal experience.

• Facebook: 416 uTesters found 243 bugs (.58 bugs/tester), including 14.40% that were deemed
  showstoppers by the tester
• LinkedIn: 399 uTesters found 250 bugs (.63 bugs/tester), including 9.78% that were deemed
  showstoppers by the tester.
• MySpace: 304 uTesters found 225 bugs (.74 bugs/tester), including 10.80% that were deemed
  showstoppers by the tester

Aside from logging each bug that they found, testers were also asked to rate each social network in a handful of categories and to detail some of the most apparent issues. Overall, the contest deemed Facebook to have the most complete feature set, but the social network was criticized for not having a high enough default security level (the options to keep things private are there, but are not always set by default).

LinkedIn took the title of “Best in Overall Quality”, but testers had issues syncing data with Outlook, which they considered “problematic for the ’professional social network’”

MySpace didn’t take the top prize in any of the bug testing categories, but was considered to have “a loyal following”, as it drew consistently high marks from a subset of users. Common complaints included “Overlapping images, backgrounds, titles, etc.” and slow page load times.

Testers’ choice for best overall quality: 
1.  LinkedIn (45%) 
2.  Facebook (37%)   
3.  MySpace (17%) 
 
Testers’ choice for best usability:  
1.  Facebook (39%) 
2.  LinkedIn (38%) 
3.  MySpace (17%) 
 
Testers’ choice for best feature set: 
1.  Facebook (46%) 
2.  LinkedIn (36%) 
3.  MySpace (18%)

uTest, the crowdsourcing QA startup that lets companies rely on external developers to help them identify bugs, has closed a $5 million Series B funding round led by Longworth Venture Partners and Egan-Managed Capital, along with existing investors Mesco Ltd. and Massachusetts Technology Development Corp. The new round brings the startup’s total funding to around $7.8 million after a $2.3 million Series A round last year and some early seed funding.

The site offers customers a web based platform and tools for monitoring testing and QA cycles, which are available to them free of charge (they only pay for the testing completed by the crowdsourced community). Community members are paid depending on the number and type of bugs they find, and the marketprice for bug finds fluctuates depending on the number of bugs left to find, the demand for testers, and other criteria. Since launching in Febuary, the uTest community has grown to 11,000 users.

The startup is trying a crowdsourcing approach to testing software bugs. Anyone can sign up to test software and make some cash. uTest estimates that its testers will be able to rake in anywhere from a few hundred to a few thousand dollars per month, depending on tester-expertise and bug pricing.

It is important to note that bug prices will fluctuate in real-time based on a variety of parameters, including: Bug type (logical, GUI), type of application (Web, desktop), number of testers that fit the required profile for the testing environment, bugs left to find, and more.

Over 2000 testers from around the world have already signed-up, so it seems the company’s pay-per-bug model is resonating well across testing professionals.

Boston-based uTest has raised $1.7M to bring to market a crowd-sourced quality assurance (QA) marketplace and community. The service has begun recruiting testers in anticipation of its official launch, expected in early 2008.

No fuzzy logic here; uTest presents clear and communicable value propositions: For companies in need of QA, uTest is providing an on-demand environment for the management of full testing cycles. For testers, uTest is providing an oDesk-like marketplace through which they can be hired and paid on a Pay-per-Bug basis.

The company’s founders recognized that QA and usability testing solutions are inefficient. QA departments are either under-utilized when waiting for versions to test, or over-extended when a new version comes out. To most SMBs, QA outsourcing is neither cost effective nor available “on-demand”. Then, of course, there are cost-cutting measures that leave companies unable to sufficiently fund QA efforts, ultimately shipping buggy applications to users like us.

The uTest testing platform itself is entirely Web-based and provides the management of complete QA cycles, from creating/loading test scripts, to selecting the target community (profile and environment) of testers required. The platform also provides test case management tools, real-time information on bugs and defects logged, statistical information on release maturity level, as well as QA coverage and market readiness. Support for bug tracking systems includes Bugzilla, Jira, FogBugs and will be expanded over time.

In a smart business move, uTest will not charge companies to use their platform to manage testing and QA cycles—this will be entirely free of charge. uTest will only charge for the services provided by the community of testers.

The uTest offering makes a lot of sense to me and I expect it will be warmly embraced for several reasons. First, consider for a moment that for many SMBs, QA management solutions such as those offered by Mercury or IBM-Rational are beyond their reach. Speaking from personal experience, the majority of the testing cycles I’ve seen have been “managed” on Word or Excel. The “on-demand” model which has been proven time and time again on the Web from CRM to ERP, should work just as well for QA. The fact that the platform will be offered free of charge, pushes the offering to “no-brainer land”.

Second, beyond the manageability aspects of uTest’s service, companies will obviously enjoy the cost saving aspects of paying testers by the bug. It’s this aspect that in my opinion will make uTest’s offering relevant not only to SMBs but to large organizations as well. Logic would dictate that if they embraced off-shoring and near-shoring, crowd-sourced QA shouldn’t be too jagged of a pill to swallow.

Third, recruiting a userbase of testers should not be difficult. There are droves of potential testers in countries such as India, China, Russia, Bulgaria, Estonia, etc. Also, getting hired through sites like oDesk, Elance and RentACoder is becoming increasingly difficult due to the growing number of service providers. These same individuals can theoretically provide testing services instead of programming.

All-in-all uTest sounds quite promising and has earned a place on my companies to watch in 2008 list.