Xamarin Launches Test Cloud Automated Mobile UI Testing Platform, Acquires Mobile Test Company LessPainful

Xamarin, the company behind the cross-platform mobile development platform with the same name, today announced that it has recently acquired the Danish mobile testing startup LessPainful. LessPainful is the company behind the popular open source cross-platform functional test platform for Android and iOS, and Xamarin has already put this acquisition to use. The Xamarin Test Cloud, the automated user interface testing platform the company launched at its Evolve 2013 developer conference in Austin today, is based on Calabash.

Test Cloud is a pretty ingenious product that solves a real problem for mobile developers. It allows users to easily write user interface tests and then see how their apps perform on real-world devices. Indeed, LessPainful – and now Xamarin – operate a lab with hundreds of devices, and the service runs these tests directly on the device and provides users with detailed logs and screenshots about how the app performed.

devices_xamarinAs Xamarin co-founder and CEO Nat Friedman told me last week, fewer than 10 percent of developers currently use automated user interface testing for their apps. The tools that are currently on the market, he noted, tend to be hard to set up and use – and once they run, the tests are often very fragile because they rely on image recognition, so even the slightest change to the wording on a button can throw them off. Developers, however, should really run these tests, given the huge fragmentation in the mobile space (and especially on Android). They only get a short amount of time to convince users that their apps are worth using, and the smoother the experience, the more time the user will spend with the app.

Test Cloud – and Calabash – takes a different approach. It provides users with a very basic language to write tests and a great user interface for running the tests and seeing the logs. Developers can choose which phones to test their apps on or just choose to run their apps on the top 10, 20 or 30 phones on the market.


As Friedman noted, one of the nifty features of Test Cloud is that users don’t even have to write a test to get some early results (he called it a “gateway drug” in our interview). The Xamarin App Explorer will automatically navigate through your app and visit every screen, press every button and try every UI element. All of the user interface elements are identified by object IDs and not image recognition or gesture recording, so as long as the ID remains the same, developers and designers can change the color of the layout of their apps at will and the tests will still work.

Because it can be integrated into popular continuous integration systems like Team Foundation Server, Jenkins and TeamCity (Xamarin offers plugins for those), as well as an API and command-line interface, it’s easy for developers to regularly test their apps.

Xamarin, Friedman told me, is growing rapidly. The company has over 15,000 paying customers and in total, over 300,000 developers are now using the service. Some of the company’s most well-known customers include Rdio, Clear Channel, the Portland Trailblazers and National Instruments. The launch of Xamarin 2.0, which allows developers to use Visual Studio to write iOS apps, has been a major factor in the recent interest in the company’s tools.

Access to the Test Cloud is currently only available by invite. During the beta phase, the service will be available for free to Xamarin users. After the beta ends later this year, the pricing will be based on device hours, but the company is still working out the details.