CrowdFlower Adds Skill Tests To Find The Right Crowdsourced Labor

CrowdFlower, a startup that helps businesses manage crowdsourced labor for tasks like image moderation, is looking to expand the types of jobs it can tackle with the launch of new Skill Tests — which, as you probably guessed, are online tests used to measure workers’ proficiency in various skills.

Until now, anyone doing work on CrowdFlower had been able to access any job. As people do the work, the company performs automatic and human checks to ensure that the results are up to par — if someone’s work isn’t adequate, then all of their results are removed.

However, CEO Lukas Biewald (a former college roommate of mine) and Tools and Platform Product Manager Joseph Childress told me that some potential customers are interested in a particular skill or knowledge set — they might, for example, want results from people who are knowledgeable about golf, or cars. The Skill Tests allow CrowdFlower to target jobs to workers with specific skills. In Biewald’s words, it’s a way to create “a fair system” where people can still access the jobs they want, while also finding “higher value crowds” within the company’s overall workforce.

“This is like how LinkedIn allows people to endorse your skills, except that instead of using your network, we actually verify that people do know about these specific areas,” he said.

The tests consist of 50-60 questions in five or six categories, the company said. The first tests focus on URL acquisition and image moderation.

Childress said the goal is to add new types of work that are more lucrative, rather than adding new requirements to existing jobs. At the same time, he said the new tests could also help workers by giving them a clear idea of skills they should develop.

With this addition, Childress said CrowdFlower is now taking a three-pronged approach to quality control, combining the automatic and human checks mentioned above with skill tests, as well as checks from the most reliable workers on a given job.