Crowdfunding platform Kickstarter has come in for a lot of plaudits for creating a new platform on which to fund startups. Certainly, since the passing of the U.S. Jobs Act earlier this year, crowdfunding is about to have its day in the sun. But it’s also had its fair share of skeptical critics.
[Update: Please scroll to the bottom for a convincing rebuttal by Kickstarter.]
A few weeks ago, Jeanne Pi of AppsBlogger wrote about Kickstarter failures that were difficult to find. Why? Because Kickstarter prevents failed campaigns from being indexed by search engines. Following that, Kickstarter began a stats page with data and basic metrics about the projects that were unsuccessful. But according to Pi and her team, Kickstarter is still hiding much of the data about failed projects.
However, Pi was criticised for her analysis, by Professor Ethan Mollick of The Wharton School of the University of Pennsylvania, for comparing the percent of projects in a category that are successful without also controlling for the size of the project. Plus, plenty of projects might not have gotten funded on Kickstarter but did not deserve to be called a failure.
So Pi and team have done more digging and come up with some research. You can read the post in full here, but meanwhile, here are some highlights.
• Pi alleges that the bulk of the missing data about projects is from failed projects. “This further supports the contention that failed projects are being made difficult to find,” she says.
• Pi and team scraped Kickstarter’s pages and found a total of 57,860 projects (or approximately 91% of all launched projects). They scraped about 99% of the successfully funded but were only able to scrape 82% of the unsuccessfully funded projects.
• They thus allege that Kickstarter “may have data issues that inflate the number of failed projects, or else are otherwise hiding these projects.”
• They also found that “projects that successfully fund tend to do so by relatively small margins. 25% of them funded at 3% or less over their goal. And 50% raised only 10% over their goal. In other words, when you succeed, it’s not by much. Projects that raised double or more over their goal is the exception.”
• In addition “Projects that fail to fund tend to fail by large margins. Only 10% were able to reach 30% of their funding goal. And only 3% made it to 50% of their goal. In other words, when you fail, you fail big; adding insult to injury. In short… Failures happen by large amounts, successes by small amounts.”
• The exception is any large project (over $10,000) that received over 10x its funding goal. “Of the 106 projects that received over 10 times their goal, only 33 were large projects. With the exception of a single music project, all of these 33 overachievers were in the hardware, software, games, or product design categories.”
• Pushing the project to a lot of friends on social networks increase the chances of the project getting funding.
• Pi and team clim that only 25% of Kickstarter projects are delivered on time – however, they admit that they only surveyed a sample of projects specifically from the Technology and Design categories (which in turn comprise only less than 5% of projects on Kickstarter overall).
• Over-funding increases the chances of the project being delayed, (perhaps because the whol thing changes in nature with more cash?).
Pi and team have produce an info-graphic to illustrate their findings.
[Responses by Kickstarter:
- The research paper’s actual findings on project delivery are very different than the headlines. Of the 471 Technology and Design projects surveyed, 96.4% had either fulfilled or were in the process of fulfilling and 67% had fulfilled to-date. While they found that delays were common, this is not unexpected. Note that it’s called the “Estimated Delivery Date” on Kickstarter.
- To generate data for the research, the blogger hired programmers to “scrape” Kickstarter. Scraping is a common but inexact tool, and the programmers hired were unable to pull all of Kickstarter’s data. Because of this, the blogger asserts that Kickstarter was either “hiding” or “inflating” its data. That’s a serious accusation. It also happens to be untrue. The only reason the blogger was even aware that they were unable to pull all data is because Kickstarter is open about sharing its numbers. The responsible conclusion is that they just didn’t do the scrape correctly.
- Much of the data in the infographic is based on “large projects, with goals of $10,000 or higher.” Because only 27% of projects have goals of $10,000 or higher, the analysis excludes 73% of projects on Kickstarter.
- Even the correct data points in the infographic are unnecessarily sensationalized. We’re eager to see more analysis on Kickstarter’s data, but it’s important for it to be approached with diligence and care.]
