DataSift: There Is No Change To Our Firehose Deal With Twitter After Gnip Acquisition

Next Story

Red Hat To Bring Docker Support To Enterprise Linux And OpenShift

Earlier today, Twitter announced that it acquired Gnip, one of its partners that uses Twitter’s firehose of data and parses it for third parties who mine the social network for market insights. One natural follow-up question: what happens to the other companies that take the firehose? The biggest of these, DataSift, has told TechCrunch that for now, it will be business as usual.

“There is no change to our relationship with Twitter,” CEO Rob Bailey told us. “We’ve worked really hard over the years to build a platform that adds value and we believe that we have been and will remain a vital part of the Twitter ecosystem.” That will include the resyndication deal that it has with some 1,000 businesses across 40 countries.

DataSift was the first company to ink a firehose and resyndication deal with Twitter, and he says that the two companies have a “long-term” contract. But he wouldn’t specify the length of it, when it would be up for renewal, or the financial terms of it.

He and DataSift founder Nick Halstead, who is the CTO, also refused to say whether Twitter had approached them for an acquisition. DataSift has raised nearly $72 million since 2010, compared to Gnip raising just over $6.6 million — so you might assume that the price would have been significantly different had Twitter looked to DataSift instead.

But the two confirmed that because of the nature of the deal, Twitter did keep them in the loop in advance of making the acquisition public.

“We’ve had a longer relationship with Twitter than anyone else,” said Halstead. “We have a firehose contract and they would talk to us before doing anything. They told us before they made the announcement.” He wouldn’t say whether that “before” was one hour, one day, one week, or one month in advance.

Unsurprisingly, DataSift, which says it’s currently growing revenues at a rate of 400% a year, is now talking up how Gnip is different to what it does.

The main difference, Halstead said, is that DataSift offers a single API that covers all of the different data sources it uses for its big data mining effort. Those sources include Twitter, but also Facebook, Tumblr, WordPress, Sina Weibo, LexisNexis, hundreds of news sources and more. It’s also the only one to offer historical data as far back as it does, Halstead noted.

Gnip requires an API for each data source, and has put the majority of its focus on Twitter, Halstead said.

“We are a huge fans of Gnip, but the differentiation in the market is that DataSift has spent four years buiding a large platform where our customers have to deal with only one API,” he said. “It’s a good purchase for Twitter because Gnip wants to focus only on Twitter data while we want to concentrate on everything.”

Bailey would not comment on whether he worries about the big data and firehose world eventually going the way of Twitter clients, which were (in)famously cut off by Twitter in 2012 when the company decided to take a stronger grip on how people interacted with its platform and developers who merely recreated the Twitter experience were no longer as welcome as they were before.

At the time, Twitter crafted a kind of magic quadrant of the kinds of companies it would like to work with more in the future, and less in the future. In that vein, big-data analytics companies like DataSift remained a valuable part of the ecosystem.

But now the goal posts have moved — perhaps because Twitter itself sees that there can be a viable business for itself in harnessing data in a smarter way. (Diversity of revenue is an important part of a company’s health, after all.) In theory, Twitter could decide to build up Gnip into its primary firehose channel. At that point, it could argue that DataSift merely provides the same experience.

It’s a valid point, Bailey said, but nothing more at this point.

“We can’t speculate on Twitter’s plans but I think that we have constantly in the last three years tried to evolve the platform and deliver a unique service that no one else can provide,” he said. “Half our company is engineers because we’ve been obsessed with adding value on top of the data. Three years ago we were very clear about this. We knew that if we syndicated only Twitter data we wouldn’t exist. We’ve been working our assess off executing on that.”