When it comes to getting access to all the data that flows through Twitter, there are the 50,000 apps that drink from Twitter’s Streaming API, which is subject to various limits. And then there are the chosen few who get the full unlimited firehose of data, the more than 50 million Tweets a day coursing through Twitter.
In the past, only select partners, particularly big search engines such as Google or Bing, got the full firehose. Search engines need it more than others to be able to index and serve up results in realtime. Today, smaller search startups are also getting the firehose. These include Ellerdale, Collecta, Kosmix, Scoopler, twazzup, CrowdEye, and Chainn Search (which has not yet launched).
In December at Le Web, Twitter indicated that the firehose would open up to more and more developers. This is a step in that direction.
And while Twitter makes deep-pocketed partners like Google and Bing pay for the firehose, it treats startups differently with more favorable (even free) terms, or at least it used to. These aren’t the first startups to get the firehose, and they won’t be the last. Once they get to a certain size though, they’ll pay if they don’t already. Apparently it works on a sliding scale.
Photo credit: Flickr/ZeroOne
Collecta monitors the update streams of popular realtime blogs and sites like Twitter, Wordpress, and Flickr, and shows results as they happen. Results can be filtered by status updates, comments, stories, or photos. The entire engine is built around the XMPP standard, which pushes out data on a continual basis, so that for every search you end up watching a stream that keeps updating itself.
Ellerdale, operating in stealth mode for most of the past year, has built an exciting site using leading-edge semantic technology that filters the real-time stream by topics, instead of keyword strings. This enables Ellerdale to automatically display trending topics in real-time within categories, e.g. trending people, trending films, trending companies, trending sports, etc. Users can select from over a million individual topics to track in real-time. The latest tweets and rss articles are tracked, as well as the most...
CrowdEye has always been proud to be on the forefront of building real-time search technology and we want to thank all of you, our amazing users, for your support. While we have been successful pushing the state of the art forward with features like location-based search, relevance sorting of results and sentiment, we have not yet built a profitable business around CrowdEye. We are hard at work innovating new real-time search ideas that have the potential for a good business model...
Austin, TX
Seattle, WA
San Diego, CA
Menlo Park, CA
Berlin, Germany
Boston, MA
