Twitter developers have long been pining for access to historical Tweets. Right now, the best they can get is 7 days worth based on keyword search. DataSift, one of Twitter’s data partners which currently provides developers and third parties with access to the full Twitter firehose in realtime, will soon make historical Tweets accessible as well. Developers can sign up for the Alpha of DataSift’s Historical Data starting today (the actual service will begin to roll out in the first quarter of next year).
DataSift’s Historical service will give developers, social media monitoring companies, marketers, and brands access to 60 days of tweets for the Alpha, which can be analyzed and filtered beyond simple keyword search. When the service is launched more broadly later next year, it will go back as far as two years. DataSift allows for all sorts of data analysis because it pours all the tweets into a structured database. So you can give it queries like: “Give me all the tweets that mention TechCrunch from people who do not follow @techcrunch” or “All females in the UK who mention fashion.”
The company is already collecting the 1 terabyte per day of data internally—that is how much is produced by the firehose of 250 million tweets a day— with 400 terabytes total so far. “This is a real ‘big data’ engine—and that we are making it simple—we are taking advantage of map reduce—but this is our own bespoke processing engine,” says DataSift founder Nick Halstead, referring to the Hadoop technology the service is partially based on.
DataSift is the leading social data platform, enabling companies to aggregate, filter and extract insights from the billions of public social conversations on Twitter, leading social networks and millions of other sources. DataSift provides access to both real-time and historical social data to uncover insights and trends that relate to brands, businesses, financial markets, news and public opinion. Delivered as a cloud platform, DataSift does the heavy lifting for companies creating social media monitoring, social CRM, business intelligence, financial trading...
Created in 2006, Twitter is a global real-time communications platform with 400 million monthly visitors to twitter.com, more than 200 million monthly active users around the world. We see a billion tweets every 2.5 days on every conceivable topic. World leaders, major athletes, star performers, news organizations and entertainment outlets are among the millions of active Twitter accounts through which users can truly get the pulse of the planet.
Austin, TX
Seattle, WA
San Diego, CA
Menlo Park, CA
Boston, MA
Disrupt Europe: Berlin Hackathon
Berlin, Germany
