Twitter Search is easily the most promising aspect of Twitter. People talk about mundane updates, or connecting with companies, or following celebrities — but that’s all small scale. The real power of Twitter lies in its aggregate data. Why do you think Google and every other company out there is interested in them? It’s not just because they are the hot ticket in town right now, despite what some would have you believe. It’s all about the data. And Twitter knows that too — and is apparently on the verge of some interesting moves with Twitter Search that will better highlight that.
Speaking on a panel today, Santosh Jayaram, Twitter’s new VP of Operations, had some very interesting things to say, Webware’s Rafe Needleman who moderated it, reports. The most interesting thing is that Twitter Search will soon begin crawling the links that people tweet out and indexing them. This immediately takes Twitter Search, which is still a very basic service, to the next level. This means that no longer will it just be a stream of textual tweets, but it will include millions of web pages as well — web pages that are more or less already curated by the individuals who tweet them out. Sure, there will be some spam, maybe even a lot of it, but this user curation should help real good content from around the web bubble up.
Apparently, Twitter Search will index the content of these pages as well. Yes, this is what Google does. So it should be no surprise when I say that Jayaram formerly worked on the search quality team for Google. [Update: We previously stated that Jayaram was a VP at Google, which is not the case.]
Of course there is no way Twitter Search will index as many pages as Google, but that’s not the point. Twitter Search isn’t meant to replace Google, that’d be dumb. At this point, no one is going to beat Google at its own game (you hear that Microsoft?). Twitter Search is meant to be a different kind of powerful search engine in its own right. A smaller, potentially curated, real-time search engine.
Twitter’s biggest trump card here is the real-time factor. It’s not entirely real-time right now, and there are often delays, but it’s faster than Google — mainly thanks to the nature of tweets (fast to send) versus the nature of webpages (slow to build). And so it’s not yet clear how indexing these linked pages would impact that aspect of Twitter Search. You’d have to assume it would slow it down, but really there’s not much point in assuming anything because it’s not yet really clear how Twitter would use this webpage data in search results.
One thing that is more clear is that Twitter is also looking at ways to better tailor search results. As I mentioned, right now the results are a very basic collection of tweets with key terms. Jayaram says that the company wants to add some sort of reputation filtering to offer better results. Now before everyone gets in a tizzy about the word “reputation” just like when Loic Le Meur brought up the idea of filtering Twitter Search by “authority,” Jayaram says Twitter’s engineers are still determining the best way to calculate this reputation. That seems to indicate that it wouldn’t be something silly like just being based on the number of followers you have.
Things should get very interesting in this space over the next several months. Unless, of course, Google buys Twitter first.