At this very moment, at this very villa in the Israeli city of Hertzeliya Pituach, the final preparations are being made for what can be best described as ‘TwitterSense’—a way to automatically filter your Twitter stream so that the most relevant Tweets come out on top. The location in question is the home of my6sense, which currently offers a powerful way to filter news feeds. It is applying its filtering technology to Twitter and by the looks of it you’ll soon be able to follow as many Twitter users as you want and still never miss out on the most important tweets.
It took insistent prodding on my part to get my6sense to spill some of the beans and give me a sneak peak. The good news is that TwitterSense (my term, not theirs) is real and it works. The bad news is that it’ll take a couple of more months to be deployed. And yes, it could greatly improve the way we consume Twitter streams.
The advent of a TwitterSense offering could not be timelier as the onslaught of noise on Twitter has increased dramatically and its manageability has become a real pain point. Even Robert “The Stream Prince” Scoble has had to take dramatic measures, namely, slashing the number of users he follows on Twitter and befriends on Facebook. I, on the other hand, keep the number of people I follow on Twitter in the neighborhood of 150. This number works well for me, but I keep wondering whether I’m missing out on users who could provide insights relevant to my personal and professional interests. That is exactly where TwitterSense would come into play.
First, a quick recap on my6sense: The company has been building out what it calls ‘digital intuition,’ a content ranking technology that to date has been applied to RSS feeds to separate the signal from the noise. My6sense’s technology translates user actions such as Web navigation within and across various streams of content, and actions taken with various pieces of information in different contexts, into semantically-sensible implicit user feedback. The real beauty is that it requires zero intervention other than using the app itself. Here’s how I described my experience with the alpha release:
The “A-ha moment” took a couple of days of interacting with the product, but it came. Suddenly, very relevant info was floated to the top of the main “TOP MESSAGES” pane. By relevant, I mean posts I would absolutely have clicked on through my Reader, but would have had to sift through hundreds of posts before doing so.
A couple of weeks ago my6sense announced its new native iPhone app (iTunes link), which along with a few new features, presented a major user experience improvement over the original iPhone web app version. So far there is nothing seemingly compelling beyond our previous in-depth look into the company’s technology. But looks can be deceiving. Underneath the surface lies what could transform the way my6sense users consume Twitter.
TwitterSense in an extension of my6sense’s ranking technology and in this respect treats a user’s Twitter stream like an ordinary content source, much like an RSS feed. To begin with, my6sense has to differentiate between simple status updates/personal tweets and tweets which link to content. The differentiation is a must because its ranking algorithms require further optimization to be able to correctly float important simple/status tweets. In the short-term they have no plans to solve this particular challenge. Instead, the company is focusing on ranking tweets with links—and we all get quite a few of those. From my6sense’s perspective, your friends provide the first level of filtering. It then provides the second level by taking it upon itself to re-rank these Tweets so a users’ focus is directed to the information that is most important to them.
If you tend to click on links from specific friends on Twitter, those will get a boost in the rankings. But my6sense also looks at the underlying pages behind the links and figures out what topics those pages are about using its semantic engine. If those topics match your interests, as determined by your past reading and clicking behavior on the app, then those links rise to the top as well.
So the obvious question to ask is, why then if it rests upon my6sense’s existing technology isn’t it deployed already? First, there are challenges in ranking the content behind the link. A typical web page includes not only the post/article itself, but additional data and content as well. my6sense wants to make sure it ranks the intended content and this isn’t always trivial.
Second, there are scalability challenges. On average, a Twitter stream encompasses a greater mass of content than an average RSS feed. This means that my6sense has to go out and parse every piece of content behind every link in a user’s steam so it can analyze it based on the user’s ranking model. This requires extra processing power in order to avoid significant delays in ranking. My6sense did close a round of funding recently, but it can’t just throw money at the problem and solve it via brute force (i.e. just buy more machines).
I asked Barak Hachamov, the company’s founder and president, whether they’ll be offering TwitterSense integration for Twitter clients. His answer was that they do have such plans but it’s far too early to talk about them now.
My6sense plans to make TwitterSense publicly available in a couple of months or so. In the meantime, if you want to experience what it will behave like I suggest downloading my6sense’s native iPhone app to see how it works on RSS feeds. You won’t have to spend very long waiting to see the ranking magic since some backend improvements were made that get users to achieve the ‘A-ha moment’ I mentioned above much quicker, even within one or two brief sessions. There’s also a new digital intuition meter that provides users with feedback regarding the status of their preference model and indicates how strong their digital intuition is at that point in time.
We’ll be keeping a close tab on the upcoming release of this so called TwitterSense and reexamine it when it’s made publicly available in a couple of months.