Twitter Can Be Liberated – Here's How

Over the last few days a number of popular bloggers have complained, loudly, that it’s time to ditch Twitter and move to a decentralized version of the service that won’t go down every time usage spikes. Generally services like Twitter, once they reach a critical mass of users, can’t really be stopped because the network effect kicks in as a massive barrier to entry. But one aspect of Twitter – it’s openness – could also be its Achilles heel.

Scott Hanselman mused on how such a system might work yesterday. Dave Winer has also written extensively about this, although he’s more focused on simply backing up Twitter messages so that they are still available when the site goes down. He isn’t paying attention to the real benefit of Twitter – the spontaneous, asynchronous conversations that pop up between every changing groups of people. Instead, he just wants to make sure the data is secure.

Marc Canter also weighs in in a surprisingly lucid post. Infrastructure needs to be decentralized to be reliable, he says, pointing to DNS as an example.

Decentralizing Twitter isn’t about having backups of content if the service goes down. It’s about making sure that the service as a whole can’t go down, and allowing all those conversations to continue unabated no matter how popular the service gets.

Here’s How To Decentralize Twitter

Chris Saad, co-founder of DataPortability and founder of startup Faraday Media, thinks he knows one way Twitter can be decentralized effectively.

The key weakness in Twitter (and therefore opportunity for a new decentralized approach) is the fact that so much Twitter activity occurs off Users are getting very used to using desktop clients (Twitterific, Alert Thingy, Twhirl, etc.), IM, SMS, and other interfaces to talk to Twitter. Those third party applications can be tuned to lock in to the new decentralized Twitter-like service instead or in addition to Twitter itself.

Users on the new system will post to a microblog. Third parties can create platforms for these blogs, and have them be certified as compliant with the microblogging standard – posts of 140 characters, no titles, etc. Users could also install compliant software on their own servers – much as they do with today. There would certainly be an open source project around this shortly.

The hard part is putting these microblog posts together into a Twitter-like conversation where people subscribe to those writers they like, can respond via an “@[username]” mechanism, etc.

This can’t be done efficiently just via RSS because rapid and excessive polling would bring servers to a halt. Instead, Saad thinks wrapping RSS in XMPP, an open standards based instant messaging protocol that was originally created for Jabber and is now used in various applications including Google Talk, is the answer. XMPP allows for pushing of messages to subscribers, which removes the need for constant polling. For more of Saad’s thinking, see his site on their product SyncStream, and they’ve already written code that will do this based on their proposed standard called “GetPingd.” Twitter uses XMPP in their API already; third party applications like Google Talk integrate with Twitter via XMPP already.

If users begin to “twitter” using this new system, applications like Alert Thingy can simply add it to their functionality. Instead of using Alert Thingy to sign into just Twitter, you would also create an account at Alerty Thingy, too, which stores your subscription lists. This is analogous to what feed readers like Google Reader do today with RSS subscriptions.

XMPP already has a mechanism for tracking subscribers which performs the same funtion as the “followers” list in Twitter. Users can therefore have a list and count of followers back on their microblog.

Handling replies is a little more complicated (the @[username] feature on Twitter). One way to do this is to use the existing RSS infrastructure specifically with Google or Technorati Blog search to monitor for @replies and feed this to the user applicaitons. Results wouldn’t be limited to just microblogs, but I’m not sure that matters. It would also be simple to block spammy stuff by simply clicking on a button, the same way Twitter works today.

The net effect of this theoretical platform is to move the publishing to a completely decentralized network, and move the hard part of Twitter into the third party aggregators (which is already a competitive space). There would be no one central bottleneck that could fail.

Twitter (the company, the service, the site or the software) isn’t part of this new decentralized platform, of course, so they’ll oppose it. But can it happen? Absolutely, because it doesn’t require a hard reset to a new Twitter-free world. Existing Twitter clients could add support for GetPingd and the rest of the infrastructure and it would work seamlessly with the existing Twitter world. Anyone could create a website that duplicates what Twitter does today, but supporting the new decentralized framework. And a limitless number of microblogging applications could emerge to join in the fun.

And we’d never have to deal with outages again.