The Mayor of Realtime

ginjointsIf you believe the noise emanating from the retweetsphere, this realtime thing is something we don’t need, don’t want, destroys our sense of normalcy, prevents real thought from emerging, is populated by charlatans and idiots with more time than sense on their hands, and besides it causes seizures.

I went to Scoble’s blog on the recommendation of some retweet and found myself watching a realtime updating Twitter list of Tech Smart Guys or something of that nature. Scoble evidently has spent considerable time compiling these lists, running into limits like 500 geniuses on any one list. There are problems with lists, I’ve heard, but none more pronounced than the question of why one would like to produce multiple Twitter home pages to navigate between when the Home page is already useless.

I’ve certainly read numerous explanations of why lists get around the Follow problem by allowing you to create imaginary follow lists (hat tip to the late great FriendFeed’s imaginary friends concept.) Indeed, without Track all Follows are imaginary in that you are stuck waiting around for people to randomly say something interesting on a freakin’ Web page. These are the same Web pages we ran away from when RSS gave us the opportunity to request updates of blog posts when they were published.

But RSS has no social metadata to speak of, and no business model to keep the pipeline flowing. And if RSS detractors are to be believed, the technology never got significant adoption anyway. In fact, RSS scraped the cream of the attentionrati off the top of the Web page model and forced publishers into a race for space in a diminishing window of consumption time. Thus micro-messages were invented as a hybrid of the 10-second spot and texting crowd. Besides, most RSS posts wrapped 1 or 2 seconds of information in a stream of self-promotion — like this one.

It turns out that we missed the good ole Web pages more than we thought, because as information junkies we actually enjoy noise as much as signal. The noise surrounding an RSS item is stripped of the kinds of metadata cues that help us understand not just what is being fed us but by whom and for what reason. Jumping briefly to the so-called present, Twitter’s new house retweet destroys the messy noise surrounding multiple cascading citations that emanates the acrid smell of smoke and lurking fire.

RSS makes Twitter look positively Studio 54-ish with its Bianca, Grace, and Andy avatars flashing down the non-updating page. RSS tells you nothing about what sub-group of the A-List subscribes, nothing about what the cloud thinks about the stream to date, nothing about anything other than what you already knew when you subscribed in the first place. And Twitter lists are an updated version of the same lousy deal, taken out of the famous River of News or email-style reader and dumped back on the freakin’ Web page.

How do these Web pages work again? Lessee, if I get bored I can click on a link and go to some other page. Or I can hit the back button and — oh wait, there is no Back because the list opens another tab or page instance. At least I’m not just sitting here waiting for some factoid to scroll down on Scoble’s auto-updating list widget, or worse yet clicking on the new all-improved click me something’s happening text on Twitter Web pages.

If RSS sucks (there is no back button at all because you can’t address RSS items by URL except through a reader) and lists suck because they also strip away social cues, then why are we so happy to be back in Web page land and its sea of comments, ads, come-ons, opt-outs, identity honey pots, etc. We’re not; we’re looking for some safe haven which respects the noise for its social acuity while allowing group dynamics to accrue. Put simpler, give me clues as to what people I care about think about what’s going on Now.

That is the reason realtime is important: not the aggregate value of the stream, not the authority mapping of lists or retweets or location, not the personal garden of Facebook, the professional business district of LinkedIn. Realtime is important because we are still wired up to make decisions with the most amount of information at the last possible second.

Is Twitter realtime? No, because it doesn’t support Track which allows people to alert people in realtime. If we are more or less confident that we will be alerted when something game-changing has or will happen, we don’t need to sit around watching Web pages not update. Track is social alerts, and Twitter has disabled it for a very long time. Can Microsoft or Google provide alerts under the terms of their firehose deals? We don’t know nothing except to assume that the terms of the deals prohibit us knowing that answer.

But for Track to work it needs an unencumbered micro-bus where anyone can jump in in realtime and engage with any node or group of nodes. A dynamic list if you must. That’s why FriendFeed got its hose cut off when Facebook bought them. If there’s no apparent latency between Twitter and a third party (we are the second party) then there is no perceived value add for either service. Result: an effectively unencumbered micro-bus where gestures can be transmitted and received in realtime. This is the only real stream filtering possible.

Stop and think about that for a micro-second. If semantic analysis, location constraint, and relevance farming are the primary tools for filtering, the results will inevitably fall apart in competition with social mining of affinities. Adding hashtags and other codes serves only to provide SEO-like gamers the tools to tarnish the value of the system. Traditional journalists fall back on the priesthood of editorial liturgy, which is endlessly vulnerable to new and exciting voices. Witness the Beatles and the Nixon administration. One generation’s insolence becomes the next’s anthem.

Do we have any reason to expect Facebook will harness realtime any more than Twitter has failed? Could happen, but more likely is something coming up from below. Foursquare has a whiff of this insolence, but depending on going out every night to get “on the map” skews the sample in ways Twitter avoided. There are new streamreaders checking in every day, but who will become the Mayor of Realtime is not so much the question as if it’s anybody at all.

Ever since RSS took hold, we’ve had an ongoing joke that the winner will be the first application to most quickly confirm that nothing is going on. Google Reader held that title for years, then gave way to the onslaught of the micro-bus and url shorteners. Now every startup and platform is busy reinventing the same dumb wheel, spinning and twisting to avoid looking like they’re building the Next Big Ping Server.

In fact, that is the business model for every single service out there. Scrape away all the shiny promises and what you’re being pitched is something that does not include unencumbered Track. That’s because if it’s open (anybody can ping anybody in realtime) then anyone can mean a competitor. If you get multiple competitors working off the same backbone, then the carriers become the owners of the NBPS.

That’s why who controls mobile controls the micro-bus, because mobile is where business gets away with micropayments. They’ve got this SIM chip with your name on it, tied to a credit card, and when the credit runs dry, you’re done. If the carriers own the NBPS, then what does Twitter or Facebook or even Google have to offer you if you run out of credit?

Working backwards, unless you’re independently wealthy, your network access is controlled by whatever entity pays you enough to cover rent, food, medicine, and phone. What starts to become clear is that a microcommunity has a small but real chance of looking like a small business, which in turn can establish its own currency and create enough wealth to put up its own ping server/identity hub. That would be great if every single social media play wasn’t trying to get in the middle between you and access to your phone.

So far, Facebook has done the best job of being our Ping Server buddy, but they’ve made a huge mess of understanding how to unbundle the news stream. Every time I talk to a Facebook executive, they say how difficult it is to do this without threatening the privacy of their existing customers. So what you’re saying is that it makes no sense to offer a service just like Twitter except that we’ll guarantee no rate limiting or metadata quarantining or list camp.

That would be because why? There’s no money in the stream? [Wrong buzzer sound] People don’t want to aggregate power by banding together in microcommunities where the group owns the keys to the ping server? [Buzzer] Grandma will get confused. We’re doing fine thanks. We’re working hard to get there, here’s a lollipop in the meantime.

Bottom line: we’re never going to get there, just like Twitter will never restore Track. We’ll go through wave after wave of third party hacks on these fundamentals, with each vendor performing the ball and three cups trick explaining how it provides what we want without giving us control over how we configure the engine. Again, these companies do not want to give us access to our own social identity for fear that we are in fact competitors looking to own the ping server.

This is an immutable fact: we absolutely do want to own the ping server and enable Track. We just don’t want to front the money for the server. Interestingly, cloud computing means we don’t have to. The app we’re building runs on several clouds, in fact — App Engine, Amazon, and Facebook/FriendFeed — with Rackspace and Azure coming online as well. Each of these clouds has its own terms and conditions, but in aggregate they provide an unencumbered simulcrum of Track and gestural integrity.

But why believe me or Dave Winer or anybody who promises an open federated architecture where identity will flow freely across silos? You shouldn’t. Like everybody in realtime, we’re hoping you just make a choice because things are moving so quickly you want to cut your losses on what you’re missing. The truth is that realtime is no different than any other preceding era. No matter how deliberative you might be, following time-tested rules of journalistic process and tenured argument, you still have to make that last leap into the unknown where you put your chips on red or black. Those who suggest this is not a job for amateurs are just trying to own the ping server.

Finally, much is made of the devious lock-in of the url shortener, that it inserts another layer of possible breakage, that it’s a stealth ping server. All that is true, especially that it is another ping server. But in a realtime world, if one piece breaks, you route around it. If Twitter goes down, we move to FriendBook. What begins to scale is not the individual service but the fail-over system of competitive nodes. Blaming the url shortener is like realtime for all the bullshit written every second on the net, professional or otherwise.

Url shorteners exist because the url is the real payload of the micro-bus. The rest of the 140 characters are metadata about the link; if there’s no link then the url defaults to the page you’re currently on. Urls are a trigger for the next realtime fork, the moment when we deliver our most valuable contribution. Of all the gin joints in the world, you had to walk into this one.

Url shorteners go by many different names: Bit.ly, FriendFeed, Silverlight, Java, Wave, Techmeme, YouTube, Url shorteners are ping servers. We want to run Track on them to alert us in realtime. Behind every link is a url shortener. What’s valuable is recording the moment when they expand into something. Those who honor that implicit contract with users will prosper.