Hey You Get Off of My Cloud

Next Story

Sony prepping touchscreen Vaios for Windows 7

cldhubSandwiched neatly between the RealTime is God and RealTime Who Needs It crowds is a new group that embraces both positions while moving forward rapidly. These folks include Brett Slatkin of the PubSubHubub effort and Dave Winer of rssCloud.org. Slatkin and fellow conspirator Brad Fitzpatrick demoed the PSHBB architecture at the RealTime Stream Crunchup, and Winer quickly jumped in with his own implementation.

While Winer attacked PSHBB as a Google (and TechCrunch) conspiracy, the rhetoric seemed mostly in service of his own plans for a decentralized open platform not controlled by bigcos. The charges don’t make sense — PSHBB is not a Google effort but rather one started by Fitzpatrick and Slatkin on their own — and like Winer’s, proposed and delivered as an open architecture that can be spread across multiple players. In other words, they’re saying the same thing.

Digging deeper, it could be that Winer’s concern is that adoption by Google or other platform players will create a bandwagon effect that could lock out smaller independent developers. There’s certainly some truth to that dynamic, as PSHBB is already running in a number of cooperating platform implementations including FeedBurner, WordPress, and even FriendFeed, where Slatkin is active in tracking user issues. But ironically, by offering a competing framework, Winer takes away the argument of unfairness. If developers and publishers want to adopt one platform over another because it’s got more mainstream adoption, they’re free to do so. In the past, Winer has been the beneficiary of just such choices, with RSS 2.0 proliferating in no small part due to support from Microsoft on the platform side and the New York Times on the publishing side.

Both PSHBB and rssCloud sidestep the issue of whether RSS is on the decline, preferring to co-opt whatever momentum the technology has into the swarming viral nature of the realtime micromessaging disruption. Whether there’s consensus about the underlying value of the new platform, there’s no denying the uptake and attendant opportunities for gaining mind and perhaps market share of the new wave of approaches to managing the stream. If that were not the case, why would PSHBB be gaining adoption and Winer be spending time on his own cloud? Our own experience in managing the CrunchUp suggests an explosive market for startups; we spent most of the last few weeks trying to fit people into panels and demos that had already expanded to the breaking point.

If realtime is simply the latest buzzword, a placeholder for getting attention and perhaps venture money until the next buzzword comes along, then we’d expect to see some slowdown in those realtime projects that are out at the bleeding edge. Facebook certainly seems to have stopped short of a realtime updating stream, though I’m seeing more Facebook messages appearing in aggregate tools such as Seesmic and FriendFeed. But mostly I’ve seen complaints about intermittent slow updating from Twitter and an apparent move to a one-hour pinging of new WordPress items in the latest update. In other words, speed wins even if no one has yet figured out how to manage that speed efficiently.

Then there’s WiFi on planes, where I bet we’ll soon see numbers reflecting the impact on what airlines we choose. If realtime is now the new normal in publishing, then losing a working day to a cross-country flight is no longer viable. With price discounts for mobile access, the *Phone becomes the easy way to keep in touch with short, bursty messages the simplest way of managing the infostream. And the fundamental result: the URL payload becomes the lingua franca of the message bus.

It’s here where the damage is being done to RSS as a transport. As micromessage citation grows in importance, the sheer number of Web pages cited begins to overwhelm the RSS payloads. In recent days, Google Reader Likes have been added in an attempt to map some sort of social graph to the consumption of RSS items, but it’s still unreachable except from within the Reader UI. The trend moves away from RSS items and toward the original pages, which for publishers means a much greater opportunity for cross-linking, advertising, engagement, and analytics.

Podcasting is similarly disemboweled, given the ease with which YouTube videos are streamed to all always-on devices. It’s easy to decry Google market force here, but it’s no accident that Microsoft’s IIS Media Server now supports H.264 as does Silverlight cross-platform. I believe Silverlight becomes the default NetBook media server, based not on enclosures but rather streaming. We’re already seeing popular podcasts such as Leo Laporte’s TWiT gaining large realtime audiences as the number of streaming services proliferates. It’s only partially random that Twitter emerged out of the ashes of a podcasting service.

One argument against this future is that it tends to rely on centralized storage and a gated cloud of services. Winer personalizes this to the notion of TwitterCorp, that no single company should control this uber stream, that they won’t be able to, that if they could it would be a bad thing. He has a point, but whether rssCloud will solve this problem is not clear. He’s using Amazon EC2 and the open source Identica project for parts of his platform, both services provided by commercial companies.

In his rssCloud Roadmap, Winer asks us to think of rssCloud as Loosely coupled 140-character networks, and his goal “to instantaneously flow 140-character messages around, with metadata, including links, tags, enclosures, and whatever else (it’s as open architecture as RSS 2.0 is).” I’ll quote with interleaved commentary:

A network that works alongside Twitter, but outside the control of a company. You can be confident that no company will control it because it is being started by a person, not a company. Even if I wanted to crush you, I couldn’t. Even if I wanted to control all the users’ data, I can’t — since most of it won’t flow through my servers.

I’m not convinced there is much of a distinction between services offered by an individual or a company; they both can be “controlled” to the extent that users offer their data up. Perhaps the caveat of “most” lessens that control but to that extent the remainder of the data flowing through Winer’s servers would be the sum of his control.

Re Google’s PubSubHubBub, their goal appears to flow updates of blog posts in association with Feedburner. This is a good goal. I hope to help them get RSS compatibility into it. If your goal is to optimize the polling of existing RSS or Atom feeds, you can use the rssCloud network, but that’s not something I want to get involved in personally. My interested is in the micro-blogging application, at least at the outset.

Again, characterizing PubSubHubBub as “Google’s” is inaccurate. He offers to help provide RSS compatibility, and suggests that while rssCLoud can be used to optimize polling of RSS feeds, he’s personally only interested in the micro-blogging application for now. He’s surely threading the needle here, but how that will help spread adoption of his solution depends on the degree to which his solution appears more open and counter to the TwitterCorp threat he sees.

As with RSS, it is always a mistake to underestimate the power Winer’s bearhugging of technology can muster. But the context in which RSS emerged has as many differences as similarities with today’s circumstances. In particular, RSS was supported by two kinds of media — the emerging blogosphere and, eventually, the mainstream media. Today, these two medias have merged, primarily as a result of RSS and its democratic impact. It’s as though the New York State Thruway suddenly smashed all of its toll booths and allowed free travel. The impact was seismic, like the Guttenberg press. Shelf space became infinite.

That revolution took time and luck and plenty of political pressure to gain enough momentum. By contrast, Twitter bootstrapped RSS, broadband, social dynamics, and the combined medias. What took several years with RSS has taken a fraction of that time precisely because RSS proved you could tip markets without the dominance of bigcos. In effect, RSS proved we don’t need to fear the locked trunk, at least not in the classical way it’s been defined. How do we defend ourselves against bigcos like Google and Microsoft from using RSS tactics themselves?

At the Fortune conference in Pasadena today, Michael Arrington reported Twitter’s Biz Stone responding to a question about Discovery as saying “they need to assign weight to users or specific Tweets to do a better job of surfacing Tweets.” Inherent in the comment are the tools of control: “assign weight” and “surfacing Tweets.” These words imply ownership of the stream, just as Page Rank became the control point of authority in the search era.

Here’s Ray Ozzie on the same point:

I think it’s going to impact search, because the nature of search ranking is based on — at least classically is based on looking at what’s relevant by what it’s linked to, and if a lot of the links are moving from more static media into more dynamic media —

Me: Oh, so you agree with my thesis.

Ozzie: Yeah.

Me: Excellent.

Ozzie: Yeah. I don’t know to what degree the ranking will be impacted in that realm, but I think that’s a fairly important thing. And the Tweet itself is more or less the anchor text of the link and so on.

But wait, here’s Microsoft talking about how disruptive the stream will be, and it’s a littleco talking about controlling the high ground. Can Twitter establish control of the message stream to the point where no other stream can produce efficient filtered results to users? Again, Ozzie:

[W]hat I learned in the Groove experience is that if you have lots of things that notify things, the user wants to suddenly aggregate them and tune which ones they want high bandwidth instant notification and which ones they only want to occur every once in a while. So, having messages go through something where they can say, okay, I’m going to have all of these different classes of messages, move these classes into this bucket, these into this bucket, notify these on my phone, these somewhere else, it’s useful for the user….

[A]s humans we have these issues. And certain of the events, certain classes of the events we want to treat, as Dave [Winer] says, like a river where you don’t really care if you miss something. It’s where you’re not trying to keep up every little thing. Maybe it’s an amusement, maybe it’s just a background activity.

Some types of events you just want to see them. You just don’t want to miss even a single one in this big flood of notifications. And so we just need better tools.

It stands to reason that the more endpoints, the better the chance of no single company controlling the stream. Today Twitter talks like a bigco, but in a world of streams, they must compete on the quality of their discovery model, compete on how they “assign weight,” compete on how they “surface Tweets” to their cloud of third party clients and analytic engines.

Viewed in this context, PSHBB promises to bring Feedburner feeds, Google Reader shares and Likes, and various smaller RSS clouds such as WordPress up to speed and into the weighting and surfacing game. The coalition also benefits from bridges to services such as FriendFeed that Winer has eschewed because they have offered solutions such as SUP. rssCloud certainly serves as a reminder of the underlying values of simplicity, transparency, developer independence, and user control, but then again that’s what PubSubHubBub is selling too.

  • http://superfeedr.com Julien GENESTOUX

    Great write-up. In the end, I think what matters most is not what “technology” is being used by who, but really that everybody sorts of agree that RSS/Atom are too “static” and they both need velocity.

    • http://wyman.us/ Bob Wyman

      There is nothing inherently “too static” about Atom. In fact, the issues of PubSub distribution (like what is done via PSHB and XMPP PubSub) were explicitly considered at great length during the definition of the Atom format. For instance, Atom includes a number of features that make this sort of application much easier:
      * Atom defines both “feed” and “entry” as top-level objects. This was done to permit the distribution of single entries and to permit entry aggregation as well as the construction of streams entries from multiple sources without loss of information.
      * Atom provides a robust “source” element to permit entries to explicitly describe their source feeds. Thus, an entry on its own is semantically equivalent to an entry in a feed.
      * Atom explicitly prohibits considering the order of entries in a feed to have semantic meaning. Thus, an entry with a source element can be properly evaluated, in isolation, outside a feed.
      * Atom provides mechanisms to digitally “sign” both entries and feeds. This allows you to state and determine the author of a feed or entry without polling the source site.
      * etc.. (many other minor details are included)

      RSS, on the other hand, is much more problematic when used in real-time applications largely because it became “mature” in an environment where only polling was considered useful. When we were developing Atom, those of us at PubSub.com and others were working hard to make the real-time web a reality and thus could see the things that Atom needed but that RSS had failed to anticipate.

      bob wyman

      • Matthew Terenzio

        Those are all great things, most if not all of which can be accomplished by a namespaced extension.

        I do appreciate all the effort and genius that went to baking them in to the original spec though. Don’t take me wrong here.

  • http://newgadgets.dailytidbit.com/new-gadgets/hey-you-get-off-of-my-cloud/ New Gadgets | Hey You Get Off of My Cloud

    […] Original post by Hostpundit – Hosting and Gadgets […]

  • http://siliconangle.com/ver2/sabackchan/2009/07/24/cloudiness-for-the-day-so-far-can-the-u/ Cloudiness for the day so far: Can the U… « /SAbackchan
  • Matthew Terenzio

    There are a few subtle differences. Pubsubbhubbub actually delivers the feed payload, while rssCloud just notfies the subscriber that a change has occured. Actually, maybe that’s not so subtle.

    • http://wyman.us/ Bob Wyman

      It’s not a “subtle” difference. Pushing payload, rather than just update notifications, drastically improves the scalability of these systems.

      A system that only delivers notification is little more than a “thin” ping generator and almost inevitably suffers from the “thundering herd” problem. (i.e. As soon as notifications are distributed, the publishing site gets crushed by polling requests for the updated feeds. This is basically the same as the “Slashdot effect” and makes service management very difficult.) Note: Because the feeds contain more than one entry, serving feed requests usually consumes vastly more bandwidth than simply distributing updated entries.

      A system that pushes the updated content as messages payload is able to drastically reduce the burstiness of its polling traffic and can much better manage its request load. This is true since most systems will trust that the payload is correct and will not poll-back to verify the contents. (This was part of the premise behind the old “fat-ping” proposals like FeedMesh that we pursued years ago.) Note: If “fake” updates become a problem, the Atom format provides for digital signatures that will allow the receivers of messages to validate received content without needing to poll the source feed.

      bob wyman

      • Matthew Terenzio

        Yup, it’s more complex and more efficient.

        I was being a little too subtle in my humor in the original comment. It’s a huge difference.

        And in the pubsubhubbub method, the hub is definitely right in between the publisher and the subscriber.

        Good place to collect data and place semantic and behavioral based ads. ; )

  • http://james.wheare.org James Wheare

    Why must Twitter learn to surface tweets more effectively. They seem to have done a good job satisfying massive user demand by just focusing on making the interpersonal thing happen. I’d hate to see that side take a back seat in favour of assigning a global weighting.

    Put control into the hands of users, let me decide who I care about the most, not your algorithms.

  • http://blog.broadbandmechanics.com/2009/07/26/blogging-from-boca-raton-florida/ Marc's Voice » Blogging from Boca Raton, Florida

    […] And thanks to Steve Gillmor for sorting out the so-called conspiracies and BigCo versus small guy is… […]

  • http://garyburge.com/ Gary Burge

    Thanks, Steve. I like this division of labor: You handle the personalities, spit ball fights and politics. I’ll just implement the products.

  • http://www.brianalvey.com/ Brian Alvey

    “It’s only partially random that Twitter emerged out of the ashes of a podcasting service.”

    Poetic insight.

  • http://www.geo-sms.blogspot.com laalay ki jaan

    There are a few subtle differences.

  • http://blog.feedly.com Edwin Khodabakchian

    This is a good right up. I am wondering why there is only 13 comments. You might be a little ahead of your time.

  • http://thepaisano.com paisano

    Hmm. Interesting title… I did this for Sarah Lacy a year ago

    http://www.sarahlacy.com/sarahlacy/2008/07/hey-you-get-off.html

    I know… just a coincidence…

  • http://twitter.com/sull sull

    I think a better post about this stuff can be found here:

    http://dashes.com/anil/2009/07/the-pushbutton-web-realtime-becomes-real.html

    It’s about “Push Button” without the author actually pushing buttons ;)

    @sull

  • http://secondthoughts.typepad.com Prokofy Neva

    Good job of reporting these issues.

    I agree with James that the beauty of Twitter is that it is not yet spoiled by weight and Google like algorithms, but the user decides whom to follow and makes his own weights.

    If Twitter or anyone else thinks they are doing the user a favour by boosting for the users’ view the twitter accounts that have the most followers or the most RTs or whatever the algorithm is, they are wrong. It serves only the ad agency Google and others that want to profit. I think we’re at the point now at all these free services is that we want to see companies monetarize and sustain them, but not by imposing their ideologies on the content, and that means whatever is first in view.

    Internet search as it is now is dominated by the “bigco” Google, biggest of them all, and its friends, like Wikipedia, a “bigorg” that dominates first place in every search not because it’s necessarily good, or non-biased, or helpful, but because loads of people link to it because…loads of other people link to it because…it is always pushed and shoved up first by search algorithms.

    That there are supposed to be correctives in Google that means that “actually doesn’t happen” doesn’t persuade me, simply because now I have Twitter to see how it works without the Silicon Valley finger on the weight scale. (Of course some services have twiterati to see, and Ev and Biz try to sway it with their favoured accounts but most users ignore it).

    I’m still trying to figure out why I need RSS, given that I have twitter, except, of course, it’s useful for reading accounts like @davewiner that have follow-blocked me. I can search for their name and then RSS feed the Twitter search to my Google reader.

  • http://www.techcrunch.com/2009/09/09/rsscloud-vs-pubsubhubbub-why-the-fat-pings-win/ RSSCloud Vs. PubSubHubbub: Why The Fat Pings Win

    […] note: With all of the debate lately between RSSCloud versus PubSubHubbub, we wanted to hear from a developer who could actually […]

  • http://www.submitteronline.com/blog/2009/09/rsscloud-vs-pubsubhubbub-why-the-fat-pings-win.html RSSCloud Vs. PubSubHubbub: Why The Fat Pings Win | Submitter

    […] note: With all of the debate lately between RSSCloud versus PubSubHubbub, we wanted to hear from a developer who could actually […]

  • http://www.upoff.com/2009/09/10/rsscloud-vs-pubsubhubbub-why-the-fat-pings-win/ RSSCloud Vs. PubSubHubbub: Why The Fat Pings Win | UpOff.com

    […] note: With all of the debate lately between RSSCloud versus PubSubHubbub, we wanted to hear from a developer who could actually […]

  • http://www.emediaone.net/index.php/2009/09/09/rsscloud-vs-pubsubhubbub-why-the-fat-pings-win/ RSSCloud Vs. PubSubHubbub: Why The Fat Pings Win | eMediaOne

    […] note: With all of the debate lately between RSSCloud versus PubSubHubbub, we wanted to hear from a developer who could actually […]

  • http://www.dreamnest.in/technology/rsscloud-vs-pubsubhubbub-why-the-fat-pings-win.html RSSCloud Vs. PubSubHubbub: Why The Fat Pings Win | Technology

    […] note: With all of the debate lately between RSSCloud versus PubSubHubbub, we wanted to hear from a developer who could actually […]

  • http://busybb.us/2009/09/rsscloud-vs-pubsubhubbub-why-the-fat-pings-win-2/ BusyBB.us » RSSCloud Vs. PubSubHubbub: Why The Fat Pings Win

    […] note: With all of the debate lately between RSSCloud versus PubSubHubbub, we wanted to hear from a developer who could actually […]

  • http://jp.techcrunch.com/archives/20090909rsscloud-vs-pubsubhubbub-why-the-fat-pings-win/ RSSCloudとPubSubHubbub, どっちがいい?: fat pingが勝つのはどうして?

    […] 編集者注記: 最近は、RSSCloudかPubSubHubbubかという議論が騒々しくなっているので、どちらにも詳しいデベロッパから話を聞きたくなった。この記事を書いたJosh Fraserは、EventVueの協同ファウンダで、空き時間にはPubSubHubbubに積極的に貢献している。彼はとくに、WordPressのプラグインをはじめ、PubSubHubbubのクライアントライブラリをいくつか作った。彼は、どっちに軍配を上げるだろうか。 […]

  • http://link Ganry35

    Please don’t think we’re all rich? ,

blog comments powered by Disqus