Gnip is a new web services platform being launched today by the former founders of MyBlogLog (see previous Techcrunch announcement of Gnip here). The goal of Gnip is to serve as a web services proxy to enable consuming services to easily access user data from a variety of sources. Currently an application consuming user data must write-in direct support for each API for every service it requires, which usually requires a lot of development effort in terms of protocol and format support, maintenance of each API integration and other issues such as uptime and availability. With Gnip, developers can write once for the Gnip API and gain access to user data for a number of supported publishing services.
Starting this morning, Gnip has support for publishers such as Digg, Flickr, del.icio.us, MyBlogLog, Six Apart and many more. Gnip have established a formal relationship with some of these providers, such as Digg, so that they are able to retrieve all data and make it available to consuming applications using Gnip. In the case of Digg, this involved allowing Gnip unlimited access to their data API. Current consumers of the service include Plaxo, Lijit, Spokeo and MyBlogLog. Gnip is launching with either publisher or consumer partnerships or support for 15 different products.
A publisher can either push data to Gnip using their API’s, or Gnip can poll the latest user data. For consumers, Gnip offers a standards-based API to access all the data across the different publishers. A key advantage of Gnip is that new events are pushed to the consumer, rather than relying on the consuming application to poll the publishers multiple times as a way of finding new events. For example, instead of polling Digg every few seconds for a new event for a particular user, Gnip can ping the consuming service – saving multiple round-trip API requests and resolving a large-scale problem that exists with current web services infrastructure. With a ping-based notification mechanism for new events via Gnip the publisher can be spared the load of multiple polling requests from multiple consuming applications.
On the publishing side Gnip supports formats such as standard RSS, Atom or XMPP. A consuming application can pick up a number of the Gnip-supplied client library toolkits (written for Php, RubyOnRails, Java etc.) and hook into the Gnip API and receive notifications of new events or request data. With Gnip as a proxy, data formats can be normalized and standardized, with load being taken off of the publishing source and instead placed on Gnip and its event notification service.
What Gnip are doing for web services is very similar to ping services for weblogs, except that multiple formats and types of services are supported. A service such as Twitter can easily offload almost all of their API traffic to Gnip and consuming applications can make use of a notification-based web service rather than a polling-based service. With a notification based service, clients can be informed of user updates within seconds, rather than minutes or hours – meaning more up-to-date information and all via a standards-based interface.
For output, Gnip will support Atom initially, with plans to extend support to XMPP and other formats in the near future. Also in the pipeline is support for private data feeds using oAuth as the authentication mechanism – a problem that is hard to solve across multiple data streams but one that once solved will be implemented within Gnip. Future services via Gnip include transforming data types and providing an identity services, both features which are due within the next few months.
This off course depends on Gnip being able to handle the load and traffic for all such services, and because it is notification based rather than polling-based, they already cut a large number of the API calls being used today. Gnip aims to act as a proxy for web services traffic, allowing the publisher to push data once to Gnip, from where it is either pushed or polled by the various consumers. While today Gnip only supports public data streams, the service already has a good level of support both amongst publishers as well as consumers and offers a compelling value proposition for both (eg. imagine Gnip taking care of API traffic for twitter). The initial version of Gnip was developed in co-operation with Pivotal Labs, who are well known for their app development experise (and who have recently been bought in at Twitter)
In an ideal world, all services would support standard formats and protocols such as XMPP, and there would be no requirement for proxies. Gnip not only offers the standard interface, but also provides other services to publishers and consumers which make it an almost invaluable part of the new web services infrastructure.