Three weeks ago, we had announced that CrunchBase has partnered with Mashery to move the CrunchBase API to a managed solution. At the time, we simply cutover the API calls to their proxy servers. Over the past three weeks, we have been monitoring the API usage and were quite frankly blown away by the volume of calls to the API. The API numbers average nearly 100K calls per hour! Today, we are ready to announce the CrunchBase developer portal and a transition plan to help migrate third party developers over to registering their applications.
We have now opened up a dedicated portal at http://developer.crunchbase.com. The portal offers improved documentation including IO Docs, a forum for developers to ask questions, identify bugs, and make API requests. In addition to this, we’d like developers to start registering their applications and use an API key. As with any API, we’d like to prevent abuse, allocate and scale resources more efficiently, and potentially offer custom endpoints of the CrunchBase API in the future. Our target date to have all CrunchBase applications use access keys is December 1st, 2012 — approximately two months from now. To get there in the least intrusive (to your application) and effective manner, here is our migration plan:
Access Keys Required From December 1, 2012
Unregistered (using no access key) Applications Throttling
Starting today, all unregistered calls will fall under the ‘Unregistered’ bucket. A rate limit of aggregate 100K calls per hour will be instituted on this bucket. This means the sum total of all calls in this bucket will be capped, not per application. This rate limit will decrease by 10K calls/hour for every week thereafter.
Essentially on 9/24 (i.e., today), the aggregate call volume for all unregistered apps is capped at 100K/hour. On 10/1, this will be capped at 90K/hour, on 10/8 to 80K/hour, and so forth until December 1st, when this limit will effectively drop to 0.
What happens if my app hits the rate ceiling? If a call from your unregistered app is made once the aggregate cap (across all apps) is reached for the hour, your app will receive a 403 response as shown below.
This application has exceeded the hourly rate limit. API calls are being made without an access key. Your application must start using access keys by December 1, 2012 in order to continue using the API. Please register for free at http://developer.crunchbase.com/member/register and start using access keys to avoid being rate limited. More details at http://developer.crunchbase.com/.
If you start seeing failures in your app, please check your server logs for this message. To protect against this, simply register for a key (see the section below) and use it in every call.
If you register your existing (or new) application and start appending the access key in every call to the API, you will _not_ be throttled/rate limited. We therefore highly encourage you to register as soon as possible to create your developer account and start using keys. Registration is free.
The API calls will remain the same except that you will need to append the access key in every call. Example below:
Summary Of Changes
To summarize, these are the changes coming to the CrunchBase API
- on December 1st, 2012, required use of access key for every endpoint and application
- unregistered applications to be throttled at aggregate call volume of 100K calls/hour, progressively degraded by 10K calls/week, and effectively dropping to 0 on 12/1/2012
- no rate limiting for registered applications
Quick Start Guide
Here is a quick start guide to the portal and to help you migrate your application to using an API key:
Register — http://developer.crunchbase.com/member/register
Documentation — http://developer.crunchbase.com/docs
Interactive IO DOcs — http://developer.crunchbase.com/io-docs
Developer Forum — http://developer.crunchbase.com/forum
We value our developer community and have thought about this transition carefully. If you have any feedback, please leave it in the comments below.