New Relic Makes A Bet On A Real-Time Service That Processes Billions Of Data Points For Ad Hoc Analysis

Every couple of months, New Relic CEO Lew Cirne has coded on a secret project, away from the family and mostly anyone else. He has been a code monk, cloistered in a cabin, developing a real-time analytics engine code-named Rubicon that uses live application data to do ad hoc analysis.

Cirne says the service he developed is essentially a database that processes billions of data points on a daily basis. With the data, Rubicon does analysis for insights into how people are using the app according to any number of criteria.

Rubicon, named for Rubicon Peak where he had the initial idea for the project, will be available next year. A beta version will go live later this quarter. The price for the service has not been announced.

The hope here is that Rubicon will help New Relic be a company that lives beyond the lifetime of its core service: application lifecycle management. By focusing on data analytics, Cirne believes the company can serve a budding ecosystem in entirely new dimensions.

rubiconproject

Rubicon is built from the ground up, based upon the data from its 70,000 customers. It is now running on 100 million phones. On the backend, the service runs on a distributed cluster with Map Reduce running in parallel, storing and aggregating trillions of time-stamped events. It primarily leverages in-memory technology he developed that is tuned to time-stamp events by the billions. It promises sub-second results for the user.

Cine said the company expects the number of customers to double in the next year. With this expanding data foundation, the company will have a service that it can open to third-party application providers who can add additional data and create different types of services.

Every minute, New Relic collects eight to nine million data points. As that number grows, New Relic will have a service that customers can leverage in a way that very few companies have been historically capable of doing.

Rubicon does remind me of data analytics platforms that have emerged in the past year that bypass the need for a data warehouse. Platfora fits into this category as do others that offer real-time analysis. Rubicon’s distinctiveness is in the database Cirne developed. If the company’s ecosystem can be truly leveraged then Rubicon could very well be the foundation for the next phase of the company’s growth and future potentially as a publicly traded company.

Cirne will show Rubicon at FutureStack today the company’s first user conference. The company has other announcements about its mobile service and general availability for Node.js code monitoring. But you can tell that Cirne’s burning interest is in Rubicon. That does pose its own set of risks as a new effort can take resources from the company. And if it is not a success it could leave the company well-exposed and potentially be a detriment to its efforts to become a publicly traded company.