The open source Kafka real-time stream data service was incubated at LinkedIn over five years ago. Since then, Kafka has become an Apache open-source project and the original developers left LinkedIn last year to launch Confluent, an enterprise startup that — unsurprisingly — focuses on Kafka.
The Confluent team today announced that it has closed a $24 million Series B round led by Index Ventures with participation from prior investor Benchmark Ventures, which led the company’s $7 million Series A round.
The company plans to use this new capital to build out its platform. This includes improving security features and management tools, as well as the stream processing capabilities of Kafka. In addition, the company also wants to expand its lineup of plugins for existing databases.
At its core, Apache Kafka is a real-time messaging service. At LinkedIn, for example, it handled over 800 billion messages every day (that’s a combination of backend tools and user-facing notifications). Current users of the technology include the likes of LinkedIn, Netflix, Uber, Cisco and Goldman Sachs. Its real-time capabilities make it an ideal solution for emerging Internet of Things services that have to ingest large amounts of data or even large online gaming services, but the current trend toward microservices means that Kafka’s can become the central hub and buffer for ingesting and moving data between users and databases. It could be used for ingesting data that’s destined for a Hadoop install, for example, or for feeding stream processing services based on Apache Samza (which also came out of LinkedIn). Using Kafka’s plugins, enterprises can also connect their existing database systems with these tools, which allows them to leverage these new technologies without having to switch to a new database platform.
Confluent CEO Jay Kreps and head of engineering Neha Narkhede (who both co-founded the company together with told me that the Jun Rao) told me that the team decided to go with Index as the lead investor because of the firm’s experience in working with similar open source companies, including Hortonworks and Elastic (previously elasticsearch), for example. Index partner Mike Volpi will join Confluent’s board.
“Confluent’s platform can also support hundreds of applications built by disparate teams and is scalable, reliable enough to handle critical updates, and features a stream-processing framework that integrates easily and makes data available for real-time processing,” Volpi said in today’s announcement. “Jay, Neha and Jun have created a highly intelligent product and we are excited to support their continued growth.”
Confluent will offer Kafka as a supported commercial product, but the company also offers services around the platform, including technical consulting and training.