Scaling CockroachDB in the red ocean of relational databases

Most database startups avoid building relational databases, since that market is dominated by a few goliaths. Oracle, MySQL and Microsoft SQL Server have embedded themselves into the technical fabric of large- and medium-size companies going back decades. These established companies have a lot of market share and a lot of money to quash the competition.

So rather than trying to compete in the relational database market, over the past decade, many database startups focused on alternative architectures such as document-centric databases (like MongoDB), key-value stores (like Redis) and graph databases (like Neo4J). But Cockroach Labs went against conventional wisdom with CockroachDB: It intentionally competed in the relational database market with its relational database product.

While it did face an uphill battle to penetrate the market, Cockroach Labs saw a surprising benefit: It didn’t have to invent a market. All it needed to do was grab a share of a market that also happened to be growing rapidly.

In previous parts of this EC-1, I looked at the origins of CockroachDB, presented an in-depth technical description of its product as well as an analysis of the company’s developer relations and cloud service, CockroachCloud. In this final installment, we’ll look at the future of the company, the competitive landscape within the relational database market, its ability to retain talent as it looks toward a potential IPO or acquisition, and the risks it faces.

CockroachDB’s success is not guaranteed. It has to overcome significant hurdles to secure a profitable place for itself among a set of well-established database technologies that are owned by companies with very deep pockets.

It’s not impossible, though. We’ll first look at MongoDB as an example of how a company can break through the barriers for database startups competing with incumbents.

When life gives you Mongos, make MongoDB

MongoDB is a good example of the risks that come with trying to invent a new database market. The company started out as a purely document-centric database at a time when that approach was the exception rather than the rule.

Web developers like document-centric databases because they address a number of common use cases in their work. For example, a document-centric database works well for storing comments to a blog post or a customer’s entire order history and profile.

Relational database developers shun the document-centric approach because of its tendency to create redundant data that can introduce errors and complexity into application engineering. In a document-centric database, the same data is often stored in multiple locations, so when it needs to be updated, each document in which the data appears needs to be synced.

When you have millions of documents in play, it’s expensive and computationally intensive to make updates compared to traditional relational databases.

MongoDB’s plan for success was to popularize and grow the document-centric database market. So instead of focusing on banks, insurance companies, and large manufacturers and retailers — the typical users of relational databases — the company focused on startups.

Startups had no legacy storage infrastructure to maintain, and a document-centric database such as MongoDB would address the needs of these startups. As the startups grew, so did MongoDB.

Eventually, the big legacy companies took notice as well. They started using MongoDB in low-risk, skunkworks projects where a document-centric database was appropriate. This created demand for MongoDB within the mature Fortune 1000 companies, but it also created a problem.

The skunkworks developers had little problem understanding the true nature of document-centric architectures, but the relational database developers were slower to catch on and demanded MongoDB fit their way of thinking. So the big companies said something along the lines of, “If you really want us to adopt MongoDB for production, you need to give us some of the features of a relational database, such as transactions.” (We talked more about transactions back in part 1).

So MongoDB added transactions to the product. Reactions were mixed. Banks and insurance companies were pleased, but the document-centric purists were incensed. For many of them, transactions are antithetical to the true nature of a document-centric database.

For better or worse, the move away from being a pure document-centric database changed the company and its technology. Some say the change diluted MongoDB. Satisfying both the document-centric and relational database worlds incurred additional overhead for the company — conceptually, operationally and culturally. MongoDB started out as one type of company and turned into another.

Nonetheless, MongoDB’s hybrid approach was a success, and it went on to become a publicly traded company with a market capitalization of nearly $23 billion. Today, it represents one of the clearest examples of success in this generation of database startups, even though it tread a convoluted path to get there.

Leveraging the novelty of CockroachDB

Cockroach Labs doesn’t want to straddle multiple markets or paradigms. Instead, the company is just as focused on relational database technology today as it was at its inception seven years ago. That’s always been the plan.

Cockroach Labs co-founder and CEO Spencer Kimball says, “Relational databases remain a large part of the market in software, so any successful effort to redefine them is bound to be an attractive business.”

But to see true success, Cockroach Labs has to hit two targets. First, it needs to keep attracting and growing the number of greenfield startups that use its technology since they have no legacy tech stack. Second, the company has to garner a big enough market share of the Fortune 1000 segment to prosper. The question is, how?

CockroachDB was designed from inception to be a truly globally distributed, relational database. While other relational databases can be deployed worldwide, CockroachDB is designed so that data access and data resiliency are implemented globally. It’s a fine but important distinction.

While other databases’ strategy for globalizing data around the world is to deploy redundant database servers to particular global regions, CockroachDB has the mechanism to ensure that data is distributed to regions so that users in a particular region are getting the data that is as close as possible to them. CockroachDB’s fine-grain approach makes it faster for users to access data.

Any distributed database can achieve this result, but older products require a good deal of expertise and programming to make it happen. What makes CockroachDB special is that it does this type of globalization automatically. This is a big deal, since CockroachDB can fill a void for companies that need access to global data faster than their current relational databases can provide.

So it’s not very surprising that the company is growing at more than 100% a year, according to Chief Revenue Officer Jeff Miller.

It’s telling that CockroachDB advertises Chinese search and internet company, Baidu, as one of its success stories. Baidu moved over to CockroachDB from MySQL to support its 1 billion users and 40 TB of data.

The Baidu case isn’t a matter of a startup just going with CockroachDB because there was nothing to stop it from doing so. Rather, an established company made a change from a well-known relational database to CockroachDB. Baidu is exactly the type of company Cockroach Labs wants as a customer.

CockroachDB is designed to go up against the likes of Oracle, Postgres, MySQL, Microsoft SQL Server, AWS Aurora and Google Cloud Spanner. And Cockroach Labs has both the financial and technical resources required to make the product competitive in the relational database marketplace in terms of feature set, price point and support.

These are significant achievements, no doubt, but they create another set of challenges for the company: finding and retaining the talent required to sustain itself.

A database of talent in New York City and globally

Like all companies competing in the technology space in general — and databases in particular — Cockroach Labs wants to attract the most capable talent around and is willing to pay for it. According to Glassdoor analysis, Cockroach Labs pays its technical staff an average of ~$169,000 a year, which places it at the higher end of the salary spectrum excluding equity compensation.

To get a headstart on the competition over talent, Cockroach Labs has a clever way of retaining employees even before they’re hired: It extends stock options to its college interns, and when these interns return to Cockroach Labs as employees after graduation, their vesting period will include their time working at the company as an intern.

In terms of the factors usual to HR analysis, Cockroach Labs is doing well. Chief People Officer Linsday Grenawalt reports a 10% annual attrition rate among the workforce, which is below the reported industry average of 13.2%. It’s one of .Inc Magazine’s Best Places to work in 2021, and it continued to attract enough employees to increase its head count to 285 in 2021, an increase of 182%, according to LinkedIn.

Co-founder Spencer Kimball, who has a Glassdoor CEO rating of 4.8 out of a possible 5, seems to be the secret sauce. As per private interviews with a number of senior executives, he is a reason important senior staff, all of whom have a significant history in the software industry at an enterprise level, decided to come to Cockroach Labs.

Even more telling is the fact that the company’s other two co-founders are still active seven years later, despite the prominence of Kimball in the Cockroach Labs milieu. Creating an environment that has enough room to meet everybody’s need for recognition and gratification, particularly among C-level executives, is difficult. The startup seems to be successful at creating an attractive, growing workplace while also accommodating the necessary egos of its higher-ups.

In short, what makes Cockroach Labs interesting is that it’s an engineering company run by engineers and has a mature set of senior executives with deep experience in the world of distributed data management solutions.

Its founders come from Google, where they had the benefit of learning what does and what does not work in terms of motivating and managing technical talent. As Grenawalt, also a former Google employee, says, “Google is the gold standard for running modern engineering companies. We learned from our experiences at Google and built something new, influenced by Google’s best practices in order to make working at Cockroach Labs a world-class employment experience.”

Bold exits and risks

Cockroach Labs has a bright future, compelling technology, a lot of money in the bank and has an experienced, technically astute executive team. The company presents itself as an engineering firm that continues to have an engineer’s bent even within its sales and marketing teams.

Many companies that start off with a strong technical focus get diluted when the need to keep the lights on overcomes the desire to build technology that matters. To date, Cockroach Labs has avoided this problem.

The company is doing many, if not all, of the right things that go with growing an outstanding startup. However, the road ahead will not be easy. The company will face challenges, particularly as it grows to become a real threat to some gigantic competitors with the resources to combat CockroachDB’s invasion. Companies do not act politely when their market share is compromised.

There are risks at hand, in this writer’s opinion. The most obvious one is the risk of a premature acquisition, which might be instigated by a goliath company that needs a relational database on the order of CockroachDB to fill out its product line.

Another type of suitor might be a company that simply wants to take CockroachDB out of the market to eliminate the threat. It’s a long shot, but it could happen.

While a few competitors have globally distributed database technologies that are equal to CockroachDB in general and CockroachCloud in particular, there are some major multibillion cloud providers out there that do not. This makes CockroachDB an attractive acquisition target.

A cloud giant in need of what Cockroach Labs has might make it an offer it can’t refuse. Once acquired, CockroachDB could continue to prosper or it could languish. More than one company has been absorbed by a Big Fish in the throes of an acquisition frenzy only to see its technology hidden away, never to see the light of day again.

An additional risk is Cockroach Labs failing to realize the full potential of CockroachCloud in cloud computing. Right now, CockroachCloud is positioned as a way to make CockroachDB available to companies that don’t want to incur the costs of standing up and supporting their own on-premise clusters at massive scale.

Taking this approach is very much inline with current market trends. But as the companies that have matured on the cloud have revealed, providing a single service is but an entry point to a wider presence.

Today, we’re witnessing the evolution of intelligent cloud platforms. For example, AWS and Google Cloud started out as a way to provide computing and storage on demand as a service. But they’ve since evolved to offer a large variety of cloud services in a number of areas.

While CockroachCloud implements the essential intelligence required to spin up, monitor and control CockroachDB clusters, these clusters are running on AWS or Google Cloud. Having the wherewithal to implement CockroachDB as a cloud service is impressive, but it’s not unique. Google and AWS have similar products.

Evolving CockroachCloud into an intelligent cloud service that goes beyond providing just massive database capabilities on a global scale will make Cockroach Labs more than just a database company. Implementing a wide array of intelligent services would give the company the chance to define the data landscape for the entire planet.

It’s a grand ambition, no doubt. Nonetheless it’s the type of ambition that’s well within the imagination of big thinkers, many of whom are hard at work at Cockroach Labs’s competitors. Time will tell if Cockroach Labs will be a player in a game of this scale.

Ultimately, it seems inevitable that Cockroach Labs is looking forward to an IPO. Going public has always been part of the plan — one could construe this as the reasoning behind the enormous amount of venture capital fueling the company. As for when the IPO will actually happen, that subject is still private to Cockroach Labs. But rest assured, it’s coming unless the company is acquired before then.

CockroachDB has come a long way since version 1.0 appeared as an open-source project on GitHub in May 2017. How the company fares moving forward will depend on how it evolves its technology and how it reacts to the transformations and pressures that follow in the wake of its eventual IPO.

---

CockroachDB EC-1 Table of Contents

• Introduction
• Part 1: Origin story
• Part 2: Technical design
• Part 3: Developer relations and business
• Part 4: Competitive landscape and future

Also check out other EC-1s on Extra Crunch.

---