How Kubernetes came to rule the world

Open source has become the de facto standard for building the software that underpins the complex infrastructure that runs everything from your favorite mobile apps to your company’s barely usable expense tool. Over the course of the last few years, a lot of new software is being deployed on top of Kubernetes, the tool for managing large server clusters running containers that Google open-sourced five years ago.

Today, Kubernetes is the fastest growing open-source project, and earlier this month, the bi-annual KubeCon+CloudNativeCon conference attracted almost 8,000 developers to sunny Barcelona, Spain, making the event the largest open-source conference in Europe yet.

To talk about how Kubernetes came to be, I sat down with Craig McLuckie, one of the co-founders of Kubernetes at Google (who then went on to his own startup, Heptio, which he sold to VMware); Tim Hockin, another Googler who was an early member on the project and was also on Google’s Borg team; and Gabe Monroy, who co-founded Deis, one of the first successful Kubernetes startups, and then sold it to Microsoft, where he is now the lead PM for Azure Container Compute (and often the public face of Microsoft’s efforts in this area).

Google’s cloud and the rise of containers

To set the stage a bit, it’s worth remembering where Google Cloud and container management were five years ago.

It’s no secret that Google Cloud always had good technology, which could hold its own against any competitor, but the company was slow in expanding its product lineup and data center availability, all while Amazon’s AWS had already become the go-to platform for all things cloud.

At the same time, Docker, which launched in 2013, had brought containers from a niche technology into the spotlight by making them extremely easy to use. That success spawned competing container management solutions from the likes of CoreOS, Mesosphere and others. Software development now happens on a compressed timeline, and 2014 was pretty much the Wild West of container development.

And then Google joined the fray in June 2014 by announcing Kubernetes in a short blog post that focused more on Docker image support in Compute Engine than Kubernetes.

While the success of Kubernetes now seems almost pre-determined, simply getting Google to release it as an open-source project was a bit of a struggle, even though the company knew that it needed a magic bullet if it wanted to compete with AWS — and as McLuckie stressed, the launch of Kubernetes was very much about that competition.

“For me, the journey started with Compute Engine. Joe [Beda — one of the other co-founders of Kubernetes] and I were working on our infrastructure service offering and we constructed what we felt was a world-class piece of infrastructure service that had incredibly favorable consistency, network properties, the block device was really good,” McLuckie, who is now at VMware and freed from whatever shackles the Google PR team put on him during his tenure there, said. “But it was also pretty clear that, without a significant change in the ecosystem, Amazon had pretty much achieved escape velocity — just on a pure VM-based focus on things.”

To get backing for the Kubernetes project, Google’s executive team needed some good business reasons — and being able to better compete with AWS was surely a good reason. “If you asked Tim [Hockin] or Joe [Beda], they’ll be like: it was an opportunity to change the world and create great technology, and it certainly was, but it’s hard to sponsor a project on that basis,” McLuckie said.

We are the Borg

Google itself had always used containers internally for a very long time. That was never really a secret, and, as it disclosed a year after the launch of Kubernetes, the so-called Borg cluster and container management system was essentially the predecessor to Kubernetes (though the two don’t share any code).

“We’d long talked about the idea that whoever nails the Linux OS level abstraction and creates a kind of Borg-like capability will do really well,” McLuckie said. At the same time, the Borg team, including Hockin, was looking to evolve the existing stack inside the company, and those two groups finally came together with the Kubernetes project.

When Docker launched, the actual technology wasn’t all that interesting to those working inside of Google, given that Borg already offered them all of its features. Still, Docker brought many of these ideas to the masses for the first time.

“Working on the Borg team, every now and then I had these thoughts of ‘what am I going to do when I eventually leave Google? How am I going to live without Borg? Holy smokes, how do people do this? That is so stone age compared to what we can do with Borg,'” Hockin said. “When we saw Docker come out, from an internal point of view it wasn’t very exciting, but when you look at it through the lens of what the rest of the world is doing, suddenly it became really compelling.”

Specifically, Hockin noted that Docker really got the developer experience right. Given that Google’s Borg was used internally, the company never really needed to perfect the onboarding experience. But the Google team also saw that the businesses that adopted Docker and containers more generally would eventually need tools to orchestrate all of these deployments.

“I saw it as an opportunity to change the world a little bit; to bring forth what Google had learned; to help people not make the same mistakes, but also to put Google in a position where we were able to demonstrate that we have stuff, too, that we have thought leaders in this space, that it’s more than just about VMs and that the opportunities are bigger here,” Hocking explained.

The reason Docker suddenly became so popular wasn’t a surprise either. As Monroy noted, platform-as-a-service (PaaS) offerings like Heroku and Cloud Foundry were all the rage at the time. Those provided developers with abstractions that made them more productive, but the moment those failed, they had to fall back to VMs. “It was clear to me that there was at least one layer in between where if we could build better plumbing around containers and orchestration […] it just wasn’t an open layer where there was any collaboration in the ecosystem.”

Unsurprisingly, McLuckie agreed with this. “We were really excited about this idea of creating this Goldilocks abstraction — high enough that you can [run] pretty much anything, low enough that even if you broke glass, it didn’t create all kinds of issues — and then you could tweak and customize as much as you wanted to.”

Open-sourcing Kubernetes

So inside of Google, the team had built a great piece of technology that solved a lot of the problems that developers were struggling with at the time, but that wasn’t available to anyone outside of the company. Meanwhile, Docker was getting a lot of buzz by focusing on the developer experience and building a community around its products. “The community was a big factor,” McLuckie said. “So as we’re thinking about doing Kubernetes and trying to get the list of ideas flowing, it became obvious that the only way to succeed and create that moment of disruption we were looking for was to lean in really hard on open-source technology.”

Google, of course, wasn’t new to open source, but five years ago, open source wasn’t yet the de facto standard for infrastructure projects as it is today. Sure, there was OpenStack and Cloud Foundry, but at Google, the standard way to talk about some of these fundamental technologies that made its services tick was to release papers about topics like MapReduce and BigTable — not to give away the code itself.

Open-sourcing Kubernetes then was a bit of a struggle, a challenge that nobody really acknowledged at the time. It came down to a final meeting with the executive team — a meeting where McLuckie at least wasn’t all that sure he’d get the go-ahead for the project.

“There was a perception that we were literally giving away the golden goose and that we couldn’t teach people how to do this,” Hockin said. “They had to learn it on their own. That was a statement that was said. We shouldn’t have to pay the development cost and then give it to everyone. We made an argument that this was going to happen, regardless of what Google’s position on it was, except that we had a perspective that was informed by 10-plus years of experience doing almost literally this exact thing — that it would be kind of dumb for us to be left out of that and to not be able to bring that perspective in a way that was both useful to the community and useful to what we wanted to do.”

“It was a non-trivial set of work to get people to get comfortable,” McLuckie added. “I think between the time when we started really pushing on just getting people’s heads around it — it took us about as long to get the approval as it did to actually build the first version of the technology.”

The team actually conceived multiple plans for taking these ideas to market, including the open-source version, but also a hosted product and some in-between options. Early on, the team actually called Kubernetes an SDK, Hockin said.

Even with all of this, though, there was a good chance that Kubernetes was never going to happen. Indeed, McLuckie said that by the time the final meeting came about, he had almost given up. “At that point, I was like, I’m not sure we’re going to succeed,” he said and acknowledged that he was pretty nervous going into the meeting. Eventually, though, the team got support from various camps inside the company and got approval to go forward.

“At the end, the final results speak for themselves. It has catapulted Google’s relevance in the cloud space forward in ways that we anticipated — and it certainly moved the industry forward in ways that I don’t think even we anticipated,” said McLuckie.

From open-source project to popularity

Getting approval was one thing, actually turning Kubernetes into a success was another, though. In what’s an interesting wrinkle to this story, Google actually released Kubernetes at DockerCon 2014. Google VP Eric Brewer, who was instrumental in getting the Kubernetes project approved, introduced the project (and clearly wasn’t quite sure how to pronounce Kubernetes yet). In his keynote, Brewer joked how Kubernetes was basically yet another orchestration platform — and indeed, DockerCon 2014 probably saw the launch of half a dozen of these, including Docker’s own Docker Swarm. Not all of these were open source, but Facebook, Spotify and others were all getting into this game at the time.

“We were all laughing. There was at least six and most of them were very much Borg lookalikes,” Hocking said. “But what they were all missing was the last page of the Borg paper, which was the mistakes we made.”

It took the team three months to go from getting approval to shipping that first version, something that McLuckie attributes to the coding skills of Brendan Burns, the third co-founder of Kubernetes, together with McLuckie’s Heptio co-founder Joe Beda. Some of those early parts weren’t pretty, and Burns wrote chunks of it in Java before the team eventually moved to Go.

Still, Kubernetes was not an instant hit. Indeed, Monroy argued that it actually “landed a bit poorly.” The reason for that was mostly technical about the networking model, which was somewhat biased to how Google did things internally — something that was hard to replicate for everybody else. Google bet on programmable networks and software-defined networks, a model it was already comfortable with but had not been adopted by the rest of the industry. That turned out to be the right model in the long run, but it took a while to see adoption, and the community had to rally around this and improve on the original implementation.

“One of the key points of the ethos of Kubernetes was: design in the open, develop in the open, embrace an open community. The key concepts still hold,” McLuckie said.

With all of the competition and the ongoing popularity of Docker, those early design decisions could have done in Kubernetes, though. “I sweated a long time about Docker and Docker Swarm,” Hockin said. “Docker is relentlessly fantastic about the developer experience and I used Docker Swarm — the third generation of Docker Swarm — and it was really good.”

The community rallied around Kubernetes, though, in part because of its Google pedigree, but also because it was open source and offered developers the experience they wanted. “The first time I realized that communities was going to win was the first time that I saw the API extended to do something that the founders never would have envisioned,” Monroy said. That community needed guidance, though, and with Google backing and leading the project, other large cloud players remained hesitant to join in the efforts, even if its technical merits weren’t really up for discussion. Microsoft and Amazon, after all, weren’t going to bet their cloud strategies on a Google product.

Stewarding Kubernetes through a foundation

Eventually, Kubernetes ended up in the Cloud Native Compute Foundation (CNCF), under the umbrella of the Linux Foundation, which was essentially created to host Kubernetes and other adjacent “cloud-native” projects.

“The thing I was most excited about was creating disruption in the industry and we didn’t have to hold on to the project,” McLuckie said. He had noticed how some of the tensions around Docker started emerging around vendor relationships, for example. “[What] we really needed to do is make sure that this disruption happened systematically,” he said. “And it was clear that we were not going to be able to do it alone. There was no practical way for us to construct an outcome that was going to be based on our understanding of enterprise organizations.”

Naturally, then, Google had to bring in other players like Red Hat that understood this market and had the necessary relationships.

“I spent a lot of time kind of negotiating through the industry and trying to get a conceptual agreement around what things should look like, and then put a lot of effort into designing a foundation structure that I thought would avoid some of the issues we’ve seen with other open-source foundations in the past,” McLuckie said. “I was really worried about balkanization. […] We wanted to make sure that some of those influences were not there.” That meant making sure that there were people in those organizations that would stick to the overall ethos and to set up an organization that would remain autonomous.

Traditionally, foundations like the CNCF exist to house intellectual property and to provide marketing through events and other means. The CNCF board clearly had bigger ambitions, though. It was essentially looking to bring together all of the building blocks that would allow enterprises to set up their own Google-like cloud-native data centers.

That is one of McLuckie’s regrets, too, though. Today, the CNCF plays host to a plethora of projects —including some that compete with each other. That creates some confusion because it’s hard to know what the organization is actually focused on — yet it’s hard to whittle it down at this point without upsetting a lot of people.

Five years in, the project keeps on moving at a rapid clip, though, and today, Kubernetes is the de facto standard for deploying modern software. Indeed, it has changed how software is written as developers and operators in large and small companies adapt to this new reality that lets them deploy updates faster and manage their infrastructure with far more flexibility than ever before — no matter whether that’s in a massive public cloud or in their own data center.

Some members would also like the CNCF to offer something more akin to a complete solution for building a cloud-native platform with Kubernetes at its core. Instead, though, the foundation seems to be happy to offer bits and pieces and leave it to the ecosystem to integrate them into a coherent solution. Time will tell if that’s the right approach, but there can be no doubt that Kubernetes has spawned a massive ecosystem of vendors that are filling in the gaps, no matter whether that’s security, storage solutions or full platforms.

With that, there’s a good chance that Kubernetes will still be around five years from now. But maybe it’ll be boring by then. Maybe it’ll be as exciting as a new release of the Linux kernel, McLuckie joked. The technology itself will fade into the background and the things it will enable will move to the foreground.