The History of Kubernetes

Kubernetes prehistory: Google Borg and Docker

Before diving into a discussion of the history of Kubernetes since the platform’s official introduction in 2014, let’s talk about what you might call Kubernetes pre-history – specifically, two key platforms or technologies that played a major role in preparing the way for Kubernetes.

Borg, the Kubernetes predecessor

Google announced Kubernetes as an open source project on June 6, 2014, the date that most people regard as Kubernetes’s birthday. However, viewed from a broader perspective, Kubernetes has a longer history because it traces its conceptual origins back to a platform called Borg.

Borg was a cluster manager that Google used internally to orchestrate workloads across massive numbers of servers. Google hasn’t publicly shared many technical details about how Borg works, and it doesn’t seem that Kubernetes currently shares any code with Borg. But according to a blog post published on the Kubernetes website in 2015, many of the concepts at the core of Kubernetes – such as Pods, Services, and Labels – originated with Borg.

So, we can thank Google engineers for distinctive Kubernetes ideas, like organizing workloads into Pods and exposing network applications as Services.

The rise of Docker

A history of Kubernetes would also be incomplete if it didn’t mention Docker. Docker isn’t a part of Kubernetes or a direct predecessor. Still, as the platform that helped bring container technology mainstream, Docker played an important role in creating the need for container orchestration technology that Kubernetes was born to solve.

Containers have a long history that stretches back to the 1970s, and there were main containerization frameworks or technologies available before Docker came along in 2013. However, Docker made containers very easy to work with, helping to popularize the idea of packaging and deploying applications inside containers.

As more and more people began taking advantage of Docker to develop and run applications inside containers, they realized they needed a way to orchestrate those containers efficiently. This led to the rise of Kubernetes and a variety of other container orchestration tools – including an orchestrator from Docker, called Swarm, which didn’t appear until after the launch of Kubernetes, and therefore wasn’t available early-on to solve the orchestration needs of people who had adopted Docker.

Kubernetes’s early days

Now that we’ve covered the historical context from which Kubernetes arose, let’s talk about the debut of Kubernetes itself.

The initial Kubernetes release

Although Kubernetes was announced to the world in June 2014, it took more than a year before the project issued its first stable release. Kubernetes 1.0 arrived on July 21, 2015, the date at which Kubernetes developers deemed the platform stable enough for real-world use.

At the time, Kubernetes supported all of the core functionality that is still familiar to Kubernetes admins today. You could deploy sets of containers as Pods, distribute them across nodes, and expose applications over the network as Services. You could also manage everything with kubectl, which is still the main CLI tool in most Kubernetes distributions today.

A rough release

However, Kubernetes circa 2015 was rough around the edges, to put it mildly. Part of the challenge was simply that many of the concepts behind Kubernetes – and cloud native computing in general – remained novel to many engineers. For instance, the idea of not knowing (or even caring) where which server hosted a given application, or the notion of treating infrastructure as “cattle” instead of “pets,” were still new to most people accustomed to deploying applications directly on carefully managed servers. Thus, Kubernetes presented something of a learning curve.

Lack of extensive Kubernetes documentation, integrations, and add-ons also made the platform hard to use early on. Today, the Kubernetes documentation base is arguably one of the most thorough and best-written documentation resources in the world, and there are hundreds of extensions and integrations that add to the core capabilities of Kubernetes. But back in July 2015, there was just Kubernetes 1.0.

Kubernetes grows up: The late 2010s

Between the mid-2010s and the late 2010s, Kubernetes did a lot of growing, and it ended up becoming the most popular container orchestrator. However, it wasn’t all that clear as of July 2015 that this would happen.

Let’s explore some of the factors that explain how Kubernetes matured so quickly during the pivotal first years of its history.

Kubernetes and the CNCF

One key factor was the backing that Kubernetes enjoyed from influential companies and organizations.

The same day that Kubernetes 1.0 debuted, Google and the Linux Foundation (a nonprofit that supports the development of the Linux kernel and other major open source technologies) announced the formation of the Cloud Native Computing Foundation (CNCF), an organization overseen by the Linux Foundation to advance a new breed of cloud-centric open source platforms. Kubernetes was the CNCF’s first project.

The link between the CNCF and Kubernetes is not especially important in a technical sense, but it does matter in explaining how Kubernetes gained so much momentum so quickly. Because a variety of major companies – such as Red Hat, VMware and IBM – belonged to the CNCF as consortium members, the CNCF association helped ensure a certain legitimacy for Kubernetes from the start. In addition, the CNCF helped build momentum within the fledgling Kubernetes community by organizing KubeCon events, where thousands of Kubernetes developers and users gathered on a regular basis, starting in 2015.

Without the backing of the CNCF and its link to Google, Kubernetes might have been viewed by many as just another random open source project – or as another orchestration platform whose purpose was unclear given that alternative orchestration tools, like Apache Mesos, were already available. (Docker Swarm appeared in autumn 2015, a few months after Kubernetes 1.0 came out, so Swarm wasn’t actually on the scene when Kubernetes first debuted.)

Orchestrator Wars: 2016-2017

The “orchestrator wars” era of 2016-2017 was a pivotal time in the evolution of container orchestration. During this period, organizations sought to scale their operations and had multiple options to consider. Apache Mesos, another open-source project, was a popular choice for those needing robust scalability, but wasn’t purpose built for containers. Kubernetes was still in its infancy, while Docker was vigorously promoting Docker Swarm as a way to monetize their platform. 

However, by late 2017 and early 2018, it became increasingly clear that Kubernetes would emerge as the dominant force in the field. This shift was driven by strong support from the CNCF and major industry players like Microsoft, AWS, VMware, Red Hat, and Dell. Their backing signaled that Kubernetes (whether the best solution or not) would become the de facto industry standard, akin to the VHS triumph in the videotape format wars of the 1980s. This consensus solidified Kubernetes’ position as the leading container orchestration platform, shaping the future of cloud-native applications.

Novel Kubernetes features and functionality

Another key factor in explaining Kubernetes’s rapid rise to fame was that it gained important new features.

At the outset, as we mentioned, Kubernetes was pretty basic. It could orchestrate workloads across servers, but that was about it. It lacked sophisticated support for persistent data storage, its network functionality was limited and it didn’t offer much in the way of built-in security controls.

This changed in good time, however, as Kubernetes gained important features like the following:

• A native role-based access control (RBAC) framework, which debuted as a generally available feature with Kubernetes 1.8 in 2017. RBAC features make it possible to manage who can do what within Kubernetes, reducing many security risks related to improper access controls.
• Advanced networking capabilities, such as support for cluster load balancing and CoreDNS, both of which appeared with the release of Kubernetes 1.11 in 2018.
• Support for persistent volumes, which make it easier to store data persistently on Kubernetes – and, by extension, to run stateful workloads. Persistent volumes became generally available with Kubernetes 1.14 in 2019.

Capabilities like these smoothed over some of Kubernetes’s rough edges and made the platform a viable choice for supporting a wider array of production workload types.

Tech behemoths take notice

At the same time that Kubernetes was gaining critical new features, it was also coming onto the radars of cloud providers, who were busy creating cloud-based Kubernetes services.

In 2017, Microsoft released Azure Kubernetes Service (AKS), a managed Kubernetes service in the Azure cloud. This was notable because Microsoft had initially forged a close relationship with Docker, which it reportedly even tried to buy in the mid-2010s. The release of AKS signaled that Microsoft saw Kubernetes, not Docker Swarm or other Docker tooling, as the future of container orchestration.

Likewise, in 2018, Amazon introduced Elastic Kubernetes Service (EKS), a managed Kubernetes service in the Amazon Web Services (AWS) cloud. This was also a notable move given that Amazon had previously created its own, custom-managed container orchestration service, Elastic Container Service (ECS). The company didn’t abandon ECS (which it still supports today), but Amazon’s decision to build a container management platform based on Kubernetes was also a reflection of the prominence that Kubernetes had gained in the few short years since its initial release.

It wasn’t just cloud providers that made important early bets on Kubernetes. Red Hat, too, signaled its confidence in the future of the platform starting in 2015, when it swapped in Kubernetes to serve as the foundation for OpenShift, Red Hat’s flagship application development and deployment platform. Previously, OpenShift had run on custom technology. The subsequent acquisition of Red Hat by IBM in 2019 added even more momentum to OpenShift, since it forged a close association between Kubernetes and another major technology brand name.

A similar evolution occurred with Rancher. Initially, Rancher was known for RancherOS, a lean Linux distribution, and its own container orchestrator, Rancher. However, around 2018, Rancher made a strategic pivot to standardize on Kubernetes, integrating it as the core of their platform. This decision allowed Rancher to leverage the growing popularity and robust ecosystem of Kubernetes. 

Additionally, Rancher developed K3s, a lightweight Kubernetes distribution designed for resource-constrained environments like edge devices and IoT applications. K3s has since become a popular choice for these specialized use cases, maintaining Rancher’s commitment to simplifying container orchestration while adapting to industry trends.

VMware set up its own Kubernetes distribution (TKG) and in 2018 acquired Heptio to beef up its platform. Heptio was a Kubernetes services firm that was then only 2 years old, but had the pedigree of having two of the original creators of Kubernetes (at Google) as its founders.

Kubernetes since 2020

By around 2020, or perhaps a bit before, it was clear that Kubernetes had won what some people have called the container orchestration wars – meaning competition between Kubernetes and alternative orchestrators, like Docker Swarm and Mesos, to become the de facto platforms for managing containerized applications across clusters of servers.

Data from 2022 showed that by that time, about 60 percent of organizations that used containers were orchestrating them with Kubernetes. The rate of Kubernetes adoption has only increased since then.

Meanwhile, although Kubernetes has continued to gain new features, the changes have not been as momentous as they were during the 2010s, when the platform grew by leaps and bounds. Most enhancements since about 2020 have been refinements upon or improved versions of earlier features – such as the introduction of security contexts to replace security policies.

Likewise, while the most recent major Kubernetes release, version 1.30, doesn’t include any earth-shattering new features, it does offer several new capabilities that, when used properly, can help admins build and operate more secure clusters. These include features such as:

• Support for using Common Expression Language (CEL) for admission control, which makes it possible to govern access requests in a granular way directly through the Kubernetes API.
• Beta support for Linux-based user namespacing in Pods, which can help contain security risks.

Changes like these are not huge on their own. But they do introduce valuable new capabilities that, when used collectively, open powerful opportunities for improving Kubernetes security. However, the multitude of Kubernetes distributions and installers adds complexity to the market and creates potential security gaps. This fragmentation can lead to inconsistencies and vulnerabilities that can be exploited. That’s why it’s important to keep following Kubernetes as it continues to evolve – as we do here at Aqua, where we’re always on the lookout for ways to make our Kubernetes security solutions even better.

What’s next for Kubernetes?

As Kubernetes enters into its second decade, one thing is clear: The orchestration platform is not going away. On the contrary, Kubernetes has become so deeply embedded into the cloud-native computing realm by this point that it’s hard to imagine living without it. Unless a major disruptive new technology becomes ready for prime time (did we hear someone say “unikernels“?) that makes Kubernetes irrelevant, it’s likely to become a cornerstone of the digital world as we know it, in a fashion similar to technologies like Linux.

That said, there remains plenty of opportunity for Kubernetes to keep growing and improving, while maintaining the balance between richness of capabilities and simplicity of operation. For example, one feature that would arguably create a great deal of value for Kubernetes is full support for multi-cloud clusters, meaning the ability to operate a single Kubernetes cluster whose nodes are distributed across multiple clouds or data centers. This is currently possible to do, but it’s hard to do well because Kubernetes wasn’t designed for this type of use case.

Greater consistency between Kubernetes distributions would also be a nice addition. The diversity of distributions, including some with proprietary extensions or features that only work within a given vendor ecosystem, that have arisen over the past decade has made it challenging to migrate from one distribution to another in some cases. That said, this type of division tends to happen with any type of major open source platform (think of the many different Linux distributions, for instance), so perhaps there’s not much the community can do about it.

Enhancing Kubernetes security will no doubt also be a place for growth for the foreseeable future. While Kubernetes offers several important native security features, like support for RBAC, more fine-grained security controls would be a welcome enhancement for Kubernetes admins seeking to stay ahead of Kubernetes security threats.

Aqua Security was founded in 2015 as one of the first platforms to focus on securing Kubernetes, containers, and other cloud-native technologies – we’ve been following Kubernetes closely (to put it mildly) from the start. More importantly, we’ve been creating security solutions to address K8s security challenges since 2017 and inventing Kubernetes Security Posture Management (KSPM) a comprehensive set of policies and controls to automate secure configuration and compliance, in addition to extensive runtime protection controls for Kubernetes applications. It is this innovation that keeps us looking forward and continuing to keep Kubernetes workloads secure – over the next decade and beyond.