Presentation Recap: Kubernetes as a Universal Control Plane – Joe Beda, VMware

We were delighted to have Joe Beda, one of the original creators of Kubernetes, and Principal Engineer at VMware, give the opening keynote at this year’s 2020 Distributed SQL Summit. If you weren’t able to attend, here’s the summary and playback of his keynote presentation.

Kubernetes as a Universal Control Plane

Oftentimes systems start out with efficiency in mind (as Joe states, “to try and get as much out of your computers as you possibly could“), but new innovations are unlocked along the way. Similarly, Kubernetes started out with the main job of running a container across a pool of machines. But over time, Kubernetes emerged as a new abstraction layer and new way of dealing with infrastructure for application teams. Additional innovations in and around Kubernetes continue today.

Starting with the basics of what’s under the hood, Joe describes the core of how Kubernetes works.

“At its core, Kubernetes is a database,” says Joe. In the image above, you can see etcd as the core database. The API server talks to etcd, and everything else calls into the API server. The scheduler that decides which container will run on which node. The controller manager does “higher order things“. Joe explains, “the whole idea of a controller is to make relentless forward progress. The idea is you take this desired state, you then do whatever you need to try and take some steps, one foot in front of the other, to try and actually work toward that desired state.” Including managing the unexpected.

The Kubelet does what it needs to do to get the pod running. The components aren’t talking to each other directly, “but essentially leave breadcrumbs for each other by writing data into etcd,” notes Joe.

So while some people describe Kubernetes as a container orchestrator, Joe sees it a bit differently, “orchestration implies that there’s a plan, that there’s a score, that there’s a conductor and everything is actually pre-planned from the get go.” But because Kubernetes is flexible and durable enough to do self-healing, Joe suggests “in reality, this is more like jazz improv.”

Joe provides additional detail on the controllers: “The goal out of Kubernetes when we were creating it was to create a Unix philosophy type of thing – a bunch of primitives that can be used with each other and layered on top of each other to be able to produce a result. And for me, I think the mark of a durable platform over time is when people can use the thing in unexpected ways. If you build something and then somebody uses it in a way that you never actually even thought about or considered, that is the mark of a successful platform. Layering these primitives, these composable systems like infrastructure Lego blocks, that I think is the heart of what makes cloud cloud from an application person’s point of view.”

Joe also describes additional Lego blocks in play in a Kubernetes environment, including attached storage, replica sets, deployment, stateful sets, and custom resource definitions (CRDs).

But operators is where Joe thinks “things really start to get exciting.” Describing further that “the definition of operator is a domain specific controller that knows something about the workload that it’s actually controlling. And so what we find is that operators are a way to build on top of everything that I talked about here around desired state and bring that into the application, or at least the service domain. One example would be, how can we take a database system like Postgres or MySQL, and then start adapting that to Kubernetes? Well, you could build an operator to do that. Take all of the runbook, all the institutional knowledge for what it takes to run a database, start to distill that down into code, build that as a way that adapts those things into Kubernetes.”

Although Kubernetes is about six years old now, the future remains bright. Joe explains that “as we look towards the future, we’re going to start seeing people start to use these level-triggered systems, these controllers, to start controlling more than just software systems.

Video Playback

Want to See More?

Check out all the talks from this year’s Distributed SQL Summit including Twitter, Pinterest, Comcast, Kroger, and more on our Vimeo channel.