A lighter foundation for Kubernetes

Kubernetes has become the standard control plane for modern applications, but the way we deploy it still forces infrastructure teams into a bad set of tradeoffs.

At one end, Kubernetes can run directly on bare metal. That gives you excellent performance and removes a virtualization layer, but it also ties the cluster closely to the physical infrastructure underneath it. Highly available deployments often require multiple physical servers before the first application is even scheduled. Hardware failures become cluster events. Rebuilding a node means rebuilding a server. Scaling often means adding more physical machines. And while Kubernetes provides strong workload controls, the workloads on a node still ultimately share that node's kernel.

At the other end, Kubernetes can run inside traditional virtual machines. That solves some of the lifecycle and isolation problems, but now Kubernetes depends on another infrastructure platform underneath it. That might be VMware, Nutanix, OpenStack, or a public cloud such as AWS or GCP. In each case, the virtualization layer brings its own costs, management plane, networking, storage, patching, monitoring, licensing, security controls, and operational burden.

The cost of that virtualization layer is not just CPU and memory. Someone has to secure it. Someone has to patch it. Someone has to manage templates, virtual networks, storage, access controls, lifecycle, and capacity. In commercial environments, someone also has to pay for it. Kubernetes becomes an application running on top of another large infrastructure system.

For organizations already invested in those platforms, that may be acceptable. But securely running Kubernetes should not require building or buying another virtualization stack first.

That is the problem we are addressing with x0, Starlight's Kubernetes microVM orchestrator.

The node itself becomes lightweight

x0 uses lightweight microVMs to turn Kubernetes nodes into portable, isolated infrastructure that can be created, placed, scaled, and replaced as needed. Instead of dedicating a physical server to every Kubernetes node, or placing each node inside a traditional VM, x0 creates controllers and workers inside lightweight libkrun microVMs directly on Starlight.

The important idea is simple: the Kubernetes node itself becomes lightweight.

A microVM still gives the node its own kernel and a hardware virtualization boundary, but it avoids much of the machinery associated with a traditional VM. There is no need for a general-purpose virtualization management platform underneath the cluster. The guest can remain intentionally small and focused on running Kubernetes.

x0 starts with k0s and k3s because both distributions are particularly well suited to this model. They package Kubernetes into compact, operationally simple forms that fit naturally inside lightweight virtualized nodes. More distributions can be added over time through the same driver model.

The goal, however, is not simply to put a k0s or k3s binary inside a small VM.

x0 owns the lifecycle around it.

When a cluster is created, x0 determines where its controllers and workers should run, creates the microVMs, configures the Kubernetes distribution, establishes the cluster, applies the required networking and security controls, and brings the nodes online. As the cluster grows, x0 can add workers where capacity exists instead of forcing operators to think in terms of one server per node.

That separation creates two scheduling layers with very different responsibilities. Starlight decides where Kubernetes nodes should exist across the physical infrastructure. Kubernetes decides where pods should run inside those nodes.

Especially useful at the edge

A remote site may have one or two capable servers, not five or ten machines reserved for Kubernetes topology. With x0, those systems become pools of compute capacity. Multiple isolated Kubernetes nodes can run on the same physical host, while different hosts can contribute controllers and workers to the same cluster.

Instead of thinking that one physical server equals one Kubernetes node, x0 lets us think of the hardware as capacity and the Kubernetes nodes as scheduled workloads.

That also changes the economics of scaling. Adding capacity does not require rebuilding the cluster topology around new physical servers. New workers can be created where resources are available. Clusters can become smaller, denser, and easier to reproduce.

Security is a major reason for microVMs

Each x0 Kubernetes node receives its own kernel. That provides a stronger boundary between nodes than running everything directly on the same host operating system. A compromised worker does not simply become another set of processes sharing the Starlight host kernel.

The microVM boundary is only one part of the security model. x0 also benefits from the security controls already present in Starlight.

On Starlight Secure, the host is immutable and built around a hardened RHEL-based operating system with SELinux enforcing, STIG-oriented configuration, FIPS support, and tightly controlled host services. The components responsible for creating and managing x0 microVMs can be constrained by host policy, while the guest itself can apply its own Kubernetes and operating-system security controls.

SELinux remains important here because hardware virtualization and mandatory access control solve different problems. The microVM provides the guest with a separate kernel and a hardware-backed boundary. SELinux constrains what processes can do inside their respective environments and limits what privileged host-side components are allowed to access.

The guest environment can also be substantially smaller than a general-purpose VM. It does not need the same collection of packages, services, agents, and administrative tools normally installed on a long-lived Linux server. Reducing the amount of software inside the Kubernetes node reduces both operational surface area and attack surface.

OCI without giving up the VM boundary

x0 uses OCI in two different places, and the distinction matters.

The first OCI layer defines the Kubernetes node itself. Each x0 controller or worker is packaged from a signed, digest-addressed OCI image, but it does not run as a container on the Starlight host. x0 uses that image as the immutable userspace for a real libkrun microVM with its own Linux kernel, dedicated CPU and memory, a virtual network device, persistent node identity, and a separate writable state disk.

For the initial k0s implementation, that guest image is built from RHEL UBI Init 10.2 and includes the pinned k0s binary, its bundled containerd runtime, the Kubernetes images needed for offline operation, Starlight guest-management components, systemd integration, SELinux policy, and the artifacts needed to verify how the image was built.

That gives x0 many of the operational advantages that made containers successful without reducing the Kubernetes node to a host container. The node image is versioned, immutable, portable, and verifiable. x0 can identify exactly which image should run, verify it by digest and policy, materialize the filesystem, attach it to a microVM, and boot it with the approved guest kernel.

The Kubernetes node's durable state lives separately from that immutable image. Data such as k0s state, kubelet state, and x0 bootstrap state is kept on a dedicated persistent disk. This means the guest operating system can be upgraded or replaced without treating the node's Kubernetes state as part of the operating-system image. Temporary writes can remain temporary, while arbitrary host filesystem access is prohibited.

There is then a second OCI layer inside the microVM.

k0s runs its own containerd instance inside the guest, and Kubernetes uses that runtime to pull and execute ordinary OCI workload images. The architecture therefore looks roughly like this:

Signed OCI Kubernetes-node image
            |
            v
      libkrun microVM
            |
            v
  independent guest kernel
            |
            v
      k0s + containerd
            |
            v
  Kubernetes OCI workloads

Those two OCI layers serve very different purposes. The outer OCI image describes the Kubernetes node operating environment and is controlled by x0. The inner OCI images are application workloads controlled by Kubernetes.

That separation gives us a useful security property. Kubernetes workload containers still get the packaging and ecosystem developers expect, but their container runtime lives behind the Kubernetes node's own guest kernel. They do not run directly against the Starlight host kernel.

In other words, x0 keeps OCI where it is strongest, as a packaging and distribution format, while moving the Kubernetes node itself behind a hardware virtualization boundary.

A controlled path into the guest

x0 also avoids turning every Kubernetes node into a conventional remotely administered server.

Each guest includes starlight-guestd, a narrow management service that communicates with x0 over virtio-vsock. It handles bootstrap, lifecycle operations, health, readiness, bounded diagnostics, and infrastructure telemetry without requiring a broad management network or general-purpose SSH access.

Enrollment material and cluster credentials are intentionally kept out of the OCI image. They can be provided to a machine when it is created and scoped to that machine rather than baked into a reusable image, stored in logs, or carried around in environment variables.

This is another place where the microVM model is different from simply cloning VM templates. The image contains the software needed to become a Kubernetes node. The identity and authority needed to join a specific cluster arrive separately.

The result is a Kubernetes node that is reproducible without being pre-trusted, persistent without making its operating system mutable, and compatible with the OCI ecosystem without sharing the Starlight host kernel.

Confidential compute

The architecture also provides a natural path toward confidential computing.

Starlight supports confidential-compute technologies such as Intel TDX and AMD SEV-SNP. In defense environments, the value is less about protecting workloads from a cloud provider and more about anti-tamper and protecting sensitive data in use when an adversary gains physical access to the system.

Edge systems may be deployed in vehicles, aircraft, ships, or remote sites where hardware can be captured or directly probed. Disk encryption protects data at rest, but sensitive workloads, credentials, and operational data can still exist in memory while the system is running. Confidential compute adds protection at that point by encrypting and isolating guest memory.

With x0, that protection can be applied at the individual Kubernetes controller or worker boundary, making direct memory inspection and cold-data extraction significantly more difficult even if an attacker gets hands on the hardware.

Still just Kubernetes

None of this is useful if the resulting cluster stops behaving like Kubernetes.

One of the core design goals for x0 is therefore to preserve the existing Kubernetes ecosystem. kubectl should still work. Helm should still work. Operators should still work. CNI and CSI integrations should still work. Applications should not need to know that their Kubernetes nodes happen to be lightweight microVMs.

x0 sits underneath that experience.

The goal is not to replace Kubernetes or create another Kubernetes-compatible API. The goal is to improve the infrastructure Kubernetes runs on.

This becomes even more important in disconnected and intermittently connected environments. Edge deployments may operate across unreliable WAN links, isolated facilities, ships, aircraft, remote sites, or contested networks. A centralized virtualization management plane can become another dependency that operators must keep reachable.

Starlight is designed around a more distributed infrastructure model. x0 can use Starlight's shared inventory and distributed state to reason about capacity and placement across participating hosts without requiring every infrastructure decision to pass through a permanently available central management server.

Kubernetes still has its own control plane. What changes is the infrastructure underneath it.

Kubernetes becomes a workload

That leads to one of the most important architectural ideas in x0: Kubernetes itself becomes a workload. Traditionally, we think of Kubernetes as the infrastructure layer that everything else runs on. From Starlight's perspective, a Kubernetes controller or worker is another piece of desired state that can be created, placed, scaled, replaced, and moved as conditions change.

To make that work, Starlight maintains a distributed view of infrastructure state across participating hosts. Each host contributes information such as available CPU, memory, storage, health, locality, and other resource characteristics into a shared data model that is replicated across the environment. That state is exchanged peer to peer through gossip, allowing participating hosts to converge on a stable view of available capacity without depending on a single central database or scheduler.

That shared capacity view is what helps x0 decide where Kubernetes nodes should run. It is an infrastructure placement decision made below Kubernetes, not a Kubernetes scheduling decision. Kubernetes still decides where pods run inside the nodes it owns. x0 decides where those nodes themselves should exist across the Starlight fleet.

Because the state is replicated and reconciled peer to peer, this architecture does not require leader election or quorum to maintain that fleet-wide capacity view. There is no single scheduler or authoritative database that every host must reach before it can reason about available resources. Each participating host can continue working from its local copy of the shared state, exchange updates when peers are reachable, and converge again as connectivity changes.

That matters in distributed and edge environments. A site can become disconnected without losing its entire infrastructure placement model, and reconnecting does not require re-establishing a central consensus path before useful work can continue. The result is a more resilient substrate for x0, particularly across intermittently connected, geographically distributed, or contested environments.

That means a controller can be created where capacity exists. A worker can be added near the workload that needs it. A failed node can be replaced elsewhere. Different Kubernetes distributions can coexist. Small clusters can run at edge sites, while larger clusters can spread across broader pools of hardware.

The physical machines underneath simply provide capacity.

That is a much cleaner boundary. Bare metal Kubernetes gives you excellent performance, but forces Kubernetes topology onto physical infrastructure. Traditional VM Kubernetes gives you isolation and flexibility, but introduces another expensive platform underneath the cluster. x0 separates those concerns, allowing Kubernetes nodes to become schedulable infrastructure without requiring a heavyweight virtualization management layer.