Improve Kubernetes network performance with Cilium and eBPF

Why to use Cilium and eBPF for Kubernetes networking

Cilium is an open source CNI plugin that provides networking, security and monitoring features for Kubernetes. Unlike many similar tools, Cilium uses eBPF under the hood to access the data and controls necessary to manage Kubernetes environments, which improves efficiency and visibility.

Greater efficiency

The eBPF mechanism lets programs run directly inside the Linux kernel. Those programs run very efficiently because they operate in kernel space rather than user space, where monitoring and security software traditionally run.

Most other Kubernetes networking tools rely on sidecar containers to host software agents, which run as user-space applications on the nodes they monitor. This leads to higher resource consumption and lower visibility levels compared with Cilium, which uses eBPF to do the heavy lifting necessary to monitor and manage networking within Kubernetes clusters.

Better visibility into Kubernetes workloads

Managing Kubernetes networking and other features within the kernel also provides greater visibility into Kubernetes-based workloads compared with traditional Kubernetes networking software.

Because all processes and events that happen on a node inside a Kubernetes cluster are visible to the node's kernel, admins using eBPF can capture almost any data they desire about a workload. This improves visibility compared with traditional network monitoring tools, which are usually restricted to basic network logs and metrics rather than kernel-level details about network operations.

Limitations of Cilium for Kubernetes networking

That said, some users might not want or be able to use Cilium and eBPF to manage networking in Kubernetes.

In a practical sense, eBPF currently only works on Linux. Although Microsoft has implemented limited eBPF functionality for Windows, it only supports certain workloads and is not yet a realistic option for production environments. Consequently, Cilium isn't useful for those running Windows-based nodes inside their Kubernetes clusters.

In addition, for Linux users, eBPF requires kernel version 4.19.57 or later at time of publication. Kubernetes nodes running earlier versions of Linux won't be able to benefit from Cilium and eBPF.

Cilium alternatives

Calico is another Kubernetes networking tool that also supports eBPF. However, unlike Cilium, Calico makes eBPF-based monitoring optional. The tool can also collect Kubernetes networking data using the traditional approach.

Because Cilium was designed specifically for eBPF, it's a good choice if you know you want to observe Kubernetes networking operations using eBPF. On the other hand, Calico might be best if you want to monitor networking using other approaches in addition to eBPF or if some of your nodes aren't compatible with eBPF -- for example, because they're running older kernel versions.

Can you use eBPF without Cilium?

It's possible to use eBPF without Cilium. However, to do so, you'd need to write eBPF programs from scratch, which is difficult and time consuming.

Cilium lets you deploy eBPF-based network monitoring and management tooling into your cluster. Plus, most of Cilium's eBPF tooling is preconfigured. You can use it out of the box without writing or deploying custom eBPF programs.

How to install Cilium for Kubernetes

The process of installing Cilium on Kubernetes varies somewhat depending on which Kubernetes distribution you use. For distribution-specific details, see the Cilium documentation.

This tutorial walks you through a Cilium installation on a local K3s cluster hosted on Ubuntu version 22.04.

Step 1: Install the Cilium CLI tool

Before installing Cilium itself, install the Cilium CLI tool using the following code.

CILIUM_CLI_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/cilium-cli/master/stable.txt)

CLI_ARCH=amd64

if [ "$(uname -m)" = "aarch64" ]; then CLI_ARCH=arm64; fi
curl -L --fail --remote-name-all https://github.com/cilium/cilium-cli/releases/download/${CILIUM_CLI_VERSION}/cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}
sha256sum --check cilium-linux-${CLI_ARCH}.tar.gz.sha256sum
sudo tar xzvfC cilium-linux-${CLI_ARCH}.tar.gz /usr/local/bin
rm cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}

The tool should now be available at /usr/local/bin/cilium.

Step 2: Install Cilium

The next step is to install Cilium itself.

To ensure that Cilium will work properly, disable support for other Kubernetes networking plugins before starting your cluster. Use the command below to start a K3s cluster without other networking plugins.

curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC='--flannel-backend=none --disable-network-policy' sh -

You can now install Cilium using the cilium install command.

If you get a "command not found" error, you most likely need to specify the path to the Cilium binary explicitly.

/usr/local/bin/cilium install

Check that Cilium was successfully installed using the cilium status --wait command.

Step 3: Use Cilium with Hubble

At this point, Cilium is installed in your cluster. But to make full use of its networking and security capabilities, you'll want to install Hubble, an observability platform that runs on top of Cilium and eBPF.

Set up Hubble with the cilium hubble enable command. Hubble also provides its own CLI tool, which you can install using the following code.

export HUBBLE_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/hubble/master/stable.txt)

HUBBLE_ARCH=amd64
if [ "$(uname -m)" = "aarch64" ]; then HUBBLE_ARCH=arm64; fi
curl -L --fail --remote-name-all https://github.com/cilium/hubble/releases/download/$HUBBLE_VERSION/hubble-linux-${HUBBLE_ARCH}.tar.gz{,.sha256sum}
sha256sum --check hubble-linux-${HUBBLE_ARCH}.tar.gz.sha256sum
sudo tar xzvfC hubble-linux-${HUBBLE_ARCH}.tar.gz /usr/local/bin
rm hubble-linux-${HUBBLE_ARCH}.tar.gz{,.sha256sum}

Once Hubble is up and running, you can begin monitoring Kubernetes networking in the command line or via an optional web-based UI that Hubble provides.