How to load eBPF programs in DaemonSets?

To allow a Kubernetes DaemonSet to load eBPF programs, inspect host processes, attach to host network interfaces, and read kernel BTF (BPF Type Format) and tracefs metadata, you need to configure three things in your manifest:

1. Pod-level Host Namespaces (hostPID, hostNetwork) in spec.template.spec.
2. Container-level Capabilities (CAP_BPF, CAP_PERFMON, CAP_NET_ADMIN, CAP_SYS_ADMIN, CAP_SYS_RESOURCE) and SELinux/AppArmor unconfined overrides in securityContext.
3. hostPath Volume Mounts for /sys/kernel/btf, /sys/kernel/tracing, /sys/fs/bpf, and /sys/fs/cgroup.

Production-Ready eBPF DaemonSet Manifest

Below is a complete manifest using least-privilege eBPF capabilities (with SYS_ADMIN included as requested, plus notes on when privileged: true or Bidirectional mount propagation is needed):

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: ebpf-agent
  namespace: kube-system
  labels:
    app: ebpf-agent
spec:
  selector:
    matchLabels:
      app: ebpf-agent
  template:
    metadata:
      labels:
        app: ebpf-agent
    spec:
      # 1. Share the host's PID and Network namespaces
      hostPID: true
      hostNetwork: true
      # Required when hostNetwork is true if the pod still needs to resolve Kubernetes ClusterIP DNS names
      dnsPolicy: ClusterFirstWithHostNet

      containers:
        - name: ebpf-agent
          image: ghcr.io/your-org/ebpf-agent:latest
          securityContext:
            # Run as root user inside the container so capabilities take effect in the effective set
            runAsUser: 0
            # Set privileged: true ONLY if your container also needs to auto-mount tracefs/bpffs
            # or access raw host devices; otherwise explicit capabilities below are cleaner:
            privileged: false
            # Prevent AppArmor or SELinux on the node from blocking bpf() or tracefs reads
            seLinuxOptions:
              type: spc_t
            appArmorProfile:
              type: Unconfined
            capabilities:
              drop:
                - ALL
              add:
                - BPF # Load eBPF programs and create maps (Linux 5.8+)
                - PERFMON # Attach kprobes, fentry, tracepoints, perf_events (Linux 5.8+)
                - NET_ADMIN # Attach XDP, TC, sock_ops, cgroup_skb programs
                - SYS_ADMIN # Required for BPF LSM, kernel symbol lookup (/proc/kallsyms), and older kernels
                - SYS_RESOURCE # Required on kernels < 5.11 to raise RLIMIT_MEMLOCK for eBPF maps
                - SYS_PTRACE # Required if attaching uprobes across host PIDs via /proc/<pid>/root

          volumeMounts:
            # 2. Read-only access to CO-RE BTF type info (/sys/kernel/btf/vmlinux)
            - name: sys-kernel-btf
              mountPath: /sys/kernel/btf
              readOnly: true

            # 3. Access to tracepoint format files & kprobe event interfaces
            - name: sys-kernel-tracing
              mountPath: /sys/kernel/tracing
              readOnly: false
            - name: sys-kernel-debug
              mountPath: /sys/kernel/debug
              readOnly: false

            # 4. Persistent BPF filesystem (bpffs) for pinning maps/links across pod restarts
            - name: bpf-maps
              mountPath: /sys/fs/bpf
              mountPropagation: Bidirectional

            # 5. Cgroup v2 hierarchy (needed if attaching CGROUP_SKB / CGROUP_SOCK_ADDR programs)
            - name: cgroup
              mountPath: /sys/fs/cgroup
              readOnly: true

      volumes:
        - name: sys-kernel-btf
          hostPath:
            path: /sys/kernel/btf
            type: Directory
        - name: sys-kernel-tracing
          hostPath:
            path: /sys/kernel/tracing
            type: DirectoryOrCreate
        - name: sys-kernel-debug
          hostPath:
            path: /sys/kernel/debug
            type: Directory
        - name: bpf-maps
          hostPath:
            path: /sys/fs/bpf
            type: DirectoryOrCreate
        - name: cgroup
          hostPath:
            path: /sys/fs/cgroup
            type: Directory

Breaking Down the Key Settings & Gotchas

A. Why CAP_SYS_ADMIN Alone Isn't Always Enough in Kubernetes

Even though CAP_SYS_ADMIN is the "catch-all" capability in Linux, container runtimes (containerd / runc) drop capabilities that are not in the default allowlist unless you either set privileged: true or explicitly list them under capabilities.add:

• BPF + PERFMON (Linux 5.8+): The kernel split many eBPF checks out of CAP_SYS_ADMIN into CAP_BPF and CAP_PERFMON. Some kernel subsystems check capable(CAP_BPF) or perfmon_capable() directly, so always add BPF, PERFMON, and SYS_ADMIN together.
• SYS_RESOURCE (RLIMIT_MEMLOCK): On Linux 5.11+, eBPF memory accounting moved to cgroup memcg, but if your DaemonSet runs on any older 5.4/5.10 nodes or uses libraries (libbpf / cilium/ebpf) that call setrlimit(RLIMIT_MEMLOCK, RLIM_INFINITY) at startup, the call will fail with EPERM without CAP_SYS_RESOURCE.
• seLinuxOptions: { type: spc_t } & appArmorProfile: { type: Unconfined }: Even with CAP_SYS_ADMIN, a standard container runs under a restricted SELinux label (container_t) or the default runtime/default AppArmor profile, which blocks bpf() syscalls and writes to /sys/kernel/tracing unless you set privileged: true or explicitly mark the security profile unconfined (spc_t = Super Privileged Container in SELinux).

B. Accessing the Host Mount Namespace (hostMount Alternatives)

Kubernetes Pod specs have hostPID: true, hostIPC: true, and hostNetwork: true, but there is no hostMount: true field in the Kubernetes Pod API because a container's root filesystem (rootfs) must be isolated via pivot_root to run the container image.

You have two ways to interact with the host mount namespace:

1. hostPath Volumes + mountPropagation: Bidirectional (Standard): Mounting /sys/kernel/btf, /sys/kernel/tracing, and /sys/fs/bpf via hostPath brings those specific host kernel filesystems into the container. Adding mountPropagation: Bidirectional (Note: Kubernetes requires privileged: true on the container if you enable Bidirectional mount propagation) allows your container to run mount -t bpf bpf /sys/fs/bpf if the node hasn't mounted bpffs yet, and have that mount propagate back to the host.
2. Entering the Host Mount Namespace via /proc/1/root or nsenter: Because you enabled hostPID: true and **CAP_SYS_ADMIN + CAP_SYS_PTRACE**, PID 1 inside your container is the host's systemd/init process.

• Your eBPF loader can inspect any host file directly at /proc/1/root/sys/kernel/btf/vmlinux—even without a hostPath volume!
• When attaching uprobes to libraries inside other containers on the node, your agent can open /proc/<target_pid>/root/usr/lib/... to resolve the exact inode and ELF offset in that container's mount namespace.
• Or, your entrypoint can call setns(open("/proc/1/ns/mnt"), CLONE_NEWNS) (or nsenter -t 1 -m) to switch a worker thread directly into the host's root mount namespace.

C. Kubernetes Pod Security Admission (PSA) Requirement

Because this DaemonSet uses hostPID, hostNetwork, hostPath, and elevated capabilities, Kubernetes' built-in Pod Security Admission (PSA) controller will reject the pod if your target namespace is set to baseline or restricted. Ensure the namespace where you deploy the DaemonSet (kube-system has this by default on most clusters) is labeled for the privileged policy:

kubectl label namespace <your-namespace> pod-security.kubernetes.io/enforce=privileged --overwrite

Frequently Asked Questions

Why does UID 0 inside a Kubernetes container not automatically grant CAP_BPF, CAP_PERFMON, or CAP_SYS_ADMIN?

On a standard Linux host, user ID 0 (root) possesses every capability enabled by default. However, Kubernetes and container runtimes deliberately break this rule as a core security isolation feature.

Even if your container runs as root (UID 0), the container runtime (containerd/runc) drops nearly all powerful capabilities from the process's bounding set upon startup to prevent containers from modifying the host kernel or network stack. Running as container root only grants file-ownership permissions and a small default set of safe capabilities (like CAP_KILL or CAP_NET_BIND_SERVICE); advanced privileges like CAP_BPF or CAP_SYS_ADMIN must always be explicitly requested via securityContext.capabilities.add in your Pod manifest.