logo
CHEATSHEETSTOOLSABOUT中文

gVisor - systrap vs KVM

In gVisor, systrap and kvm are the two primary platform engines. The platform engine is the low-level component responsible for intercepting application system calls, isolating memory, and context-switching execution between the untrusted application (the Sentry's application threads) and the gVisor kernel (the Sentry itself).

Because they use entirely different virtualization abstractions—one relying on Linux seccomp and the other on hardware-assisted virtualization—their observable footprints on the host operating system differ significantly.

Process Count on the Host (ps aux)

If you run ps aux or pstree on the host system while a gVisor container is running, the observable process count is one of the most stark differences between the two engines:

Platform Engine Process Count Visible on Host
systrap 1 + N processes/threads (1 Sentry process + 1 stub thread per sandboxed app thread).
kvm 1 + V processes/threads (1 Sentry process + 1 thread per virtual CPU core allocated).

Under systrap:

systrap uses a lightweight, specialized process stub mechanism combined with seccomp filters.

  • Every time the application inside the container spawns a new thread or process (e.g., via fork or pthread_create), the Sentry creates a corresponding host stub thread/process.
  • Host Visibility: If you run ps on the host, you will see a 1:1 mapping between threads running inside the gVisor container and host processes/threads under the gVisor sandbox daemon.
  • If your container runs 50 application threads, you will see roughly 50+ host processes/threads tied to the gVisor sandbox.

Under kvm:

kvm runs the Sentry as a guest operating system inside a hardware-assisted KVM virtual machine.

  • The application processes inside the container exist entirely within the virtualized guest address space managed by the Sentry's internal page tables.
  • Host Visibility: The host OS only sees the gVisor Sentry process and its allocated vCPU threads (matching the -cpus flag configured for the sandbox).
  • Whether your container runs 1 process or 1,000 application threads, you will only see the fixed number of vCPU threads on the host ps output. The application threads inside the container are completely invisible to the host process tree.

/proc Memory and Mapping Footprint

If you inspect /proc/<pid>/maps or /proc/<pid>/status for the gVisor process on the host:

systrap

  • Address Space Separation: Uses mmap with MAP_SHARED and file descriptors (memfd_create) to share memory regions between the Sentry process and stub processes.
  • Seccomp Filters: You will see active seccomp filters attached to the stub processes (Seccomp: 2 in /proc/<pid>/status). The stubs are restricted via seccomp so they can only execute sys_futex, sys_read, sys_write, or trigger a trap back to the Sentry.

kvm

  • /dev/kvm Handle: The Sentry process holds an open file descriptor to /dev/kvm.
  • KVM Memory Regions: The Sentry allocates host anonymous memory (mmap) and registers it with KVM via KVM_SET_USER_MEMORY_REGION.
  • Guest Page Tables: The Sentry manages Extended Page Tables (EPT on Intel) or Nested Page Tables (NPT on AMD). The host kernel page tables only see a single large virtual memory area assigned to the VM instance, not individual application memory segments.

Host System Call Overhead & Tracing (strace)

If you run strace -p <pid> on the gVisor process on the host:

systrap

  • You will observe heavy communication over shared memory ring buffers between the stub threads and the Sentry.
  • System calls triggered inside the container cause the stub to hit a seccomp filter, generating SIGSYS or signal events visible on the host.

kvm

  • You will primarily see ioctl(fd, KVM_RUN, ...) system calls being executed repeatedly by the vCPU threads.
  • When an application thread inside the container makes a system call, it triggers a VM-Exit (handled directly by kvm.ko in hardware), switching execution from Guest Ring 3/0 back to the Sentry in Host Ring 3 without going through standard Linux system call entry points or signal delivery loops.

Summary Matrix

Observable Attribute systrap kvm
Host ps Process Count Scaled to application threads (1 + N app threads) Fixed to vCPU allocation (1 + V vCPU threads)
Host /proc Visibility Each app thread appears as a distinct host stub thread App threads are completely hidden inside guest VM memory
Primary Host Device/API Shared memory (memfd), seccomp, signal traps /dev/kvm (ioctl interface)
Hardware Requirements Any x86_64 or ARM64 Linux host (no virtualization required) Requires hardware virtualization support (Intel VT-x / AMD-V or nested virt)
System Call Interception seccomp-bpf + sys_futex stub ring-buffer switching Hardware VM-Exits (KVM_RUN)
Loading...