gVisor - CPU Stunning

In the context of gVisor, CPU Stunning is a "Stop-the-World" mechanism used by the Sentry (the gVisor kernel) to temporarily freeze all executing application threads within a sandbox.

It is very similar to how a Garbage Collector (GC) in languages like Java or Go pauses all threads to perform memory cleanup, but gVisor does it for kernel-level management tasks.

Why does gVisor need to "Stun" CPUs?

Because the Sentry is a kernel running in user-space, it sometimes needs to perform operations that must be atomic (happening all at once) across all virtual CPUs. If one thread is modifying memory while another thread is trying to read it, the sandbox could become unstable.

Common reasons for a "Stun":

• Memory Mapping Changes: When a process calls mmap or mprotect, the Sentry must update the page tables for all threads to ensure they all see the same memory layout.
• Signal Delivery: Delivering certain signals that affect the entire process group.
• Thread Creation/Exiting: Synchronizing the state of the process tree.
• Checkpointing: When saving the state of a container to disk, every thread must be frozen at a precise moment so the saved state is consistent.

How it Works (The Mechanism)

When the Sentry decides a "Stun" is necessary, it follows these steps:

1. Initiation: A single thread (the one performing the system call) triggers the stun.
2. Notification: The Sentry sends a signal or an interrupt to all other running application threads in that sandbox.
3. Trapping: All application threads "trap" back into the Sentry. Instead of executing application code, they enter a "wait" state controlled by the Sentry.
4. The Work: Once all threads are confirmed to be paused (stunned), the Sentry performs the global operation (e.g., updating memory maps).
5. Resume: The Sentry releases the threads, allowing them to go back to executing application code.

The Performance Cost

CPU Stunning is one of the primary sources of latency tail-end (p99) spikes in gVisor.

• "The Stun Latency": The time it takes for all threads to stop is not instantaneous. If you have a container with 100 threads, the Sentry has to wait for the 100th thread to stop before it can do its work.
• Context Switching: Forcing every thread to move from user-space to the Sentry and back creates significant overhead.
• Scaling Issues: The more CPU cores/threads a sandbox uses, the more expensive "Stunning" becomes. This is why gVisor is generally better for "many small sandboxes" rather than "one massive sandbox with 128 cores."

CPU Stunning vs. Platforms

The way a stun is performed depends on the Platform gVisor is using:

• ptrace Platform: Uses PTRACE_INTERRUPT. This is the slowest method because ptrace itself is slow.
• systrap Platform: Uses specialized signals (like SIGSYS) to force threads back into the Sentry.
• kvm Platform: Uses hardware-level VM exits. This is the fastest method because it uses the CPU’s built-in virtualization features to pause execution.

Summary

• Definition: A "Stop-the-World" pause of all application threads.
• Triggered by: Memory changes (mmap), process-wide signals, or checkpointing.
• Main Drawback: Increases latency (jitter) in high-concurrency applications.
• Best Mitigation: Use the kvm platform where possible, as hardware-accelerated "traps" are much faster than software-based ones.