gVisor - 9P

In the context of gVisor, 9P (specifically the 9P2000.L variant) is the communication protocol used between the Sentry (the sandboxed kernel) and the Gofer (the file system proxy).

What is 9P?

9P is a network filesystem protocol originally developed for the Plan 9 operating system from Bell Labs. Its core philosophy is that "everything is a file," including hardware, network connections, and processes.

It is a simple, request-response protocol where:

• The Client sends "T-messages" (e.g., "Open this file").
• The Server responds with "R-messages" (e.g., "Here is the file handle").

Why gVisor Chose 9P

gVisor's architecture is built on the principle of Defense in Depth. To keep the Sentry secure, it is strictly forbidden from talking to the host's hardware or filesystem directly. This is where 9P and the Gofer come in.

1. Security (Attack Surface Reduction)

The Sentry is wrapped in a tight seccomp filter that prevents it from making host system calls like open(), mkdir(), or unlink().

• If the Sentry needs to open /etc/passwd, it cannot call the host kernel.
• Instead, it sends a 9P message over a Unix domain socket to the Gofer.
• The Gofer (which has slightly more permissions) performs the actual open() on the host and passes the result back.
• Result: Even if a hacker compromises the Sentry, they still don't have the permission to open files on your host machine. They are limited to the 9P "vocabulary."

2. Protocol Simplicity

9P is significantly simpler than other filesystem protocols like NFS (Network File System) or SMB.

• A simpler protocol means the code used to parse it is smaller and easier to audit.
• In security software like gVisor, less code = fewer bugs = better security.

3. The Linux Dialect (9P2000.L)

While the original 9P was generic, the Linux community created a specific version called 9P2000.L.

• This version added support for Linux-specific features like specific error codes (errno), file attributes, and symlinks.
• Because gVisor aims to run Linux applications, having a 9P version that maps directly to Linux filesystem behavior made it a perfect fit.

4. Transport Agnostic

9P doesn't care how the data gets from Point A to Point B. It can run over Ethernet, serial ports, or (in gVisor's case) Unix Domain Sockets. This flexibility allowed gVisor to implement the communication between Sentry and Gofer very efficiently without needing a network stack.

The Architecture in Action

1. Application: Calls open("config.json").
2. Sentry: Intercepts the call. It sees it doesn't have the file.
3. 9P Message: Sentry sends a Twalk and Topen 9P message to the Gofer.
4. Gofer: Receives the message, verifies the Sentry is allowed to see that file, and calls the real open() on the host disk.
5. Response: Gofer sends an Ropen message back to the Sentry with a handle.
6. Application: Receives a file descriptor and continues.

Other alternatives

To understand why gVisor chose 9P, we have to look at the alternatives available when gVisor was being designed. The goal was to find a way for the Sentry (the sandbox) to access files on the host through the Gofer (the proxy) without compromising security.

1. Direct System Calls (The "No Sandbox" Approach)

The simplest way to access files is for the Sentry to just call the host kernel’s open(), read(), and write().

• Why it was rejected: This violates the core security principle of gVisor. If a hacker compromises the Sentry, they would have direct access to the host's filesystem. gVisor uses seccomp to specifically block these calls. The Sentry is "blind" to the host's files by design.

2. Virtio-fs

Virtio-fs is a modern, high-performance way to share files between a host and a virtual machine (KVM).

• Why it was rejected (initially): When gVisor was created, Virtio-fs didn't exist (it was proposed around 2019). Furthermore, Virtio-fs is tightly coupled to KVM and shared memory. gVisor wanted to be "platform-agnostic"—it needed to work just as well on ptrace (which doesn't have VM-style shared memory) as it does on KVM.
• 9P advantage: 9P is just a stream of bytes. It can run over a simple Unix domain socket, making it work on any platform.

3. FUSE (Filesystem in Userspace)

FUSE allows you to create a filesystem as a regular user-space program.

• Why it was rejected: FUSE is very complex. It requires a specific kernel module (/dev/fuse) and involves many context switches: Application $\to$ Host Kernel $\to$ FUSE Daemon $\to$ Host Kernel $\to$ Application.
• Security: Giving the Sentry access to the host's /dev/fuse device would significantly increase the "attack surface" (more ways for a hacker to trick the host kernel).

4. NFS (Network File System) or SMB

These are the standard protocols for sharing files over a network.

• Why they were rejected: They are monstrously complex. The specification for NFS or SMB is hundreds of pages long.
• The "Goldilocks" Problem: NFS/SMB handles things gVisor doesn't need, like network latency, authentication, and complex file locking across different computers. Including an NFS client in the Sentry would add thousands of lines of code, and in security, more code = more bugs.

5. Custom Shared Memory Protocol

gVisor could have invented its own protocol from day one that uses shared memory buffers to move file data.

• Why it was rejected (initially): Engineering effort. 9P was a "known quantity." It had a clear specification, was already supported by the Linux kernel (the v9fs driver), and was simple enough to implement quickly in Go. It allowed the gVisor team to focus on the "hard" parts (like the Sentry kernel logic) rather than reinventing file transport.

The "9P" Trade-off: Security vs. Speed

While 9P won on Security and Simplicity, it eventually lost on Performance.

The Problem with 9P: It is "chatty." To read a file, you have to send a message to "walk" to the directory, a message to "open" the file, a message to "read," and a message to "close." Each of these is a round-trip between the Sentry and the Gofer.

The New Alternative: LISAFS

While 9P was the original choice and is still fundamental to understanding gVisor, it has a downside: Performance. 9P is "chatty" (it requires many round-trips for simple tasks).

In recent years, the gVisor team developed a successor called LISAFS (Linux Shared File System). LISAFS is inspired by 9P but designed specifically for gVisor to reduce the number of messages sent between the Sentry and Gofer, significantly speeding up file-heavy workloads.

Feature	9P	LISAFS (The new alternative)
Origin	Plan 9 OS (Generic)	Built by Google for gVisor
Complexity	Very Low	Moderate
Performance	Slower (Many round-trips)	Much Faster (Batching & Shmem)
Security	High (Small surface)	High (Custom-fit for gVisor)

Summary: gVisor chose 9P because it was the simplest, most platform-independent, and most secure way to move file data without trusting the Sentry. Once they proved the gVisor concept worked, they replaced the "standard" 9P with their own "LISAFS" to get the speed back.