epoll vs io_uring

For nearly two decades, epoll has been the undisputed bedrock of high-performance Linux network servers, powering engines like Nginx, Node.js, Redis, and Netty. However, the introduction of io_uring in Linux kernel 5.1 (authored by Jens Axboe) fundamentally transformed Linux asynchronous I/O.

While both interfaces allow an application to multiplex I/O across thousands of file descriptors, their core architectural paradigms, system call costs, and device support are drastically different.

Readiness Notification vs. Completion-Based I/O

The primary philosophical divergence between epoll and io_uring lies in readiness notification versus completion-based notification:

• epoll is Readiness-Based: It alerts your application when a file descriptor is ready to perform an operation without blocking (e.g., "Socket FD 14 has data available to read"). The application must then execute a subsequent synchronous system call (read(), recv()) to actually move the bytes.
• io_uring is Completion-Based: Your application instructs the kernel what operation to perform in advance (e.g., "Read 4KB from Socket FD 14 into buffer pointer 0x7fff00"). The kernel executes the operation asynchronously and alerts your application only when the transfer has already completed.

Architecture & Mechanics

How epoll Works

An epoll instance is maintained in the kernel using a red-black tree (epitem) to index monitored file descriptors, and a doubly-linked ready list to collect triggered events.

1. Setup: The application registers file descriptors via epoll_ctl(epfd, EPOLL_CTL_ADD, fd, &event).
2. Wait: The application enters the kernel via epoll_wait(), blocking until events arrive or a timeout occurs.
3. Execution: For each ready event returned, the application must issue separate system calls (read(), write(), accept()) to process the payload.
4. Context Switching Overhead: In high-throughput scenarios processing hundreds of thousands of requests per second, the constant CPU context switching between userspace and kernel space (epoll_wait -> read -> write -> epoll_wait) creates significant CPU instruction cache pollution and system call overhead.

How io_uring Works

io_uring completely bypasses recurring system calls by sharing two circular ring buffers between userspace and the kernel via a single mmap() call:

1. Submission Queue (SQ): A lockless ring buffer where the userspace application writes Submission Queue Entries (SQEs). Each SQE describes an operation: opcode (IORING_OP_READ, IORING_OP_WRITE, IORING_OP_ACCEPT, etc.), target file descriptor, memory buffer address, offset, and user data token.
2. Completion Queue (CQ): A lockless ring buffer where the kernel writes Completion Queue Events (CQEs) as operations finish. Each CQE contains the return status (e.g., number of bytes read or error code) and the user data token passed in the SQE.
3. Kernel Polling Mode (IORING_SETUP_SQPOLL): When enabled, a dedicated kernel thread polls the Submission Queue. The application simply appends SQEs and advances the tail pointer using atomic memory operations—zero system calls are issued during steady-state request processing.

The Regular File I/O Dilemma

One of the biggest historical flaws of epoll is its inability to handle regular disk files asynchronously:

• In Linux, regular files on disk are always considered "ready" by epoll because they do not have the same event notification hooks as sockets or pipes.
• As a result, calling read() or write() on a file descriptor monitored by epoll will still block the calling thread whenever the requested page is not in the Linux Page Cache (requiring synchronous disk read from NVMe/SSD).
• High-performance web servers (such as Nginx) historically had to maintain separate background thread pools (aio_threads) just to prevent disk I/O from stalling the main event loop.

io_uring natively supports all I/O types under a single, unified interface: regular files, block devices, sockets, epoll file descriptors, and even timer operations. Disk reads never block the application thread; the kernel schedules them asynchronously against the underlying storage driver.

Key Comparison Table

Feature	epoll	io_uring
Model	Readiness notification	Asynchronous completion
Linux Kernel Version	Linux 2.5.44+ (2002)	Linux 5.1+ (2019)
Data Exchange	System calls (epoll_ctl, epoll_wait)	Shared mmap Ring Buffers (SQ and CQ)
Syscall Cost	Multiple syscalls per event	Can achieve 0 syscalls via IORING_SETUP_SQPOLL
Regular File / Disk I/O	❌ Synchronous / Blocks thread	✅ Fully asynchronous
Network Socket I/O	✅ Excellent	✅ Excellent
Registered / Fixed Buffers	❌ No (buffer pinned per call)	✅ Yes (IORING_REGISTER_BUFFERS)
Operation Chaining	❌ Manual userspace coordination	✅ Supported (IOSQE_IO_LINK)
Security Surface	Mature, low attack surface	Broader kernel attack surface (recent CVEs)
Adoption Complexity	Simple, ubiquitous library support	Higher (liburing or raw ring buffer management)

Advanced io_uring Features

Fixed Buffers (IORING_REGISTER_BUFFERS)

With traditional system calls, the kernel must map and pin userspace memory pages into kernel address space for every single read or write, then unpin them when finished.

With io_uring, you can pre-register a pool of memory buffers with io_uring_register(). The kernel maps the pages once and keeps them pinned, eliminating memory mapping overhead on hot I/O paths.

Fixed File Descriptors (IORING_REGISTER_FILES)

Similarly, retrieving and checking file descriptor references in the process file table (struct files_struct) incurs atomic reference count increments on every system call. By registering an array of file descriptors upfront, io_uring accesses files by direct index without atomic descriptor table overhead.

Linked Operations (IOSQE_IO_LINK)

io_uring allows you to chain dependent operations directly in the Submission Queue. For example, you can submit an OPENAT SQE linked to a READ SQE, followed by a CLOSE SQE. The kernel executes them in sequence and aborts the chain if any step fails, all without returning control to userspace between steps.

When to Use Which?

Stick with epoll when:

• Broad Kernel Compatibility: Your workloads run on older enterprise Linux kernels (CentOS 7/8, RHEL 7/8, Ubuntu 18.04) where io_uring is either absent or incomplete.
• Strict Security Baselines: In multi-tenant environments with hardened seccomp profiles, io_uring may be disabled or restricted due to kernel vulnerabilities identified in earlier 5.x releases.
• Pure Network Proxying: For standard reverse proxies where network sockets are the sole bottleneck and workloads are adequately served by epoll-based frameworks.

Migrate to io_uring when:

• Heavy Disk I/O or Mixed Workloads: Applications like databases, object stores, and storage engines that read and write gigabytes of file data alongside network traffic.
• Extreme Low-Latency & High IOPS: When pushing millions of IOPS on high-performance NVMe drives or 100GbE+ networks where system call transitions consume measurable CPU cycles.
• Next-Generation Runtime Engines: Modern projects such as Tokio (Rust), Seastar (C++), and Netty (Java via native transport) increasingly leverage io_uring for peak performance.