Linux - readv / writev

readv() and writev() are system calls for vectored I/O (also known as scatter-gather I/O). Instead of reading into or writing from a single contiguous buffer like read() and write(), they allow you to transfer data between a single file descriptor and an array of non-contiguous memory buffers (struct iovec) in a single atomic system call.

• readv() ("Scatter Input"): Reads a contiguous stream of bytes from a file or socket and scatters them sequentially across multiple separate buffers in user space (filling iov[0] completely before spilling into iov[1], and so on).
• writev() ("Gather Output"): Gathers bytes from multiple separate buffers in user space (iov[0], iov[1], iov[2]) and writes them out as a single contiguous stream to the file or socket.

The System Call Signature & struct iovec

#include <sys/uio.h>

struct iovec {
    void  *iov_base;    /* Starting address of the buffer */
    size_t iov_len;     /* Number of bytes to transfer to/from this buffer */
};

ssize_t readv(int fd, const struct iovec *iov, int iovcnt);
ssize_t writev(int fd, const struct iovec *iov, int iovcnt);

• iov: Pointer to an array of struct iovec descriptors.
• iovcnt: Number of elements in the array (on Linux, capped at UIO_MAXIOV / IOV_MAX, which is 1024).
• Return value: Total number of bytes read or written across all vectors combined (or -1 on error).

Why Not Just Use read() / write()?

Suppose you are building an HTTP server or an RPC protocol where the Header (128 bytes) lives in a stack struct, and the Payload (64 KB) lives in a separate heap or page-aligned buffer. To send both over a TCP socket without writev(), you have two flawed choices:

• Option A: Two write() calls (write(fd, hdr, 128) + write(fd, body, 65536))

  • Under the hood: Makes two separate user-to-kernel context switches, acquiring and releasing the file/socket lock twice.
  • The penalty: Doubles your syscall and CPU mitigation overhead, allows bytes from concurrent threads to interleave between your header and body, and risks triggering the TCP Nagle + Delayed ACK trap (where the first write() sends a tiny packet and the second write() stalls for 40ms–200ms waiting for an ACK if TCP_NODELAY isn't set).

• Option B: User-space memcpy() + one write() call

  • Under the hood: Allocates a temporary contiguous buffer in user space (128 + 65536 bytes), copies both the header and payload into it via memcpy(), and makes a single write() syscall.
  • The penalty: Avoids multiple syscalls and TCP fragmentation, but wastes CPU cycles, memory bandwidth, and L1/L2 cache capacity copying data from user space to user space—only for the kernel to copy that exact same data a second time into kernel page cache or socket buffers (sk_buff).

• Option C: One writev() call (writev(fd, iov, 2))

  • Under the hood: Crosses the syscall boundary once. The kernel wraps your iovec array in a struct iov_iter, locks the descriptor once, and copies directly from your separate user-space addresses into the contiguous kernel buffer or network scatterlist.
  • The Result: Zero user-space copies, a single syscall transition, atomic ordering across all buffers, and optimal TCP packet packing.

How the Linux Kernel Unifies read/write and readv/writev

Inside the modern Linux VFS layer (fs/read_write.c), the kernel no longer maintains separate driver implementations for single-buffer vs. vectored I/O:

1. Both write() and writev() wrap the user buffer(s) into a unified kernel iterator called struct iov_iter (ITER_UBUF for a single buffer, ITER_IOVEC for an array of iovecs).
2. The VFS invokes the file or socket's .write_iter() or .read_iter() method (vfs_writev → call_write_iter).
3. Atomicity Guarantee: For regular files, POSIX guarantees that readv() and writev() acquire the inode/pos lock once for the entire vector array, so concurrent threads writing to the same file descriptor will never interleave their vector segments.

Modern Variants (preadv/pwritev and preadv2/pwritev2)

Linux extends classic readv/writev with two newer syscall pairs you should know about:

• preadv(fd, iov, iovcnt, offset) / pwritev(...): Combines vectored I/O with an explicit file offset (like pread/pwrite), so it does not read or modify the file descriptor's current seek pointer—crucial for multithreaded database engines reading/writing the same file concurrently.
• preadv2(fd, iov, iovcnt, offset, flags) / pwritev2(...): Adds a per-call flags bitmask:
• RWF_NOWAIT: Returns -EAGAIN immediately if the read/write would block on disk I/O, lock contention, or page cache allocation (heavily used by async runtimes and io_uring).
• RWF_DSYNC / RWF_SYNC: Applies O_DSYNC or O_SYNC durability to just that single write call without modifying the open file description flags.
• RWF_APPEND: Performs an atomic append regardless of the offset passed.

The Classic Pitfall: Handling Partial writev() / readv()

Just like read() and write(), readv() and writev() on sockets or pipes can return a short count (fewer bytes transferred than the sum of iov_len, which might stop right in the middle of iov[1]).

If a short write occurs and you need to retry in a loop, you must advance the iov pointer past any fully written vectors and adjust iov_base + iov_len on the partially written vector before calling writev() again.