Linux - struct msghdr vs. struct cmsghdr

In Linux socket programming, when developers transition from basic read() and write() calls to advanced socket messaging via sendmsg() and recvmsg(), they immediately encounter two similarly named, closely coupled C data structures: struct msghdr and struct cmsghdr.

While both are used together to transmit data and metadata across sockets, they operate at fundamentally different structural levels:

┌────────────────────────────────────────────────────────────────────────┐
│ struct msghdr (Outer Envelope)                                         │
│ Passed directly as an argument to sendmsg() or recvmsg().              │
│                                                                        │
│ ├── msg_name    ──► Destination/Source Address (struct sockaddr)       │
│ ├── msg_iov     ──► Normal Payload Data (struct iovec array)           │
│ └── msg_control ──► Points to raw memory buffer containing:            │
│                     ┌────────────────────────────────────────────┐     │
│                     │ struct cmsghdr #1 (First Certified Item)   │     │
│                     │ ├── cmsg_len   = 20 bytes                  │     │
│                     │ ├── cmsg_level = SOL_SOCKET                │     │
│                     │ ├── cmsg_type  = SCM_RIGHTS                │     │
│                     │ └── CMSG_DATA: [ int file_descriptor = 4 ] │     │
│                     ├────────────────────────────────────────────┤     │
│                     │ struct cmsghdr #2 (Second Certified Item)  │     │
│                     │ ├── cmsg_len   = 28 bytes                  │     │
│                     │ ├── cmsg_level = SOL_SOCKET                │     │
│                     │ ├── cmsg_type  = SCM_CREDENTIALS           │     │
│                     │ └── CMSG_DATA: [ pid, uid, gid ]           │     │
│                     └────────────────────────────────────────────┘     │
└────────────────────────────────────────────────────────────────────────┘

• struct msghdr is the Outer Envelope: It describes the entire transmission package (recipient address, scatter-gather data buffers, and transmission flags). You pass a pointer to a single struct msghdr directly to the sendmsg() or recvmsg() system call.
• struct cmsghdr is an Enclosed Control Item: It describes an individual piece of ancillary control data (such as passing a file descriptor, verifying process credentials, or reading packet timestamps). It never touches a system call directly; instead, one or more struct cmsghdr blocks live packed inside the buffer pointed to by msghdr.msg_control.

Structural Anatomy

struct msghdr: The Top-Level Message Descriptor

Defined in <sys/socket.h>:

struct msghdr {
    void         *msg_name;       /* Optional address (struct sockaddr *) */
    socklen_t     msg_namelen;    /* Size of address */
    struct iovec *msg_iov;        /* Scatter-gather array (payload data) */
    size_t        msg_iovlen;     /* Number of elements in msg_iov */
    void         *msg_control;    /* Pointer to ancillary data buffer */
    size_t        msg_controllen; /* Length of ancillary data buffer */
    int           msg_flags;      /* Flags on received message (e.g. MSG_TRUNC) */
};

• msg_name / msg_namelen: Specifies the target peer for connectionless datagrams (sendto style), or stores the sender address on recvmsg(). For connected stream sockets, set these to NULL and 0.
• msg_iov / msg_iovlen: An array of struct iovec pointers enabling vectored I/O (sending headers and bodies from separate memory regions in one syscall without user-space copying).
• msg_control / msg_controllen: Points to a contiguous byte buffer allocated in user space where one or more struct cmsghdr records are packed.
• msg_flags: Output flags populated by the kernel upon return from recvmsg() (e.g., MSG_EOR, MSG_TRUNC, MSG_CTRUNC).

struct cmsghdr: The Ancillary Data Header

Also defined in <sys/socket.h>:

struct cmsghdr {
    size_t cmsg_len;    /* Data byte count, including this header */
    int    cmsg_level;  /* Originating protocol layer (e.g., SOL_SOCKET, IPPROTO_IP) */
    int    cmsg_type;   /* Protocol-specific option type (e.g., SCM_RIGHTS) */
    /* Flexible array / payload data follows immediately after this header */
};

• cmsg_len: The total byte length of the control message, which equals sizeof(struct cmsghdr) plus the length of the payload data. (Crucial: this length does not include trailing padding bytes required for alignment).
• cmsg_level: Specifies the protocol tier responsible for this control message:
  • SOL_SOCKET: Socket-level control (file descriptors, credentials, timestamps).
  • IPPROTO_IP / IPPROTO_IPV6: IP-level metadata (packet interface info IP_PKTINFO, hop limits).
• cmsg_type: The specific capability or telemetry being transferred:
  • SCM_RIGHTS: Clones an open file descriptor into the receiving process's table.
  • SCM_CREDENTIALS: Transmits kernel-verified PID, UID, and GID (struct ucred).
  • SO_TIMESTAMP / SO_TIMESTAMPING: Hardware or kernel packet reception timestamps.

The Kernel's View: Opaque Payload vs. Structured Control Plane

To understand why POSIX and the Linux kernel separate msg_iov from msg_name and msg_control inside struct msghdr, look at how the kernel treats each component internally:

Kernel Processing: Opaque Stream vs. Structured Control Plane

       ┌────────────────────────────────────────────────────────┐
       │ struct msghdr                                          │
       │                                                        │
       │ ├── msg_iov (The Payload: HTTP, JSON, Binary Blobs)    │
       │     ► Completely OPAQUE to the kernel socket layer     │
       │     ► Kernel treats it as an uninterpreted byte stream │
       │     ► Copied verbatim into sk_buff / network frames    │
       │                                                        │
       │ ├── msg_name (The Envelope Address: IP/Port / Path)    │
       │     ► STRUCTURED and parsed by kernel network stack    │
       │     ► Triggers routing table lookups and ACL checks    │
       │                                                        │
       │ └── msg_control (Ancillary cmsg Headers: SCM_RIGHTS)   │
       │     ► DEEPLY UNDERSTOOD and executed by the kernel     │
       │     ► Mutates kernel data structures (refcounts, FDs)  │
       │     ► Injects authenticated process credentials        │
       └────────────────────────────────────────────────────────┘

msg_iov is Opaque to the Socket Layer

The memory buffers pointed to by msg_iov contain the actual message content/payload. To the Linux kernel socket layer, this data is completely opaque:

• The kernel does not inspect, parse, validate, or restrict what is inside msg_iov. Whether your buffers contain HTTP/2 frames, a Protobuf payload, encrypted TLS records, or random binary garbage, the socket layer treats it strictly as an uninterpreted byte stream.
• The kernel's only job with msg_iov is transport: it uses scatter-gather routines (iov_iter) to copy the bytes from user-space memory directly into kernel socket buffers (sk_buff), generates transport framing (TCP headers, checksums) if needed, and transmits the bytes over the wire or local IPC queue.

msg_name and msg_control are Structured and Understood by the Kernel

In stark contrast, msg_name (the envelope address) and msg_control (ancillary control messages) are not opaque data. They represent the control plane of the system call, and the kernel actively parses, verifies, and executes privileged logic based on them:

• msg_name: The kernel casts this memory to the corresponding struct sockaddr family (sockaddr_in, sockaddr_in6, sockaddr_un), validates destination routing, checks firewall rules, and determines which socket endpoint or network interface will handle the traffic.
• msg_control: The kernel steps through each struct cmsghdr using internal alignment helpers. When it encounters known types, it mutates operating system state:
  • SCM_RIGHTS: The kernel does not simply copy integer bytes across memory. It looks up each integer file descriptor in the sender's private files_struct, verifies that it points to a valid open file, increments the underlying struct file reference count, allocates an available slot in the receiving process's descriptor table, and links the handle.
  • SCM_CREDENTIALS: The kernel actively rejects user-space spoofing. It extracts the caller's actual, verified PID, UID, and GID directly from the task's task_struct and writes this certified identity into the receiver's control buffer.
  • SO_TIMESTAMP: The kernel reads the hardware or network subsystem packet arrival clock and writes the nanosecond timestamp descriptor.

Why You Cannot Use Direct Pointer Arithmetic

A critical pitfall in C socket programming is attempting to allocate or navigate cmsghdr records using standard sizeof() and pointer addition:

// BUGGY: Violates CPU word alignment! Will crash or corrupt on 64-bit architectures!
struct cmsghdr *cmsg = (struct cmsghdr *)buffer;
int *fd_ptr = (int *)(cmsg + 1); // WRONG

On 64-bit systems, CPU hardware architectures require structure headers and payloads to be aligned on 8-byte boundaries. If a control payload is 4 bytes (like an integer fd), the next cmsghdr cannot start immediately at byte offset 4; it must be padded with 4 silent bytes.

To guarantee portable, architecture-safe alignment, POSIX provides five mandatory CMSG Macros:

CMSG_SPACE() vs. CMSG_LEN() Memory Alignment:

┌───────────────────┬───────────────────┬───────────────────────┐
│ struct cmsghdr    │ Payload Data      │ Silent Padding Bytes  │
│ (Header, 16 bytes)│ (e.g. int fd = 4) │ (Aligned to 8 bytes)  │
└───────────────────┴───────────────────┴───────────────────────┘
◄───────────────── CMSG_LEN(4) ────────►
◄─────────────────────────── CMSG_SPACE(4) ────────────────────►

Macro	Purpose	Formula / Behavior
CMSG_SPACE(length)	Calculates total buffer size needed to store the header, payload, and alignment padding.	Used to size the char msg_control[...] buffer.
CMSG_LEN(length)	Calculates the value to assign to cmsg->cmsg_len.	Header size + payload size (excludes trailing padding).
CMSG_DATA(cmsg)	Returns an aligned unsigned char * pointer to the first byte of payload data.	Used to cast and write/read the payload (e.g. *(int *)CMSG_DATA(cmsg)).
CMSG_FIRSTHDR(&msg)	Returns a pointer to the first struct cmsghdr inside msg.msg_control.	Returns NULL if msg_control is empty or invalid.
CMSG_NXTHDR(&msg, cmsg)	Steps forward to the next struct cmsghdr in the chain.	Handles internal padding automatically; returns NULL at the end.

Practical Code Example: Passing an FD via SCM_RIGHTS

Here is how struct msghdr and struct cmsghdr coordinate in real production code:

#include <sys/socket.h>
#include <fcntl.h>
#include <unistd.h>
#include <string.h>

int send_file_descriptor(int unix_socket_fd, int fd_to_send) {
    // 1. msghdr requires at least 1 byte of normal payload to trigger reception
    char dummy_byte = 'D';
    struct iovec iov = {
        .iov_base = &dummy_byte,
        .iov_len  = 1,
    };

    // 2. Allocate the control buffer using CMSG_SPACE to ensure padding space
    char control_buf[CMSG_SPACE(sizeof(int))];
    memset(control_buf, 0, sizeof(control_buf));

    // 3. Populate the outer envelope (struct msghdr)
    struct msghdr msg = {
        .msg_name       = NULL,
        .msg_namelen    = 0,
        .msg_iov        = &iov,
        .msg_iovlen     = 1,
        .msg_control    = control_buf,
        .msg_controllen = sizeof(control_buf),
    };

    // 4. Populate the inner control item (struct cmsghdr) using macros
    struct cmsghdr *cmsg = CMSG_FIRSTHDR(&msg);
    cmsg->cmsg_level = SOL_SOCKET;
    cmsg->cmsg_type  = SCM_RIGHTS;
    cmsg->cmsg_len   = CMSG_LEN(sizeof(int));

    // 5. Inject the raw file descriptor into the aligned data area
    *((int *)CMSG_DATA(cmsg)) = fd_to_send;

    // 6. Transmit the outer envelope
    return sendmsg(unix_socket_fd, &msg, 0);
}

When the receiving process calls recvmsg() with an identically sized buffer, the Linux kernel allocates a new file descriptor in the receiver's file table pointing to the sender's existing Open File Description (struct file), returning the new descriptor integer inside CMSG_DATA(cmsg).

Comparison Matrix

Feature	struct msghdr	struct cmsghdr
Primary Responsibility	Outer message wrapper for the entire I/O operation.	Header for an individual out-of-band metadata record.
Syscall Usage	Passed directly to sendmsg(fd, &msg, flags) and recvmsg().	Embedded inside msg_control; never passed directly to syscalls.
Quantity per Operation	Exactly one per syscall.	Zero, one, or multiple chained sequentially in memory.
Key Members	msg_name, msg_iov, msg_control, msg_flags.	cmsg_len, cmsg_level, cmsg_type.
Payload Storage	Points to regular user buffers (msg_iov).	Follows immediately after the header in memory (CMSG_DATA).
Traversing Technique	Standard C structure field access.	Must use CMSG_FIRSTHDR and CMSG_NXTHDR macros.
Typical Use Cases	Sending an HTTP request, batching headers and body.	Passing file descriptors (SCM_RIGHTS), PID authentication (SCM_CREDENTIALS).

Frequently Asked Questions (FAQ)

Can a single struct msghdr carry multiple struct cmsghdr control messages?

Yes. A single struct msghdr can pack multiple struct cmsghdr records contiguously inside its msg_control buffer. For instance, an application can pass a file descriptor (SCM_RIGHTS) and process credentials (SCM_CREDENTIALS) simultaneously in the same sendmsg() call. The receiver steps through each record using CMSG_FIRSTHDR(&msg) and CMSG_NXTHDR(&msg, cmsg) until NULL is returned.

Why is CMSG_SPACE() larger than CMSG_LEN()?

CMSG_SPACE() includes the trailing padding bytes required to align the start of the next cmsghdr structure on an architecture-specific boundary, whereas CMSG_LEN() only counts the header and payload. On 64-bit architectures, struct cmsghdr must be aligned to 8-byte boundaries. If the payload length is 4 bytes (such as an int file descriptor), CMSG_LEN(4) is 20 bytes (16 bytes header + 4 bytes payload), while CMSG_SPACE(4) rounds this up to 24 bytes. Always use CMSG_SPACE() when sizing msg_control buffers and CMSG_LEN() when populating cmsg->cmsg_len.

Does passing a file descriptor via SCM_RIGHTS clone the underlying struct file?

No. SCM_RIGHTS does not clone struct file; both sender and receiver processes share the exact same Open File Description. In net/core/scm.c, the Linux kernel calls get_file() to increment f_count on the sender's existing struct file and calls fd_install() in the receiver to bind a newly allocated integer descriptor to it. Consequently, both processes share the same file offset cursor (f_pos) and file status flags (O_NONBLOCK, O_APPEND), while descriptor flags (FD_CLOEXEC) remain private to each process.

Related Guides

• Linux - Socket I/O Syscalls (send, recv, sendmsg, recvmsg): Detailed flag semantics and multi-packet batching with recvmmsg.
• Linux - Socket Control Messages (SCM): Deep dive into descriptor sharing, zero-trust security, and nanosecond timestamping.
• Linux - Sockets Overview: Architectural comparison of Unix domain sockets vs. TCP/IP network sockets.
• Linux - Inode vs. File Descriptor: Understanding how file handles exist across process tables.