Linux syscalls - ftruncate

SYS_ftruncate (system call number 92 on x86_64) is the kernel-space ABI entry point where execution control transitions from unprivileged Ring 3 (userspace) into Ring 0 (kernel space) to modify a file's registered size.

Userspace (Ring 3)                          Kernel Space (Ring 0)
──────────────────                          ─────────────────────
ftruncate(fd, len)
   │
   ├──> libc wrapper
   │       │
   │       └──> syscall 92 ───────TRAP───────> sys_ftruncate(fd, length)
                                                      │
                                                      ├──> fget() [VFS FD Lookup]
                                                      └──> do_ftruncate()
                                                              └──> inode->i_op->truncate

The Kernel Signature & Macro Entry Point

In the Linux kernel source (under fs/open.c), SYS_ftruncate is defined using the kernel's system call definition macros:

SYSCALL_DEFINE2(ftruncate, unsigned int, fd, unsigned long, length)
{
    return do_sys_ftruncate(fd, length, 1);
}

Entry Assembly: When a CPU executes syscall with %rax = 92, the kernel's system call dispatcher traps the interrupt and routes arguments directly from registers (%rdi for fd, %rsi for length) into sys_ftruncate.
32-bit Compatibility (SYS_ftruncate64): On 32-bit architectures where a single 32-bit register cannot hold a 64-bit off_t, the kernel exposes SYS_ftruncate64 (taking the 64-bit length split across two register arguments).

What `sys_ftruncate` Does Inside the Kernel

When SYS_ftruncate handles the trap, the kernel executes a precise validation and VFS (Virtual File System) execution sequence:

Step 1: File Descriptor & Permission Lookup (`fget`)

The kernel inspects the calling process's file descriptor table (current->files):

Validates fd: If the integer doesn't map to an active struct file, it instantly aborts and returns -EBADF.
Checks Write Mode: The kernel verifies file->f_mode & FMODE_WRITE. If the descriptor was opened read-only (O_RDONLY), it returns -EBADF or -EINVAL.
Type & Seal Validation: Verifies the file is a regular file or shared memory object (not a directory or socket, returning -EISDIR or -EINVAL). It also checks file seals (e.g., F_SEAL_GROW or F_SEAL_SHRINK set via memfd_create/fcntl).

Step 2: VFS Inode Operations (`do_ftruncate`)

Once validated, the kernel delegates the operation to the Virtual File System layer:

Shrinking (length < inode->i_size):
- The VFS invokes truncate_pagecache() to unmap and drop cached memory pages beyond length from the page cache.
- It calls the filesystem-specific driver function (inode->i_op->truncate or i_op->setattr for ext4, btrfs, xfs, etc.).
- The filesystem releases the unneeded disk blocks back to its free-block bitmap.
Extending (length > inode->i_size):
- The kernel updates the inode's metadata (inode->i_size = length).
- On modern block filesystems, no physical blocks are allocated immediately. The kernel marks the new region as a hole, turning the file into a sparse file. Reading from this range returns zeroes directly from memory without triggering block I/O.

Step 3: Metadata & Timestamp Updates

Before returning, sys_ftruncate updates the inode’s timestamps (mtime and ctime) and clears SUID/SGID bits if necessary.

Kernel Error Returns vs. Userspace `errno`

The SYS_ftruncate kernel function does not set errno. Instead, it returns negative integer error codes directly to the CPU's %rax register. The userspace C library (glibc or musl) intercepts this negative value, copies the positive equivalent into errno, and returns -1 to the user program:

Kernel Return Value (`sys_ftruncate`)	C Library Action	Meaning to Process
`0`	Returns `0`	Success.
`-EBADF`	Sets `errno = EBADF`, returns `-1`	Invalid or non-writable file descriptor.
`-EINVAL`	Sets `errno = EINVAL`, returns `-1`	`length` is negative, or target is an invalid type.
`-EFBIG`	Sets `errno = EFBIG`, returns `-1`	`length` exceeds process `RLIMIT_FSIZE` or maximum filesystem size.
`-EPERM`	Sets `errno = EPERM`, returns `-1`	Operation blocked by file seal or read-only filesystem.

`ftruncate()` vs. `SYS_ftruncate`

ftruncate() is a POSIX C library wrapper declared in <unistd.h> that abstracts OS differences.
SYS_ftruncate is the Linux kernel syscall number (Assembly vector).
sys_ftruncate() is the C function inside the Linux kernel source that handles the trap, checks FMODE_WRITE, and calls the underlying storage driver to adjust file metadata and disk blocks.

Can ftruncate expand a file?

Yes! Despite the name ("truncate" historically meant only cutting a file shorter in early Unix, specifically 4.2BSD where it was introduced), modern ftruncate sets the file's size to any exact length:

If length < current_size: it shrinks the file and discards the extra bytes.
If length > current_size: it extends the file's logical size (i_size on the inode), and any byte between the old size and the new length is logically zero-filled (implemented as a sparse hole on filesystems that support sparse files, including tmpfs/memfd).