logo
CHEATSHEETSTOOLSABOUT中文

execve vs mmap

While both execve() and mmap(..., PROT_EXEC) result in executable machine code occupying a process's virtual address space, they operate at fundamentally different levels of the process lifecycle: execve() is a process-wiping transformation, whereas mmap() is an additive memory allocation.

Core Differences

Feature execve() System Call mmap(..., PROT_EXEC) System Call
Primary Purpose Replaces the current process image with a completely new binary program. Maps a file or memory region into the existing process with execute permissions.
Address Space Impact Destructive. Destroys the existing virtual address space (clears heap, stack, page tables, BSS, and mapping tables). Additive. Modifies the existing virtual address space layout without disturbing existing heap/stack/code.
Thread Context Terminates all other threads in the process; resets execution state to the new binary's entry point. Retains all existing threads, registers, stacks, and execution flows.
Process State Preserves Process ID (PID), file descriptors (unless marked FD_CLOEXEC), parent PID, and credentials. Preserves everything.
Entry Mechanism Handled by kernel binary loaders (binfmt_elf) which set up the ELF dynamic linker (ld.so), environment vectors, and stack. Requires manual handling (e.g., parsing ELF headers, relocations, dynamic linking) if loading a binary library/executable.

Execution Flow & Address Space Transformation

What execve() Does

When you call execve("/usr/bin/python3", argv, envp):

  1. Kernel Binary Handler: The kernel checks the binary format (e.g., binfmt_elf.c or script #! shebang).
  2. Address Space Flush: The kernel calls flush_old_exec(), tearing down all existing virtual memory mappings (vm_area_struct instances), user stacks, thread structures, and signal handlers.
  3. ELF Parsing & Dynamic Linker: The kernel parses the ELF file headers, loads the program segments (PT_LOAD), and sets up the interpreter path (/lib64/ld-linux-x86-64.so.2).
  4. Setup & Jump: It constructs the initial stack frame containing argc, argv, envp, and the Auxiliary Vector (auxv), then resets CPU registers and sets the Instruction Pointer (%rip) to the program/interpreter entry point.

What mmap(..., PROT_EXEC) Does

When you execute mmap(addr, len, PROT_READ | PROT_EXEC, MAP_PRIVATE, fd, offset):

  1. Memory Allocation: The kernel creates a new Virtual Memory Area (vm_area_struct) inside the current process address space marked with VM_EXEC permissions.
  2. Page Table Update: Page table entries are allocated for that address range, mapping virtual pages to page cache entries of the underlying file (or anonymous memory).
  3. Control Flow: No code is executed automatically. The CPU continues running the thread's current instruction sequence. Executing code inside the mapped region requires explicitly jumping or calling into that address (e.g., via a C function pointer cast: ((void(*)())mapped_ptr)();).

Under the Hood: execve() Uses mmap() Internally

To see how these two concepts intersect, look at how execve() constructs a process.

execve() does not copy the executable binary directly into RAM. Instead, the Linux kernel ELF loader (binfmt_elf) and the dynamic linker (ld.so) call internal kernel equivalents of mmap(..., PROT_EXEC) to map the executable's .text (code) and .rodata (read-only data) sections directly into the new address space:

Process Address Space after execve()
┌───────────────────────────────────────────────┐ 0x7FFF...
│ User Stack                                    │
├───────────────────────────────────────────────┤
│ [Mapped Libraries via mmap(..., PROT_EXEC)]   │ <─── ld.so maps libc.so here
├───────────────────────────────────────────────┤
│ Heap                                          │
├───────────────────────────────────────────────┤
│ [Executable .text via mmap(..., PROT_EXEC)]   │ <─── Kernel binfmt_elf maps app binary here
├───────────────────────────────────────────────┤
│ Reserved / Null Pointer Guard                 │ 0x0000...
└───────────────────────────────────────────────┘

Practical Use Cases

When execve() is used:

  • Spawning New Programs: Shells executing CLI utilities (ls, grep), process supervisors (systemd, supervisord), or container runtimes (runc spawning a containerized entrypoint).

When mmap(..., PROT_EXEC) is used:

  • Dynamic Library Loading (dlopen): Loading a shared object (.so plugin) at runtime. dlopen() opens the library file, maps its executable segments using mmap(..., PROT_EXEC), resolves symbols, and executes initialization routines.
  • JIT Compilers (Just-In-Time Compilation): Languages like V8 (JavaScript), JVM (Java), or PyTorch allocate memory, write generated machine code into it, change/set permissions using mmap() or mprotect() with PROT_EXEC, and jump into the compiled buffer.
  • In-Memory Executable Loaders / Sandboxing: Running embedded sub-routines or sandboxed foreign code directly without replacing the main host process.

Security Implications: W^X (Write XOR Execute)

Modern OS security policies enforce W^X (Write XOR Execute): a memory page should be writable or executable, but never both simultaneously.

  • execve() complies naturally because the kernel maps the executable code as PROT_READ | PROT_EXEC (read-only executable), while mutable data segments (.data, .bss) are mapped separately as PROT_READ | PROT_WRITE.
  • Arbitrary mmap() calls with PROT_WRITE | PROT_EXEC are blocked or flagged by modern security mitigations (SELinux, SELinux execmem, macOS Hardened Runtime) to prevent code-injection and Return-Oriented Programming (ROP) exploits. JIT engines typically map memory as PROT_READ | PROT_WRITE, write code, and then call mprotect() to swap the permission to PROT_READ | PROT_EXEC before execution.
Loading...