Operating Systems (OS)

An Operating System (OS) is the software layer that mediates between application software and physical hardware. It is responsible for process scheduling, virtual memory management, hardware device abstractions, inter-process communication (IPC), and security isolation.

Operating System Case Studies

Explore comparative architectural analyses of modern operating systems, examining how different kernels solve concurrency, IPC, memory safety, and hardware constraints under the hood:

  • Unix Architecture & Design Case Study: The foundational abstractions that shaped modern computing—byte streams, uniform file descriptors (int fd), composable pipes, and the fork-exec process model.
  • Linux Architecture: The cloud and enterprise standard—monolithic kernel design, completely fair scheduling (CFS/EEVDF), namespaces, cgroups, virtual memory, and eBPF.
  • macOS & Darwin (XNU) Case Study: The XNU hybrid kernel fusing Mach 3.0 microkernel primitives with a FreeBSD POSIX personality, Mach ports, XPC, compressed memory, and Signed System Volumes (SSV).
  • Windows NT Architecture Case Study: David Cutler's hybrid NT architecture—object-oriented executive, handle tables, asynchronous I/O Request Packets (IRPs), I/O Completion Ports (IOCP), and Hyper-V/WSL2.
  • Android OS Architecture Case Study: Specialized mobile-first Linux kernel offshoot evolving to desktop—Generic Kernel Image (GKI), Binder single-copy IPC, ZRAM/lmkd memory pressure management, per-app UID sandboxing, and Protected KVM (pKVM/AVF).
  • Real-Time Operating Systems (RTOS) Case Study: Deterministic scheduling algorithms (FPPS, RMS, EDF), resolving the Mars Pathfinder priority inversion crisis, zero-jitter memory pools, and case studies of FreeRTOS and QNX Neutrino.

Cross-Cutting OS Subsystems

Explore the fundamental engineering mechanisms and hardware abstractions that span modern operating systems:

  • Booting & Firmware: Firmware handoff (BIOS vs UEFI), Master Boot Record (MBR) vs GUID Partition Table (GPT), secure boot chains, and kernel initialization (systemd, launchd).
  • Virtual Memory & Paging: Hardware MMU translation, 4-level/5-level page tables, TLB shootdowns, PCID/ASID context caching, demand paging, Copy-on-Write (CoW), and KPTI side-channel isolation.
  • Process Scheduling & CPU Virtualization: Preemption, context switch cache penalties, Linux CFS/EEVDF virtual runtime, Windows dynamic priority boosting, macOS QoS asymmetric core scheduling, and RTOS deadline determinism.
  • System Calls & Interrupt Handling: Hardware interrupt routing (APIC/MSI-X), top-half vs bottom-half deferral, fast syscall entry (SYSCALL/SYSRET, SVC), vDSO user-mode acceleration, and io_uring.
  • Inter-Process Communication (IPC): Double-copy pipes vs single-copy Binder vs zero-copy shared memory (memfd_create), SCM_RIGHTS file descriptor passing, Mach ports, ALPC, and futex fast userspace locking.
  • File Systems: Inode structures, directory entries, journaling vs Copy-on-Write (CoW), block allocation, and RAM filesystems (ramfs vs tmpfs).
  • FUSE (Filesystem in Userspace): Architecture, /dev/fuse kernel-to-userspace protocol loops, context switching overhead, and building custom virtual filesystems.

Kernel Archetypes: Monolithic vs Microkernel vs Hybrid

Archetype Core Concept Ring 0 Kernel Responsibilities Advantages Disadvantages Production Examples
Monolithic All core subsystems run within a single shared address space. Scheduling, Memory, VFS, Network, Device Drivers, Filesystems. Maximum execution throughput; zero IPC context-switch penalty. Driver crashes can panic the entire system; large attack surface. Linux, FreeBSD, OpenBSD.
Microkernel Minimizes Ring 0 code to foundational primitives; runs drivers in user mode. Thread scheduling, inter-process communication (IPC), address space management. Extreme fault isolation; crashed drivers restart without OS reboot. IPC message-passing latency and context-switching overhead. QNX Neutrino, Minix, seL4.
Hybrid Combines microkernel abstraction boundaries with monolithic kernel-space execution. Microkernel core combined with in-kernel performance subsystems (e.g. graphics/VFS). Balances microkernel modularity with monolithic graphics and driver speed. Complexity; bugs in subsystems still impact kernel stability. Windows NT, macOS / iOS (XNU).
UNIX Classic
Sponsored
The Design of the UNIX Operating System
Kernel Architecture
The Design of the UNIX Operating System
By Maurice J. BachView on Amazon