Golang - G-M-P Model

Go's concurrency model is built on goroutines—lightweight threads managed by the Go runtime rather than the operating system. To manage thousands (or millions) of goroutines efficiently across a small number of machine cores, Go uses a customized M:N multiplexing scheduler called the G-M-P model.

Core Components: G, M, and P

The model gets its name from three fundamental abstractions in the Go runtime:

+-----------------------------------------------------------------+
|                        Go Runtime                               |
|                                                                 |
|   +----+  +----+  +----+     Global Run Queue (GRQ)             |
|   | G  |  | G  |  | G  |  <-------------------------+           |
|   +----+  +----+  +----+                            |           |
|                                                     |           |
|        +-------------------+     +------------------|-----+     |
|        |    Processor (P)  |     |   Processor (P)  |     |     |
|        |                   |     |                  v     |     |
|        |  +----+  +----+   |     |  +----+  +----+  +----+|     |
|        |  | G  |  | G  |   |     |  | G  |  | G  |  | G  ||     |
|        |  +----+  +----+   |     |  +----+  +----+  +----+|     |
|        |  Local Run Queue  |     |  Local Run Queue       |     |
|        +--------+----------+     +--------+---------------+     |
|                 |                         |                     |
|                 v                         v                     |
|        +-------------------+     +--------------------+         |
|        |    Machine (M)    |     |    Machine (M)     |         |
|        +--------+----------+     +--------+-----------+         |
+-----------------|-------------------------|---------------------+
                  v                         v
           OS Thread (OS)            OS Thread (OS)

Component	Entity	What It Represents	Key Characteristics
G	Goroutine	A unit of execution	Contains the stack, current program counter (PC), and scheduling state. Uses dynamic stack sizes (starting at ~2KB) and costs almost nothing to create.
M	Machine	OS Thread	Managed directly by the OS kernel. Executes code attached to a P. By default, Go creates M threads as needed (up to a limit of 10,000).
P	Processor	Logical Resource / Context	Represents the resources required to execute Go code (such as the memory allocator cache). The count of P instances defaults to GOMAXPROCS.

Key Rule: An M (OS thread) cannot execute a G (goroutine) without holding a P (logical processor context).

Queues and Scheduling Architecture

To manage execution order, Go uses two tiers of run queues:

1. Local Run Queue (LRQ): Each P owns a lock-free, fixed-size queue (up to 256 goroutines). Because an M executing on a P accesses its local queue without locking, execution is extremely fast and cache-friendly.
2. Global Run Queue (GRQ): A single fallback queue guarded by a mutex lock. Goroutines end up here when an LRQ overflows, or during specific system operations.

How the Scheduler Stays Efficient

The G-M-P scheduler uses several smart strategies to prevent thread starvation, reduce lock contention, and keep CPU cores fully saturated:

1. Work Stealing

When an M finishes all work in its P's local run queue, it tries to find more work in this order:

1. Check the Global Run Queue (checked periodically every 61 ticks to avoid GRQ starvation).
2. Check the Network Poller for I/O-ready goroutines.
3. Steal work: It randomly picks another P and steals half of the goroutines from that P's local run queue.

2. Hand Off (Handling Blocking System Calls)

When a goroutine (G) makes a synchronous system call (e.g., waiting on disk I/O):

• The M running G enters the kernel and blocks.
• To prevent other goroutines on that P from stalling, the runtime detaches P from M.
• Another thread (M) takes over P and continues running the remaining goroutines in its local queue.
• Once the blocking system call completes, the original G is placed back on a P's local queue or the global run queue, and the idle M goes to sleep.

3. Non-blocking I/O (Network Poller)

For network I/O, Go uses asynchronous OS primitives (epoll on Linux, kqueue on macOS, IOCP on Windows) via the Network Poller:

• When G blocks on a network read/write, G is detached from M and moved to the Network Poller.
• M immediately picks up another G from P's queue without blocking the OS thread.
• Once the socket becomes ready, the Network Poller wakes G up and puts it back into a P's run queue.

4. Preemption (Cooperativeness to Signal-based)

• Pre-Go 1.14 (Cooperative): Preemption occurred at function call checks (stack allocation checks). Infinite loops without function calls could starve the CPU.
• Go 1.14+ (Asynchronous Signal-Based): The runtime uses OS signals (SIGURG) to preempt long-running goroutines asynchronously after ~10ms of continuous execution, ensuring tight loops don't hog threads.

g0

In Go's runtime, g0 (or the g0 goroutine) is a special system goroutine allocated for every OS thread (M). Unlike standard user goroutines, g0 does not execute user code; it executes internal runtime management tasks such as scheduling, stack allocation, garbage collection, and deferred functions.

Why Is g0 Needed?

Standard user goroutines start with a very small, dynamic stack (typically starting around 2 KB) that grows and shrinks as needed.

If runtime operations—like allocating memory for a new stack, triggering garbage collection, or running the scheduler—were executed on a user goroutine's small stack, they could easily cause a stack overflow.

To prevent this, Go switches to g0's dedicated stack whenever the runtime needs to perform administrative work.

Core Characteristics of g0

• One per M (OS Thread): Every OS thread managed by Go has its own dedicated g0 instance.
• Fixed, Large Stack: The g0 stack is allocated directly using system memory (usually around 8 KB on Linux/Unix systems, or larger depending on the platform). Its stack does not grow or shrink dynamically.
• System Call Stack (gsignal): Beside g0, each M also maintains gsignal for handling OS signals, which operates similarly with its own isolated stack.

How Stack Switching to g0 Works

When a goroutine (G) needs to perform a runtime task (e.g., calling runtime.Gosched(), creating a new goroutine via go func(), or allocating a channel), the CPU switches execution context from the user goroutine to g0:

 [User Goroutine: G]  ----(Needs scheduling / GC)---->  [System Goroutine: g0]
  - Small stack (~2KB)                                   - Fixed OS stack (~8KB)
  - Runs user code                                       - Runs runtime.schedule()
                                                                  |
                                                                  v
 [User Goroutine: G2] <---(Switches context to next G)------------+
  - Resumes execution

1. Save State: The CPU registers and Program Counter (PC) of the current user goroutine G are saved in its gbuf structure.
2. Switch Stack Pointer: The SP (Stack Pointer) register is switched to g0's stack address (m.g0.stack).
3. Execute Runtime Code: The runtime function (such as mcall() or systemstack()) executes on the g0 stack.
4. Switch Back: Once the runtime task finishes, the SP is set back to the next target user goroutine's stack, and execution resumes.

Primary Tasks Executed on g0

Category	Specific Functions
Scheduling	Executing schedule(), context switching between user goroutines, work stealing, and handling goroutine park/unpark operations.
Stack Management	Allocating new stacks, re-allocating/growing user stacks (morestack), and copying stack frames.
Garbage Collection	STW (Stop-The-World) phases, mark/sweep setup, and GC coordination tasks.
Goroutine Creation	Allocating and initializing new G structures (newproc).

gsignal

gsignal is a distinct g struct instance from g0.

Both g0 and gsignal are special, non-user goroutines tied directly to an OS thread (M), but they serve entirely different roles and use separate stacks.

g0 vs. gsignal Comparison

Feature	g0	gsignal
Primary Purpose	Internal runtime tasks (scheduling, stack growth, GC)	Handling OS signals (SIGSEGV, SIGALRM, SIGURG, etc.)
Trigger Mechanism	Explicit runtime calls (e.g., systemstack, mcall, newproc)	Asynchronous OS hardware/software signal interrupts
Stack Allocation	Allocated on the heap/OS stack during M creation (~8 KB)	Alternate signal stack set via OS system call (sigaltstack)
Count	Exactly 1 per M	Exactly 1 per M
User Goroutine Interruption	Cooperatively/explicitly yields control to g0	Involuntarily interrupts whatever thread/routine is running

Why gsignal Needs Its Own Stack

When an OS signal arrives (such as a segmentation fault SIGSEGV or an asynchronous preemption signal SIGURG), the OS kernel interrupts the running thread and invokes Go's signal handler.

Go cannot process signals on normal stacks for two critical reasons:

1. User Stack Overflow: If a user goroutine triggers a signal due to stack overflow, attempting to run the signal handler on that same exhausted stack will instantly crash the process with an unrecoverable double-fault.
2. g0 Nesting Constraints: If a signal arrives while the thread is already executing runtime code inside g0, executing the signal handler directly on g0's stack could corrupt g0's state or overflow its fixed allocation.

To handle this safely, Go calls the Unix system call sigaltstack() during thread initialization (M startup). This tells the OS kernel: "Whenever a signal interrupts this thread, switch the CPU stack pointer to m.gsignal.stack before invoking our signal handler."

How They Work Together (e.g., Preemption via SIGURG)

Since Go 1.14, asynchronous preemption relies on both gsignal and g0 working in tandem:

[User Goroutine G1]
        |
        v  (OS interrupts with SIGURG signal)
[gsignal Stack]  --> Executes sighandler(), detects preemption request,
        |            modifies G1's register state to inject a call to asyncPreempt()
        v  (Signal handler returns, resumes G1 at asyncPreempt)
[User Goroutine G1] --> Calls asyncPreempt() -> Calls mcall()
        |
        v  (Switches stack)
[g0 Stack]       --> Executes schedule(), places G1 on local run queue,
                     and context-switches to next available G2