logo

What is cgroup?

  • cgroup = control group.
  • System for resource management on Linux.
  • Directory hierarchy at /sys/fs/cgroup, called the root control group. This root cgroup is the cgroup to which all processes belong.
  • Limit, throttle, and account for resource usage per control group.
  • Each resource interface is provided by a controller.
  • Used to constrain resources that are allocated to processes.

When systemd is chosen as the init system for a Linux distribution, the init process generates and consumes a root control group (cgroup) and acts as a cgroup manager. systemd has a tight integration with cgroups and allocates a cgroup per systemd unit.

cgroup v1 vs cgroup v2

cgroupv1 has a hierarchy per-resource (resource = cpu, devices, memory, pids, etc):

  • Separate hierarchy/cgroups for each resource.
  • cgroups can be nested inside each other.
  • Limits and accounting are performed per-cgroup.
  • One PID is in exactly one cgroup per resource.
/sys/fs/cgroup => resource => cgroup => pid

cgroupv2 has a unified hierarchy, Each cgroup can support multiple resource domains.

  • cgroups are "global": not limited to one resource.
  • Resources are now opt-in for cgroups.
  • Granularity at TGID (PID), not TID level.
  • Focus on simplicity / clarity over ultimate flexibility.

How to identify the cgroup version?

To check which cgroup version your distribution uses:

$ stat -fc %T /sys/fs/cgroup/
  • For cgroup v2, the output is cgroup2fs.
  • For cgroup v1, the output is tmpfs.

To check the layout:

$ ls /sys/fs/cgroup

cgroup2 is mounted to /sys/fs/cgroup

The /sys/fs/cgroup is called root control group:

$ mount | grep cgroup
cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot)

How to check cgroups of a process?

For example, find a process and its pid:

$ ps -e
...
   4284 ?        00:00:00 dnsmasq
...

Check cgroup by ps -o cgroup PID:

$ ps -o cgroup 4284
CGROUP
0::/system.slice/libvirtd.service

Check details in /sys/fs/cgroup:

$ ls /sys/fs/cgroup/system.slice/libvirtd.service
cgroup.controllers      cgroup.pressure         cpu.pressure    memory.events.local  memory.oom.group     memory.swap.events    memory.zswap.writeback
cgroup.events           cgroup.procs            cpu.stat        memory.high          memory.peak          memory.swap.high      pids.current
cgroup.freeze           cgroup.stat             cpu.stat.local  memory.low           memory.pressure      memory.swap.max       pids.events
cgroup.kill             cgroup.subtree_control  io.pressure     memory.max           memory.reclaim       memory.swap.peak      pids.max
cgroup.max.depth        cgroup.threads          memory.current  memory.min           memory.stat          memory.zswap.current  pids.peak
cgroup.max.descendants  cgroup.type             memory.events   memory.numa_stat     memory.swap.current  memory.zswap.max

Is cgroup a fs?

Yes, cgroup (Control Groups) is a pseudo-filesystem (also called a virtual filesystem).

Just like /proc or /sys, it does not exist on your hard drive. It is an interface provided by the Linux kernel that allows you to manage resources (CPU, memory, I/O) by manipulating files and directories.

The Interface: "Folders as Groups"

The kernel uses a filesystem because it is the most intuitive way to represent a hierarchy.

  • Creating a Group: When you use mkdir inside a cgroup directory, the kernel automatically creates a new control group.
  • Removing a Group: When you use rmdir, the kernel deletes the group (provided no processes are left in it).
  • Assigning Processes: To move a process into a group, you simply write its PID into a file named cgroup.procs.
  • Setting Limits: To limit a group to 1GB of RAM, you write 1G into the memory.max file.

Where is it located?

On almost all modern Linux distributions (using systemd), the cgroup filesystem is mounted at: /sys/fs/cgroup

Why use a Filesystem instead of System Calls?

The Linux philosophy is "Everything is a file." Using a filesystem for cgroups provides several advantages:

  1. Permission Management: You can use standard Linux permissions (chmod, chown) to allow a specific user to manage their own sub-groups without needing root access.
  2. Tool Compatibility: You can manage complex container resources using simple shell scripts and basic commands like ls, echo, and mkdir.
  3. Visibility: It is very easy for a human to "browse" the state of the system by just looking through the directories.

Avoid modifying the folder directly when using systemd

If you are using systemd, you should avoid modifying files under /sys/fs/cgroup directly.

While the kernel allows you to echo values into those files, doing so creates a "conflict of interest" between you and systemd.

Cgroup v2 (the modern standard) follows a principle called the "Single Writer" rule. This means that for any given cgroup directory, only one application should be responsible for managing it. In most Linux distros, systemd is that single writer.