Linux Control Groups Version 2

Control groups (cgroups) account for and control sets of Linux tasks. A cgroup is not a process-tree node: a task inherits its parent’s cgroup when it forks, but a manager can subsequently migrate it. Consequently, cgroups provide the resource boundary behind service managers and containers, while namespaces provide views of global resources.

This article describes cgroup version 2 (cgroup v2). Its documented interface is the cgroup2 filesystem; files and supported controllers depend on the running kernel configuration and available hardware.

Unified Hierarchy and Membership #

Cgroup v2 has one unified hierarchy. Every task belongs to exactly one cgroup in that hierarchy, including the unnamed root cgroup. A child cgroup inherits the constraints and controller configuration permitted by its ancestors, so a child cannot obtain CPU, memory, or devices that its parent has not delegated.

The hierarchy is represented by a cgroup2 mount, conventionally /sys/fs/cgroup:

findmnt -t cgroup2
cat /proc/self/cgroup
ps -o pid,cgroup,comm -p $$

Each directory is a cgroup. Creating a directory creates a child cgroup, and writing a PID to its cgroup.procs migrates the process’s thread group:

sudo mkdir /sys/fs/cgroup/build
printf '%s\n' "$$" | sudo tee /sys/fs/cgroup/build/cgroup.procs

Use a service manager rather than moving its own processes by hand. For example, systemd creates and manages cgroups for slices, scopes, and services; manual migration can violate the manager’s accounting and lifecycle assumptions.

Domain and Threaded Cgroups #

The normal hierarchy consists of domain cgroups, whose controllers apply to a process as a unit. The no-internal-process rule prevents a non-root domain cgroup from both containing processes and distributing domain controllers to children. Move the workload into a leaf and use an empty interior node to allocate a controller among child cgroups:

The root is exempt. A threaded subtree, configured through cgroup.type, permits individual threads to be placed with cgroup.threads. It is intended for thread-aware controllers and has additional structural constraints; it does not turn every controller into a per-thread limit. Inspect cgroup.type and the kernel documentation before mixing threaded and domain subtrees.

Controllers and Enablement #

A controller is kernel code that implements a resource policy. Availability and enablement are distinct:

• cgroup.controllers lists controllers that the parent has made available for the cgroup’s children.
• cgroup.subtree_control lists controllers enabled to control those children. Write +name or -name to alter it.
• Controller interface files appear only where their controller, configuration, and device context make them applicable.

For a new subtree, first enable controllers on the parent, then configure child cgroups. A non-root domain cgroup must not contain processes while enabling a domain controller for its children.

cd /sys/fs/cgroup
cat cgroup.controllers
sudo sh -c 'printf "+cpu +memory +pids\n" > cgroup.subtree_control'
sudo mkdir build
sudo sh -c 'printf "50000 100000\n" > build/cpu.max'
sudo sh -c 'printf "2G\n" > build/memory.max'
sudo sh -c 'printf "512\n" > build/pids.max'

cpu.max in the example allows 50,000 microseconds of CPU time per 100,000 microsecond period. It throttles aggregate CPU bandwidth; it does not reserve a dedicated CPU. The effective controller set remains bounded by every ancestor.

Controller Interfaces #

Controller names at the root identify what a particular kernel exposes. Current v2 documentation covers the following resource-controller families:

memory.high is a reclaim-and-throttle boundary, while memory.max is a hard limit that can lead to cgroup-local OOM handling when reclaim cannot make progress. memory.oom.group can request group-oriented killing for a workload. Read memory.events, memory.stat, and pressure-stall information before changing a limit; a limit that fits steady-state RSS can still fail during page faults, fork, or cache growth.

cpuset.cpus and cpuset.mems express requests subject to parent constraints. Their *.effective counterparts show the CPUs and NUMA nodes actually granted, including hotplug effects. CPU affinity and cpusets are intersected, so either mechanism can reduce where a task runs.

Common Cgroup Files #

In addition to controller files, cgroup v2 supplies hierarchy-wide files:

• cgroup.procs lists thread-group IDs in a cgroup and accepts a PID migration.
• cgroup.threads lists threads in a threaded subtree and accepts thread IDs.
• cgroup.events reports state such as whether the cgroup or its descendants are populated; poll or inotify can observe changes.
• cgroup.freeze stops and resumes execution in a non-root subtree, and cgroup.kill kills all processes in a non-root subtree.
• cgroup.max.depth and cgroup.max.descendants bound hierarchy growth.
• cgroup.stat, cgroup.type, and cgroup.controllers expose hierarchy and controller state.

File writes are operations, not configuration transactions. A manager must handle EBUSY, permission failures, hotplug changes, concurrent migration, and partial setup. Limits may be written as max where the interface permits an unbounded value.

Delegation, Namespaces, and Containers #

Delegation gives a subordinate manager control of a subtree without granting control of its ancestors. A delegator creates an empty cgroup, enables the necessary controllers on its parent, assigns ownership or access rights to the subtree, and allows the delegate to create and manage descendants. The delegate must operate only below that boundary because ancestor configuration determines its available resources.

systemd’s Delegate=yes is a common way to establish such a boundary. A container runtime then composes several mechanisms:

1. create or select the cgroup and apply host resource policy;
2. create user, mount, PID, network, and other namespaces;
3. arrange a private mount view and mount /proc after the PID namespace exists;
4. execute the workload and retain a host-side handle for lifecycle operations.

A cgroup namespace changes the cgroup-path view seen by a process: /proc/PID/cgroup and cgroup filesystem paths are rooted relative to that namespace. It does not create a separate controller hierarchy, relax ancestor limits, or isolate resource accounting. Host-side managers retain the authoritative hierarchy.

Kernel Representation #

The user-visible tree is backed by cgroup objects in the kernel. task_struct points to a css_set; a css_set groups the task’s cgroup_subsys_state (CSS) references for the active controllers. A controller embeds or associates its state with a CSS and uses that association in its accounting and enforcement paths. Tasks can share a css_set, which avoids duplicating an identical controller-state combination for every task.

These implementation details explain why migration must update controller membership consistently, but they are not a stable userspace ABI. Consult kernel/cgroup/cgroup.c and include/linux/cgroup-defs.h for a particular kernel revision.

Contrast with Cgroup Version 1 #

Version 1 permits multiple independent hierarchies and controller attachment to different hierarchies. Therefore a task can have distinct memberships for different controllers, which makes an application’s resource boundary difficult to reason about. Cgroup v2 uses one hierarchy, common delegation rules, and consistent controller interfaces. Systems can mount v1 and v2 during migration, but a controller cannot be simultaneously bound to both versions.

The authoritative, version-specific reference is the cgroup v2 kernel documentation. Check the target kernel’s cgroup.controllers and that document rather than assuming a controller or interface file exists.