Linux Process Model

The kernel represents every schedulable flow of execution with struct task_struct, declared in the kernel source’s include/linux/sched.h. Distribution headers in /usr/include/linux are exported userspace ABI headers, not a copy of every kernel internal definition.

Tasks, Threads, and IDs #

A pid_t is an integer type; its width is architecture and ABI dependent. A struct pid represents an identity across PID namespaces and associates numeric PIDs with their namespaces. A task’s pid identifies that individual task in its active namespace. Its tgid is the thread-group ID: for the leader it equals pid, and getpid(2) returns the thread-group ID.

Linux uses the same task representation for processes and threads. clone(2) selects which resources are shared; threads share a thread group, address space, and other resources as requested, while fork(2) creates a child with separate process state before a possible execve(2).

Parentage #

parent is the task that receives parent-oriented events such as SIGCHLD; it can change when a child is reparented. real_parent initially identifies the creator and normally remains that creator, although kernel facilities can change it in special cases. The children and sibling lists express the current parent-child tree. Sessions and process groups form additional job-control relationships independent of the thread-group relationship.

Migrated Reference Material #

Task is the only unit of execution flow scheduling in Linux. This article explains the mysteries around Linux tasks by directly checking the Linux source.

A task is represented by a struct task_struct instance. In the Linux source tree, this structure is defined by include/linux/sched.h:

(Don’t get confused with /usr/include/linux/sched.h on your distro, which are userspace APIs generated at kernel’s build time. Structures internal to the kernel, including struct task_struct, won’t be there.)

// include/linux/sched.h
struct task_struct {
    // ...

task_struct is a large structure. Its fields will be summarized below.

Types #

Before looking at the definition of task_struct, some basics need to be set up.

A pid_t is a platform int. On x86_64 that’s 32 bit:

// include/linux/types.h
typedef __kernel_pid_t		pid_t;

// include/uapi/asm-generic/posix_types.h
typedef int		__kernel_pid_t;

A struct pid instance identifies an individual task, a process group, or a session. The latter 2 are sets of tasks and exist for a reason, see Process Group & Session

The same task, process group, or session, can have different numeric ID in different pid namespaces. Thus a struct pid instance can have multiple upid.

// include/linux/pid.h


/*
 * What is struct pid?
 *
 * A struct pid is the kernel's internal notion of a process identifier.
 * It refers to individual tasks, process groups, and sessions.  While
 * there are processes attached to it the struct pid lives in a hash
 * table, so it and then the processes that it refers to can be found
 * quickly from the numeric pid value.  The attached processes may be
 * quickly accessed by following pointers from struct pid.
 *
 * Storing pid_t values in the kernel and referring to them later has a
 * problem.  The process originally with that pid may have exited and the
 * pid allocator wrapped, and another process could have come along
 * and been assigned that pid.
 *
 * Referring to user space processes by holding a reference to struct
 * task_struct has a problem.  When the user space process exits
 * the now useless task_struct is still kept.  A task_struct plus a
 * stack consumes around 10K of low kernel memory.  More precisely
 * this is THREAD_SIZE + sizeof(struct task_struct).  By comparison
 * a struct pid is about 64 bytes.
 *
 * Holding a reference to struct pid solves both of these problems.
 * It is small so holding a reference does not consume a lot of
 * resources, and since a new struct pid is allocated when the numeric pid
 * value is reused (when pids wrap around) we don't mistakenly refer to new
 * processes.
 */


/*
 * struct upid is used to get the id of the struct pid, as it is
 * seen in particular namespace. Later the struct pid is found with
 * find_pid_ns() using the int nr and struct pid_namespace *ns.
 */

struct upid {
	int nr;
	struct pid_namespace *ns;
};

struct pid
{
	refcount_t count;
	unsigned int level;
	spinlock_t lock;
	/* lists of tasks that use this pid */
	struct hlist_head tasks[PIDTYPE_MAX];
	struct hlist_head inodes;
	/* wait queue for pidfd notifications */
	wait_queue_head_t wait_pidfd;
	struct rcu_head rcu;
	struct upid numbers[];
};

The `task_struct` #

pid_t				pid;
pid_t				tgid;

pid is a number that uniquely marks this task_struct instance. This number is not necessarily
tgid is a number returned by the getpid(2) syscall, and is numerically the same as the pid of the thread group leader.

See Threading below.

/* Recipient of SIGCHLD, wait4() reports: */
struct task_struct __rcu	*parent;

/* Real parent process: */
struct task_struct __rcu	*real_parent;

Current parent, which may change due to [reparenting](#documented behavior); and the original parent that clones/forks this task that never changes once the task is created.

/*
* Children/sibling form the list of natural children:
*/
struct list_head		children;
struct list_head		sibling;
struct task_struct		*group_leader;

Pointers to the youngest child, the youngest sibling, and the oldest sibling. See Process Group & Session.

#ifdef CONFIG_THREAD_INFO_IN_TASK
	/*
	 * For reasons of header soup (see current_thread_info()), this
	 * must be the first element of task_struct.
	 */
	struct thread_info		thread_info;
#endif

	/* PID/PID hash table linkage. */
	struct pid			*thread_pid;
	struct hlist_node		pid_links[PIDTYPE_MAX];
	struct list_head		thread_node;

    /* CPU-specific state of this task: */
	struct thread_struct		thread;

	/*
	 * WARNING: on x86, 'thread_struct' contains a variable-sized
	 * structure.  It *MUST* be at the end of 'task_struct'.
	 *
	 * Do not put anything below here!
	 */

/* Signal handlers: */
	struct signal_struct		*signal;
	struct sighand_struct __rcu		*sighand;
	sigset_t			blocked;
	sigset_t			real_blocked;
	/* Restored if set_restore_sigmask() was used: */
	sigset_t			saved_sigmask;
	struct sigpending		pending;
	unsigned long			sas_ss_sp;
	size_t				sas_ss_size;
	unsigned int			sas_ss_flags;

A task represents a thread in the process-thread terminology, and thus belongs to a process (i.e. a thread group).
A process (i.e. thread group) belongs to a process group,

A task belongs to a namespace.
A task belongs to a cgroup (control group).

IDs #

For the current kernel design, there are 4 numeric IDs directly associated with a task_struct instance: thread ID, thread group (i.e. process) ID, process group ID, and session ID; as opposed to indirect ones, e.g. parent process ID obtained as the thread group ID of p->real_parent pointer.

The numeric values of all 4 IDs as observed from the userspace are subject to the pid namespace (see man 7 namespaces) the process is currently in, thus simply storing numbers in task_struct and returing them from syscalls won’t work.

//include/linux/pid_types.h
enum pid_type {
	PIDTYPE_PID,
	PIDTYPE_TGID,
	PIDTYPE_PGID,
	PIDTYPE_SID,
	PIDTYPE_MAX,
};


// kernel/pid.c

/*
 * the helpers to get the task's different pids as they are seen
 * from various namespaces
 *
 * task_xid_nr()     : global id, i.e. the id seen from the init namespace;
 * task_xid_vnr()    : virtual id, i.e. the id seen from the pid namespace of
 *                     current.
 * task_xid_nr_ns()  : id seen from the ns specified;
 *
 * see also pid_nr() etc in include/linux/pid.h
 */

static inline pid_t task_pid_nr(struct task_struct *tsk)
static inline pid_t task_pid_vnr(struct task_struct *tsk)
static inline pid_t task_pid_nr_ns(struct task_struct *tsk, struct pid_namespace *ns)

static inline pid_t task_tgid_nr(struct task_struct *tsk)
static inline pid_t task_tgid_nr_ns(struct task_struct *tsk, struct pid_namespace *ns)
static inline pid_t task_tgid_vnr(struct task_struct *tsk)

static inline pid_t task_pgrp_nr(struct task_struct *tsk)
static inline pid_t task_pgrp_vnr(struct task_struct *tsk)
static inline pid_t task_pgrp_nr_ns(struct task_struct *tsk, struct pid_namespace *ns)

static inline pid_t task_session_nr_ns(struct task_struct *tsk, struct pid_namespace *ns)
static inline pid_t task_session_vnr(struct task_struct *tsk)

static inline pid_t task_ppid_nr(const struct task_struct *tsk)
static inline pid_t task_ppid_nr_ns(const struct task_struct *tsk, struct pid_namespace *ns)


pid_t __task_pid_nr_ns(struct task_struct *task, enum pid_type type, struct pid_namespace *ns);

From the userspace’s perspective:

getpid(2) syscall returns a number same to tgid if in the default namespaces
gettid syscall returns a number same to pid if in the default namespaces

If the task is in a new pid namespace (via CLONE_NEWPID clone flag), its ID in the corresponding namespace are looked up via the _vnr functions.

Process Group, Session, and Controlling Terminal #

Process group and session are concepts built into the kernel’s task model to support a feature of shell known as the job control

In the normal operation of a shell:

A process group models all processes in the same pipeline
A session models the lifecycle of a shell from a user’s login to logout

Models #

A process group is simply a group of tasks. The choice of the word “process” is rather unfortunate and can’t change now due to the inertia of history.

From the userspace’s perspective:

A process group has a leader task, known as the group leader.
A process group has an ID that is numerically the same as the pid_t of the leader task.
A task has a pgid which is the ID of its belonging process group; Any code can query the pgid of any task by getpgid(2) or reading /proc/<pid>/stat. Any code can change the process group of a task in the same session by calling setpgid(2).
A child task by default is in the same process group as its parent.

Internally, the kernel doesn’t have dedicated structure for a process group. A task remembers which group it is in by holding a pointer to the leader of its group: struct task_struct *group_leader, instead of holding a number pgid. This field is copied when a child task is fork/clone-ed, thus we say a child task by default is in the same process group as its parent, or inherits the pgid of its parent. Such descriptions can be indeed misleading.

A process that is not a group leader (*) may call setsid(2) to start a new session and become the session leader of that new session. It also becomes the group leader of the only process group in that new session.

(* This restriction is imposed to avoid the complexity where process group members will lose their group leader.)

When a session leader open(2) a device node backed by a tty driver (with character device major 4 (for real tty) or 136 to 143 (for pseudo tty)), this terminal (specifically, the struct cdev instance registered by that tty driver), is said to be the controlling terminal of that session (unless O_NOCTTY flag is used).

When the controlling terminal of a session hang ups the modem connection, the kernel will send a SIGHUP signal to the session leader. (unless CLOCAL flag is set for that terminal, see termios(3))
It’s up to that session leader what to do with that signal, by properly setting up a signal handler. Almost always, session leaders are shell program. See SIGNALS section for bash.
When a session leader exits, the kernel will send a SIGHUP to all processes in the foreground process group of that session.
The kernel is aware of session and which process group is currently the foreground process group of a session. You can check this from the kernel’s procfs by checking the tpgid field of the /proc/<pid>/stat file. See man 5 proc_pid_stat.

Controlling terminal is very different from the terminal associated with stdin (if any) of a process.

Reparenting, Zombie and Orphan #

A task has a pointer to its original parent (the one that forked it) and a pointer to its current parent (the one that receives SIGCHLD)

There is no setppid() - the change of parent only happens when the parent process exit without wait()-ing on childern. Those orphan child processes change their current parent to the pid 1 i.e. init or systemd.

After a child task exit, if its parent is alive and has not call wait() on it, the child is known as a zombie. Zombie process appeared in the system is the problem of their parents. There is no specific name for such a parent afaik.
regardless a child task exit-ed or not, if if its parent is exit already (indicating no wait() on the child is ever called), the child is known as an orphan. Orphan processes are always re-parented to pid 1.

Threading #

Each flow of execution, i.e. process, is represented as a task_struct instance,

Back then the only id for task_struct instance was the pid field. A process can retrieve its own pid by getpid(2).

The first standardization for thread API is pthread in POSIX 1996. Several attempts of pure userspace implementation was made: thread library schedules application threads on top of a single kernel process.

struct task_struct {
#ifdef CONFIG_THREAD_INFO_IN_TASK
	/*
	 * For reasons of header soup (see current_thread_info()), this
	 * must be the first element of task_struct.
	 */
	struct thread_info		thread_info;
#endif

The kernel build option CONFIG_THREAD_INFO_IN_TASK determines whether the struct thread_info instance is embedded in task_struct or found in a hash table linked by a pointer (see PID/PID hash table linkage below).

struct thread_info is a CPU-specific structure defined in arch/<archtecture>/. For x86, it is:

// arch/x86/include/asm/thread_info.h

struct thread_info {
	unsigned long		flags;		/* low level flags */
	unsigned long		syscall_work;	/* SYSCALL_WORK_ flags */
	u32			status;		/* thread synchronous flags */
#ifdef CONFIG_SMP
	u32			cpu;		/* current CPU */
#endif
};

	/* PID/PID hash table linkage. */
	struct pid			*thread_pid;
	struct hlist_node		pid_links[PIDTYPE_MAX];
	struct list_head		thread_node;

Syscalls #

Clone / Clone3 #

clone3 is clone’s extensible successor. There was once a syscall, clone2, only available on the IA64 architecture. It’s now permanently removed from the mainstream kernel release.

#ifdef __ARCH_WANT_SYS_CLONE
#ifdef CONFIG_CLONE_BACKWARDS
SYSCALL_DEFINE5(clone, unsigned long, clone_flags, unsigned long, newsp,
		 int __user *, parent_tidptr,
		 unsigned long, tls,
		 int __user *, child_tidptr)
#elif defined(CONFIG_CLONE_BACKWARDS2)
SYSCALL_DEFINE5(clone, unsigned long, newsp, unsigned long, clone_flags,
		 int __user *, parent_tidptr,
		 int __user *, child_tidptr,
		 unsigned long, tls)
#elif defined(CONFIG_CLONE_BACKWARDS3)
SYSCALL_DEFINE6(clone, unsigned long, clone_flags, unsigned long, newsp,
		int, stack_size,
		int __user *, parent_tidptr,
		int __user *, child_tidptr,
		unsigned long, tls)
#else
SYSCALL_DEFINE5(clone, unsigned long, clone_flags, unsigned long, newsp,
		 int __user *, parent_tidptr,
		 int __user *, child_tidptr,
		 unsigned long, tls)
#endif
{
	struct kernel_clone_args args = {
		.flags		= (lower_32_bits(clone_flags) & ~CSIGNAL),
		.pidfd		= parent_tidptr,
		.child_tid	= child_tidptr,
		.parent_tid	= parent_tidptr,
		.exit_signal	= (lower_32_bits(clone_flags) & CSIGNAL),
		.stack		= newsp,
		.tls		= tls,
	};

	return kernel_clone(&args);
}
#endif

/**
 * sys_clone3 - create a new process with specific properties
 * @uargs: argument structure
 * @size:  size of @uargs
 *
 * clone3() is the extensible successor to clone()/clone2().
 * It takes a struct as argument that is versioned by its size.
 *
 * Return: On success, a positive PID for the child process.
 *         On error, a negative errno number.
 */
SYSCALL_DEFINE2(clone3, struct clone_args __user *, uargs, size_t, size)
{
	int err;

	struct kernel_clone_args kargs;
	pid_t set_tid[MAX_PID_NS_LEVEL];

	kargs.set_tid = set_tid;

	err = copy_clone_args_from_user(&kargs, uargs, size);
	if (err)
		return err;

	if (!clone3_args_valid(&kargs))
		return -EINVAL;

	return kernel_clone(&kargs);
}

Info.

The glibc’s wrapper use variadic function in signature:

// source:
extern int clone (
    int (*__fn) (void *__arg),
    void *__child_stack,
    int __flags,
    void *__arg,
    ...
) __attribute__ ((__nothrow__ , __leaf__));


// man page:
int clone(int (*fn)(void *_Nullable), void *stack, int flags,
                 void *_Nullable arg, ...  /* pid_t *_Nullable parent_tid,
                                              void *_Nullable tls,
                                              pid_t *_Nullable child_tid */ );

Both clone() and clone3() simply assembles a struct kernel_clone_args instance and delegates to kernel_clone(); This structure

struct kernel_clone_args {
	u64 flags;					//
	int __user *pidfd;
	int __user *child_tid;
	int __user *parent_tid;
	const char *name;
	int exit_signal;
	u32 kthread:1;
	u32 io_thread:1;
	u32 user_worker:1;
	u32 no_files:1;
	unsigned long stack;
	unsigned long stack_size;
	unsigned long tls;
	pid_t *set_tid;
	/* Number of elements in *set_tid */
	size_t set_tid_size;
	int cgroup;
	int idle;
	int (*fn)(void *);
	void *fn_arg;
	struct cgroup *cgrp;
	struct css_set *cset;
};

`clone` Flags #

/* CLONE_PARENT re-uses the old parent */
if (clone_flags & (CLONE_PARENT|CLONE_THREAD)) {
    p->real_parent = current->real_parent;
    p->parent_exec_id = current->parent_exec_id;
    if (clone_flags & CLONE_THREAD)
        p->exit_signal = -1;
    else
        p->exit_signal = current->group_leader->exit_signal;
} else {
    p->real_parent = current;
    p->parent_exec_id = current->self_exec_id;
    p->exit_signal = args->exit_signal;
}

Linux Process Model

Tasks, Threads, and IDs #

Parentage #

Migrated Reference Material #

Types #

The `task_struct` #

IDs #

Process Group, Session, and Controlling Terminal #

Models #

Reparenting, Zombie and Orphan #

Threading #

Syscalls #

Clone / Clone3 #

`clone` Flags #

Pctrl #

Unshare #

Comments

Linux Process Model

Tasks, Threads, and IDs #

Parentage #

Migrated Reference Material #

Types #

The task_struct #

IDs #

Process Group, Session, and Controlling Terminal #

Models #

Reparenting, Zombie and Orphan #

Threading #

Syscalls #

Clone / Clone3 #

clone Flags #

Pctrl #

Unshare #

Comments

The `task_struct` #

`clone` Flags #