x86 Memory Organization

Last update: 2024-07-08 12:41:37 +0800 +0800
Available translations: 日本語 / 中文(简体) / 中文(繁體)

x86 memory organization developed in response to successive limits on register width, address-bus width, protection, and multitasking. Physical memory remains a byte-addressed space: after translation, the processor places a physical address on the memory interface. The path to that address, however, has changed substantially.

An x86 instruction first computes an effective address (EA), also called an offset. In modes with usable segmentation, a selected segment contributes a base, producing a linear address. With paging enabled, page tables translate the linear address to a physical address. The stages are therefore:

instruction+-->effectiveaddress+-->segmentation+--p>h|yspiacgailngad|dress|

Real-address mode has no paging, so segment arithmetic produces the physical address directly. Modern 64-bit systems retain the same conceptual sequence, but long mode disables ordinary segmentation for address generation and requires paging.

Programming with Physical Addresses #

Early processors such as the Intel 8080 and 8085 exposed their physical address space directly to instructions. Their instructions are at most three bytes long: the opcode occupies one byte, leaving at most two bytes for an immediate operand. Because those processors have 16 address lines, a two-byte immediate can represent any physical address.

Assembly can refer to a memory location in two ways:

  1. Encode a known address in the instruction. An assembler resolves a label to the address of the target, or the programmer supplies a numeric address. For example, SHLD 0x0055 stores the HL register pair at physical address 0x0055, and JMP foo transfers control to the address assigned to foo. Labels make the source maintainable, but the generated instruction still contains a fixed address.
  2. Dereference an address held in registers. The 8080 and 8085 have 8-bit general-purpose registers, whereas an address is 16 bits. Register pairs BC, DE, and HL consequently hold addresses. STAX B and STAX D store A through BC and DE, respectively; MOV M,A stores through HL.

The direct model is simple, but fixed addresses bind code to its load location. Relocation requires changing every embedded address, and independently loaded programs must coordinate their physical ranges.

8086/8088: Segmentation v1 and Why #

The 8086 and 8088 have 20 address pins and can address 1 MiB, but their general-purpose and segment registers are 16 bits wide. They have neither virtual memory nor paging. Memory references still reach physical memory, but the instruction encoding no longer treats a 20-bit physical address as the normal addressing form.

Encoding a 20-bit immediate would consume three whole bytes and waste four bits. It would also preserve the inflexibility of a fixed load address. Loading an address into registers can express dynamically computed locations, so a dedicated 20-bit immediate form would add hardware and encoding complexity without covering new use cases. Some later x86 instructions retain a direct offset form (moffs), but it is still interpreted with a segment; it is not an unsegmented physical-address operand.

Segmentation v1 #

The 8086 solution combines a 16-bit segment value with a 16-bit offset:

  1. Four segment registers exist: CS, SS, DS, and ES.
  2. Every memory reference has a default segment. Instruction fetches use CS; ordinary data references use DS; references whose effective address uses BP, and stack operations, use SS. String instructions use prescribed source and destination segments. A segment-override prefix can select another permitted segment for many references.
  3. An instruction computes an offset from an optional base register (BX or BP), an optional index register (SI or DI), and an optional displacement. Legal 16-bit addressing combinations are constrained by the encoding, but the resulting arithmetic is base + index + displacement. At least one encoded component identifies a memory operand. Integer instructions normally encode at most one explicit memory operand; string instructions are an important exception because they contain implicit source and destination accesses.

For example, MOV AX, [4] uses the default DS segment. An assembler can encode an override such as MOV AX, CS:[4]; in the common moffs encoding the latter has an additional 0x2E prefix. Assembler spelling differs, but the prefix has the same architectural meaning.

The execution unit computes the EA. The bus-interface unit then shifts the selected segment value left by four bits and adds the EA:

text
segment     4 5 0 0
offset        F C 3 2
-------------------
sum         5 4 C 3 2

physical address = (segment << 4) + effective address
seefgfmeecnttivreegoifsftseertshiftleft4segmentbaadsdephysicaladdress

The 8086 exposes only 20 address bits. If the addition produces a carry beyond bit 19, the bus address wraps modulo 1 MiB; the familiar FFFF:FFFF calculation is therefore not a way to reach a unique address above 1 MiB on an original 8086. Consequently, a reference written as [BX] or [40] never bypasses segment arithmetic, even though the offset can look like an absolute address. “Real physical addressing” in descriptions of real mode means that paging does not virtualize the final address; it does not mean that instructions encode arbitrary physical addresses without a segment.

Segment values are paragraph numbers. They provide 4,096 possible bases, each 16 bytes apart, and each offset can span a 64 KiB window. Segments can be adjacent, disjoint, partially overlapping, or identical. A physical location can therefore have many segment:offset representations.

Conclusion: 8086/8088 Memory Segmentation #

Segmentation primarily compensated for registers that were narrower than the address bus. An implicit segment also shortened many instructions because only the offset needed encoding. Intel considered shifting segment values by eight rather than four bits, which would have created a 24-bit address space and 256-byte segment alignment. The four-bit shift preserved finer placement granularity while keeping the 1 MiB design target.

The arrangement complicates software because a pointer is not a single flat number. That cost became increasingly difficult to justify once general-purpose registers reached the width of the address space, but compatibility requirements kept segmentation in the architecture.

80286: Segmentation v2 #

The 80286 retained 16-bit general-purpose registers while widening the physical address bus to 24 bits, supporting up to 16 MiB. It introduced real-address mode and protected mode. On reset, it begins in real-address mode.

The 80286’s MSW.PE bit enables protected mode. Unlike later x86 processors, an 80286 cannot return from protected mode to real-address mode simply by clearing that bit; software conventionally resets the processor to return. The limitation matters to early operating systems, but it does not alter the address-translation rules in either mode.

Real-Address Mode #

Real-address mode preserves 8086-compatible segment:offset addressing. Segment registers contain 16-bit segment values, and the processor forms addresses by shifting the segment by four and adding the 16-bit offset. Unlike the original 8086 package, the 80286 exposes address line A20. With A20 enabled, references near FFFF:FFFF can reach the high-memory area above 1 MiB; compatibility hardware can mask A20 to reproduce 20-bit wraparound. Programs written for an 8086 can therefore run without source changes, subject to their other hardware assumptions.

The 80286 also makes new instructions available in real-address mode. Those instructions do not turn real-address segmentation into protected segmentation or add paging.

Protected Virtual Address Mode #

The 80286 protected mode adds protected, variable-sized segments, but it does not yet implement page-table virtual memory. “Protected virtual address mode” is thus protected segmentation, not the demand-paged virtual memory model associated with the 80386 and later processors.

Two memory-resident descriptor tables define segments:

  • The Global Descriptor Table (GDT) is system-wide.
  • A Local Descriptor Table (LDT) can describe segments associated with a task.

The visible portion of each segment register is now a 16-bit selector, not a shifted base. Its format is:

15index(13bits)|TI(1)|2RPL(2bits)0
  • index selects an eight-byte descriptor.
  • TI selects the GDT when clear and the current LDT when set.
  • RPL is the requestor privilege level and participates in protection checks.

Loading a selector validates it and loads the selected descriptor into an architecturally invisible descriptor cache associated with the segment register. The cache contains the base, limit, and access rights needed by later references. Reading a segment register exposes only its selector; software cannot read or write its cached fields directly. Caching avoids a descriptor-table memory access for every memory reference.

Global Descriptor Table (GDT) #

GDTR locates the GDT. On the 80286, it contains a 24-bit physical base and a 16-bit limit:

39GDTbase(24bits)1615limit(16bit0s)

The limit is the table size in bytes minus one, which permits a maximum encoded size of 65,536 bytes. Because every descriptor is eight bytes, a valid full table ends on a descriptor boundary. Privileged LGDT and SGDT load and store this register; ordinary data instructions cannot address GDTR.

The GDT is an array of descriptors. A descriptor with S = 1 is a code or data segment descriptor. A descriptor with S = 0 is a system descriptor, such as an LDT or task-state descriptor. The 80286 segment descriptor stores a 24-bit base, a 16-bit byte-granular limit, and an access byte:

bytrees7ervedbytrees6ervedbytaecc5essbytbeas4e23..16bytbeasse3.1.50..0,limit|

The diagram summarizes field placement rather than byte order within the final four-byte group: the descriptor’s low 16 bits are the limit, followed by base bits 15 through 0. A segment is now a contiguous range with an arbitrary 24-bit base and a variable limit; no 16-byte shift is involved. Because the 80286 has no page translation, the descriptor base identifies physical memory.

GDT entry zero must be an all-zero null descriptor. It makes selector index zero invalid when TI selects the GDT.

Local Descriptor Table (LDT) #

An LDT has the same descriptor layout as a GDT, but its first entry may be valid. Several LDTs can exist in memory, whereas the machine uses one GDT at a time. The LDTR visible selector identifies an S = 0 LDT descriptor in the GDT; its invisible cache supplies that LDT’s base and limit. LLDT and SLDT load and store the visible LDTR selector.

The 80286 hardware task mechanism can reload LDTR while switching tasks, making an LDT task-specific. Operating systems commonly avoid hardware task switching because software context switching is more flexible and portable. In IA-32e mode, hardware task switching is unavailable, although the task-state segment still serves other purposes such as privilege-stack and interrupt-stack management.

Segment Registers #

At any instant, selectors can refer to the GDT and to the LDT selected by LDTR. CS, SS, DS, and ES each consist architecturally of a visible 16-bit selector and an inaccessible cached descriptor. Loading one of them, whether by MOV, POP, LDS, a far control transfer, or another relevant instruction, causes the processor to select from:

v1i5ssieblleecttoorsoftware0inbvaissei,blleimciatc,heaccessrights|

The cache is not a second software-visible register layout: the diagram identifies the values retained by the processor after a successful selector load. A later memory reference uses those cached values, rather than rereading the descriptor table.

  • the GDT when TI is zero; or
  • the current LDT when TI is one.

The processor then checks type, presence, limit, and privilege rules before installing the descriptor cache. A far JMP, CALL, return, interrupt, or gate can load CS; ordinary MOV cannot.

A selector with index zero and TI = 0 is a null selector. Loading a null selector into CS or SS faults. DS and ES may hold a null selector, but any attempt to use that register for a memory reference faults. Later processors apply the same useful invalid-data-segment convention to FS and GS.

Conclusion: 80286 Segmentation without Paging #

The 80286 adds descriptor-based protection, privilege checking, and variable segment sizes. It also has an Interrupt Descriptor Table and IDTR, but interrupt dispatch does not change the ordinary data-address path. Protected-mode memory is still segmented physical memory because no page-table stage exists.

Since 80386: Segmentation and Virtual Paging #

The 80386 introduced 32-bit general-purpose registers, 32-bit segment bases, and page-table virtual memory. Its model applies to 32-bit protected-mode software on later processors as well. It defines three relevant execution modes:

  1. Real-address mode remains 8086-compatible.
  2. Protected mode extends 80286 protected segmentation and can enable paging.
  3. Virtual-8086 mode runs a real-mode-style program as a protected-mode task. Paging and protection belong to the surrounding protected-mode environment, so a virtual-8086 task cannot escape other processes’ mappings.

Protected Mode, Refurbished #

In 32-bit protected mode, the general-purpose registers, offsets, linear address space, segment bases, and physical address bus of the original 80386 are all 32 bits wide. Descriptor entries expand their base field to 32 bits and add flags, including granularity and default operand/address size. The high descriptor bytes that were reserved on the 80286 acquire these meanings.

Segmentation remains fully functional, including nonzero bases, limits, expand-down data segments, and privilege checks. Yet a 32-bit operating system can give its code and data segments base zero and a 4 GiB limit. Such flat segmentation makes the offset equal the linear address and lets paging perform process isolation. The segment selectors are not themselves zero: kernels commonly load non-null selectors such as 0x08 or 0x10 for valid flat descriptors.

The 80386 also adds the SIB (scale-index-base) encoding. It extends EA formation to:

text
effective address = base + index × scale + displacement
scale = 1, 2, 4, or 8

The ModR/M and optional SIB bytes select the participating registers and displacement size. The scale is useful for arrays of 2-, 4-, or 8-byte elements, but it changes only EA formation, not the segmentation or paging model.

bidanisdseepxlraercgeegm+e-n-t-+-->---s-c-a-l-e------>addeafdfdercetsisve

Paging #

Setting CR0.PG enables paging in protected mode. Address formation then has four steps:

  1. The instruction computes its EA from the applicable ModR/M, SIB, registers, and displacement.
  2. The selected segment’s cached descriptor validates the offset and contributes its base. The result is a 32-bit linear address.
  3. Paging uses that linear address to walk tables rooted at the physical address in CR3.
  4. The selected page entry contributes a physical page-frame base; adding the page offset produces the physical address.

The processor checks descriptor permissions before or during the segmented-address stage and page permissions during translation. A not-present entry or prohibited access produces a page fault. Translation lookaside buffers (TLBs) cache successful linear-to-physical translations, so the operating system must invalidate or otherwise synchronize cached translations after changing active page tables.

Legacy 32-Bit Paging #

The original 80386 scheme uses 4 KiB pages and two levels. CR3 identifies the page directory for the current address space; there is not one page directory for the entire system. A context switch can load another CR3, and the operating system may share selected kernel mappings among many directories.

A page directory and every page table occupy one 4 KiB page. Each contains 1,024 four-byte entries:

31physicalbaseaddress1211flags0

A page-directory entry (PDE) points to a page table; a page-table entry (PTE) points to a 4 KiB physical page frame. Their low bits carry related but level-specific flags, including present, read/write, user/supervisor, accessed, and cache-control attributes. A PTE also has dirty state; a PDE can instead represent a large page when page-size extensions are enabled.

The 32-bit linear address divides as follows:

31d1i0rebcittosry2221t1a0blbeits1211p1a2gebiotfsfset0

directory selects a PDE, table selects a PTE in its page table, and offset selects the byte within the resulting 4 KiB frame. With CR4.PSE enabled, a PDE whose page-size bit is set can map a 4 MiB page directly, replacing the page-table step.

seloefcftsoertdbeasscer+iopftfosretlaidnderaersspavgieatCaRb3lespahdydsriecsasl
Physical Address Extension (PAE) #

PAE retains 32-bit linear addresses but changes the paging hierarchy and uses 64-bit paging entries. It is enabled with CR4.PAE. Early PAE implementations extended physical addresses to 36 bits; the architectural physical-address width is implementation-dependent on later processors.

In PAE paging, CR3 points to a four-entry Page Directory Pointer Table (PDPT). Each selected page directory has 512 entries, and each selected page table also has 512 entries. A 32-bit linear address is divided into:

31P2D3P0T29d9irbeicttsory2120t9abbliets1211p1a2gebiotfsfset0

The page-directory pointer chooses a PDPTE; the next two fields choose a PDE and PTE. A PAE PDE with its page-size bit set maps a 2 MiB page. PAE also supplies the entry format required for the no-execute permission bit when EFER.NXE is enabled. Without NXE, the high bit is reserved rather than an executable-policy control.

IA-32e and Long-Mode Paging #

IA-32e mode is the 64-bit-capable execution environment. It contains 64-bit mode and compatibility mode. Both use IA-32e paging, which is mandatory: CR0.PG cannot be cleared while IA-32e mode is active.

To enter IA-32e mode, privileged software constructs valid PAE-format page tables, sets CR4.PAE, sets EFER.LME, and then enables CR0.PG. Enabling paging with LME set latches EFER.LMA, the read-only indication that IA-32e mode is active. A far control transfer to a code segment whose L bit is set enters 64-bit mode; a code segment with L = 0 and an appropriate default size enters compatibility mode. Transition code must establish mappings before enabling paging because the next instruction fetch is translated through the new hierarchy.

Four-level paging uses a PML4, a page-directory-pointer table, a page directory, and a page table. Each table has 512 eight-byte entries, so each index is nine bits:

47P9MLb4its3938P9DPbTits3029d9irbeicttsory2120t9abbliets1211p1a2gebiotfsfse0t

Four-level implementations use 48-bit virtual addresses. Bits 63 through 48 must copy bit 47, making an address canonical. A noncanonical address faults before a page-table walk. A processor with five-level paging (CR4.LA57) adds a nine-bit PML5 index, supports 57-bit virtual addresses, and requires bits 63 through 57 to copy bit 56. Physical-address width remains processor-dependent and is reported by CPUID; it must not be inferred from the linear-address width.

IA-32e page tables can map 4 KiB pages at the PTE level, 2 MiB pages at a PDE with the page-size bit set, and 1 GiB pages at a PDPTE with that bit set when the processor advertises support. Page entries contain physical-frame address bits and permission, memory-type, accessed, dirty where applicable, global, and execute-disable controls. An execute-disable bit works only when EFER.NXE is enabled.

Segmentation in IA-32e Mode #

Long mode changes segmentation without removing segment selectors or descriptors:

  • In 64-bit mode, CS, DS, ES, and SS contribute a base of zero for ordinary address generation. Their limits are not used for ordinary references. FS and GS retain useful bases, supplied by dedicated model-specific registers, and operating systems use them for thread-local or processor-local data. Canonical-address checking and paging still apply after adding an FS or GS base.
  • In compatibility mode, 16- and 32-bit protected-mode applications retain their protected-mode segmentation semantics while using IA-32e paging.
  • Virtual-8086 mode and hardware task switching are unavailable while IA-32e mode is active. The task-state segment remains necessary for privilege transitions and interrupt-stack handling.

Therefore, a long-mode kernel does not obtain a flat address space merely by loading zero-valued segment registers. It obtains ordinary zero bases from 64-bit-mode rules, selects valid descriptors where required, and relies on canonical-address rules plus paging for address-space isolation and permissions.

Address Translation across Modes #

The historical progression explains the layers visible today:

Processor and modeEffective address and segmentationPagingResult
8086/8088 real mode16-bit EA plus segment << 4None20-bit physical address
80286 real mode8086-compatible segment arithmeticNoneReal-address physical access
80286 protected modeSelector resolves to a 24-bit-base descriptorNoneProtected segmented physical access
80386 protected mode32-bit descriptor base plus EAOptional legacy pagingLinear address translated to physical memory
32-bit PAE mode32-bit protected segmentationRequired PAE hierarchyWider physical-address translation
IA-32e 64-bit modeZero bases except FS and GSRequired four- or five-level pagingCanonical linear address translated to physical memory

Address formation is consequently not a generic “x86 memory model.” The 8086 uses segmentation to extend a 16-bit offset into a 20-bit physical address. The 80286 turns segment values into protected descriptor selectors. The 80386 adds linear-address paging. PAE changes the paging entry width and hierarchy, while IA-32e makes paging mandatory and reduces ordinary segmentation in 64-bit mode.

Descriptor details are covered in x86 protected mode and privilege, and the instruction fields that encode effective addresses are covered in x86 instruction encoding.

Comments

Read or comment on GitHub (sign-in required to comment)