15-213
“The course that gives CMU its Zip!”
Virtual Memory
November 2, 2007
Topics



class19.ppt
Motivations for virtual memory
Address translation
Accelerating translation with TLBs
Why Virtual Memory?
(1) VM uses main memory efficiently



Main memory is a cache for the contents of a virtual address
space stored on disk.
Keep only active areas of virtual address space in memory
Transfer data back and forth as needed.
(2) VM simplifies memory management

Each process gets the same linear address space.
(3) VM protects address spaces

One process can’t interfere with another.
 Because they operate in different address spaces.

User process cannot access privileged information
 Different sections of address spaces have different permissions.
–2–
15-213, F’07
Motivation 1: DRAM a “Cache” for Disk
The full address space is quite large:


32-bit addresses:
~4,000,000,000 (4 billion) bytes
64-bit addresses: ~16,000,000,000,000,000,000 (16 quintillion) bytes
Disk storage is ~100X cheaper than DRAM storage


1 TB of DRAM: ~ $30,000
1 TB of disk:
~ $300
To access large amounts of data in a cost-effective
manner, the bulk of the data must be stored on disk
4 MB: ~$300
SRAM
–3–
8 GB: ~$300
DRAM
1 TB: ~$300
Disk
15-213, F’07
Levels in Memory Hierarchy
Smaller,
faster,
and
costlier
(per byte)
storage
devices
L0:
registers
L1: on-chip L1
cache (SRAM)
L2:
L3:
Larger,
slower,
and
cheaper
(per byte)
storage
devices
L5:
–4–
CPU registers hold words retrieved
from L1 cache.
L4:
off-chip L2
cache (SRAM)
L1 cache holds cache lines retrieved
from the L2 cache memory.
L2 cache holds cache lines
retrieved from main memory.
main memory
(DRAM)
Main memory holds disk
blocks retrieved from local
disks.
local secondary storage
(local disks)
Local disks hold files
retrieved from disks on
remote network servers.
remote secondary storage
(tapes, distributed file systems, Web servers)
15-213, F’07
DRAM vs. SRAM as a “Cache”
DRAM vs. disk is more extreme than SRAM vs. DRAM

access latencies:
 DRAM is ~10X slower than SRAM
 disk is ~100,000X slower than DRAM

importance of exploiting spatial locality:
 first byte is ~100,000X slower than successive bytes on disk
» vs. ~4X improvement for page-mode vs. regular accesses to DRAM

“cache” size:
 main memory is ~1000X larger than an SRAM cache

addressing for disk is based on sector address, not memory
address
SRAM
–5–
DRAM
Disk
15-213, F’07
Impact of These Properties on Design
If DRAM was to be organized similar to an SRAM cache,
how would we set the following design parameters?

Line size?

Associativity?

Replacement policy (if associative)?

Write through or write back?
What would the impact of these choices be on:




–6–
miss rate
hit time
miss latency
tag overhead
15-213, F’07
Locating an Object in a “Cache”
“Cache”
1. Search for matching tag

SRAM cache
Object Name
X
= X?
Tag
Data
0:
D
243
1:
X
•
•
•
J
17
•
•
•
105
N-1:
2. Use indirection to look up actual object location

virtual memory Lookup Table
“Cache”
Location
Object Name
D:
0
0:
243
X
J:
N-1
•
•
•
1
1:
17
•
•
•
105
X:
–7–
Data
N-1:
15-213, F’07
A System with Physical Memory
Only
Examples:

most Cray machines, early PCs, nearly all embedded
Memory
systems (phones, PDAs, etc.)
Store 0x10
0:
1:
CPU
Load 0xf0
N-1:
CPU’s load or store addresses used directly to access memory.
–8–
15-213, F’07
A System with Virtual Memory
Examples:

Memory
laptops, servers, modern PCs, etc.
Page Table (MMU)
Virtual
Physical
Addresses
0:
Addresses
1:
Store 0x10
0:
1:
CPU
Load 0xf0
P-1:
N-1:
Disk
Address Translation: the hardware converts virtual addresses into
physical addresses via an OS-managed lookup table (page table)
–9–
15-213, F’07
Page Faults (Similar to “Cache Misses”)
What if an object is on disk rather than in memory?


Page table entry indicates that the virtual address is not in memory
An OS trap handler is invoked, moving data from disk into memory
 current process suspends, others can resume
 OS has full control over placement, etc.
Virtual
Addresses
CPU
Page Table (MMU)
Physical
0:
Addresses
1:
Memory
0:
1:
Load 0x05
Store 0xf8
P-1:
N-1:
Disk
– 10 –
15-213, F’07
Servicing a Page Fault
(1) Processor signals
controller

Read block of length P
starting at disk address X and
store starting at memory
address Y
(2) Read occurs


Direct Memory Access (DMA)
Under control of I/O controller
(3) Controller signals
completion


– 11 –
Interrupt processor
OS resumes suspended
process
(1) Initiate Block Read
Processor
Reg
(3) Read
Done
Cache
Memory-I/O bus
(2) DMA
Transfer
Memory
I/O
controller
disk
Disk
disk
Disk
15-213, F’07
Locality to the Rescue
Virtual memory works because of locality.
At any point in time, programs tend to access a set of
active virtual pages called the working set.

Programs with better temporal locality will have smaller
working sets.
If working set size < main memory size

Good performance after initial compulsory misses.
If working set size > main memory size

– 12 –
Thrashing: Performance meltdown where pages are
swapped (copied) in and out continuously
15-213, F’07
(2) VM as a Tool for Memory Mgmt
Key idea: Each process has its own virtual address space

Simplifies memory allocation, sharing, linking, and loading.
0
Virtual
Address
Space for
Process 1:
Address Translation
0
VP 1
VP 2
PP 2
...
N-1
PP 7
Virtual
Address
Space for
Process 2:
– 13 –
Physical
Address
Space
(DRAM)
0
VP 1
VP 2
PP 10
...
N-1
(e.g., read/only
library code)
M-1
15-213, F’07
Simplifying Sharing and Allocation
Sharing code and data among processes

Map virtual pages to the same physical page (PP 7)
Memory allocation

Virtual page can be mapped to any physical page
0
Virtual
Address
Space for
Process 1:
Address Translation
0
VP 1
VP 2
PP 2
...
N-1
PP 7
Virtual
Address
Space for
Process 2:
– 14 –
Physical
Address
Space
(DRAM)
0
VP 1
VP 2
PP 10
...
N-1
(e.g., read/only
library code)
M-1
15-213, F’07
Simplifying Linking and Loading
0xc0000000
User stack
(created at runtime)
Linking


Kernel virtual memory
Each program has similar
virtual address space
Code, stack, and shared
Memory mapped region for
libraries always start at the
shared libraries
same address.
0x40000000
Loading


– 15 –
Memory
invisible to
user code
%esp (stack ptr)
brk
Run-time heap
execve() maps PTEs to
(created at runtime by malloc)
the appropriate location in
the executable binary file.
Read/write segment
(.data, .bss)
The .text and .data
sections are copied, page
Read-only segment
by page, on demand by the
(.init, .text, .rodata)
0x08048000
virtual memory system.
0
Unused
Loaded from
executable file
15-213, F’07
(3)VM as a Tool for Memory Protection
Extend PTEs with permission bits.
Page fault handler checks these before remapping.

If violated, send process SIGSEGV (segmentation fault)
Page tables with permission bits
SUP READ WRITE
Process i:
Address
Physical memory
VP 0:
No
Yes
No
PP 6
VP 1:
No
Yes
Yes
PP 4
PP 0
VP 2: Yes
Yes
Yes
•
•
•
PP 2
PP 2
PP 4
PP 6
SUP READ WRITE
Process j:
– 16 –
Address
VP 0:
No
Yes
No
PP 9
VP 1:
Yes
Yes
Yes
PP 6
VP 2:
No
Yes
Yes
PP 11
PP 9
•
•
•
PP 11
15-213, F’07
Address Spaces
A linear address space is an ordered set of contiguous
nonnegative integer addresses:
{0, 1, 2, 3, … }
A virtual address space is a set of N = 2n virtual addresses:
{0, 1, 2, …, N-1}
A physical address space is a set of M = 2m (for convenience)
physical addresses:
{0, 1, 2, …, M-1}
In a system based on virtual addressing, each byte of main
memory has a virtual address and a physical address.
– 17 –
15-213, F’07
VM Address Translation
Virtual Address Space

V = {0, 1, …, N–1}
Physical Address Space


P = {0, 1, …, M–1}
M < N (usually, but >=4 Gbyte on an IA32 possible)
Address Translation


MAP: V  P U {}
For virtual address a:
 MAP(a) = a’ if data at virtual address a at physical address a’ in P
 MAP(a) =  if data at virtual address a not in physical memory
» Either invalid or stored on disk
– 18 –
15-213, F’07
Address Translation with a Page Table
VIRTUAL ADDRESS
Page table
base register
(PTBR)
n–1
p p–1
Virtual page number (VPN)
Valid
Virtual page offset (VPO)
Physical page number (PPN)
Page
table
The VPN acts
as index into
the page table
If valid=0
then page
not in memory
(page fault)
0
m–1
p p–1
Physical page number (PPN)
0
Physical page offset (PPO)
PHYSICAL ADDRESS
– 19 –
15-213, F’07
Address Translation: Page Hit
2
CPU chip
PTEA
PTE
1
Processor
3
MMU
VA
PA
Cache/
memory
4
Data
5
1) Processor sends virtual address to MMU
2-3) MMU fetches PTE from page table in memory
4) MMU sends physical address to L1 cache
5) L1 cache sends data word to processor
– 20 –
15-213, F’07
Address Translation: Page Fault
4
Exception
CPU chip
Page fault exception handler
2
PTEA
1
Processor
VA
7
MMU
PTE
3
Victim page
Cache/
memory
5
Disk
New page
6
1) Processor sends virtual address to MMU
2-3) MMU fetches PTE from page table in memory
4) Valid bit is zero, so MMU triggers page fault exception
5) Handler identifies victim, and if dirty pages it out to disk
6) Handler pages in new page and updates PTE in memory
7) Handler returns to original process, restarting faulting instruction.
– 21 –
15-213, F’07
Integrating VM and Cache
PTE
CPU chip
PTE
PTEA
hit
PTEA
Processor
VA
PTEA
miss
PTEA
MMU
Memory
PA
PA
miss
PA
hit
Data
PA
Data
L1
cache
Page table entries (PTEs) are cached in L1 like any other memory word.


PTEs can be evicted by other data references
PTE hit still requires a 1-cycle delay
Solution: Cache PTEs in a small fast memory in the MMU.

– 22 –
Translation Lookaside Buffer (TLB)
15-213, F’07
Speeding up Translation with a TLB
Translation Lookaside Buffer (TLB)



– 23 –
Small hardware cache in MMU
Maps virtual page numbers to physical page numbers
Contains complete page table entries for small number of
pages
15-213, F’07
TLB Hit
CPU chip
TLB
2
1
Processor
VA
VPN
PTE
Translation
5
3
4
PA
Cache/
memory
Data
A TLB hit eliminates a memory access.
– 24 –
15-213, F’07
TLB Miss
CPU chip
TLB
4
2
VPN
PTE
3
PTEA
1
Processor
VA
Translation
PA
Cache/
memory
5
Data
6
A TLB miss incurs an additional memory access (the PTE).
Fortunately, TLB misses are rare. Why?
– 25 –
15-213, F’07
Simple Memory System Example
Addressing



14-bit virtual addresses
12-bit physical address
Page size = 64 bytes
13
12
11
10
9
8
7
6
5
4
VPN
– 26 –
10
2
1
0
VPO
(Virtual Page Offset)
(Virtual Page Number)
11
3
9
8
7
6
5
4
3
2
1
PPN
PPO
(Physical Page Number)
(Physical Page Offset)
0
15-213, F’07
Simple Memory System Page Table

– 27 –
Only show first 16 entries (out of 256)
VPN
PPN
Valid
VPN
PPN
Valid
00
28
1
08
13
1
01
–
0
09
17
1
02
33
1
0A
09
1
03
02
1
0B
–
0
04
–
0
0C
–
0
05
16
1
0D
2D
1
06
–
0
0E
11
1
07
–
0
0F
0D
1
15-213, F’07
Simple Memory System TLB
TLB


16 entries
4-way associative
TLBT
13
12
11
10
TLBI
9
8
7
6
5
4
3
VPN
2
1
0
VPO
Set
Tag
PPN
Valid
Tag
PPN
Valid
Tag
PPN
Valid
Tag
PPN
Valid
0
03
–
0
09
0D
1
00
–
0
07
02
1
1
03
2D
1
02
–
0
04
–
0
0A
–
0
2
02
–
0
08
–
0
06
–
0
03
–
0
3
07
–
0
03
0D
1
0A
34
1
02
–
0
– 28 –
15-213, F’07
Simple Memory System Cache
Cache



16 lines
4-byte line size
Direct mapped
CI
CT
11
10
9
8
7
6
5
4
PPN
CO
3
2
1
0
PPO
Idx
Tag
Valid
B0
B1
B2
B3
Idx
Tag
Valid
B0
B1
B2
B3
0
19
1
99
11
23
11
8
24
1
3A
00
51
89
1
15
0
–
–
–
–
9
2D
0
–
–
–
–
2
1B
1
00
02
04
08
A
2D
1
93
15
DA
3B
3
36
0
–
–
–
–
B
0B
0
–
–
–
–
4
32
1
43
6D
8F
09
C
12
0
–
–
–
–
5
0D
1
36
72
F0
1D
D
16
1
04
96
34
15
6
31
0
–
–
–
–
E
13
1
83
77
1B
D3
7
16
1
11
C2
DF
03
F
14
0
–
–
–
–
– 29 –
15-213, F’07
Address Translation Example #1
Virtual Address 0x03D4
TLBT
TLBI
13
12
11
10
9
8
7
6
5
4
3
2
1
0
0
0
0
0
1
1
1
1
0
1
0
1
0
0
VPN
0x0F
VPN ___
VPO
3 TLBT ____
0x03
TLBI ___
Y
TLB Hit? __
Page Fault? NO
__ PPN:0x0D
____
Physical Address
CI
CT
11
10
9
8
7
6
5
4
3
2
1
0
0
0
1
1
0
1
0
1
0
1
0
0
PPN
0
Offset ___
– 30 –
CO
0x5
CI___
CT 0x0D
____
PPO
Y
Hit? __
Byte: 0x36
____
15-213, F’07
Address Translation Example #2
Virtual Address 0x0B8F
TLBT
TLBI
13
12
11
10
9
8
7
6
5
4
3
2
1
0
0
0
1
0
1
1
1
0
0
0
1
1
1
1
VPN
VPO
0x2E TLBI ___
2 TLBT 0x0B
NO Page Fault?YES
VPN ___
____ TLB Hit? __
__ PPN:TBD
____
Physical Address
CI
CT
11
10
9
8
7
6
PPN
Offset ___
– 31 –
CI___
CT ____
5
4
CO
3
2
1
0
PPO
Hit? __
Byte: ____
15-213, F’07
Address Translation Example #3
Virtual Address 0x0020
TLBT
TLBI
13
12
11
10
9
8
7
6
5
4
3
2
1
0
0
0
0
0
0
0
0
0
1
0
0
0
0
0
VPN
0x00
VPN ___
VPO
0 TLBT ____
0x00
TLBI ___
NO Page Fault? NO
TLB Hit? __
__ PPN:0x28
____
Physical Address
CI
CT
11
10
9
8
7
6
5
4
3
2
1
0
1
0
1
0
0
0
1
0
0
0
0
0
PPN
0
Offset ___
– 32 –
CO
0x8
CI___
0x28
CT ____
PPO
Hit? NO
__
Byte: MEM
____
15-213, F’07
Multi-Level Page Tables
Level 2
Tables
Given:



4KB (212) page size
48-bit address space
4-byte PTE
Problem:

Level 1
Table
Would need a 256 GB page table!
Common solution


Multi-level page tables
Example: 2-level page table
...
 248 * 2-12 * 22 = 238 bytes
...
 Level 1 table: each PTE points to a
page table (memory resident)
 Level 2 table: Each PTE points to a
page (paged in and out like other data)
– 33 –
15-213, F’07
A Two-Level Page Table Hierarchy
Level 1
page table
Level 2
page tables
Virtual
memory
0
VP 0
PTE 0
...
PTE 0
...
VP 1023
PTE 1
PTE 1023
VP 1024
PTE 2 (null)
2K allocated VM pages
for code and data
...
PTE 3 (null)
VP 2047
PTE 4 (null)
PTE 0
PTE 5 (null)
...
PTE 6 (null)
PTE 1023
Gap
PTE 7 (null)
6K unallocated VM pages
PTE 8
(1K - 9)
null PTEs
PTE 1023
1023
unallocated
pages
VP 9215
...
– 34 –
1023 null
PTEs
1023 unallocated pages
1 allocated VM page
for the stack
15-213, F’07
Translating with a k-level Page Table
VIRTUAL ADDRESS
n-1
VPN 1
VPN 2
Level 2
page table
Level 1
page table
...
... ...
p-1
VPN k
0
VPO
Level k
page table
PPN
m-1
p-1
PPN
0
PPO
PHYSICAL ADDRESS
– 35 –
15-213, F’07
Summary
Programmer’s View of Virtual Memory


Each process has its own private linear address space
Cannot be corrupted by other processes
System View of Virtual Memory

Uses memory efficiently by caching virtual memory pages
stored on disk.
 Efficient only because of locality


– 36 –
Simplifies memory management in general, linking, loading,
sharing, and memory allocation in particular.
Simplifies protection by providing a convenient
interpositioning point to check permissions.
15-213, F’07