2nd Half Review
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
Silberschatz, Galvin and Gagne ©2007
What Have We Covered in the 2nd Half?






Chapter 8: Memory Management
Chapter 9: Virtual Memory
Chapter 10: File Systems
Chapter 11: FS Implementation
Chapter 12: Storage
Chapter 13: I/O
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.2
Silberschatz, Galvin and Gagne ©2007
Binding of Instructions and Data to Memory

Address binding of instructions and
data to memory addresses can
happen at three different stages
 Compile time: If memory location
known a priori, absolute code
can be generated; must recompile
code if starting location changes
 Load time: Must generate
relocatable code if memory
location is not known at compile
time
 Execution time: Binding delayed
until run time if the process can be
moved during its execution from
one memory segment to another.
Need hardware support for
address maps (e.g., base and limit
registers)
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.3
Silberschatz, Galvin and Gagne ©2007
Logical vs. Physical Address Space


The concept of a logical address space that is
bound to a separate physical address space
is central to proper memory management
 Logical address – generated by the CPU;
also referred to as virtual address
 Physical address – address seen by the
memory unit
Logical and physical addresses are the same
in compile-time and load-time address-binding
schemes; logical (virtual) and physical
addresses differ in execution-time addressbinding scheme
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.4
Silberschatz, Galvin and Gagne ©2007
Memory-Management Unit (MMU)



Hardware device that
maps virtual to physical
address
In MMU scheme, the
value in the relocation
register is added to every
address generated by a
user process at the time it
is sent to memory
The user program deals
with logical addresses; it
never sees the real
physical addresses
Dynamic relocation using a relocation register
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.5
Silberschatz, Galvin and Gagne ©2007
Dynamic Loading and Linking


Dynamic Loading:
 Routine is not loaded until it is called
 Better memory-space utilization; unused routine is
never loaded
 Useful when large amounts of code are needed to
handle infrequently occurring cases
Dynamic Linking:
 Linking postponed until execution time
 Small piece of code, stub, used to locate the
appropriate memory-resident library routine
 Stub replaces itself with the address of the routine,
and executes the routine
 Operating system needed to check if routine is in
processes’ memory address
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.6
Silberschatz, Galvin and Gagne ©2007
Swapping

A process can be
swapped temporarily
out of memory to a
backing store, and
then brought back
into memory for
continued execution

Backing store – fast
disk large enough to
accommodate copies
of all memory images
for all users; must
provide direct access
to these memory
images
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.7
Silberschatz, Galvin and Gagne ©2007
Contiguous Allocation

Main memory usually into two partitions:
 Resident operating system, usually held in low memory
with interrupt vector
 User processes then held in high memory

Relocation registers used to protect user processes from
each other, and from changing operating-system code and
data
 Base register contains value of smallest physical
address
 Limit register contains range of logical addresses –
each logical address must be less than the limit register
 MMU maps logical address dynamically
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.8
Silberschatz, Galvin and Gagne ©2007
Contiguous Allocation (Cont.)

Multiple-partition allocation
 Hole – block of available memory; holes of various size
are scattered throughout memory
 When a process arrives, it is allocated memory from a
hole large enough to accommodate it
 Operating system maintains information about:
a) allocated partitions b) free partitions (hole)
OS
OS
OS
OS
process 5
process 5
process 5
process 5
process 9
process 9
process 8
process 2
process 10
process 2
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
process 2
9.9
process 2
Silberschatz, Galvin and Gagne ©2007
Dynamic Storage-Allocation Problem
How to satisfy a request of size n from a list of free holes



First-fit: Allocate the first hole that is big enough
Best-fit: Allocate the smallest hole that is big enough;
must search entire list, unless ordered by size
 Produces the smallest leftover hole
Worst-fit: Allocate the largest hole; must also search
entire list
 Produces the largest leftover hole
First-fit and best-fit better than worst-fit in terms of
speed and storage utilization
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.10
Silberschatz, Galvin and Gagne ©2007
Fragmentation



External Fragmentation – total memory space exists to
satisfy a request, but it is not contiguous
Internal Fragmentation – allocated memory may be slightly
larger than requested memory; this size difference is
memory internal to a partition, but not being used
Reduce external fragmentation by compaction
 Shuffle memory contents to place all free memory
together in one large block
 Compaction is possible only if relocation is dynamic, and
is done at execution time
 I/O problem
 Hold job in memory while it is involved in I/O
 Do I/O only into OS buffers
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.11
Silberschatz, Galvin and Gagne ©2007
Paging







Logical address space of a process can be
noncontiguous; process is allocated physical memory
whenever the latter is available
Divide physical memory into fixed-sized blocks called
frames (size is power of 2, between 512 bytes and
8,192 bytes)
Divide logical memory into blocks of same size called
pages
Keep track of all free frames
To run a program of size n pages, need to find n free
frames and load program
 Pages do not need to be contiguous!
Set up a page table to translate logical to physical
addresses
Avoids external fragmentation for internal fragmentation
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.12
Silberschatz, Galvin and Gagne ©2007
Paging HW and Address Translation

Address generated by CPU is divided into:
 Page number (p) – used as an index into a page table which
contains base address of each page in physical memory
 Page offset (d) – combined with base address to define the
physical memory address that is sent to the memory unit
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.13
Silberschatz, Galvin and Gagne ©2007
Implementation of Page Table






Page table is kept in main memory
Page-table base register (PTBR) points to the page table
Page-table length register (PRLR) indicates size of the
page table
In this scheme every data/instruction access requires two
memory accesses. One for the page table and one for the
data/instruction.
The two memory access problem can be solved by the use of
a special fast-lookup hardware cache called associative
memory or translation look-aside buffers (TLBs)
Some TLBs store address-space identifiers (ASIDs) in each
TLB entry – uniquely identifies each process to provide
address-space protection for that process
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.14
Silberschatz, Galvin and Gagne ©2007
Paging Hardware With TLB
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.15
Silberschatz, Galvin and Gagne ©2007
Effective Access Time





Associative Lookup =  time unit
Assume memory cycle time is 1 microsecond
Hit ratio – percentage of times that a page
number is found in the associative registers;
ratio related to number of associative registers
Hit ratio = 
Effective Access Time (EAT)
EAT = (1 + )  + (2 + )(1 – )
=  +  + 2 +  -  - 2
=2+–
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.16
Silberschatz, Galvin and Gagne ©2007
Shared Pages Example

Shared code
 One copy of read-only
(reentrant) code shared
among processes (i.e., text
editors, compilers, window
systems).
 Shared code must appear in
same location in the logical
address space of all
processes

Private code and data
 Each process keeps a
separate copy of the code and
data
 The pages for the private code
and data can appear
anywhere in the logical
address space
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.17
Silberschatz, Galvin and Gagne ©2007
Page Table Size Issues




Given:
 A 32 bit address space (1 GB)
 1 KB pages
 A page table entry of 4 bytes
Implication:
 Page table is 4 MB per process!
 Huge size and context switching costs
Address space usage tends to be sparce
 Most programs don’t use all of 32 bits
Approaches:
 Hierarchical Page Tables
 Hashed Page Tables
 Inverted Page Tables
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.18
Silberschatz, Galvin and Gagne ©2007
Hierarchical Page Tables

Break up the
logical address
space into
multiple page
tables

A simple
technique is a
two-level page
table
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.19
Silberschatz, Galvin and Gagne ©2007
Hashed Page Table



Common in address spaces > 32 bits
The virtual page number is hashed into a page table. This
page table contains a chain of elements hashing to the
same location.
Virtual page numbers are compared in this chain searching
for a match. If a match is found, the corresponding physical
frame is extracted.
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.20
Silberschatz, Galvin and Gagne ©2007
Inverted Page Table Architecture




One entry for each real
page of memory
Entry consists of the virtual
address of the page stored
in that real memory
location, with information
about the process that
owns that page
Decreases memory
needed to store each page
table, but increases time
needed to search the table
when a page reference
occurs
Use hash table to limit the
search to one — or at most
a few — page-table entries
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.21
Silberschatz, Galvin and Gagne ©2007
Segmentation
1
4
1
2
2
3
4
3
user space
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
physical memory space
9.22
Silberschatz, Galvin and Gagne ©2007
Segmentation Arch & Hardware




Logical address consists of a two tuple:
<segment-number, offset>,
Segment table – maps two-dimensional
physical addresses; each table entry has:
 base – contains the starting physical
address where the segments reside in
memory
 limit – specifies the length of the
segment
Segment-table base register (STBR)
points to the segment table’s location in
memory
Segment-table length register (STLR)
indicates number of segments used by a
program; segment number s is legal if
s < STLR
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.23
Silberschatz, Galvin and Gagne ©2007
Pentium Address Translation


Supports both segmentation and
segmentation with paging
CPU generates logical address
 Given to segmentation unit
 Which produces linear addresses
 Linear address given to paging unit
 Which generates physical address in main memory
 Paging units form equivalent of MMU
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.24
Silberschatz, Galvin and Gagne ©2007
Three-level Paging in Linux
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.25
Silberschatz, Galvin and Gagne ©2007
Virtual Memory That is Larger Than Physical Memory


Virtual memory – separation of user
logical memory from physical memory.
 Only part of the program needs to be
in memory for execution
 Logical address space can therefore
be much larger than physical
address space
 Allows address spaces to be shared

by several processes
 Allows for more efficient process
creation
Virtual memory can be implemented via:
 Demand paging
 Demand segmentation
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.26
Silberschatz, Galvin and Gagne ©2007
Shared Library Using Virtual Memory
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.27
Silberschatz, Galvin and Gagne ©2007
Demand Paging



Bring a page into memory only
when it is needed
 Less I/O needed
 Less memory needed
 Faster response
 More users
Page is needed  reference to it
 invalid reference  abort
 not-in-memory  bring to
memory
Lazy swapper – never swaps a
page into memory unless page
will be needed
 Swapper that deals with
pages is a pager
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.28
Silberschatz, Galvin and Gagne ©2007
Page Fault

1.
2.
3.
4.
5.
6.
If there is a reference to a page,
first reference to that page will trap
to operating system:
page fault
Operating system looks at another
table to decide:
 Invalid reference  abort
 Just not in memory
Get empty frame
Swap page into frame
Reset tables
Set validation bit = v
Restart the instruction that caused
the page fault
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.29
Silberschatz, Galvin and Gagne ©2007
Performance of Demand Paging

Page Fault Rate 0  p  1.0
 if p = 0 no page faults
 if p = 1, every reference is a fault

Effective Access Time (EAT)
EAT = (1 – p) x memory access
+ p (page fault overhead
+ swap page out
+ swap page in
+ restart overhead )
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.30
Silberschatz, Galvin and Gagne ©2007
Copy on Write (COW)




Copy-on-Write (COW)
allows both parent and
child processes to
initially share the same
pages in memory
If either process
modifies a shared
page, only then is the
page copied
COW allows more
efficient process
creation as only
modified pages are
copied
Free pages are
allocated from a pool
of zeroed-out pages
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.31
Silberschatz, Galvin and Gagne ©2007
Page Replacement
1.
2.
3.
4.
Find the location of the
desired page on disk
Find a free frame:
- If there is a free
frame, use it
- If there is no free
frame, use a page
replacement algorithm
to select a victim
frame
Bring the desired
page into the (newly)
free frame; update the
page and frame tables
Restart the process
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.32
Silberschatz, Galvin and Gagne ©2007
Page Replacement Algorithms







Want lowest page-fault rate
Evaluate algorithm by running it on a
particular string of memory references
(reference string) and computing the
number of page faults on that string
FIFO
OPT
LRU
Approximating LRU
MFU, MRU
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.33
Silberschatz, Galvin and Gagne ©2007
FIFO Illustrating Belady’s Anomaly

Reference string: 1, 2, 3, 4, 1, 2, 5, 1, 2, 3, 4, 5
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.34
Silberschatz, Galvin and Gagne ©2007
LRU: Concept vs. Reality



LRU is considered to be a reasonably good algorithm
Problem is in implementing it
 Counter implementation: counter per page, copied
per memory reference, have to search pages on page
replacement to find oldest
 Stack implementation: no search, but pointer swap on
each memory reference
Thus the efforts to design efficient implementations that
approximate LRU
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.35
Silberschatz, Galvin and Gagne ©2007
LRU Approximation Algorithms

Reference bit




With each page associate a bit, initially = 0
When page is referenced bit set to 1
Replace the one which is 0 (if one exists)
 We do not know the order, however
Second chance



Need reference bit
Clock replacement
If page to be replaced (in clock order) has reference bit = 1
then:
 set reference bit 0
 leave page in memory
 replace next page (in clock order), subject to same rules
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.36
Silberschatz, Galvin and Gagne ©2007
Second-Chance (clock) Page-Replacement Algorithm
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.37
Silberschatz, Galvin and Gagne ©2007
Allocation of Frames




Each process needs minimum number of pages
Example: IBM 370 – 6 pages to handle SS MOVE
instruction:
 instruction is 6 bytes, might span 2 pages
 2 pages to handle from
 2 pages to handle to
Two major allocation schemes
 fixed allocation
 priority allocation
Ways to do allocation
 Local allocation (within process)
 Global allocation (across processes)
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.38
Silberschatz, Galvin and Gagne ©2007
Thrashing


If a process does not have “enough” pages, the page-fault rate
is very high. This leads to:
 low CPU utilization
 operating system thinks that it needs to increase the degree
of multiprogramming
 another process added to the system
Thrashing  a process is busy swapping pages in and out
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.39
Silberschatz, Galvin and Gagne ©2007
Locality In A Memory-Reference Pattern
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.40
Silberschatz, Galvin and Gagne ©2007
Working-Set Model





  working-set window  a fixed number of page references
Example: 10,000 instruction
WSSi (working set of Process Pi) =
total number of pages referenced in the most recent  (varies in
time)
 if  too small will not encompass entire locality
 if  too large will encompass several localities
 if  =   will encompass entire program
D =  WSSi  total demand frames
if D > m  Thrashing
Policy if D > m, then suspend one of the processes
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.41
Silberschatz, Galvin and Gagne ©2007
Implementing Working Set




Approximate with interval timer + a reference bit
Example:  = 10,000
 Each memory reference sets reference bit
 Timer interrupts after every 5000 time units
 Keep in memory 2 history bits for each page
 Whenever a timer interrupts, copy reference bit to history and
set the values of all reference bits to 0
 If one of reference or history bits = 1  page in working set
Why is this not completely accurate?
 Can’t tell (within interval of 5000) where reference occurred
Improvement = 10 bits and interrupt every 1000 time units
 But interrupt overhead starts getting substantial
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.42
Silberschatz, Galvin and Gagne ©2007
Page-Fault Frequency Scheme


Establish “acceptable” page-fault rate
PFF: Page Fault Frequency (page faults /
instructions executed)
 If actual rate too low, process loses frame
 If actual rate too high, process gains frame
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.43
Silberschatz, Galvin and Gagne ©2007
Memory Mapped Files



Memory-mapped file I/O
allows file I/O to be
treated as routine memory
access by mapping a
disk block to a page in
memory
Simplifies file access by
treating file I/O through
memory rather than
read() write()
system calls
Also allows several
processes to map the
same file allowing the
pages in memory to be
shared
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.44
Silberschatz, Galvin and Gagne ©2007
Allocating Kernel Memory


Treated differently from user memory
Often allocated from a free-memory pool
 Kernel requests memory for structures of
varying sizes
 Some kernel memory needs to be
contiguous
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.45
Silberschatz, Galvin and Gagne ©2007
Buddy System Allocator

Memory allocated using
power-of-2 allocator
 Satisfies requests in
units sized as power
of 2
 Request rounded up
to next highest power
of 2
 When smaller
allocation needed
than is available,
current chunk split
into two buddies of
next-lower power of 2
 Continue until
appropriate sized
chunk available
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.46
Silberschatz, Galvin and Gagne ©2007
Slab Allocator







Slab is one or more physically
contiguous pages
Cache consists of one or more
slabs
Single cache for each unique
kernel data structure
 Each cache filled with objects
– instantiations of the data
structure
When cache created, filled with
objects marked as free
When structures stored, objects
marked as used
If slab is full of used objects, next
object allocated from empty slab
 If no empty slabs, new slab
allocated
Benefits include no fragmentation,
fast memory request satisfaction
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.47
Silberschatz, Galvin and Gagne ©2007
Other Issues





Prepaging
Page Size
TLB Reach
Program Structure
I/O Interlock (Wiring)
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.48
Silberschatz, Galvin and Gagne ©2007
File Attributes








Name – only information kept in human-readable form
Identifier – unique tag (number) identifies file within file
system
Type – needed for systems that support different types
Location – pointer to file location on device
Size – current file size
Protection – controls who can do reading, writing,
executing
Time, date, and user identification – data for protection,
security, and usage monitoring
Information about files are kept in the directory structure,
which is maintained on the disk
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.49
Silberschatz, Galvin and Gagne ©2007
File Operations








Create
Write
Read
Reposition within file
Delete
Truncate
Open(Fi) – search the directory structure on disk for
entry Fi, and move the content of entry to memory
Close (Fi) – move the content of entry Fi in memory
to directory structure on disk
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.50
Silberschatz, Galvin and Gagne ©2007
Directory Structure

A collection of nodes containing information about all files
Directory
Files
F1
F2
F4
F3
Fn
Both the directory structure and the files reside on disk
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.51
Silberschatz, Galvin and Gagne ©2007
Operations Performed on Directory






List directory contents
Search for a file
Create a file
Delete a file
Rename a file
Traverse the file system
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.52
Silberschatz, Galvin and Gagne ©2007
Single-Level Directory

A single directory for all users
 Called the root directory
 Pros: Simple, easy to quickly locate files
 Cons: inconvenient naming (uniqueness), no grouping
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.53
Silberschatz, Galvin and Gagne ©2007
Two-Level Directory

Separate directory for each user
 Introduces the notion of a path name
 Can have the same file name for different user
 Efficient searching
 No grouping capability
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.54
Silberschatz, Galvin and Gagne ©2007
Tree-Structured Directories
 Directories can now contain files and subdirectories
 Efficient searching, allows grouping
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.55
Silberschatz, Galvin and Gagne ©2007
Acyclic-Graph Directories

Allow sharing of subdirectories and files
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.56
Silberschatz, Galvin and Gagne ©2007
General Graph Directory
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.57
Silberschatz, Galvin and Gagne ©2007
File System Mounting

Mount allows two FSes to be merged into one
 For example you insert your floppy into the root FS:
mount(“/dev/fd0”, “/mnt”, 0)
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.58
Silberschatz, Galvin and Gagne ©2007
Protection


File owner/creator should be able to control:
 what can be done
 by whom
Types of access
 Read
 Write
 Execute
 Append
 Delete
 List
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.59
Silberschatz, Galvin and Gagne ©2007
Issues with UNIX Access Rights


Just a single owner, a single group and the public
 Pro: Compact enough to fit in just a few bytes
 Con: Not very expressive
Access Control List: This is a per-file list that tells who
can access that file
 Pro: Highly expressive
 Con: Harder to represent in a compact way
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.60
Silberschatz, Galvin and Gagne ©2007
Layered File System




Operating System Concepts with Java – 7th Edition, Nov 15, 2006
I/O Control: device drivers and
interrupt handlers that talk to the
disk
Basic file system: Generic block
reads and writes
 e.g., read cylinder 73, track 2,
sector 10
File organization: Files and logical
blocks
 Translate logical blocks to
physical
 Manage free space
Logical file system: Metadata
information
 e.g., owner, permissions, size,
etc.
9.61
Silberschatz, Galvin and Gagne ©2007
A Typical File Control Block

FCB has all meta-information about a file
 Linux calls these i-nodes
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.62
Silberschatz, Galvin and Gagne ©2007
In-Memory File System Structures
opening a file
reading a file
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.63
Silberschatz, Galvin and Gagne ©2007
Virtual File System (VFS)



Virtual File Systems
(VFS) provide an objectoriented way of
implementing file
systems.
VFS allows the same
system call interface
(the API) to be used for
different types of file
systems.
The API is to the VFS
interface, rather than
any specific type of file
system.
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.64
Silberschatz, Galvin and Gagne ©2007
Contiguous Allocation



Pros:
 Simple: state required per file is start block and size
 Performance: entire file can be read with one seek
Cons:
 Files can’t grow
 Fragmentation: external frag is bigger problem
 Wastes space
Used in CDROMs, DVDs
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.65
Silberschatz, Galvin and Gagne ©2007
Linked List Allocation



Pros:
 No space lost to
external
fragmentation
 Disk only needs to
maintain first block
of each file
Cons:
 Random access is
costly
 Overheads of
pointers
Used in MS-DOS
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.66
Silberschatz, Galvin and Gagne ©2007
Indexed Allocation

Brings all pointers to data blocks together into an
index block.
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.67
Silberschatz, Galvin and Gagne ©2007
Combined Scheme: UNIX (4K bytes per block)


If data blocks are 4K …
 First 48K reachable
from the inode
 Next 4MB available
from single-indirect
 Next 4GB available
from double-indirect
 Next 4TB available
through the tripleindirect block
Any block can be found
with at most 3 disk
accesses
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.68
Silberschatz, Galvin and Gagne ©2007
Implementing Directories



Directory: map ASCII file name to file attributes & location
When a file is opened, OS uses path name to find dir
 Directory has information about the file’s disk blocks
 Whole file (contiguous), first block (linked-list) or Inode (indexed allocation)
 Directory also has attributes of each file
2 options: entries have all attributes, or point to file I-node
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.69
Silberschatz, Galvin and Gagne ©2007
Managing file names: Example
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.70
Silberschatz, Galvin and Gagne ©2007
Hard Links
File name
Inode#
Foo.txt
2433
Hard.lnk
2433
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
Inode
Inode
#2433
9.71
Silberschatz, Galvin and Gagne ©2007
Soft Links
Soft.lnk
43234
Inode
#43234
/path/to/Foo.txt
..and then redirects to Inode #2433 at open() time..
Foo.txt
2433
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
Inode
#2433
9.72
Silberschatz, Galvin and Gagne ©2007
Disk Space Management


Files are stored as fixed-size blocks
What is a good block size?
 If 131,072 bytes/track, rotation 8.33 ms, seek 10 ms
 To read k bytes block: 10+ 4.165 + (k/131072)*8.33 ms
 Median file size: 2 KB
Block size
Operating System Concepts with Java –
7th
Edition, Nov 15, 2006
9.73
Silberschatz, Galvin and Gagne ©2007
Managing Free Disk Space

2 approaches to keep track of free disk blocks
 Linked list and bitmap approach
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.74
Silberschatz, Galvin and Gagne ©2007
Free-Space Management Tradeoffs


Bit Map:
 Pro: Easy to get contiguous files
 Con: Bit map requires extra space
 Example:
block size = 212 bytes
disk size = 230 bytes (1 gigabyte)
n = 230/212 = 218 bits (or 32K bytes)
Linked list:
 Pro: no wasted extra space
 Use free blocks themselves to store free block list
 Con: Cannot get contiguous space easily
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.75
Silberschatz, Galvin and Gagne ©2007
I/O Using a Page Cache



A page cache caches
pages rather than disk
blocks using virtual
memory techniques
Memory-mapped I/O
uses a page cache
Routine I/O through
the file system uses
the buffer (disk) cache
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.76
Silberschatz, Galvin and Gagne ©2007
I/O Using a Unified Buffer Cache

A unified buffer cache uses the same page cache
to cache both memory-mapped pages and
ordinary file system I/O
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.77
Silberschatz, Galvin and Gagne ©2007
Log Structured File System (LFS)
inode
file2
file1
directory
dir1
data
dir2
Unix File
System
dir2
dir1
Log
file1
inode map
file2
Log-Structured
File System
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.78
Blocks written to
create two 1-block
files: dir1/file1 and
dir2/file2, in UFS and
LFS
Silberschatz, Galvin and Gagne ©2007
Network File System (NFS)
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.79
Silberschatz, Galvin and Gagne ©2007
NFS Architecture



System Call Layer
Virtual File System
(VFS) layer
NFS service layer
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.80
Silberschatz, Galvin and Gagne ©2007
Chapter 12: Mass-Storage Systems
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
Silberschatz, Galvin and Gagne ©2007
What Does The Disk Look Like?
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.82
Silberschatz, Galvin and Gagne ©2007
Disk Overheads


To read from disk, we must specify:
 cylinder #, surface #, sector #, transfer size, memory address
Latency includes:
 Seek time: to get to the track
 Latency time: to get to the sector (rotation delay)
 Transfer time: get bits off the disk
Track
Sector
Rotation
Delay
Seek Time
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.83
Silberschatz, Galvin and Gagne ©2007
Disk Structure



Disk drives addressed as 1-dim arrays of logical blocks

the logical block is the smallest unit of transfer

Logical block addressing
This array mapped sequentially onto disk sectors

Address 0 is 1st sector of 1st track of the outermost
cylinder

Addresses incremented within track, then within
tracks of the cylinder, then across cylinders, from
innermost to outermost
Translation is theoretically possible, but usually difficult

Some sectors might be defective

Number of sectors per track is not a constant
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.84
Silberschatz, Galvin and Gagne ©2007
Non-Uniform # Sectors / Track


Maintain same data rate with Constant Linear Velocity
Approaches:
 Reduce bit density per track for outer layers
 Have more sectors per track on the outer layers (virtual
geometry)
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.85
Silberschatz, Galvin and Gagne ©2007
How is the Disk Formatted?



After manufacturing disk has no information
 Is stack of platters coated with magnetizable metal oxide
Before use, each platter receives low-level format
 Format has series of concentric tracks
 Each track contains some sectors
 There is a short gap between sectors
Preamble allows h/w to recognize start of sector
 Also contains cylinder and sector numbers
 Data is usually 512 bytes
 ECC field used to detect and recover from read errors
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.86
Silberschatz, Galvin and Gagne ©2007
Disk Partitioning




Each partition is like a separate disk
Sector 0 is Master Boot Record (MBR)
 Contains boot code + partition table (starting sector and size)
High-level formatting
 Done for each partition
 Specifies boot block, free list, root directory, empty file system
What happens on boot?
 BIOS loads MBR, boot program checks to see active partition
 Reads boot sector from that partition that then boots OS kernel,
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.87
Silberschatz, Galvin and Gagne ©2007
Handling Errors


A disk track with a bad sector
Solutions:
 Substitute a spare for the bad sector (sector sparing)
 Shift all sectors to bypass bad one (sector forwarding)
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.88
Silberschatz, Galvin and Gagne ©2007
Disk Scheduling





The operating system tries to use hardware efficiently
 for disk drives  having fast access time, disk
bandwidth
Access time has two major components
 Seek time is time to move the heads to the cylinder
containing the desired sector
 Rotational latency is additional time waiting to rotate
the desired sector to the disk head.
Want to minimize seek time
Seek time  seek distance
Disk bandwidth is total number of bytes transferred,
divided by the total time between the first request for
service and the completion of the last transfer.
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.89
Silberschatz, Galvin and Gagne ©2007
FCFS (First-Come First Serve)
Illustration shows total head movement of 640 cylinders.
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.90
Silberschatz, Galvin and Gagne ©2007
SSTF (Shortest Seek Time First)


Selects request with minimum seek time from current head position
SSTF scheduling is a form of SJF scheduling
 may cause starvation of some requests.
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.91
Silberschatz, Galvin and Gagne ©2007
SCAN (Elevator)

The disk arm starts at one end of the disk,
 moves toward the other end, servicing requests
 head movement is reversed when it gets to the other end of disk
 servicing continues.
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.92
Silberschatz, Galvin and Gagne ©2007
C-SCAN

The head moves from one end of the disk to the other.
 servicing requests as it goes.
 When it reaches other end immediately returns to beginning of disk
 No requests serviced on the return trip.
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.93
Silberschatz, Galvin and Gagne ©2007
C-LOOK (Version of C-SCAN)

Arm only goes as far as last request in each direction,
 then reverses direction immediately,
 without first going all the way to the end of the disk.
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.94
Silberschatz, Galvin and Gagne ©2007
Selecting a Good Algorithm

SSTF is common and has a natural appeal

SCAN and C-SCAN perform better under heavy load

Performance depends on number and types of requests

Requests for disk service can be influenced by the fileallocation method.

Disk-scheduling algo should be a separate OS module


allowing it to be replaced with a different algorithm if
necessary.
Either SSTF or C-LOOK is a reasonable default algo
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.95
Silberschatz, Galvin and Gagne ©2007
RAID





RAID: Redundant Array of Inexpensive Disks
 In industry, “I” is for “Independent”
 Alternative: SLED, single large expensive disk
Disks are small and cheap, so it’s easy to put lots of
disks (10s to 100s) in one box for increased storage,
performance, and availability
The RAID box with a RAID controller looks just like a
SLED to the computer
Data plus some redundant information is striped
across the disks in some way
How that striping is done is key to performance and
reliability.
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.96
Silberschatz, Galvin and Gagne ©2007
Various RAID Levels
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.97
Silberschatz, Galvin and Gagne ©2007
Nested RAID: RAID 0 + 1, 1 + 0
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.98
Silberschatz, Galvin and Gagne ©2007
Stable Storage: Approach


Use 2 identical disks
 corresponding blocks on both drives are the same
3 operations:
 Stable write: retry on 1st until successful, then try 2nd disk
 Stable read: read from 1st. If ECC error, then try 2nd
 Crash recovery: scan corresponding blocks on both disks
 If one block is bad, replace with good one
 If both are good, replace block in 2nd with the one in 1st
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.99
Silberschatz, Galvin and Gagne ©2007
NAS vs. SAN
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.100
Silberschatz, Galvin and Gagne ©2007
Swap-Space Management



Swap-space — Virtual memory uses disk space as an
extension of main memory.
Swap-space can be carved out of the normal file system,or,
more commonly, it can be in a separate disk partition.
Swap-space management
 4.3BSD allocates swap space when process starts; holds
text segment (the program) and data segment.
 Kernel uses swap maps to track swap-space use.
 Solaris 2 allocates swap space only when a page is
forced out of physical memory, not when the virtual
memory page is first created.
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.101
Silberschatz, Galvin and Gagne ©2007
I/O Devices

Device Rates vary over
many orders of magnitude
 System better be able
to handle this wide
range
 Better not have high
overhead/byte for fast
devices!
 Better not waste time
waiting for slow
devices
Some typical device,
network, and bus data
rates.
Table from Tanenbaum, Modern Operating Systems
3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved.
0-13-6006639
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.102
Silberschatz, Galvin and Gagne ©2007
Intel Chipset Components: Pentium 4


Northbridge:
 Handles memory
 Graphics
Southbridge:
 PCI bus
 Disk controllers
 USB controllers
 Audio
 Serial I/O
 Interrupt controller
 Timers
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.103
Silberschatz, Galvin and Gagne ©2007
How does processor talk to device?
Memory / I/O Bus
CPU
Interrupt
Controller


Bus
Adaptor
Other Devices
or Buses
Regular
Memory
Bus
Adaptor
Address+
Data
Interrupt Request
Device
Controller
Bus
Interface
read
write
control
status
Registers
(port 0x20)
Hardware
Controller
Addressable
Memory
and/or
Queues
CPU interacts with a Controller
 Contains a set of registers that
can be read and written
Memory Mapped
 May contain memory for request
Region: 0x8f008020
queues or bit-mapped images
Regardless of the complexity of the connections and buses, processor accesses
registers in two ways:
 I/O instructions: explicit in/out instructions
 Example from the Intel architecture: out 0x21,AB
 Memory mapped I/O: load/store instructions
 Registers/memory appear in physical address space
 I/O accomplished with load and store instructions
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.104
Silberschatz, Galvin and Gagne ©2007
Memory-Mapped I/O vs. Explicit I/O Instructions

Explicit I/O Instructions:
 Must use assembly language
 Prevents user-mode I/O

Memory-Mapped I/O:
 No need for special instructions (can use in C)
 Allows user-based I/O
 Caching addresses must be prevented
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.105
Silberschatz, Galvin and Gagne ©2007
Notifying the OS: Polling vs. Interrupts


Polling:
 OS periodically checks a device-specific status
register
 Busy-wait cycle to wait for I/O from device
 Pro: simple, potentially low overhead
 Con: may waste many cycles on polling if
infrequent, expensive, or unpredictable I/O
operations
Interrupts:
 Device generates an interrupt whenever it needs
service
 Pro: handles unpredictable events well
 Con: interrupts relatively high overhead
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.106
Silberschatz, Galvin and Gagne ©2007
Precise and Imprecise Interrupts
(a) A precise interrupt. (b) An imprecise interrupt.
Picture from Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved. 0-13-6006639
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.107
Silberschatz, Galvin and Gagne ©2007
Transfering Data To/From Controller


Programmed I/O:
 Each byte transferred via processor in/out or
load/store
 Pro: Simple hardware, easy to program
 Con: Consumes processor cycles proportional
to data size
Direct Memory Access:
 Give controller access to memory bus
 Ask it to transfer data to/from memory directly
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.108
Silberschatz, Galvin and Gagne ©2007
Direct Memory Access (DMA)



Used to avoid programmed I/O for large data movement
Requires DMA controller
Bypasses CPU to transfer data directly between I/O device and
memory
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.109
Silberschatz, Galvin and Gagne ©2007
Device Drivers

Device-specific code
in the kernel that
interacts directly with
the device hardware
 Supports a
standard, internal
interface
 Same kernel I/O
system can
interact easily with
different device
drivers
 Special devicespecific
configuration
supported with the
ioctl() system
call
Picture from Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved. 0-13-6006639
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.110
Silberschatz, Galvin and Gagne ©2007
Device Types


Block devices include disk drives
 Commands include read, write, seek
 Raw I/O or file-system access
 Memory-mapped file access possible
Character devices include keyboards, mice, serial ports
 Single character at a time
 Commands include get, put
Libraries layered on top allow line editing
Network Devices
 Both block and character-like
 Socket interface common


Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.111
Silberschatz, Galvin and Gagne ©2007
Clock Devices and Timers






Maintaining the time of day
Accounting for CPU usage
Preventing processes from running longer than they
are allowed to
Handling alarm system call made by user
processes
Providing watchdog timers for parts of the system
itself.
Doing profiling, monitoring, statistics gathering
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.112
Silberschatz, Galvin and Gagne ©2007
Blocking and Nonblocking I/O



Blocking Interface: “Wait”
 When request data (e.g. read() system call), put process
to sleep until data is ready
 When write data (e.g. write() system call), put process
to sleep until device is ready for data
Non-blocking Interface: “Don’t Wait”
 Returns quickly from read or write request with count of
bytes successfully transferred
 Read may return nothing, write may write nothing
Asynchronous Interface: “Tell Me Later”
 When request data, take pointer to user’s buffer, return
immediately; later kernel fills buffer and notifies user
 When send data, take pointer to user’s buffer, return
immediately; later kernel takes data and notifies user
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.113
Silberschatz, Galvin and Gagne ©2007
Life Cycle of An I/O Request
User
Program
Kernel I/O
Subsystem
Device Driver
Top Half
Device Driver
Bottom Half
Device
Hardware
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.114
Silberschatz, Galvin and Gagne ©2007
I/O Performance

I/O a major factor in system performance:
 Demands CPU to execute device driver,
kernel I/O code
 Context switches due to interrupts
 Data copying
 Network traffic especially stressful

Improving performance:
 Use DMA
 Reduce/eliminate data copying
 Reduce number of context switches
 Reduce interrupts by using large transfers,
smart controllers, polling
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.115
Silberschatz, Galvin and Gagne ©2007
Interaction of I/O and VM





The kernel deals with (kernel) virtual addresses
These do not necessarily correspond to physical
addresses
Contiguous virtual addresses are probably not
contiguous in physical memory
Some systems have an I/O map — the I/O bus manager
has a (version of) the virtual memory map
More often, the kernel has to translate the virtual
addresses itself
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.116
Silberschatz, Galvin and Gagne ©2007
Other I/O Issues

Scatter/Gather I/O
 Suppose we’re reading a single packet or disk block into two or
more non-contiguous pages
 The I/O transfer has to use more than one (address, length) pair
for that transfer to scatter the data around memory
 The same applies on output, where it has to be gathered from
different physical pages

Direct I/O
 For efficiency, we may want to avoid copying data to/from user
space
 Sometimes possible to do direct I/O
 Must consult user virtual memory map for translation
 Must lock pages in physical memory during I/O
Operating System Concepts with Java – 7th Edition, Nov 15, 2006
9.117
Silberschatz, Galvin and Gagne ©2007