MODERN
OPERATING SYSTEMS
FIFTH EDITION
This page intentionally left blank
MODERN
OPERATING SYSTEMS
FIFTH EDITION
ANDREW S. TANENBAUM
HERBERT BOS
Vrije Universiteit
Amsterdam, The Netherlands
Content Management: Tracy Johnson, Erin Sullivan
Content Production: Carole Snyder, Pallavi Pandit
Product Management: Holly Stark
Product Marketing: Krista Clark
Rights and Permissions: Anjali Singh
Please contact https://support.pearson.com/getsupport/s/ with any queries on this content.
Cover Image by Jason Consalvo.
Microsoft and/or its respective suppliers make no representations about the suitability of the information
contained in the documents and related graphics published as part of the services for any purpose. All such
documents and related graphics are provided “as is” without warranty of any kind. Microsoft and/or its
respective suppliers hereby disclaim all warranties and conditions with regard to this information, including
all warranties and conditions of merchantability, whether express, implied or statutory, fitness for a particular
purpose, title and non-infringement. In no event shall Microsoft and/or its respective suppliers be liable for
any special, indirect or consequential damages or any damages whatsoever resulting from loss of use, data or
profits, whether in an action of contract, negligence or other tortious action, arising out of or in connection with
the use or performance of information available from the services.
The documents and related graphics contained herein could include technical inaccuracies or typographical
errors. Changes are periodically added to the information herein. Microsoft and/or its respective suppliers
may make improvements and/or changes in the product(s) and/or the program(s) described herein at any time.
Partial screen shots may be viewed in full within the software version specified.
Microsoft® and Windows® are registered trademarks of the Microsoft Corporation in the U.S.A. and other
countries. This book is not sponsored or endorsed by or affiliated with the Microsoft Corporation.
Copyright © 2023, 2014, 2008 by Pearson Education, Inc. or its affiliates, 221 River Street, Hoboken, NJ 07030.
All Rights Reserved. Manufactured in the United States of America. This publication is protected by copyright, and
permission should be obtained from the publisher prior to any prohibited reproduction, storage in a retrieval system,
or transmission in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise. For
information regarding permissions, request forms, and the appropriate contacts within the Pearson Education Global
Rights and Permissions department, please visit www.pearsoned.com/permissions/.
Acknowledgments of third-party content appear on the appropriate page within the text.
PEARSON is an exclusive trademark owned by Pearson Education, Inc. or its affiliates in the U.S. and/or other
countries.
Unless otherwise indicated herein, any third-party trademarks, logos, or icons that may appear in this work are the
property of their respective owners, and any references to third-party trademarks, logos, icons, or other trade dress
are for demonstrative or descriptive purposes only. Such references are not intended to imply any sponsorship,
endorsement, authorization, or promotion of Pearson’s products by the owners of such marks, or any relationship
between the owner and Pearson Education, Inc., or its affiliates, authors, licensees, or distributors.
Library of Congress Cataloging-in-Publication Data
MODERN OPERATING SYSTEMS
Library of Congress Control Number: 2022902564
ScoutAutomatedPrintCode
Rental
ISBN-10:
0-13-761887-5
ISBN-13: 978-0-13-761887-3
To Suzanne, Barbara, Aron, Nathan, Marvin, Matilde, Olivia and Mirte. (AST)
To Marieke, Duko, Jip, Loeka and Spot. (HB)
Pearson’s Commitment
to Diversity, Equity,
and Inclusion
Pearson is dedicated to creating bias-free content that reflects the diversity,
depth, and breadth of all learners’ lived experiences.
We embrace the many dimensions of diversity, including but not limited to race, ethnicity, gender,
sex, sexual orientation, socioeconomic status, ability, age, and religious or political beliefs.
Education is a powerful force for equity and change in our world. It has the potential to deliver
opportunities that improve lives and enable economic mobility. As we work with authors to create
content for every product and service, we acknowledge our responsibility to demonstrate inclusivity
and incorporate diverse scholarship so that everyone can achieve their potential through learning.
As the world’s leading learning company, we have a duty to help drive change and live up to our
purpose to help more people create a better life for themselves and to create a better world.
Our ambition is to purposefully contribute to a world where:
• Everyone has an equitable and lifelong
opportunity to succeed through learning.
• Our educational products and services are inclusive
and represent the rich diversity of learners.
• Our educational content accurately reflects
the histories and lived experiences of the
learners we serve.
• Our educational content prompts deeper
discussions with students and motivates them to
expand their own learning (and worldview).
Accessibility
Contact Us
We are also committed to providing products
that are fully accessible to all learners. As per
Pearson’s guidelines for accessible educational
Web media, we test and retest the capabilities
of our products against the highest standards
for every release, following the WCAG guidelines
in developing new products for copyright year
2022 and beyond.
While we work hard to present unbiased, fully
accessible content, we want to hear from you about
any concerns or needs with this Pearson product so
that we can investigate and address them.
You can learn more about Pearson’s
commitment to accessibility at
https://www.pearson.com/us/accessibility.html
Please contact us with concerns about any
potential bias at
https://www.pearson.com/report-bias.html
For accessibility-related issues, such as using
assistive technology with Pearson products,
alternative text requests, or accessibility
documentation, email the Pearson Disability Support
team at disability.support@pearson.com
CONTENTS
PREFACE
1
xxiii
1
INTRODUCTION
1.1
WHAT IS AN OPERATING SYSTEM? 4
1.1.1 The Operating System as an Extended Machine 4
1.1.2 The Operating System as a Resource Manager 6
1.2
HISTORY OF OPERATING SYSTEMS 7
1.2.1 The First Generation (1945–1955): Vacuum Tubes 8
1.2.2 The Second Generation (1955–1965): Transistors and Batch Systems 8
1.2.3 The Third Generation (1965–1980): ICs and Multiprogramming 10
1.2.4 The Fourth Generation (1980–Present): Personal Computers 15
1.2.5 The Fifth Generation (1990–Present): Mobile Computers 19
1.3
COMPUTER HARDWARE REVIEW 20
1.3.1 Processors 20
1.3.2 Memory 24
1.3.3 Nonvolatile Storage 28
1.3.4 I/O Devices 29
1.3.5 Buses 33
1.3.6 Booting the Computer 34
vii
viii
CONTENTS
1.4
THE OPERATING SYSTEM ZOO 36
1.4.1 Mainframe Operating Systems 36
1.4.2 Server Operating Systems 37
1.4.3 Personal Computer Operating Systems 37
1.4.4 Smartphone and Handheld Computer Operating Systems 37
1.4.5 The Internet of Things and Embedded Operating Systems 37
1.4.6 Real-Time Operating Systems 38
1.4.7 Smart Card Operating Systems 39
1.5
OPERATING SYSTEM CONCEPTS 39
1.5.1 Processes 39
1.5.2 Address Spaces 41
1.5.3 Files 42
1.5.4 Input/Output 45
1.5.5 Protection 46
1.5.6 The Shell 46
1.5.7 Ontogeny Recapitulates Phylogeny 47
1.6
SYSTEM CALLS 50
1.6.1 System Calls for Process Management 53
1.6.2 System Calls for File Management 57
1.6.3 System Calls for Directory Management 58
1.6.4 Miscellaneous System Calls 60
1.6.5 The Windows API 60
1.7
OPERATING SYSTEM STRUCTURE 63
1.7.1 Monolithic Systems 63
1.7.2 Layered Systems 64
1.7.3 Microkernels 66
1.7.4 Client-Server Model 68
1.7.5 Virtual Machines 69
1.7.6 Exokernels and Unikernels 73
1.8
THE WORLD ACCORDING TO C 74
1.8.1 The C Language 74
1.8.2 Header Files 75
1.8.3 Large Programming Projects 76
1.8.4 The Model of Run Time 78
1.9
RESEARCH ON OPERATING SYSTEMS 78
1.10 OUTLINE OF THE REST OF THIS BOOK 79
CONTENTS
ix
1.11 METRIC UNITS 80
1.12 SUMMARY 81
2
PROCESSES AND THREADS
85
2.1
PROCESSES 85
2.1.1 The Process Model 86
2.1.2 Process Creation 88
2.1.3 Process Termination 90
2.1.4 Process Hierarchies 91
2.1.5 Process States 92
2.1.6 Implementation of Processes 94
2.1.7 Modeling Multiprogramming 95
2.2
THREADS 97
2.2.1 Thread Usage 97
2.2.2 The Classical Thread Model 102
2.2.3 POSIX Threads 106
2.2.4 Implementing Threads in User Space 107
2.2.5 Implementing Threads in the Kernel 111
2.2.6 Hybrid Implementations 112
2.2.7 Making Single-Threaded Code Multithreaded 113
2.3
EVENT-DRIVEN SERVERS 116
2.4
SYNCHRONIZATION AND INTERPROCESS COMMUNICATION 119
2.4.1 Race Conditions 119
2.4.2 Critical Regions 120
2.4.3 Mutual Exclusion with Busy Waiting 121
2.4.4 Sleep and Wakeup 127
2.4.5 Semaphores 129
2.4.6 Mutexes 134
2.4.7 Monitors 138
2.4.8 Message Passing 145
2.4.9 Barriers 148
2.4.10 Priority Inversion 150
2.4.11 Avoiding Locks: Read-Copy-Update 151
x
3
CONTENTS
2.5
SCHEDULING 152
2.5.1 Introduction to Scheduling 153
2.5.2 Scheduling in Batch Systems 160
2.5.3 Scheduling in Interactive Systems 162
2.5.4 Scheduling in Real-Time Systems 168
2.5.5 Policy Versus Mechanism 169
2.5.6 Thread Scheduling 169
2.6
RESEARCH ON PROCESSES AND THREADS 171
2.7
SUMMARY 172
MEMORY MANAGEMENT
179
3.1
NO MEMORY ABSTRACTION 180
3.1.1 Running Multiple Programs Without a Memory Abstraction 181
3.2
A MEMORY ABSTRACTION: ADDRESS SPACES 183
3.2.1 The Notion of an Address Space 184
3.2.2 Swapping 185
3.2.3 Managing Free Memory 188
3.3
VIRTUAL MEMORY 192
3.3.1 Paging 193
3.3.2 Page Tables 196
3.3.3 Speeding Up Paging 200
3.3.4 Page Tables for Large Memories 203
3.4
PAGE REPLACEMENT ALGORITHMS 207
3.4.1 The Optimal Page Replacement Algorithm 208
3.4.2 The Not Recently Used Page Replacement Algorithm 209
3.4.3 The First-In, First-Out (FIFO) Page Replacement Algorithm 210
3.4.4 The Second-Chance Page Replacement Algorithm 210
3.4.5 The Clock Page Replacement Algorithm 211
3.4.6 The Least Recently Used (LRU) Page Replacement Algorithm 212
3.4.7 Simulating LRU in Software 212
3.4.8 The Working Set Page Replacement Algorithm 214
3.4.9 The WSClock Page Replacement Algorithm 218
3.4.10 Summary of Page Replacement Algorithms 220
CONTENTS
4
3.5
DESIGN ISSUES FOR PAGING SYSTEMS 221
3.5.1 Local versus Global Allocation Policies 221
3.5.2 Load Control 224
3.5.3 Cleaning Policy 225
3.5.4 Page Size 226
3.5.5 Separate Instruction and Data Spaces 227
3.5.6 Shared Pages 228
3.5.7 Shared Libraries 230
3.5.8 Mapped Files 232
3.6
IMPLEMENTATION ISSUES 232
3.6.1 Operating System Involvement with Paging 232
3.6.2 Page Fault Handling 233
3.6.3 Instruction Backup 234
3.6.4 Locking Pages in Memory 236
3.6.5 Backing Store 236
3.6.6 Separation of Policy and Mechanism 238
3.7
SEGMENTATION 240
3.7.1 Implementation of Pure Segmentation 242
3.7.2 Segmentation with Paging: MULTICS 243
3.7.3 Segmentation with Paging: The Intel x86 248
3.8
RESEARCH ON MEMORY MANAGEMENT 248
3.9
SUMMARY 250
FILE SYSTEMS
4.1
FILES 261
4.1.1 File Naming 261
4.1.2 File Structure 263
4.1.3 File Types 264
4.1.4 File Access 266
4.1.5 File Attributes 267
4.1.6 File Operations 269
4.1.7 An Example Program Using File-System Calls 270
xi
259
xii
5
CONTENTS
4.2
DIRECTORIES 272
4.2.1 Single-Level Directory Systems 272
4.2.2 Hierarchical Directory Systems 273
4.2.3 Path Names 274
4.2.4 Directory Operations 277
4.3
FILE-SYSTEM IMPLEMENTATION 278
4.3.1 File-System Layout 278
4.3.2 Implementing Files 280
4.3.3 Implementing Directories 285
4.3.4 Shared Files 288
4.3.5 Log-Structured File Systems 290
4.3.6 Journaling File Systems 292
4.3.7 Flash-based File Systems 293
4.3.8 Virtual File Systems 298
4.4
FILE-SYSTEM MANAGEMENT AND OPTIMIZATION 301
4.4.1 Disk-Space Management 301
4.4.2 File-System Backups 307
4.4.3 File-System Consistency 312
4.4.4 File-System Performance 315
4.4.5 Defragmenting Disks 320
4.4.6 Compression and Deduplication 321
4.4.7 Secure File Deletion and Disk Encryption 322
4.5
EXAMPLE FILE SYSTEMS 324
4.5.1 The MS-DOS File System 324
4.5.2 The UNIX V7 File System 327
4.6
RESEARCH ON FILE SYSTEMS 330
4.7
SUMMARY 331
INPUT/OUTPUT
5.1
PRINCIPLES OF I/O HARDWARE 337
5.1.1 I/O Devices 338
5.1.2 Device Controllers 338
5.1.3 Memory-Mapped I/O 340
337
CONTENTS
xiii
5.1.4 Direct Memory Access 344
5.1.5 Interrupts Revisited 347
5.2
PRINCIPLES OF I/O SOFTWARE 352
5.2.1 Goals of the I/O Software 352
5.2.2 Programmed I/O 354
5.2.3 Interrupt-Driven I/O 355
5.2.4 I/O Using DMA 356
5.3
I/O SOFTWARE LAYERS 357
5.3.1 Interrupt Handlers 357
5.3.2 Device Drivers 359
5.3.3 Device-Independent I/O Software 362
5.3.4 User-Space I/O Software 368
5.4
MASS STORAGE: DISK AND SSD 370
5.4.1 Magnetic Disks 370
5.4.2 Solid State Drives (SSDs) 381
5.4.3 RAID 385
5.5
CLOCKS 390
5.5.1 Clock Hardware 390
5.5.2 Clock Software 391
5.5.3 Soft Timers 394
5.6
USER INTERFACES: KEYBOARD, MOUSE, & MONITOR 395
5.6.1 Input Software 396
5.6.2 Output Software 402
5.7
THIN CLIENTS 419
5.8
POWER MANAGEMENT 420
5.8.1 Hardware Issues 421
5.8.2 Operating System Issues 422
5.8.3 Application Program Issues 428
5.9
RESEARCH ON INPUT/OUTPUT 428
5.10 SUMMARY 430
xiv
6
CONTENTS
DEADLOCKS
437
6.1
RESOURCES 438
6.1.1 Preemptable and Nonpreemptable Resources 438
6.1.2 Resource Acquisition 439
6.1.3 The Dining Philosophers Problem 440
6.2
INTRODUCTION TO DEADLOCKS 444
6.2.1 Conditions for Resource Deadlocks 445
6.2.2 Deadlock Modeling 445
6.3
THE OSTRICH ALGORITHM 447
6.4
DEADLOCK DETECTION AND RECOVERY 449
6.4.1 Deadlock Detection with One Resource of Each Type 449
6.4.2 Deadlock Detection with Multiple Resources of Each Type 451
6.4.3 Recovery from Deadlock 454
6.5
DEADLOCK AVOIDANCE 455
6.5.1 Resource Trajectories 456
6.5.2 Safe and Unsafe States 457
6.5.3 The Banker’s Algorithm for a Single Resource 458
6.5.4 The Banker’s Algorithm for Multiple Resources 459
6.6
DEADLOCK PREVENTION 461
6.6.1 Attacking the Mutual-Exclusion Condition 461
6.6.2 Attacking the Hold-and-Wait Condition 462
6.6.3 Attacking the No-Preemption Condition 462
6.6.4 Attacking the Circular Wait Condition 463
6.7
OTHER ISSUES 464
6.7.1 Two-Phase Locking 464
6.7.2 Communication Deadlocks 465
6.7.3 Livelock 467
6.7.4 Starvation 468
6.8
RESEARCH ON DEADLOCKS 469
6.9
SUMMARY 470
xv
CONTENTS
7
VIRTUALIZATION AND THE CLOUD
477
7.1
HISTORY 480
7.2
REQUIREMENTS FOR VIRTUALIZATION 482
7.3
TYPE 1 AND TYPE 2 HYPERVISORS 484
7.4
TECHNIQUES FOR EFFICIENT VIRTUALIZATION 486
7.4.1 Virtualizing the Unvirtualizable 487
7.4.2 The Cost of Virtualization 489
7.5
ARE HYPERVISORS MICROKERNELS DONE RIGHT? 490
7.6
MEMORY VIRTUALIZATION 493
7.7
I/O VIRTUALIZATION 497
7.8
VIRTUAL MACHINES ON MULTICORE CPUS 501
7.9
CLOUDS 501
7.9.1 Clouds as a Service 502
7.9.2 Virtual Machine Migration 503
7.9.3 Checkpointing 504
7.10 OS-LEVEL VIRTUALIZATION 504
7.11 CASE STUDY: VMWARE 507
7.11.1 The Early History of VMware 507
7.11.2 VMware Workstation 509
7.11.3 Challenges in Bringing Virtualization to the x86 509
7.11.4 VMware Workstation: Solution Overview 511
7.11.5 The Evolution of VMware Workstation 520
7.11.6 ESX Server: VMware’s type 1 Hypervisor 521
7.12 RESEARCH ON VIRTUALIZATION AND THE CLOUD 523
7.13 SUMMARY 524
xvi
8
9
CONTENTS
MULTIPLE PROCESSOR SYSTEMS
8.1
MULTIPROCESSORS 530
8.1.1 Multiprocessor Hardware 530
8.1.2 Multiprocessor Operating System Types 541
8.1.3 Multiprocessor Synchronization 545
8.1.4 Multiprocessor Scheduling 550
8.2
MULTICOMPUTERS 557
8.2.1 Multicomputer Hardware 558
8.2.2 Low-Level Communication Software 562
8.2.3 User-Level Communication Software 565
8.2.4 Remote Procedure Call 569
8.2.5 Distributed Shared Memory 571
8.2.6 Multicomputer Scheduling 575
8.2.7 Load Balancing 576
8.3
DISTRIBUTED SYSTEMS 579
8.3.1 Network Hardware 581
8.3.2 Network Services and Protocols 585
8.3.3 Document-Based Middleware 588
8.3.4 File-System-Based Middleware 590
8.3.5 Object-Based Middleware 594
8.3.6 Coordination-Based Middleware 595
8.4
RESEARCH ON MULTIPLE PROCESSOR SYSTEMS 598
8.5
SUMMARY 600
SECURITY
9.1
527
605
FUNDAMENTALS OF OPERATING SYSTEM SECURITY 607
9.1.1 The CIA Security Triad 608
9.1.2 Security Principles 609
9.1.3 Security of the Operating System Structure 611
9.1.4 Trusted Computing Base 612
9.1.5 Attackers 614
9.1.6 Can We Build Secure Systems? 617
CONTENTS
9.2
CONTROLLING ACCESS TO RESOURCES 618
9.2.1 Protection Domains 619
9.2.2 Access Control Lists 621
9.2.3 Capabilities 625
9.3
FORMAL MODELS OF SECURE SYSTEMS 628
9.3.1 Multilevel Security 629
9.3.2 Cryptography 632
9.3.3 Trusted Platform Modules 636
9.4
AUTHENTICATION 637
9.4.1 Passwords 637
9.4.2 Authentication Using a Physical Object 644
9.4.3 Authentication Using Biometrics 645
9.5
EXPLOITING SOFTWARE 647
9.5.1 Buffer Overflow Attacks 648
9.5.2 Format String Attacks 658
9.5.3 Use-After-Free Attacks 661
9.5.4 Type Confusion Vulnerabilities 662
9.5.5 Null Pointer Dereference Attacks 664
9.5.6 Integer Overflow Attacks 665
9.5.7 Command Injection Attacks 666
9.5.8 Time of Check to Time of Use Attacks 667
9.5.9 Double Fetch Vulnerability 668
9.6
EXPLOITING HARDWARE 668
9.6.1 Covert Channels 669
9.6.2 Side Channels 671
9.6.3 Transient Execution Attacks 674
9.7
INSIDER ATTACKS 679
9.7.1 Logic Bombs 679
9.7.2 Back Doors 680
9.7.3 Login Spoofing 681
9.8
OPERATING SYSTEM HARDENING 681
9.8.1 Fine-Grained Randomization 682
9.8.2 Control-Flow Restrictions 683
9.8.3 Access Restrictions 685
9.8.4 Code and Data Integrity Checks 689
9.8.5 Remote Attestation Using a Trusted Platform Module 690
9.8.6 Encapsulating Untrusted Code 691
xvii
xviii
CONTENTS
9.9
RESEARCH ON SECURITY 694
9.10 SUMMARY 696
10 CASE STUDY 1: UNIX,
10.1 HISTORY OF UNIX AND LINUX 704
10.1.1 UNICS 704
10.1.2 PDP-11 UNIX 705
10.1.3 Portable UNIX 706
10.1.4 Berkeley UNIX 707
10.1.5 Standard UNIX 708
10.1.6 MINIX 709
10.1.7 Linux 710
10.2 OVERVIEW OF LINUX 713
10.2.1 Linux Goals 713
10.2.2 Interfaces to Linux 714
10.2.3 The Shell 716
10.2.4 Linux Utility Programs 719
10.2.5 Kernel Structure 720
10.3 PROCESSES IN LINUX 723
10.3.1 Fundamental Concepts 724
10.3.2 Process-Management System Calls in Linux 726
10.3.3 Implementation of Processes and Threads in Linux 730
10.3.4 Scheduling in Linux 736
10.3.5 Synchronization in Linux 740
10.3.6 Booting Linux 741
10.4 MEMORY MANAGEMENT IN LINUX 743
10.4.1 Fundamental Concepts 744
10.4.2 Memory Management System Calls in Linux 746
10.4.3 Implementation of Memory Management in Linux 748
10.4.4 Paging in Linux 754
10.5 INPUT/OUTPUT IN LINUX 757
10.5.1 Fundamental Concepts 758
10.5.2 Networking 759
703
CONTENTS
xix
10.5.3 Input/Output System Calls in Linux 761
10.5.4 Implementation of Input/Output in Linux 762
10.5.5 Modules in Linux 765
10.6 THE LINUX FILE SYSTEM 766
10.6.1 Fundamental Concepts 766
10.6.2 File-System Calls in Linux 770
10.6.3 Implementation of the Linux File System 774
10.6.4 NFS: The Network File System 783
10.7 SECURITY IN LINUX 789
10.7.1 Fundamental Concepts 789
10.7.2 Security System Calls in Linux 791
10.7.3 Implementation of Security in Linux 792
10.8 ANDROID 793
10.8.1 Android and Google 794
10.8.2 History of Android 794
10.8.3 Design Goals 800
10.8.4 Android Architecture 801
10.8.5 Linux Extensions 803
10.8.6 ART 807
10.8.7 Binder IPC 809
10.8.8 Android Applications 818
10.8.9 Intents 830
10.8.10 Process Model 831
10.8.11 Security and Privacy 836
10.8.12 Background Execution and Social Engineering 856
10.9 SUMMARY 863
11 CASE STUDY 2: WINDOWS 11
11.1 HISTORY OF WINDOWS THROUGH WINDOWS 11 871
11.1.1 1980s: MS-DOS 872
11.1.2 1990s: MS-DOS-based Windows 873
11.1.3 2000s: NT-based Windows 873
11.1.4 Windows Vista 876
11.1.5 Windows 8 877
871
xx
CONTENTS
11.1.6 Windows 10 878
11.1.7 Windows 11 879
11.2 PROGRAMMING WINDOWS 880
11.2.1 Universal Windows Platform 881
11.2.2 Windows Subsystems 883
11.2.3 The Native NT Application Programming Interface 884
11.2.4 The Win32 Application Programming Interface 887
11.2.5 The Windows Registry 891
11.3 SYSTEM STRUCTURE 894
11.3.1 Operating System Structure 894
11.3.2 Booting Windows 910
11.3.3 Implementation of the Object Manager 914
11.3.4 Subsystems, DLLs, and User-Mode Services 926
11.4 PROCESSES AND THREADS IN WINDOWS 929
11.4.1 Fundamental Concepts 929
11.4.2 Job, Process, Thread, and Fiber Management API Calls 934
11.4.3 Implementation of Processes and Threads 941
11.4.4 WoW64 and Emulation 950
11.5 MEMORY MANAGEMENT 955
11.5.1 Fundamental Concepts 955
11.5.2 Memory-Management System Calls 961
11.5.3 Implementation of Memory Management 962
11.5.4 Memory Compression 973
11.5.5 Memory Partitions 976
11.6 CACHING IN WINDOWS 977
11.7 INPUT/OUTPUT IN WINDOWS 979
11.7.1 Fundamental Concepts 980
11.7.2 Input/Output API Calls 982
11.7.3 Implementation of I/O 984
11.8 THE WINDOWS NT FILE SYSTEM 989
11.8.1 Fundamental Concepts 989
11.8.2 Implementation of the NT File System 990
11.9 WINDOWS POWER MANAGEMENT 1000
CONTENTS
xxi
11.10 VIRTUALIZATION IN WINDOWS 1003
11.10.1 Hyper-V 1003
11.10.2 Containers 1011
11.10.3 Virtualization-Based Security 1017
11.11 SECURITY IN WINDOWS 1018
11.11.1 Fundamental Concepts 1020
11.11.2 Security API Calls 1022
11.11.3 Implementation of Security 1023
11.11.4 Security Mitigations 1025
11.12 SUMMARY 1035
12 OPERATING SYSTEM DESIGN
12.1 THE NATURE OF THE DESIGN PROBLEM 1042
12.1.1 Goals 1042
12.1.2 Why Is It Hard to Design an Operating System? 1043
12.2 INTERFACE DESIGN 1045
12.2.1 Guiding Principles 1045
12.2.2 Paradigms 1048
12.2.3 The System-Call Interface 1051
12.3 IMPLEMENTATION 1053
12.3.1 System Structure 1054
12.3.2 Mechanism vs. Policy 1057
12.3.3 Orthogonality 1058
12.3.4 Naming 1059
12.3.5 Binding Time 1061
12.3.6 Static vs. Dynamic Structures 1062
12.3.7 Top-Down vs. Bottom-Up Implementation 1063
12.3.8 Synchronous vs. Asynchronous Communication 1064
12.3.9 Useful Techniques 1065
12.4 PERFORMANCE 1070
12.4.1 Why Are Operating Systems Slow? 1071
12.4.2 What Should Be Optimized? 1071
12.4.3 Space-Time Trade-offs 1072
1041
xxii
CONTENTS
12.4.4 Caching 1075
12.4.5 Hints 1076
12.4.6 Exploiting Locality 1077
12.4.7 Optimize the Common Case 1077
12.5 PROJECT MANAGEMENT 1078
12.5.1 The Mythical Man Month 1078
12.5.2 Team Structure 1079
12.5.3 The Role of Experience 1081
12.5.4 No Silver Bullet 1082
13 READING LIST AND BIBLIOGRAPHY
1087
13.1 SUGGESTIONS FOR FURTHER READING 1087
13.1.1 Introduction 1088
13.1.2 Processes and Threads 1088
13.1.3 Memory Management 1089
13.1.4 File Systems 1090
13.1.5 Input/Output 1090
13.1.6 Deadlocks 1091
13.1.7 Virtualization and the Cloud 1092
13.1.8 Multiple Processor Systems 1093
13.1.9 Security 1093
13.1.10 Case Study 1: UNIX, Linux, and Android 1094
13.1.11 Case Study 2: Windows 1095
13.1.12 Operating System Design 1096
13.2 ALPHABETICAL BIBLIOGRAPHY 1097
INDEX
1121
PREFACE
The fifth edition of this book differs from the fourth edition in many ways.
There are large numbers of small changes everywhere to bring the material up to
date as operating systems are not standing still. For example, where the previous
edition focused almost exclusively on magnetic hard disks for storage, this time we
give the flash-based Solid State Drives (SSDs) the attention that befits their popularity. The chapter on Windows 8.1 has been replaced entirely by a chapter on the
new Windows 11. We have rewritten much of the security chapter, with more
focus on topics that are directly relevant for operating systems (and exciting new
attacks and defenses), while reducing the discussion of cryptography and steganography. Here is a chapter-by-chapter rundown of the changes.
•
Chapter 1 has been heavily modified and updated in many places, but
with the exception of dropping the description of CD-ROMs and
DVDs in favor of modern storage solutions such as SSDs and persistent memory, no major sections have been added or deleted.
•
In Chapter 2, we significantly expanded the discussion of event-driven
servers and included an extensive example with pseudo code. We gave
priority inversion its own section where we also discussed ways to deal
with the problem. We reordered some of the sections to clarify the discussion. For instance, we now discuss the readers-writers problem
xxiii
xxiv
PREFACE
immediately after the producer-consumer section and moved the dining philosophers to another chapter, that of deadlocks, entirely. Besides numerous smaller updates, we also dropped some older material,
such as scheduler activations and pop-up threads.
•
Chapter 3 now focuses on modern 64-bit architectures and contains
more precise explanations of many aspects of paging and TLBs. For
instance, we describe how operating systems use paging also and how
some operating systems map the kernel into the address spaces of user
processes.
•
Chapter 4 changed significantly. We dropped the lengthy descriptions
of CD-ROMs and tapes, and instead added sections about SSD-based
file systems, booting in modern UEFI-based computer systems, and
secure file deletion and disk encryption.
•
In Chapter 5, we have more information about SSDs and NVMe, and
explain input devices using a modern USB keyboard instead of the
older PS/2 one of the previous edition. In addition, we clarify the relation between interrupts, traps, exceptions, and faults.
•
As mentioned, we added the dining philosophers example to Chapter
6. Other than that, the chapter is pretty much unchanged. The topic of
deadlocks is fairly stable, with few new results.
•
In Chapter 7, we added a section about containers to the existing (and
updated) explanation of hypervisor-based virtualization. The material
on VMware has also been brought up to date.
•
Chapter 8 is an updated version of the previous material on multiprocessor systems. We added subsections on simultaneous multithreading
and discuss new types of coprocessors, while dropping sections such
as the one on the older IXP network processors and the one on the
(now dead) CORBA middleware. A new section discusses scheduling
for security.
•
Chapter 9 has been heavily revised and reorganized, with much more
focus on what is relevant for the operating system and less emphasis
on crypto. We now start the chapter with a discussion of principles for
secure design and their relevance for the operating system structure.
We discuss exciting new hardware developments, such as the Meltdown and Spectre transient execution vulnerabilities, that have come to
light since the previous edition. In addition, we describe new software
vulnerabilities that are important for the operating system. Finally, we
greatly expanded the description of the ways in which the operating
system can be hardened, with extensive discussion of control flow
integrity, fine-grained ASLR, code signing, access restrictions, and
PREFACE
xxv
attestation. Since there is much ongoing research in this area, new references have been added and the research section has been rewritten.
•
Chapter 10 has been updated with new developments in Linux and
Android. Android has evolved considerably since the previous edition,
and this chapter covers the current version in detail. This section has
been substantially rewritten.
•
Chapter 11 has changed significantly. Where the fourth edition was on
Windows 8.1, we now discuss Windows 11. It is basically a new chapter.
•
Chapter 12 is a slightly revised version of the previous edition. This
chapter covers the basic principles of system design, and they have not
changed much in the past few years.
•
Chapter 13 is a thoroughly updated list of suggested readings. In addition, the list of references has been updated, with entries to well over
100 new works published after the fourth edition of this book came
out.
•
In addition, the sections on research throughout the book have all been
redone from scratch to reflect the latest research in operating systems.
Furthermore, new problems have been added to all the chapters.
Instructor supplements (including the PowerPoint sheets) can be found at
https://www.pearsonhighered.com/cs-resources .
A number of people have been involved in the fifth edition. The material in
Chap. 7 on VMware (in Sec. 7.12) was written by Edouard Bugnion of EPFL in
Lausanne, Switzerland. Ed was one of the founders of the VMware company and
knows this material as well as anyone in the world. We thank him greatly for supplying it to us.
Ada Gavrilovska of Georgia Tech, who is an expert on Linux internals, updated Chap. 10 from the Fourth Edition, which she also wrote. The Android material in Chap. 10 was written by Dianne Hackborn of Google, one of the key
developers of the Android system. Android is the most popular operating system
on smartphones, so we are very grateful to have Dianne help us. Chapter 10 is now
quite long and detailed, but UNIX, Linux, and Android fans can learn a lot from it.
We haven’t neglected Windows, however. Mehmet Iyigun of Microsoft updated Chap. 11 from the previous edition of the book. This time the chapter covers
Windows 11 in detail. Mehmet has a great deal of knowledge of Windows and
enough vision to tell the difference between places where Microsoft got it right and
where it got it wrong. He was ably assisted by Andrea Allievi, Pedro Justo, Chris
Kleynhans, and Erick Smith. Windows fans are certain to enjoy this chapter.
The book is much better as a result of the work of all these expert contributors.
Again, we would like to thank them for their invaluable help.
xxvi
PREFACE
We were also fortunate to have several reviewers who read the manuscript and
also suggested new end-of-chapter problems. They were Jeremiah Blanchard
(University of Florida), Kate Holdener (St. Louis University), Liting Hu (Virginia
Tech), Jiang-Bo Liu (Bradley University), and Mai Zheng (Iowa State University).
We remain responsible for any remaining errors, of course.
We would also like to thank our editor, Tracy Johnson, for keeping the project
on track and herding all the cats, albeit virtually this time. Erin Sullivan managed
the review process and Carole Snyder handled production.
Finally, last but not least, Barbara, Marvin, and Matilde are still wonderful, as
usual, each in a unique and special way. Aron and Nathan are great kids and
Olivia and Mirte are treasures. And of course, I would like to thank Suzanne for
her love and patience, not to mention all the druiven, kersen, and sinaasappelsap,
as well as other agricultural products. (AST)
As always, I owe a massive thank you to Marieke, Duko, and Jip. Marieke for
just being there and for bearing with me in the countless hours that I was working
on this book, and Duko and Jip for tearing me away from it to play hoops, day and
night! I am also grateful to the neighbors for putting up with our midnight basketball games. (HB)
Andrew S. Tanenbaum
Herbert Bos
PREFACE
xxvii
ABOUT THE AUTHORS
Andrew S. Tanenbaum has an S.B. degree from M.I.T. and a Ph.D. from the
University of California at Berkeley. He is currently a Professor Emeritus of Computer Science at the Vrije Universiteit in Amsterdam, The Netherlands. He was
formerly Dean of the Advanced School for Computing and Imaging, an interuniversity graduate school doing research on advanced parallel, distributed, and imaging systems. He was also an Academy Professor of the Royal Netherlands Academy of Arts and Sciences, which has saved him from turning into a bureaucrat. He
also won a prestigious European Research Council Advanced Grant.
In the past, he has done research on compilers, reliable operating systems, networking, and distributed systems. This research has led to over 200 refereed publications in journals and conferences. Prof. Tanenbaum has also authored or coauthored five books, which have been translated into over 20 languages, ranging
from Basque to Thai. They are used at universities all over the world. There are
163 versions of his books.
Prof. Tanenbaum has also produced a considerable volume of software, notably MINIX, a small UNIX clone. It was the direct inspiration for Linux and the
platform on which Linux was initially developed. The current version of MINIX,
called MINIX 3, is now focused on being an extremely reliable and secure operating system. Prof. Tanenbaum will consider his work done when no user has any
idea what an operating system crash is. MINIX 3 is an ongoing open-source project to which you are invited to contribute. Go to www.minix3.org to download a
free copy of MINIX 3 and give it a try. Both x86 and ARM versions are available.
Prof. Tanenbaum’s Ph.D. students have gone on to greater glory after graduating. Some have become professors; others have fulfilled leading roles in government organizations and industry. He is very proud of them. In this respect, he
resembles a mother hen.
Prof. Tanenbaum is a Fellow of the ACM, a Fellow of the IEEE, and a member
of the Royal Netherlands Academy of Arts and Sciences. He has also won numerous scientific prizes from ACM, IEEE, and USENIX. If you are unbearably curious about them, see his page on Wikipedia. He also has two honorary doctorates.
Herbert Bos obtained his Master’s degree from Twente University and his
Ph.D. from Cambridge University in the United Kingdom. Since then, he has
worked extensively on dependable and efficient I/O architectures for operating systems like Linux, but also research systems based on MINIX 3. He is currently a
professor at the VUSec Systems Security Research Group in the Department. of
Computer Science at the Vrije Universiteit in Amsterdam, The Netherlands. His
main research field is systems security.
With his group, he discovered and analyzed many vulnerabilities in both hardware and software. From buggy memory chips to vulnerable CPUs, and from flaws
in operating systems to novel exploitation techniques, the research has led to fixes
xxviii
PREFACE
in most major operating systems, most popular browsers, and all modern Intel
processors. He believes that offensive research is valuable because the main reason
for today’s security problems is that systems have become so complex that we no
longer understand them. By investigating how we can make systems behave in
unintended ways, we learn more about their (real) nature. Armed with this knowledge, developers can then improve their designs in the future. Indeed, while
sophisticated new attacks tend to feature prominently in the media, Herbert spends
most of his time on developing defensive techniques to improve the security.
His (former) students are all awesome and much cleverer than he is. With
them, he has won five Pwnie Awards at the Black Hat conference in Las Vegas.
Moreover, five of his students have won the ACM SIGOPS EuroSys Roger Needham Award for best European Ph.D. thesis in systems, two of them have won the
ACM SIGSAC Doctoral Dissertation Award for best Ph.D. thesis in security, and
two more won the William C. Carter Ph.D. Dissertation Award for their work on
dependability. Herbert worries about climate change and loves the Beatles.
1
INTRODUCTION
A modern computer consists of one or more processors, some amount of main
memory, hard disks or Flash drives, printers, a keyboard, a mouse, a display, network interfaces, and various other input/output devices. All in all, a complex system. If every application programmer had to understand how all these things work
in detail, no code would ever get written. Furthermore, managing all these components and using them optimally is an exceedingly challenging job. For this reason,
computers are equipped with a layer of software called the operating system,
whose job is to provide user programs with a better, simpler, cleaner, model of the
computer and to handle managing all the resources just mentioned. Operating systems are the subject of this book.
It is important to realize that smart phones and tablets (like the Apple iPad) are
just computers in a smaller package with a touch screen. They all have operating
systems. In fact, Apple’s iOS is fairly similar to macOS, which runs on Apple’s
desktop and MacBook systems. The smaller form factor and touch screen really
doesn’t change that much about what the operating system does. Android smartphones and tablets all run Linux as the true operating system on the bare hardware.
What users perceive as ‘‘Android’’ is simply a layer of software running on top of
Linux. Since macOS (and thus iOS) is derived from Berkeley UNIX and Linux is a
clone of UNIX, by far the most popular operating system in the world is UNIX and
its variants. For this reason, we will pay a lot of attention in this book to UNIX.
Most readers probably have had some experience with an operating system
such as Windows, Linux, FreeBSD, or macOS, but appearances can be deceiving.
1
2
INTRODUCTION
CHAP. 1
The program that users interact with, usually called the shell when it is text based
and the GUI (Graphical User Interface) (which is pronounced ‘‘gooey’’) when it
uses icons, is actually not part of the operating system, although it uses the operating system to get its work done.
A simple overview of the main components under discussion here is given in
Fig. 1-1. Here we see the hardware at the bottom. The hardware consists of chips,
boards, Flash drives, disks, a keyboard, a monitor, and similar physical objects. On
top of the hardware is the software. Most computers have two modes of operation:
kernel mode and user mode. The operating system, the most fundamental piece of
software, runs in kernel mode (also called supervisor mode) for at least some of
its functionality. In this mode, it has complete access to all the hardware and can
execute any instruction the machine is capable of executing. The rest of the software runs in user mode, in which only a subset of the machine instructions is
available. In particular, those instructions that affect control of the machine, determine the security boundaries, or do I/O (Input/Output) are forbidden to usermode programs. We will come back to the difference between kernel mode and
user mode repeatedly throughout this book. It plays a crucial role in how operating
systems work.
Web
browser
E-mail
reader
Music
player
User mode
User interface program
Kernel mode
Software
Operating system
Hardware
Figure 1-1. Where the operating system fits in.
The user interface program, shell or GUI, is the lowest level of user-mode software, and allows the user to start other programs, such as a Web browser, email
reader, or music player. These programs, too, make heavy use of the operating system.
The placement of the operating system is shown in Fig. 1-1. It runs on the
bare hardware and provides the base for all the other software.
An important distinction between the operating system and normal (usermode) software is that if a user does not like a particular email reader, she is free to
get a different one or write her own if she so chooses; she is typically not free to
write her own clock interrupt handler, which is part of the operating system and is
INTRODUCTION
3
protected by hardware against attempts by users to modify it. This distinction,
however, is sometimes blurred, for instance in embedded systems (which may not
have kernel mode) or interpreted systems (such as Java-based systems that use
interpretation, not hardware, to separate the components).
Also, in many systems there are programs that run in user mode but help the
operating system or perform privileged functions. For example, there is often a
program that allows users to change their passwords. It is not part of the operating
system and does not run in kernel mode, but it clearly carries out a sensitive function and has to be protected in a special way. In some systems, this idea is carried
to an extreme, and pieces of what is traditionally considered to be the operating
system (such as the file system) run in user mode. In such systems, it is difficult to
draw a clear boundary. Everything running in kernel mode is clearly part of the
operating system, but some programs running outside it are arguably also part of it,
or at least closely associated with it.
Operating systems differ from user (i.e., application) programs in ways other
than where they reside. In particular, they are huge, complex, and very long-lived.
The source code for Windows is over 50 million lines of code. The source code for
Linux is over 20 million lines of code. Both are still growing. To conceive of what
this means, think of printing out 50 million lines in book form, with 50 lines per
page and 1000 pages per volume (about the size of this book). Each book would
contain 50,000 lines of code. It would take 1000 volumes to list an operating system of this size. Now imagine a bookcase with 20 books per shelf and seven
shelves or 140 books in all. It would take a bit over seven bookcases to hold the
full code of Windows 10. Can you imagine getting a job maintaining an operating
system and on the first day having your boss bring you to a room with these seven
bookcases of code and say: ‘‘Go learn that.’’ And this is only for the part that runs
in the kernel. No one at Microsoft understands all of Windows and probably most
programmers there, even kernel programmers, understand only a small part of it.
When essential shared libraries are included, the source code base gets much bigger. And this excludes basic application software (things like the browser, the
media player, and so on).
It should be clear now why operating systems live a long time—they are very
hard to write, and having written one, the owner is loath to throw it out and start
again. Instead, such systems evolve over long periods of time. Windows 95/98/Me
was basically one operating system and Windows NT/2000/XP/Vista/Windows
7/8/10 is a different one. They look similar to the users because Microsoft made
very sure that the user interface of Windows 2000/XP/Vista/Windows 7 was quite
similar to that of the system it was replacing, mostly Windows 98. This was not
necessarily the case for Windows 8 and 8.1 which introduced a variety of changes
in the GUI and promptly drew criticism from users who liked to keep things the
same. Windows 10 reverted some of these changes and introduced a number of
improvements. Windows 11 is built upon the framework of Windows 10. We will
study Windows in detail in Chap. 11.
4
INTRODUCTION
CHAP. 1
Besides Windows, the other main example we will use throughout this book is
UNIX and its variants and clones. It, too, has evolved over the years, with versions
like FreeBSD (and essentially, macOS) being derived from the original system,
whereas Linux is a fresh code base, although very closely modeled on UNIX and
highly compatible with it. The huge investment needed to develop a mature and
reliable operating system from scratch led Google to adopt an existing one, Linux,
as the basis of its Android operating system. We will use examples from UNIX
throughout this book and look at Linux in detail in Chap. 10.
In this chapter, we will briefly touch on a number of key aspects of operating
systems, including what they are, their history, what kinds are around, some of the
basic concepts, and their structure. We will come back to many of these important
topics in later chapters in more detail.
1.1 WHAT IS AN OPERATING SYSTEM?
It is hard to pin down what an operating system is other than saying it is the
software that runs in kernel mode—and even that is not always true. Part of the
problem is that operating systems perform two essentially unrelated functions: providing application programmers (and application programs, naturally) a clean abstract set of resources instead of the messy hardware ones and managing these
hardware resources. Depending on who is doing the talking, you might hear mostly
about one function or the other. Let us now look at both.
1.1.1 The Operating System as an Extended Machine
The architecture (instruction set, memory organization, I/O, and bus structure) of most computers at the machine-language level is primitive and awkward to
program, especially for input/output. To make this point more concrete, consider
modern SATA (Serial ATA) hard disks used on most computers. A book (Deming, 2014) describing an early version of the interface to the disk—what a programmer would have to know to use the disk—ran over 450 pages. Since then, the
interface has been revised multiple times and is even more complicated than it was
in 2014. Clearly, no sane programmer would want to deal with this disk at the
hardware level. Instead, a piece of software, called a disk driver, deals with the
hardware and provides an interface to read and write disk blocks, without getting
into the details. Operating systems contain many drivers for controlling I/O
devices.
But even this level is much too low for most applications. For this reason, all
operating systems provide yet another layer of abstraction for using disks: files.
Using this abstraction, programs can create, write, and read files, without having to
deal with the messy details of how the hardware actually works.
SEC. 1.1
WHAT IS AN OPERATING SYSTEM?
5
This abstraction is the key to managing all this complexity. Good abstractions
turn a nearly impossible task into two manageable ones. The first is defining and
implementing the abstractions. The second is using these abstractions to solve the
problem at hand. One abstraction that almost every computer user understands is
the file, as mentioned above. It is a useful piece of information, such as a digital
photo, saved email message, song, or Web page. It is much easier to deal with photos, emails, songs, and Web pages than with the details of SATA (or other) disks.
The job of the operating system is to create good abstractions and then implement
and manage the abstract objects thus created. In this book, we will talk a lot about
abstractions. They are one of the keys to understanding operating systems.
This point is so important that it is worth repeating but in different words. With
all due respect to the industrial engineers who so very carefully designed the Apple
Macintosh computers (now known simply as ‘‘Macs’’), hardware is grotesque.
Real processors, memories, Flash drives, disks, and other devices are very complicated and present difficult, awkward, idiosyncratic, and inconsistent interfaces to
the people who have to write software to use them. Sometimes this is due to the
need for backward compatibility with older hardware. Other times it is an attempt
to save money. Often, however, the hardware designers do not realize (or care) how
much trouble they are causing for the software. One of the major tasks of the operating system is to hide the hardware and present programs (and their programmers)
with nice, clean, elegant, consistent, abstractions to work with instead. Operating
systems turn the awful into the beautiful, as shown in Fig. 1-2.
Application programs
Beautiful interface
Operating system
Awful interface
Hardware
Figure 1-2. Operating systems turn awful hardware into beautiful abstractions.
It should be noted that the operating system’s real customers are the application programs (via the application programmers, of course). They are the ones
who deal directly with the operating system and its abstractions. In contrast, end
users deal with the abstractions provided by the user interface, either a command-line shell or a graphical interface. While the abstractions at the user interface
may be similar to the ones provided by the operating system, this is not always the
case. To make this point clearer, consider the normal Windows desktop and the
6
INTRODUCTION
CHAP. 1
line-oriented command prompt. Both are programs running on the Windows operating system and use the abstractions Windows provides, but they offer very different user interfaces. Similarly, a Linux user running Gnome or KDE sees a very
different interface than a Linux user working directly on top of the underlying X
Window System, but the underlying operating system abstractions are the same in
both cases.
In this book, we will study the abstractions provided to application programs in
great detail, but say rather little about user interfaces. That is a large and important
subject, but one only peripherally related to operating systems.
1.1.2 The Operating System as a Resource Manager
The concept of an operating system as primarily providing abstractions to
application programs is a top-down view. An alternative, bottom-up, view holds
that the operating system is there to manage all the pieces of a complex system.
Modern computers consist of processors, memories, timers, disks, mice, network
interfaces, printers, touch screens, touch pad, and a wide variety of other devices.
In the bottom-up view, the job of the operating system is to provide for an orderly
and controlled allocation of the processors, memories, and I/O devices among the
various programs wanting them.
Modern operating systems allow multiple programs to be in memory and run
at the same time. Imagine what would happen if three programs running on some
computer all tried to print their output simultaneously on the same printer. The first
few lines of printout might be from program 1, the next few from program 2, then
some from program 3, and so forth. The result would be utter chaos. The operating
system can bring order to the potential chaos by buffering all the output destined
for the printer on the disk or Flash drive. When one program is finished, the operating system can then copy its output from the disk file where it has been stored for
the printer, while at the same time the other program can continue generating more
output, oblivious to the fact that the output is not really going to the printer (yet).
When a computer (or network) has more than one user, managing and protecting the memory, I/O devices, and other resources is even more important since the
users might otherwise interfere with one another. In addition, users often need to
share not only hardware, but information (files, databases, etc.) as well. In short,
this view of the operating system holds that its primary task is to keep track of
which programs are using which resource, to grant resource requests, to account
for usage, and to mediate conflicting requests from different programs and users.
Resource management includes multiplexing (sharing) resources in two different ways: in time and in space. When a resource is time multiplexed, different
programs or users take turns using it. First one of them gets to use the resource,
then another, and so on. For example, with only one CPU and multiple programs
that want to run on it, the operating system first allocates the CPU to one program,
then, after it has run long enough, another program gets to use the CPU, then
SEC. 1.1
WHAT IS AN OPERATING SYSTEM?
7
another, and then eventually the first one again. Determining how the resource is
time multiplexed—who goes next and for how long—is the task of the operating
system. Another example of time multiplexing is sharing the printer. When multiple print jobs are queued up for printing on a single printer, a decision has to be
made about which one is to be printed next.
The other kind of multiplexing is space multiplexing. Instead of the customers
taking turns, each one gets part of the resource. For example, main memory is normally divided up among several running programs, so each one can be resident at
the same time (for example, in order to take turns using the CPU). Assuming there
is enough memory to hold multiple programs, it is more efficient to hold several
programs in memory at once rather than give one of them the entire mem, especially if it only needs a small fraction of the total. Of course, this raises issues of fairness, protection, and so forth, and it is up to the operating system to solve them.
Other resource that are space multiplexed are disks and Flash drives. In many systems, a single disk can hold files from many users at the same time. Allocating disk
space and keeping track of who is using which disk blocks is a typical operating
system task. By the way, people commonly refer to all nonvolatile memory as
‘‘disks,’’ but in this book we try to explicitly distinguish between disks, which have
spinning magnetic platters, and SSDs (Solid State Drives), which are based on
Flash memory and electronic rather than mechanical. Still, from a software point
of view, SSDs are similar to disks in many (but not all) ways.
1.2 HISTORY OF OPERATING SYSTEMS
Operating systems have been evolving through the years. In the following sections, we will briefly look at a few of the highlights. Since operating systems have
historically been closely tied to the architecture of the computers on which they
run, we will look at successive generations of computers to see what their operating systems were like. This mapping of operating system generations to computer
generations is crude, but it does provide some structure where there would otherwise be none. The progression given below is largely chronological, but it has been
a bumpy ride. Each development did not wait until the previous one nicely finished
before getting started. There was a lot of overlap, not to mention many false starts
and dead ends. Take this as a guide, not as the last word.
The first true digital computer was designed by the English mathematician
Charles Babbage (1792–1871). Although Babbage spent most of his life and fortune trying to build his ‘‘analytical engine,’’ he never got it working properly
because it was purely mechanical, and the technology of his day could not produce
the required wheels, gears, and cogs to the high precision that he needed. Needless
to say, the analytical engine did not have an operating system.
As an interesting historical aside, Babbage realized that he would need software for his analytical engine, so he hired a young woman named Ada Lovelace,
8
INTRODUCTION
CHAP. 1
who was the daughter of the famed British poet Lord Byron, as the world’s first
programmer. The programming language Ada® is named after her.
1.2.1 The First Generation (1945–1955): Vacuum Tubes
After Babbage’s unsuccessful efforts, little progress was made in constructing
digital computers until the World War II period, which stimulated an explosion of
activity. Professor John Atanasoff and his graduate student Clifford Berry built
what is now regarded as the first functioning digital computer at Iowa State University. It used 300 vacuum tubes. At roughly the same time, Konrad Zuse in Berlin
built the Z3 computer out of electromechanical relays. In 1944, the Colossus was
built and programmed by a group of scientists (including Alan Turing) at Bletchley
Park, England, the Mark I was built by Howard Aiken at Harvard, and the ENIAC
was built by William Mauchley and his graduate student J. Presper Eckert at the
University of Pennsylvania. Some were binary, some used vacuum tubes, some
were programmable, but all were very primitive and took seconds to perform even
the simplest calculation.
In these early days, a single group of people (usually engineers) designed,
built, programmed, operated, and maintained each machine. All programming was
done in absolute machine language, or even worse yet, by wiring up electrical circuits by connecting thousands of cables to plugboards to control the machine’s
basic functions. Programming languages were unknown (even assembly language
was unknown). Operating systems were unheard of. The usual mode of operation
was for the programmer to sign up for a block of time using the signup sheet on the
wall, then come down to the machine room, insert his or her plugboard into the
computer, and spend the next few hours hoping that none of the 20,000 or so vacuum tubes would burn out during the run. Virtually all the problems were simple
straightforward mathematical and numerical calculations, such as grinding out
tables of sines, cosines, and logarithms, or computing artillery trajectories.
By the early 1950s, the routine had improved somewhat with the introduction
of punched cards. It was now possible to write programs on cards and read them in
instead of using plugboards; otherwise, the procedure was the same.
1.2.2 The Second Generation (1955–1965): Transistors and Batch Systems
The introduction of the transistor in the mid-1950s changed the picture radically. Computers became reliable enough that they could be manufactured and sold
to paying customers with the expectation that they would continue to function long
enough to get useful work done. For the first time, there was a clear separation
between designers, builders, operators, programmers, and maintenance personnel.
These machines, which are now called mainframes, were locked away in
large, specially air-conditioned computer rooms, with staffs of professional operators to run them. Only large corporations or government agencies or universities
SEC. 1.2
9
HISTORY OF OPERATING SYSTEMS
could afford the multimillion-dollar price tag. To run a job (i.e., a program or set
of programs), a programmer would first write the program on paper (in FORTRAN
or assembler), then punch it on cards. The programmer would then bring the card
deck down to the input room, hand it to one of the operators, and go drink coffee
until the output was ready.
When the computer finished whatever job it was currently running, an operator
would go over to the printer and tear off the output and carry it over to the output
room, so that the programmer could collect it later. Then the operator would take
one of the card decks that had been brought from the input room and read it in. If
the FORTRAN compiler was needed, the operator would have to get it from a file
cabinet and read it in. Much computer time was wasted while operators were walking around the machine room.
Given the high cost of the equipment, it is not surprising that people quickly
looked for ways to reduce the wasted time. The solution generally adopted was the
batch system. The idea behind it was to collect a tray full of jobs in the input
room and then read them onto a magnetic tape using a small (relatively) inexpensive computer, such as the IBM 1401, which was quite good at reading cards,
copying tapes, and printing output, but not at all good at numerical calculations.
Other, much more expensive machines, such as the IBM 7094, were used for the
real computing. This situation is shown in Fig. 1-3.
Card
reader
Tape
drive
Input
tape
Output
tape
Printer
1401
(a)
System
tape
(b)
7094
(c)
(d)
1401
(e)
(f)
Figure 1-3. An early batch system. (a) Programmers bring cards to 1401. (b)
1401 reads batch of jobs onto tape. (c) Operator carries input tape to 7094. (d)
7094 does computing. (e) Operator carries output tape to 1401. (f) 1401 prints
output.
After about an hour of collecting a batch of jobs, the cards were read onto a
magnetic tape, which was carried into the machine room, where it was mounted on
a tape drive. The operator then loaded a special program (the ancestor of today’s
operating system), which read the first job from tape and ran it. The output was
written onto a second tape, instead of being printed. After each job finished, the
operating system automatically read the next job from the tape and began running
it. When the whole batch was done, the operator removed the input and output
10
INTRODUCTION
CHAP. 1
tapes, replaced the input tape with the next batch, and brought the output tape to a
1401 for printing off line (i.e., not connected to the main computer).
The structure of a typical input job is shown in Fig. 1-4. It started out with a
$JOB card, specifying the maximum run time in minutes, the account number to be
charged, and the programmer’s name. Then came a $FORTRAN card, telling the
operating system to load the FORTRAN compiler from the system tape. It was
directly followed by the program to be compiled, and then a $LOAD card, directing the operating system to load the object program just compiled. (Compiled programs were often written on scratch tapes and had to be loaded explicitly.) Next
came the $RUN card, telling the operating system to run the program with the data
following it. Finally, the $END card marked the end of the job. These control
cards were the forerunners of modern shells and command-line interpreters.
$END
Data for program
$RUN
$LOAD
FORTRAN program
$FORTRAN
$JOB, 10,7710802, ADA LOVELACE
Figure 1-4. Structure of a typical FMS job.
Large, second-generation computers were used mostly for scientific and engineering calculations, such as solving the partial differential equations that often
occur in physics and engineering. They were largely programmed in FORTRAN
and assembly language. Typical operating systems were FMS (the Fortran Monitor System) and IBSYS, IBM’s operating system for the 7094.
1.2.3 The Third Generation (1965–1980): ICs and Multiprogramming
By the early 1960s, most computer manufacturers had two distinct and incompatible, product lines. On the one hand, there were the word-oriented, large-scale
scientific computers, such as the 7094, which were used for industrial-strength
SEC. 1.2
HISTORY OF OPERATING SYSTEMS
11
numerical calculations in science and engineering. On the other hand, there were
the character-oriented, commercial computers, such as the 1401, which were widely used for tape sorting and printing by banks and insurance companies.
Developing and maintaining two completely different product lines was an
expensive proposition for the manufacturers. In addition, many new computer customers initially needed a small machine but later outgrew it and wanted a bigger
machine that would run all their old programs, but faster.
IBM attempted to solve both of these problems at a single stroke by introducing the System/360. The 360 was a series of software-compatible machines ranging from 1401-sized models to much larger ones, more powerful than the mighty
7094. The machines differed only in price and performance (maximum memory,
processor speed, number of I/O devices permitted, and so forth). Since they all had
the same architecture and instruction set, programs written for one machine could
run on all the others—at least in theory. (But as Yogi Berra reputedly said: ‘‘In
theory, theory and practice are the same; in practice, they are not.’’) Since the 360
was designed to handle both scientific (i.e., numerical) and commercial computing,
a single family of machines could satisfy the needs of all customers. In subsequent
years, IBM came out with backward compatible successors to the 360 line, using
more modern technology, known as the 370, 4300, 3080, and 3090. The zSeries is
the most recent descendant of this line, although it has diverged considerably from
the original.
The IBM 360 was the first major computer line to use (small-scale) ICs (Integrated Circuits), thus providing a major price/performance advantage over the
second-generation machines, which were built up from individual transistors. It
was an immediate and massive success, and the idea of a family of compatible
computers was soon adopted by all the other major manufacturers. The descendants of these machines are still in use at computer centers today. Nowadays they
are often used for managing huge databases (e.g., for airline reservation systems)
or as servers for World Wide Web sites that must process thousands of requests per
second.
The greatest strength of the ‘‘single-family’’ idea was simultaneously its greatest weakness. The original intention was that all software, including the operating
system, OS/360, had to work on all models. It had to run on small systems, which
often just replaced 1401s for copying cards to tape, and on very large systems,
which often replaced 7094s for doing weather forecasting and other heavy computing. It had to be good on systems with few peripherals and on systems with many
peripherals. It had to work in commercial environments and in scientific environments. Above all, it had to be efficient for all of these different uses.
There was no way that IBM (or anybody else for that matter) could write a
piece of software to meet all those conflicting requirements. The result was an
enormous and extraordinarily complex operating system, probably two to three
orders of magnitude larger than FMS. It consisted of millions of lines of assembly
language written by thousands of programmers, and contained thousands upon
12
INTRODUCTION
CHAP. 1
thousands of bugs, which necessitated a continuous stream of new releases in an
attempt to correct them. Each new release fixed some bugs and introduced new
ones, so the number of bugs probably remained constant over time.
One of the designers of OS/360, Fred Brooks, subsequently wrote a now-classic, witty, and incisive book (Brooks, 1995) describing his experiences with
OS/360. While it would be impossible to summarize the book here, suffice it to
say that the cover shows a herd of prehistoric beasts stuck in a tar pit. The cover of
Silberschatz et al. (2012) makes a similar point about operating systems being
dinosaurs. He also made the comment that adding programmers to a late software
project makes it even later as well saying that it takes 9 months to produce a child,
no matter how many women you assign to the project.
Despite its enormous size and problems, OS/360 and the similar third-generation operating systems produced by other computer manufacturers actually satisfied most of their customers reasonably well. They also popularized several key
techniques absent in second-generation operating systems. Probably the most
important of these was multiprogramming. On the 7094, when the current job
paused to wait for a tape or other I/O operation to complete, the CPU simply sat
idle until the I/O finished. With heavily CPU-bound scientific calculations, I/O is
infrequent, so this wasted time is not significant. With commercial data processing,
the I/O wait time can often be 80% or 90% of the total time, so something had to
be done to avoid having the (expensive) CPU be idle so much.
The solution that evolved was to partition memory into several pieces, with a
different job in each partition, as shown in Fig. 1-5. While one job was waiting for
I/O to complete, another job could be using the CPU. If enough jobs could be held
in main memory at once, the CPU could be kept busy nearly 100% of the time.
Having multiple jobs safely in memory at once requires special hardware to protect
each job against snooping and mischief by the other ones, but the 360 and other
third-generation systems were equipped with this hardware.
Job 3
Job 2
Job 1
Memory
partitions
Operating
system
Figure 1-5. A multiprogramming system with three jobs in memory.
Another major feature present in third-generation operating systems was the
ability to read jobs from cards onto the disk as soon as they were brought to the
computer room. Then, whenever a running job finished, the operating system could
load a new job from the disk into the now-empty partition and run it. This ability is
SEC. 1.2
HISTORY OF OPERATING SYSTEMS
13
called spooling (from Simultaneous Peripheral Operation On Line) and was
also used for output. With spooling, the 1401s were no longer needed, and much
carrying of tapes disappeared.
Although third-generation operating systems were well suited for big scientific
calculations and massive commercial data-processing runs, they were still basically
batch systems. Many programmers pined for the first-generation days when they
had the machine all to themselves for a few hours, so they could debug their programs quickly. With third-generation systems, the time between submitting a job
and getting back the output was often several hours, so a single misplaced comma
could cause a compilation to fail, and the programmer to waste half a day. Programmers did not like that very much.
This desire for quick response time paved the way for timesharing, a variant
of multiprogramming, in which each user has an online terminal. In a timesharing
system, if 20 users are logged in and 17 of them are thinking or talking or drinking
coffee, the CPU can be allocated in turn to the three jobs that want service. Since
people debugging programs usually issue short commands (e.g., compile a fivepage procedure rather than long ones (e.g., sort a million-record file), the computer
can provide fast, interactive service to a number of users and perhaps also work on
big batch jobs in the background when the CPU is otherwise idle. The first general-purpose timesharing system, CTSS (Compatible Time Sharing System), was
developed at M.I.T. on a specially modified 7094 (Corbató et al., 1962). However,
timesharing did not really become popular until the necessary protection hardware
became widespread during the third generation.
After the success of the CTSS system, M.I.T., Bell Labs, and General Electric
(at that time a major computer manufacturer) decided to embark on the development of a ‘‘computer utility,’’ that is, a machine that would support some hundreds
of simultaneous timesharing users. Their model was the electricity system—when
you need electric power, you just stick a plug in the wall, and within reason, as
much power as you need will be there. The designers of this system, known as
MULTICS (MULTiplexed Information and Computing Service), envisioned
one huge machine providing computing power for everyone in the Boston area.
The idea that machines 10,000 times faster than their GE-645 mainframe would be
sold (for well under $1000) by the millions only 40 years later was pure science
fiction. Sort of like the idea of supersonic trans-Atlantic undersea trains now.
MULTICS was a mixed success. It was designed to support hundreds of users
on a machine 1000× slower than a modern smartphone and with a million times
less memory. This is not quite as crazy as it sounds, since in those days people
knew how to write small, efficient programs, a skill that has subsequently been
completely lost. There were many reasons that MULTICS did not take over the
world, not the least of which is that it was written in the PL/I programming language, and the PL/I compiler was years late and barely worked at all when it finally arrived. In addition, MULTICS was enormously ambitious for its time, much
like Charles Babbage’s analytical engine in the 19th century.
14
INTRODUCTION
CHAP. 1
To make a long story short, MULTICS introduced many seminal ideas into the
computer literature, but turning it into a serious product and a major commercial
success was a lot harder than anyone had expected. Bell Labs dropped out of the
project, and General Electric quit the computer business altogether. However,
M.I.T. persisted and eventually got MULTICS working. It was ultimately sold as a
commercial product by the company (Honeywell) that bought GE’s computer business when GE got tired of it and was installed by about 80 major companies and
universities worldwide. While their numbers were small, MULTICS users were
fiercely loyal. General Motors, Ford, and the U.S. National Security Agency, for
example, shut down their MULTICS systems only in the late 1990s, 30 years after
MULTICS was released, after years of begging Honeywell to update the hardware.
The last MULTICS system was shut down amid a lot of tears in October 2000.
Can you imagine hanging onto your PC for 30 years because you think it is so
much better than everything else out there? That’s the kind of loyalty MULTICS
inspired—and for good reason. It was hugely important.
By the end of the 20th century, the concept of a computer utility had fizzled
out, but it came back in the form of cloud computing, in which relatively small
computers (including smartphones, tablets, and the like) are connected to servers in
vast and distant data centers where all the computing is done, with the local computer mostly handling the user interface. The motivation here is that most people
do not want to administrate an increasingly complex and evolving computer system and would prefer to have that work done by a team of professionals, for example, people working for the company running the data center.
Despite its lack of commercial success, MULTICS had a huge influence on
subsequent operating systems (especially UNIX and its derivatives, Linux, macOS,
iOS, and FreeBSD). It is described in several papers and a book (Corbató and
Vyssotsky, 1965; Daley and Dennis, 1968; Organick, 1972; Corbató et al., 1972;
and Saltzer, 1974). It also has an active Website, located at www.multicians.org,
with much information about the system, its designers, and its users.
Another major development during the third generation was the phenomenal
growth of minicomputers, starting with the DEC PDP-1 in 1961. The PDP-1 had
only 4K of 18-bit words, but at $120,000 per machine (less than 5% of the price of
a 7094), it sold like hotcakes. For certain kinds of nonnumerical work, it was
almost as fast as the 7094 and gave birth to a whole new industry. It was quickly
followed by a series of other PDPs (unlike IBM’s family, all incompatible) culminating in the PDP-11.
One of the computer scientists at Bell Labs who had worked on the MULTICS
project, Ken Thompson, subsequently found a small PDP-7 minicomputer that no
one was using and set out to write a stripped-down, one-user version of MULTICS.
This work later developed into the UNIX operating system, which became popular
in the academic world, with government agencies, and with many companies.
The history of UNIX has been told elsewhere (e.g., Salus, 1994). Part of that
story will be given in Chap. 10. For now, suffice it to say that because the source
SEC. 1.2
HISTORY OF OPERATING SYSTEMS
15
code was widely available, various organizations developed their own (incompatible) versions, which led to chaos. Two major versions developed, System V, from
AT&T, and BSD (Berkeley Software Distribution) from the University of California at Berkeley. These had minor variants as well. To make it possible to write
programs that could run on any UNIX system, IEEE developed a standard for
UNIX, called POSIX, that most versions of UNIX now support. POSIX defines a
minimal system-call interface that conformant UNIX systems must support. In
fact, some other operating systems now also support the POSIX interface.
As an aside, it is worth mentioning that in 1987, one of the authors (Tanenbaum) released a small clone of UNIX, called MINIX, primarily for educational
purposes. Functionally, MINIX is very similar to UNIX, including POSIX support.
Since that time, the original version of MINIX has evolved into MINIX 3, which is
highly modular and focused on very high reliability and available for free from the
Website www.minix3.org. MINIX 3 has the ability to detect and replace faulty or
even crashed modules (such as I/O device drivers) on the fly without a reboot and
without disturbing running programs. A book describing its internal operation and
listing the source code in an appendix is also available (Tanenbaum and Woodhull,
2006).
The desire for a free production (as opposed to educational) version of MINIX
led a Finnish student, Linus Torvalds, to write Linux. This system was directly
inspired by and developed on MINIX and originally supported various MINIX features (e.g., the MINIX file system). It has since been extended in many ways by
many people but still retains some underlying structure common to MINIX and to
UNIX. Readers interested in a detailed history of Linux and the open source
movement might want to read Glyn Moody’s (2001) book. Most of what will be
said about UNIX in this book thus applies to System V, MINIX, Linux, and other
versions and clones of UNIX as well.
Interestingly, both Linux and MINIX have become widely used. Linux powers
a huge share of the servers in data centers and forms the basis of Android which
dominates the smartphone market. MINIX was adapted by Intel for a separate and
somewhat secret ‘‘management’’ processor embedded in virtually all its chipsets
since 2008. In other words, if you have an Intel CPU, you also run MINIX deep in
your processor, even if your main operating system is, say, Windows or Linux.
1.2.4 The Fourth Generation (1980–Present): Personal Computers
With the development of LSI (Large Scale Integration) circuits—chips containing thousands of transistors on a square centimeter of silicon—the age of the
personal computer dawned. In terms of architecture, personal computers (initially
called microcomputers) were not all that different from minicomputers of the
PDP-11 class, but in terms of price they certainly were different. Where the minicomputer made it possible for a department in a company or university to have its
16
INTRODUCTION
CHAP. 1
own computer, the microprocessor chip made it possible for a single individual to
have his or her own personal computer.
In 1974, when Intel came out with the 8080, the first general-purpose 8-bit
CPU, it wanted an operating system for it, in part to be able to test it. Intel asked
one of its consultants, Gary Kildall, to write one. Kildall and a friend first built a
controller for the newly released Shugart Associates 8-inch floppy disk and hooked
the floppy disk up to the 8080, thus producing the first microcomputer with a disk.
Kildall then wrote a disk-based operating system called CP/M (Control Program
for Microcomputers) for it. Since Intel did not think that disk-based microcomputers had much of a future, when Kildall asked for the rights to CP/M, Intel granted his request. Kildall then formed a company, Digital Research, to further develop
and sell CP/M.
In 1977, Digital Research rewrote CP/M to make it suitable for running on the
many microcomputers using the 8080, Zilog Z80, and other CPU chips. Many
application programs were written to run on CP/M, allowing it to completely dominate the world of microcomputing for about 5 years.
In the early 1980s, IBM designed the IBM PC and looked around for software
to run on it. People from IBM contacted Bill Gates to license his BASIC interpreter. They also asked him if he knew of an operating system to run on the PC.
Gates suggested that IBM contact Digital Research, then the world’s dominant
operating systems company. Making what was without a doubt the worst business
decision in recorded history, Kildall refused to meet with IBM, sending a subordinate instead. To make matters even worse, his lawyer even refused to sign IBM’s
nondisclosure agreement covering the not-yet-announced PC. Consequently, IBM
went back to Gates asking if he could provide them with an operating system.
When IBM came back to him, Gates quickly realized that a local computer
manufacturer, Seattle Computer Products, had a suitable operating system, DOS
(Disk Operating System). He approached them and asked to buy it (allegedly for
$75,000), which they readily accepted. Gates then offered IBM a DOS/BASIC
package, which IBM accepted. IBM wanted certain modifications, so Gates hired
the person who wrote DOS, Tim Paterson, as an employee of Gates’ fledgling
company, Microsoft, to make them. The revised system was renamed MS-DOS
(MicroSoft Disk Operating System) and quickly came to dominate the IBM PC
market. A key factor here was Gates’ (in retrospect, extremely wise) decision to
sell MS-DOS to computer companies for bundling with their hardware, compared
to Kildall’s attempt to sell CP/M to end users one at a time (at least initially).
After all this transpired, Kildall died suddenly and unexpectedly from causes that
have not been fully disclosed.
By the time the successor to the IBM PC, the IBM PC/AT, came out in 1983
with the Intel 80286 CPU, MS-DOS was firmly entrenched and CP/M was on its
last legs. MS-DOS was later widely used on the 80386 and 80486. Although the
initial version of MS-DOS was fairly primitive, subsequent versions included more
advanced features, including many taken from UNIX. (Microsoft was well aware
SEC. 1.2
HISTORY OF OPERATING SYSTEMS
17
of UNIX, even selling a microcomputer version of it called XENIX during the
company’s early years.)
CP/M, MS-DOS, and other operating systems for early microcomputers were
all based on users typing in commands from the keyboard. That eventually changed due to research done by Doug Engelbart at Stanford Research Institute in the
1960s. Engelbart invented the Graphical User Interface, complete with windows,
icons, menus, and mouse. These ideas were adopted by researchers at Xerox PARC
and incorporated into machines they built.
One fine day, Steve Jobs, who co-invented the Apple computer in his garage,
visited PARC, saw a GUI, and instantly realized its potential value, something
Xerox management famously did not. This strategic blunder of incredibly gargantuan proportions led to a book entitled Fumbling the Future (Smith and Alexander,
1988). Jobs then embarked on building an Apple with a GUI. This project led to
the Lisa, which was too expensive and failed commercially. Jobs’ second attempt,
the Apple Macintosh, was an huge success, not only because it was much cheaper
than the Lisa, but also because it was user friendly, meaning that it was intended
for users who not only knew nothing about computers but furthermore had absolutely no intention whatsoever of learning. In the creative world of graphic design,
professional digital photography, and professional digital video production, Macintoshes became widely used and their users loved them. In 1999, Apple adopted a
kernel derived from Carnegie Mellon University’s Mach microkernel which was
originally developed to replace the kernel of BSD UNIX. Thus, Apple’s macOS is
a UNIX-based operating system, albeit with a distinctive interface.
When Microsoft decided to build a successor to MS-DOS, it was strongly
influenced by the success of the Macintosh. It produced a GUI-based system called Windows, which originally ran on top of MS-DOS (i.e., it was more like a shell
than a true operating system). For about 10 years, from 1985 to 1995, Windows
was just a graphical environment on top of MS-DOS. However, starting in 1995 a
freestanding version, Windows 95, was released that incorporated many operating
system features into it, using the underlying MS-DOS system only for booting and
running old MS-DOS programs. Microsoft rewrote much of the operating system
from scratch for Windows NT, a full 32-bit system. The lead designer for Windows NT was David Cutler, who was also one of the designers of the VAX VMS
operating system, so some ideas from VMS are present in NT. In fact, so many
ideas from VMS were present in it that the owner of VMS, DEC, sued Microsoft.
The case was settled out of court for an amount of money requiring many digits to
express. Version 5 of Windows NT was renamed Windows 2000 in early 1999,
and 2 years later Microsoft released a slightly upgraded version called Windows
XP which had a longer run than other versions (6 years).
After Windows 2000, Microsoft broke up the Windows family into a client and
a server line. The client line was based on XP and its successors, while the server
line produced Windows Server 2003–2019 and now Windows Server vNext.
Microsoft later also introduced a third line, for the embedded world. All of these
18
INTRODUCTION
CHAP. 1
families of Windows forked off their own variations in the form of service packs
in a dizzying proliferation of versions. It was enough to drive some administrators
(and writers of operating systems textbooks) balmy.
When in January 2007 Microsoft finally released the successor to Windows
XP, which it called Vista, it came with a new graphical interface, improved security, and many new or upgraded user programs. It bombed. Users complained about
high system requirements and restrictive licensing terms. Its successor, Windows
7, a much less resource hungry version of the operating system, quickly overtook
it. In 2012, Windows 8 came out. It had a completely new look and feel, geared
largely for touch screens. The company hoped that the new design would become
the dominant operating system on a wide variety of devices: desktops, notebooks,
tablets, phones, and home theater PCs. It did not. While Windows 8 (and especially Windows 8.1) were successful, their popularity was mostly limited to PCs. In
fact, many people did not like the new design much and Microsoft reverted it in
2015 in Windows 10. A few years later, Windows 10 overtook Windows 7 as the
most popular Windows version. Windows 11 was released in 2021.
The other major contender in the personal computer world comprises the
UNIX family. UNIX, and especially Linux, is strongest on network and enterprise
servers but also popular on desktop computers, notebooks, tablets, embedded systems, and smartphones. FreeBSD is also a popular UNIX derivative, originating
from the BSD project at Berkeley. Every modern Mac runs a modified version of
FreeBSD (macOS). UNIX derivatives are widely used on mobile devices, such as
those running iOS 7 or Android.
Many UNIX users, especially experienced programmers, prefer a command-based interface to a GUI, so nearly all UNIX systems support a windowing
system called the X Window System (also known as X11) produced at M.I.T.
This system handles the basic window management, allowing users to create,
delete, move, and resize windows using a mouse. Often a complete GUI-based
desktop environment, such as Gnome or KDE, is available to run on top of X11,
giving UNIX a look and feel something like the Macintosh or Microsoft Windows,
for those UNIX users who want such a thing.
An interesting development that began during the mid-1980s was the development of network operating systems and distributed operating systems to manage a collection of computers (Van Steen and Tanenbaum, 2017). In a network
operating system, the users are aware of the existence of multiple computers and
can log in to remote machines and copy files from one machine to another. Each
machine runs its own local operating system and has its own local user (or users).
Such systems are not fundamentally different from single-processor operating systems. They obviously need a network interface and some low-level software to
drive it, as well as programs to achieve remote login and remote file access, but
these additions do not change the essential structure of the operating system.
A distributed operating system, in contrast, is one that appears to its users as a
traditional uniprocessor system, even though it is actually composed of multiple
SEC. 1.2
HISTORY OF OPERATING SYSTEMS
19
processors. The users should not be aware of where their programs are being run or
where their files are located; that should all be handled automatically and efficiently by the operating system.
True distributed operating systems require more than just adding a little code
to a uniprocessor operating system, because distributed and centralized systems
differ in certain critical ways. Distributed systems, for example, often allow applications to run on several processors at the same time, thus requiring more complex
processor scheduling algorithms in order to optimize the amount of parallelism.
Moreover, communication delays within the network often mean that these (and
other) algorithms must run with incomplete, outdated, or even incorrect information. This situation differs radically from that in a single-processor system in
which the operating system has complete information about the system state.
1.2.5 The Fifth Generation (1990–Present): Mobile Computers
Ever since detective Dick Tracy started talking to his ‘‘two-way radio wrist
watch’’ in the 1940s comic strip, people have craved a communication device they
could carry around wherever they went. The first real mobile phone appeared in
1946 and weighed some 40 kilos. You could take it wherever you went as long as
you had a car in which to carry it.
The first true handheld phone appeared in the 1970s and, at roughly one kilogram, was positively featherweight. It was affectionately known as ‘‘the brick.’’
Pretty soon everybody wanted one. Today, mobile phone penetration in developed
countries is close to 90% of the global population. We can make calls not just with
our portable phones and wrist watches, but even with eyeglasses and other wearable items. Moreover, the phone part is no longer central. We receive email, surf
the Web, text our friends, play games, navigate around heavy traffic—and do not
even think twice about it.
While the idea of combining telephony and computing in a phone-like device
has been around since the 1970s also, the first real smartphone did not appear until
the mid-1990s when Nokia released the N9000, which literally combined two,
mostly separate devices: a phone and a Personal Digital Assistant. In 1997, Ericsson coined the term smartphone for its GS88 ‘‘Penelope.’’
Now that smartphones have become ubiquitous, the competition between the
operating systems is as fierce as in the PC world. At the time of writing, Google’s
Android is the dominant operating system with Apple’s iOS a clear second, but this
was not always the case and all may be different again in just a few years. If anything is clear in the world of smartphones, it is that it is not easy to stay king of the
mountain for long.
After all, most smartphones in the first decade after their inception were running Symbian OS. It was the operating system of choice for popular brands like
Samsung, Sony Ericsson, Motorola, and especially Nokia. However, other operating systems like RIM’s Blackberry OS (introduced for smartphones in 2002) and
20
INTRODUCTION
CHAP. 1
Apple’s iOS (released for the first iPhone in 2007) started eating into Symbian’s
market share. Many expected that RIM would dominate the business market, while
iOS would dominate on consumer devices. Symbian’s market share plummeted. In
2011, Nokia ditched Symbian and announced it would focus on Windows Phone as
its primary platform. For some time, Apple and RIM were the toast of the town
(although not nearly as dominant as Symbian had been), but it did not take very
long for Android, a Linux-based operating system released by Google in 2008, to
overtake all its rivals.
For phone manufacturers, Android had the advantage that it was open source
and available under a permissive license. As a result, they could tinker with it and
adapt it to their own hardware with ease. Also, it has a huge community of developers writing apps, mostly in the familiar Java programming language. Even so,
the past years have shown that the dominance may not last, and Android’s competitors are eager to claw back some of its market share. We will look at Android in
detail in Sec. 10.8.
1.3 COMPUTER HARDWARE REVIEW
An operating system is intimately tied to the hardware of the computer it runs
on. It extends the computer’s instruction set and manages its resources. To work,
it must know a great deal about the hardware, at least about how the hardware
appears to the programmer. For this reason, let us briefly review computer hardware as found in modern personal computers. After that, we can start getting into
the details of what operating systems do and how they work.
Conceptually, a simple personal computer can be abstracted to a model resembling that of Fig. 1-6. The CPU, memory, and I/O devices are all connected by a
system bus and communicate with one another over it. Modern personal computers
have a more complicated structure, involving multiple buses, which we will look at
later. For the time being, this model will be sufficient. In the following sections,
we will briefly review these components and examine some of the hardware issues
that are of concern to operating system designers. Needless to say, this will be a
very compact summary. Many books have been written on the subject of computer
hardware and computer organization. Two well-known ones are by Tanenbaum
and Austin (2012) and Patterson and Hennessy (2018).
1.3.1 Processors
The ‘‘brain’’ of the computer is the CPU. It fetches instructions from memory
and executes them. The basic cycle of every CPU is to fetch the first instruction
from memory, decode it to determine its type and operands, execute it, and then
fetch, decode, and execute subsequent instructions. The cycle is repeated until the
program finishes. In this way, programs are carried out.
SEC. 1.3
COMPUTER HARDWARE REVIEW
21
Monitor
CPU
MMU
Memory
Video
controller
Keyboard
USB printer
Keyboard
controller
USB
controller
Hard
disk drive
Hard
disk
controller
Bus
Figure 1-6. Some of the components of a simple personal computer.
Each CPU has a specific set of instructions that it can execute. Thus an x86
processor cannot execute ARM programs and an ARM processor cannot execute
x86 programs. Please note that we will use the term x86 to refer to all the Intel
processors descended from the 8088, which was used on the original IBM PC.
These include the 286, 386, and Pentium series, as well as the modern Intel core i3,
i5, and i7 CPUs (and their clones).
Because accessing memory to get an instruction or data word takes much longer than executing an instruction, all CPUs contain registers inside to hold key variables and temporary results. Instruction sets often contains instructions to load a
word from memory into a register, and store a word from a register into memory.
Other instructions combine two operands from registers and/or memory, into a
result, such as adding two words and storing the result in a register or in memory.
In addition to the general registers used to hold variables and temporary
results, most computers have several special registers that are visible to the programmer. One of these is the program counter, which contains the memory
address of the next instruction to be fetched. After that instruction has been
fetched, the program counter is updated to point to its successor.
Another register is the stack pointer, which points to the top of the current
stack in memory. The stack contains one frame for each procedure that has been
entered but not yet exited. A procedure’s stack frame holds those input parameters,
local variables, and temporary variables that are not kept in registers.
Yet another register is the PSW (Program Status Word). This register contains the condition code bits, which are set by comparison instructions, the CPU
priority, the mode (user or kernel), and various other control bits. User programs
may normally read the entire PSW but typically may write only some of its fields.
The PSW plays an important role in system calls and I/O.
The operating system must be fully aware of all the registers. When time multiplexing the CPU, the operating system will often stop the running program to
(re)start another one. Every time it stops a running program, the operating system
must save all the registers so they can be restored when the program runs later.
22
INTRODUCTION
CHAP. 1
In fact, we distinguish between the architecture and the micro-architecture.
The architecture consists of everything that is visible to the software such as the
instructions and the registers. The micro-architecture comprises the implementation of the architecture. Here we find data and instruction caches, translation
lookaside buffers, branch predictors, the pipelined datapath, and many other elements that should not normally be visible to the operating system or any other software.
To improve performance, CPU designers have long abandoned the simple
model of fetching, decoding, and executing one instruction at a time. Many modern
CPUs have facilities for executing more than one instruction at the same time. For
example, a CPU might have separate fetch, decode, and execute units, so that while
it is executing instruction n, it could also be decoding instruction n + 1 and fetching instruction n + 2. Such an organization is called a pipeline and is illustrated in
Fig. 1-7(a) for a pipeline with three stages. Longer pipelines are common. In most
pipeline designs, once an instruction has been fetched into the pipeline, it must be
executed, even if the preceding instruction was a conditional branch that was taken.
Pipelines cause compiler writers and operating system writers great headaches
because they expose the complexities of the underlying machine to them and they
have to deal with them.
Fetch
unit
Fetch
unit
Decode
unit
(a)
Execute
unit
Execute
unit
Decode
unit
Holding
buffer
Fetch
unit
Decode
unit
Execute
unit
Execute
unit
(b)
Figure 1-7. (a) A three-stage pipeline. (b) A superscalar CPU.
Even more advanced than a pipeline design is a superscalar CPU, shown in
Fig. 1-7(b). In this design, multiple execution units are present, for example, one
for integer arithmetic, one for floating-point arithmetic, and one for Boolean operations. Two or more instructions are fetched at once, decoded, and dumped into a
holding buffer until they can be executed. As soon as an execution unit becomes
available, it looks in the holding buffer to see if there is an instruction it can handle, and if so, it removes the instruction from the buffer and executes it. An implication of this design is that program instructions are often executed out of order. For
the most part, it is up to the hardware to make sure the result produced is the same
one a sequential implementation would have produced, but an annoying amount of
the complexity is foisted onto the operating system, as we shall see.
SEC. 1.3
COMPUTER HARDWARE REVIEW
23
Most CPUs, except very simple ones used in embedded systems, have (at least)
two modes, kernel mode and user mode, as mentioned earlier. Usually, a bit in the
PSW controls the mode. When running in kernel mode, the CPU can execute
every instruction in its instruction set and use every feature of the hardware. On
desktop, notebook, and server machines, the operating system normally runs in
kernel mode, giving it access to the complete hardware. On most embedded systems, a small piece runs in kernel mode, with the rest of the operating system running in user mode.
User programs always run in user mode, which permits only a subset of the
instructions to be executed and a subset of the features to be accessed. Generally,
all instructions involving I/O and memory protection are disallowed in user mode.
Setting the PSW mode bit to enter kernel mode is also forbidden, of course.
To obtain services from the operating system, a user program must make a system call, which traps into the kernel and invokes the operating system. The trap
instruction (e.g., syscall on x86-64 processors) switches from user mode to kernel
mode and starts the operating system. When the operating system is done, it
returns control to the user program at the instruction following the system call. We
will explain the details of the system call mechanism later in this chapter. For the
time being, think of it as a special kind of procedure call that has the additional
property of switching from user mode to kernel mode. As a note on typography,
we will use the lower-case Helvetica font to indicate system calls in running text,
like this: read.
It is worth noting that computers have traps other than the instruction for
executing a system call. Most of the other traps are caused by the hardware to warn
of an exceptional situation such as an attempt to divide by 0 or a floating-point
underflow. In all cases, the operating system gets control and must decide what to
do. Sometimes the program must be terminated with an error. Other times the
error can be ignored (an underflowed number can be set to 0). Finally, when the
program has announced in advance that it wants to handle certain kinds of conditions, control can be passed back to the program to let it deal with the problem.
Multithreaded and Multicore Chips
Moore’s law states that the number of transistors on a chip doubles every 18
months. This ‘‘law’’ is not some kind of law of physics, like conservation of
momentum, but is an observation by Intel cofounder Gordon Moore of how fast
process engineers at the semiconductor companies are able to shrink their transistors. Without wanting to enter the debate about when it will end and whether or not
the exponential is already slowing down some, we simply observe that Moore’s
law has held for half a century already and is expected to hold for at least a few
years more. After that, the number of atoms per transistor will become too small
and quantum mechanics will start to play a big role, preventing further shrinkage
of transistor sizes. Outwitting quantum mechanics will be quite a challenge.
24
INTRODUCTION
CHAP. 1
The abundance of transistors is leading to a problem: what to do with all of
them? We saw one approach above: superscalar architectures, with multiple functional units. But as the number of transistors increases, even more is possible. One
obvious thing to do is put bigger caches on the CPU chip. That is definitely happening, but eventually the point of diminishing returns will be reached.
The obvious next step is to replicate not only the functional units, but also
some of the control logic. The Intel Pentium 4 introduced this property, called
multithreading or hyperthreading (Intel’s name for it), to the x86 processor, and
several other CPU chips also have it—including the SPARC, the Power5, and some
ARM processors. To a first approximation, what it does is allow the CPU to hold
the state of two different threads and then switch back and forth on a nanosecond
time scale. (A thread is a kind of lightweight process, which, in turn, is a running
program; we will get into the details in Chap. 2.) For example, if one of the processes needs to read a word from memory (which takes many clock cycles), a multithreaded CPU can just switch to another thread. Multithreading does not offer true
parallelism. Only one process at a time is running, but thread-switching time is
reduced to the order of a nanosecond.
Multithreading has implications for the operating system because each thread
appears to the operating system as a separate CPU. Consider a system with two
actual CPUs, each with two threads. The operating system will see this as four
CPUs. If there is only enough work to keep two CPUs busy at a certain point in
time, it may inadvertently schedule two threads on the same CPU, with the other
CPU completely idle. This is far less efficient than using one thread on each CPU.
Beyond multithreading, many CPU chips now have four, eight, or more complete processors or cores on them. The multicore chips of Fig. 1-8 effectively carry
four minichips on them, each with its own independent CPU. (The caches will be
explained later in the book.) Some models of popular processors like Intel Xeon
and AMD Ryzen come with more than 50 cores, but there are also CPUs with core
counts in the hundreds. Making use of such a multicore chip will definitely require
a multiprocessor operating system.
Incidentally, in terms of sheer numbers, nothing beats a modern GPU (Graphics Processing Unit). A GPU is a processor with, literally, thousands of tiny cores.
They are very good for many small computations done in parallel, like rendering
polygons in graphics applications. They are not so good at serial tasks. They are
also hard to program. While GPUs can be useful for operating systems (e.g., for
encryption or processing of network traffic), it is not likely that much of the operating system itself will run on the GPUs.
1.3.2 Memory
The second major component in any computer is the memory. Ideally, memory
should be extremely fast (faster than executing an instruction so that the CPU is not
held up by the memory), abundantly large, and dirt cheap. No current technology
SEC. 1.3
25
COMPUTER HARDWARE REVIEW
Core 1
Core 2
L1
cache
Core 1
Core 2
L2
L2
Core 3
Core 4
L2
L2
L2 cache
Core 3
Core 4
(a)
(b)
Figure 1-8. (a) A quad-core chip with a shared L2 cache. (b) A quad-core chip
with separate L2 caches.
satisfies all of these goals, so a different approach is taken. The memory system is
constructed as a hierarchy of layers, as shown in Fig. 1-9, which would be typical
for a desktop computer or a server (notebooks use SSDs). The top layers have
higher speed, smaller capacity, and greater cost per bit than the lower ones, often
by factors of a billion or more.
Typical access time
Typical capacity
<1 nsec
Registers
<1 KB
1-8 nsec
Cache
4-8 MB
10-50 nsec
Main memory
16-64 GB
Magnetic disk / SSD
2-16+ TB
10 msec / 10s -100s usec
{
optional
persistent
memory
Figure 1-9. A typical memory hierarchy. The numbers are very rough approximations.
The top layer consists of the registers internal to the CPU. They are made of
the same material as the CPU and are thus just as fast as the CPU. Consequently,
there is no delay in accessing them. The storage capacity available in them is on
the order of 32 × 32 bits on a 32-bit CPU and 64 × 64 bits on a 64-bit CPU. Less
than 1 KB in both cases. Programs must manage the registers (i.e., decide what to
keep in them) themselves, in software.
Next comes the cache memory, which is mostly controlled by the hardware.
Main memory is divided up into cache lines, typically 64 bytes, with addresses 0
to 63 in cache line 0, 64 to 127 in cache line 1, and so on. The most heavily used
cache lines are kept in a high-speed cache located inside or very close to the CPU.
When the program needs to read a memory word, the cache hardware checks to see
if the line needed is in the cache. If it is, called a cache hit, the request is satisfied
from the cache and no memory request is sent over the bus to the main memory.
26
INTRODUCTION
CHAP. 1
Cache hits normally take only a few clock cycles. Cache misses have to go to
memory, with a substantial time penalty of tens to hundreds of cycles. Cache memory is limited in size due to its high cost. Some machines have two or even three
levels of cache, each one slower and bigger than the one before it.
Caching plays a major role in many areas of computer science, not just caching
lines of RAM. Whenever a resource can be divided into pieces, some of which are
used much more heavily than others, caching is often used to improve performance. Operating systems use it all the time. For example, most operating systems keep (pieces of) heavily used files in main memory to avoid having to fetch
them from stable storage repeatedly. Similarly, the results of converting long path
names like
/home/ast/projects/minix3/src/kernel/clock.c
into a ‘‘disk address’’ for the SSD or disk where the file is located can be cached to
avoid repeated lookups. Finally, when the address of a Web page (URL) is converted to a network address (IP address), the result can be cached for future use.
Many other uses exist.
In any caching system, several questions come up fairly soon, including:
1. When to put a new item into the cache.
2. Which cache line to put the new item in.
3. Which item to remove from the cache when a slot is needed.
4. Where to put a newly evicted item in the larger memory.
Not every question is relevant to every caching situation. For caching lines of main
memory in the CPU cache, a new item will generally be entered on every cache
miss. The cache line to use is generally computed by using some of the high-order
bits of the memory address referenced. For example, with 4096 cache lines of 64
bytes and 32 bit addresses, bits 6 through 17 might be used to specify the cache
line, with bits 0 to 5 the byte within the cache line. In this case, the item to remove
is the same one as the new data goes into, but in other systems it might not be.
Finally, when a cache line is rewritten to main memory (if it has been modified
since it was cached), the place in memory to rewrite it to is uniquely determined by
the address in question.
Caches are such a good idea that modern CPUs have two or more of them. The
first level or L1 cache is always inside the CPU and usually feeds decoded instructions into the CPU’s execution engine. Most chips have a second L1 cache for very
heavily used data words. The L1 caches are typically 32 KB each. In addition,
there is often a second cache, called the L2 cache, that holds several megabytes of
recently used memory words. The difference between the L1 and L2 caches lies in
the timing. Access to the L1 cache is done without any delay, whereas access to
the L2 cache involves a delay of several clock cycles.
SEC. 1.3
COMPUTER HARDWARE REVIEW
27
On multicore chips, the designers have to decide where to place the caches. In
Fig. 1-8(a), a single L2 cache is shared by all the cores. In contrast, in Fig. 1-8(b),
each core has its own L2 cache. Each strategy has its pros and cons. For example,
the shared L2 cache requires a more complicated cache controller but the per-core
L2 caches makes keeping the caches consistent more difficult.
Main memory comes next in the hierarchy of Fig. 1-9. This is the workhorse
of the memory system. Main memory is usually called RAM (Random Access
Memory). Old-timers sometimes call it core memory, because computers in the
1950s and 1960s used tiny magnetizable ferrite cores for main memory. They have
been gone for decades but the name persists. Currently, memories are often tens of
gigabytes on desktop or server machines. All CPU requests that cannot be satisfied out of the cache go to main memory.
In addition to the main memory, many computers have different kinds of nonvolatile random-access memory. Unlike RAM, nonvolatile memory does not lose
its contents when the power is switched off. ROM (Read Only Memory) is programmed at the factory and cannot be changed afterward. It is fast and inexpensive. On some computers, the bootstrap loader used to start the computer is contained in ROM. EEPROM (Electrically Erasable PROM) is also nonvolatile,
but in contrast to ROM can be erased and rewritten. However, writing it takes
orders of magnitude more time than writing RAM, so it is used in the same way
ROM is, except that it is now possible to correct bugs in programs by rewriting
them in the field. Boot strapping code may also be stored in Flash memory,
which is similarly nonvolatile, but in contrast to ROM can be erased and rewritten.
The boot strapping code is commonly referred to as BIOS (Basic Input/Output
System). Flash memory is also commonly used as the storage medium in portable
electronic devices such as smartphones and in SSDs to serve as a faster alternative
to hard disks. Flash memory is intermediate in speed between RAM and disk.
Also, unlike disk memory, if it is erased too many times, it wears out. Firmware
inside the device tries to mitigate this through load balancing.
Yet another kind of memory is CMOS, which is volatile. Many computers use
CMOS memory to hold the current time and date. The CMOS memory and the
clock circuit that increments the time in it are powered by a small battery, so the
time is correctly updated, even when the computer is unplugged. The CMOS memory can also hold the configuration parameters, such as which drive to boot from.
CMOS is used because it draws so little power that the original factory-installed
battery often lasts for several years. However, when it begins to fail, the computer
can appear to be losing its marbles, forgetting things that it has known for years,
like how to boot.
Incidentally, many computers today support a scheme known as virtual memory, which we will discuss at some length in Chap. 3. It makes it possible to run
programs larger than physical memory by placing them on nonvolatile storage
(SSD or disk) and using main memory as a kind of cache for the most heavily
executed parts. From time to time, the program will need data that are currently not
28
INTRODUCTION
CHAP. 1
in memory. It frees up some memory (e.g., by writing some data that have not
been used recently back to SSD or disk) and then loads the new data at this location. Because the physical address for the data and code is now no longer fixed, the
scheme remaps memory addresses on the fly to convert the address the program
generated to the physical address in RAM where the data are currently located.
This mapping is done by a part of the CPU called the MMU (Memory Management Unit), as shown in Fig. 1-6.
The MMU can have a major impact on performance as every memory access
by the program must be remapped using special data structures that are also in
memory. In a multiprogramming system, when switching from one program to
another, sometimes called a context switch, these data structures must change as
the mappings differ from process to process. Both the on-the-fly address translation and the context switch can be expensive operations.
1.3.3 Nonvolatile Storage
Next in the hierarchy are magnetic disks (hard disks), solid state drives (SSDs),
and persistent memory. Starting with the oldest and slowest, hard disk storage is
two orders of magnitude cheaper than RAM per bit and often two orders of magnitude larger as well. The only problem is that the time to randomly access data on it
is close to three orders of magnitude slower. The reason is that a disk is a mechanical device, as shown in Fig. 1-10.
Read/write head (1 per surface)
Surface 7
Surface 6
Surface 5
Surface 4
Surface 3
Direction of arm motion
Surface 2
Surface 1
Surface 0
Figure 1-10. Structure of a disk drive.
A disk consists of one or more metal platters that rotate at 5400, 7200, 10,800,
15,000 RPM or more. A mechanical arm pivots over the platters from the corner,
similar to the pickup arm on an old 33-RPM phonograph for playing vinyl records.
Information is written onto the disk in a series of concentric circles. At any given
SEC. 1.3
COMPUTER HARDWARE REVIEW
29
arm position, each of the heads can read an annular region known as a track. Together, all the tracks for a given arm position form a cylinder.
Each track is divided into some number of sectors, typically 512 bytes per sector. On modern disks, the outer cylinders contain more sectors than the inner ones.
Moving the arm from one cylinder to the next takes about 1 msec. Moving it to a
random cylinder typically takes 5–10 msec, depending on the drive. Once the arm
is on the correct track, the drive must wait for the needed sector to rotate under the
head, an additional delay of 5–10 msec, depending on the drive’s RPM. Once the
sector is under the head, reading or writing occurs at a rate of 50 MB/sec on lowend disks to 160–200 MB/sec on faster ones.
Many people also refer to SSDs as disks, even though they are physically not
disks at all and do not have platters or moving arms. They store data in electronic
(Flash) memory. The only way in which they resemble disks in terms of hardware
is that they also store a lot of data which is not lost when the power is off. But from
the operating system’s point of view, they are somewhat disk-like. SSDs are much
more expensive than rotating disks in terms of cost per byte stored, which is why
they are not so much used in data centers for bulk storage. However, they are much
faster than magnetic disks and since they have no mechanical arm to move, they
are better at accessing data at random locations. Reading data from an SSD takes
tens of microseconds instead of milliseconds as with hard disks. Writes are more
complicated as they require a full data block to be erased first and take more time.
But even if a write takes a few hundred microseconds, this is still better than a hard
disk’s performance.
The youngest and fastest member of the stable storage family is known as persistent memory. The best known example is Intel Optane which became available
in 2016. In many ways, persistent memory can be seen as an additional layer
between SSDs (or hard disks) and memory: it is both fast, only slightly slower than
regular RAM, and it holds its content across power cycles. While it can be used to
implement really fast SSDs, manufacturers may also attach it directly to the memory bus. In fact, it can be used like normal memory to store an application’s data
structures, except that the data will still be there when the power goes off. In that
case, accessing it requires no special driver and may happen at byte granularity,
obviating the need to transfer data in large blocks as in hard disks and SSDs.
1.3.4 I/O Devices
It should now be clear that CPU and memory are not the only resources that
the operating system must manage. There are many other ones. Besides disks
there many other I/O devices that interact heavily with the operating system. As
we saw in Fig. 1-6, I/O devices generally consist of two parts: a controller and the
device itself. The controller is a chip (or a set of chips) that physically controls the
device. It accepts commands from the operating system, for example, to read data
from the device, and carries them out.
30
INTRODUCTION
CHAP. 1
In many cases, the actual control of the device is complicated and detailed, so
it is the job of the controller to present a simpler (but still very complex) interface
to the operating system. For example, a hard disk controller might accept a command to read sector 11,206 from disk 2. The controller then has to convert this linear sector number to a cylinder, sector, and head. This conversion may be complicated by the fact that outer cylinders have more sectors than inner ones and that
some bad sectors have been remapped onto other ones. Then the controller has to
determine which cylinder the disk arm is on and give it a command to move in or
out the requisite number of cylinders. It has to wait until the proper sector has
rotated under the head and then start reading and storing the bits as they come off
the drive, removing the preamble and computing the checksum. Finally, it has to
assemble the incoming bits into words and store them in memory. To do all this
work, controllers often contain small embedded computers that are programmed to
do their work.
The other piece is the actual device itself. Devices have fairly simple interfaces, both because they cannot do much and to make them standard. The latter is
needed so that any SATA disk controller can handle any SATA disk, for example.
SATA stands for Serial ATA and ATA in turn stands for AT Attachment. In case
you are curious what AT stands for, this was IBM’s second generation ‘‘Personal
Computer Advanced Technology’’ built around the then-extremely-potent 6-MHz
80286 processor that the company introduced in 1984. What we learn from this is
that the computer industry has a habit of continuously enhancing existing
acronyms with new prefixes and suffixes. We also learned that an adjective like
‘‘advanced’’ should be used with great care, or you will look silly 40 years down
the line.
SATA is currently the standard type of hard disk on many computers. Since the
actual device interface is hidden behind the controller, all that the operating system
sees is the interface to the controller, which may be quite different from the interface to the device.
Because each type of controller is different, different software is needed to
control each one. The software that talks to a controller, giving it commands and
accepting responses, is called a device driver. Each controller manufacturer has to
supply a driver for each operating system it supports. Thus a scanner may come
with drivers for macOS, Windows 11, and Linux, for example.
To be used, the driver has to be put into the operating system so it can run in
kernel mode. Drivers can actually run outside the kernel, and operating systems
like Linux and Windows nowadays do offer some support for doing so, but the vast
majority of the drivers still run below the kernel boundary. Only very few current
systems, such as MINIX 3 run all drivers in user space. Drivers in user space must
be allowed to access the device in a controlled way, which is not straightforward
without some hardware support.
There are three ways the driver can be put into the kernel. The first way is to
relink the kernel with the new driver and then reboot the system. Many older UNIX
SEC. 1.3
COMPUTER HARDWARE REVIEW
31
systems work like this. The second way is to make an entry in an operating system
file telling it that it needs the driver and then reboot the system. At boot time, the
operating system goes and finds the drivers it needs and loads them. Older versions
of Windows work this way. The third way is for the operating system to be able to
accept new drivers while running and install them on the fly without the need to
reboot. This way used to be rare but is becoming much more common now. Hotpluggable devices, such as USB and Thunderbolt devices (discussed below),
always need dynamically loaded drivers.
Every controller has a small number of registers that are used to communicate
with it. For example, a minimal disk controller might have registers for specifying
the disk address, memory address, sector count, and direction (read or write). To
activate the controller, the driver gets a command from the operating system, then
translates it into the appropriate values to write into the device registers. The collection of all the device registers forms the I/O port space, a subject we will come
back to in Chap. 5.
On some computers, the device registers are mapped into the operating system’s address space (the addresses it can use), so they can be read and written like
ordinary memory words. On such computers, no special I/O instructions are
required and user programs can be kept away from the hardware by not putting
these memory addresses within their reach (e.g., by using base and limit registers).
On other computers, the device registers are put in a special I/O port space, with
each register having a port address. On these machines, special IN and OUT
instructions are available in kernel mode to allow drivers to read and write the registers. The former scheme eliminates the need for special I/O instructions but uses
up some of the address space. The latter uses no address space but requires special
instructions. Both systems are widely used.
Input and output can be done in three different ways. In the simplest method, a
user program issues a system call, which the kernel then translates into a procedure
call to the appropriate driver. The driver then starts the I/O and sits in a tight loop
continuously polling the device to see if it is done (usually there is some bit that
indicates that the device is still busy). When the I/O has completed, the driver puts
the data (if any) where they are needed and returns. The operating system then
returns control to the caller. This method is called busy waiting and has the
disadvantage of tying up the CPU polling the device until it is finished.
The second method is for the driver to start the device and ask it to give an
interrupt when it is finished. At that point, the driver returns. The operating system then blocks the caller if need be and looks for other work to do. When the controller detects the end of the transfer, it generates an interrupt to signal completion.
Interrupts are very important in operating systems, so let us examine the idea
more closely. In Fig. 1-11(a) we see a three-step process for I/O. In step 1, the
driver tells the controller what to do by writing into its device registers. The controller then starts the device. When the controller has finished reading or writing
32
INTRODUCTION
CHAP. 1
the number of bytes it has been instructed to transfer, it signals the interrupt controller chip using certain bus lines in step 2. If the interrupt controller is ready to
accept the interrupt (which it may not be if it is busy handling a higher-priority
interrupt), it asserts a pin on the CPU chip telling it, in step 3. In step 4, the interrupt controller puts the number of the device on the bus so the CPU can read it and
know which device has just finished (many devices may be running at the same
time).
Disk drive
Current instruction
CPU
1
3
Next instruction
Interrupt
controller
Disk
controller
4
2
3. Return
1. Interrupt
2. Dispatch
to handler
(a)
Interrupt handler
(b)
Figure 1-11. (a) The steps in starting an I/O device and getting an interrupt. (b)
Interrupt processing involves taking the interrupt, running the interrupt handler,
and returning to the user program.
Once the CPU has decided to take the interrupt, the program counter and PSW
are typically then pushed onto the current stack and the CPU switched into kernel
mode. The device number may be used as an index into part of memory to find the
address of the interrupt handler for this device. This part of memory is called the
interrupt vector table. Once the interrupt handler (part of the driver for the interrupting device) has started, it saves the stacked program counter, PSW, and other
registers (typically in the process table). Then it queries the device to learn its status. When the handler is all finished, it restores the context and returns to the previously running user program to the first instruction that was not yet executed. These
steps are shown in Fig. 1-11(b). We will discuss interrupt vectors further in in the
next chapter.
The third method for doing I/O makes use of special hardware: a DMA
(Direct Memory Access) chip that can control the flow of bits between memory
and some controller without constant CPU intervention. The CPU sets up the
DMA chip, telling it how many bytes to transfer, the device and memory addresses
involved, and the direction, and lets it go. When the DMA chip is done, it causes
an interrupt, which is handled as described above. DMA and I/O hardware in general will be discussed in more detail in Chap. 5.
SEC. 1.3
33
COMPUTER HARDWARE REVIEW
1.3.5 Buses
The organization of Fig. 1-6 was used on minicomputers for years and also on
the original IBM PC. However, as processors and memories got faster, the ability
of a single bus (and certainly the IBM PC bus) to handle all the traffic was strained
to the breaking point. Something had to give. As a result, additional buses were
added, both for faster I/O devices and for CPU-to-memory traffic. As a consequence of this evolution, a large x86 system currently looks something like
Fig. 1-12.
Core1
Core2
Cache
Cache
Shared cache
GPU Cores
DDR 4 Memory
PCIe
Graphics
DDR 4 Memory
Memory controllers
DMI
PCIe slot
PCIe slot
PCIe slot
SATA
Platform
Controller
Hub
PCIe slot
USB 3.2 ports
USB4
PCIe
ports
Gigabit Ethernet
More PCIe devices
Figure 1-12. The structure of a large x86 system.
This system has many buses (e.g., cache, memory, PCIe, PCI, USB, SATA, and
DMI), each with a different transfer rate and function. The operating system must
be aware of all of them for configuration and management. The main bus is the
PCIe (Peripheral Component Interconnect Express) bus.
The PCIe bus was invented by Intel as a successor to the older PCI bus, which
in turn was a replacement for the original ISA (Industry Standard Architecture)
bus. Capable of transferring tens of gigabits per second, PCIe is much faster than
its predecessors. It is also very different in nature. Up to its creation in 2004, most
buses were parallel and shared. A shared bus architecture means that multiple
devices use the same wires to transfer data. Thus, when multiple devices have data
to send, you need an arbiter to determine who can use the bus. In contrast, PCIe
makes use of dedicated, point-to-point connections. A parallel bus architecture as
used in traditional PCI means that you send each word of data over multiple wires.
For instance, in regular PCI buses, a single 32-bit number is sent over 32 parallel
wires. In contrast to this, PCIe uses a serial bus architecture and sends all bits in
34
INTRODUCTION
CHAP. 1
a message through a single connection, known as a lane, much like a network
packet. This is much simpler because you do not have to ensure that all 32 bits
arrive at the destination at exactly the same time. Parallelism is still used, because
you can have multiple lanes in parallel. For instance, we may use 32 lanes to carry
32 messages in parallel. As the speed of peripheral devices like network cards and
graphics adapters increases rapidly, the PCIe standard is upgraded every 3–5 years.
For instance, 16 lanes of PCIe 4.0 offer 256 gigabits per second. Upgrading to
PCIe 5.0 will give you twice that speed and PCIe 6.0 will double that again.
Meanwhile, we still have legacy devices for the older PCI standard. These devices
can be hooked up to a separate hub processor.
In this configuration, the CPU talks to memory over a fast DDR4 bus, to an
external graphics device over PCIe and to all other devices via a hub over a DMI
(Direct Media Interface) bus. The hub in turn connects all the other devices,
using the Universal Serial Bus to talk to USB devices, the SATA bus to interact
with hard disks and DVD drives, and PCIe to transfer Ethernet frames. We have
already mentioned the older PCI devices that use a traditional PCI bus.
Moreover, each of the cores has a dedicated cache and a much larger cache that
is shared between them. Each of these caches introduces yet another bus.
The USB (Universal Serial Bus) was invented to attach all the slow I/O devices, such as the keyboard and mouse, to the computer. However, calling a modern USB4 device humming along at 40 Gbps ‘‘slow’’ may not come naturally for
the generation that grew up with 8-Mbps ISA as the main bus in the first IBM PCs.
USB uses a small connector with 4–11 wires (depending on the version), some of
which supply electrical power to the USB devices or connect to ground. USB is a
centralized bus in which a root device polls all the I/O devices every 1 msec to see
if they have any traffic. USB 1.0 could handle an aggregate load of 12 Mbps, USB
2.0 increased the speed to 480 Mbps, USB 3.0 to 5 Gbps, USB 3.2 to 20 Gbps and
USB 4 will double that. Any USB device can be connected to a computer and it
will function immediately, without requiring a reboot, something pre-USB devices
required, much to the consternation of a generation of frustrated users.
1.3.6 Booting the Computer
Very briefly, the boot process is as follows. Every PC contains a motherboard,
which contains the CPU, slots for memory chips, and sockets for PCIe (or other)
plug-in cards. On the motherboard a small amount of flash holds a program called
the system firmware, which we commonly still refer to as the BIOS (Basic Input
Output System), even though strictly speaking the name BIOS applies only to the
firmware in somewhat older IBM PC compatible machines. Booting using the
original BIOS was slow, architecture-dependent, and limited to smaller SSDs and
disks (up to 2 TB). It was also very easy to understand. When Intel proposed what
would become UEFI (Unified Extensible Firmware Interface) as a replacement,
SEC. 1.3
COMPUTER HARDWARE REVIEW
35
it remedied all these issues: UEFI allows for fast booting, different architectures,
and storage sizes up to 8 ZiB, or 8 × 270 bytes. It is also so complex that trying to
understand it fully has sucked the happiness out of many a life. In this chapter, we
will cover both old and new style BIOS firmware, but only the essentials.
After we press the power button, the motherboard waits for the signal that the
power supply has stabilized. When the CPU starts executing, it fetches code from
a hard-coded physical address (known as the reset vector) that is mapped to the
flash memory. In other words, it executes code from the BIOS which detects and
initializes various resources, such as RAM, the Platform Controller Hub (see Fig.
Fig. 1-12), and interrupt controllers. In addition, it scans the PCI and/or PCIe buses
to detect and initialize all devices attached to them. If the devices present are different from when the system was last booted, it also configures the new devices.
Finally, it sets up the runtime firmware which offers critical services (including
low-level I/O) that can be used by the system after booting.
Next, it is time to move to the next stage of the booting process. In systems
using the old-style BIOS, this was all very straightforward. The BIOS would determine the boot device by trying a list of devices stored in the CMOS memory. The
user can change this list by entering a BIOS configuration program just after booting. For instance, you may ask the system to attempt to boot from a USB drive, if
one is present. If that fails, the system boots from the hard disk or SSD. The first
sector from the boot device is read into memory and executed. This sector, known
as the MBR (Master Boot Record), contains a program that normally examines
the partition table at the end of the boot sector to determine which partition is
active. A partition is a distinct region on the storage device that may for instance
contain its own file systems. Then a secondary boot loader is read in from that partition. This loader reads in the operating system from the active partition and starts
it. The operating system then queries the BIOS to get the configuration information. For each device, it checks to see if it has the device driver. If not, it asks
the user to install it, for instance by downloading it from the Internet. Once it has
all the device drivers, the operating system loads them into the kernel. Then it initializes its tables, creates whatever background processes are needed, and starts up
a login program or GUI.
With UEFI, things are different. First, it no longer relies on a Master Boot
Record residing in the first sector of the boot device, but it looks for the location of
the partition table in the second sector of the device. This GPT (GUID Partition
Table) contains information about the location of the various partitions on the SSD
or disk. Second, the BIOS itself has enough functionality to read file systems of
specific types. According to the UEFI standard, it should support at least the
FAT-12, FAT-16, and FAT-32 types. One such file system is placed in a special partition, known as the EFI system partition (ESP). Rather than a single magic boot
sector, the boot process can now use a proper file system containing programs,
configuration files, and anything else that may be useful during boot. Moreover,
UEFI expects the firmware to be able to execute programs in a specific format,
36
INTRODUCTION
CHAP. 1
called PE (Portable Executable). As you can see, the BIOS under UEFI very
much looks like a little operating system itself which understands partitions, file
systems, executables, etc. It even has a shell with some standard commands.
The boot code still needs to pick one of the bootloader programs to load Linux
or Windows, or whatever operating system, but there may be many partitions with
operating systems and given so much choice, which one should it pick? This is
decided by the UEFI boot manager, which you can think of as a boot menu with
different entries and a configurable order in which to try the different boot options.
Changing the menu and the default bootloader is very easy and can be done from
within the currently executing operating system. As before, the bootloader will
continue loading the operating system of choice.
This is by no means the full story. UEFI is very flexible and highly standardized, and contains many advanced features. However, this is enough for now.
In Chap. 9, we will pick up UEFI again when we discuss an interesting feature
known as Secure Boot, which allows a user to be sure that the operating system is
booted as intended and with the correct software.
1.4 THE OPERATING SYSTEM ZOO
Operating systems have been around now for over half a century. During this
time, quite a variety of them have been developed, not all of them widely known.
In this section, we will briefly touch upon nine of them. We will come back to
some of these different kinds of systems later in the book.
1.4.1 Mainframe Operating Systems
At the high end are the operating systems for mainframes, those room-sized
computers still found in major corporate data centers. These computers differ from
personal computers in terms of their I/O capacity. A mainframe with 1000 hard
disks and many terabytes of data are not unusual; a personal computer with these
specifications would be the envy of its friends. Mainframes are also making something of a comeback as high-end servers for large-scale electronic commerce sites,
banking, airline reservations, and servers for business-to-business transactions.
The operating systems for mainframes are heavily oriented toward processing
many jobs at once, most of which need prodigious amounts of I/O. They typically
offer three kinds of services: batch, transaction processing, and timesharing. A
batch system is one that processes routine jobs without any interactive user present.
Claims processing in an insurance company or sales reporting for a chain of stores
is typically done in batch mode. Transaction-processing systems handle large numbers of small requests, for example, check processing at a bank or airline reservations. Each unit of work is small, but the system must handle hundreds or thousands per second. Timesharing systems allow multiple remote users to run jobs on
the computer at once, such as querying a big database. These functions are closely
SEC. 1.4
THE OPERATING SYSTEM ZOO
37
related; mainframe operating systems often perform all of them. An example
mainframe operating system is Z/OS, the successor of OS/390, which in turn was a
direct descendant of OS/360. However, mainframe operating systems are gradually being replaced by UNIX variants such as Linux.
1.4.2 Server Operating Systems
One level down are the server operating systems. They run on servers, which
are either very large personal computers, workstations, or even mainframes. They
serve multiple users at once over a network and allow the users to share hardware
and software resources. Servers can provide print service, file service, database service, or Web service. Internet providers run many server machines to support their
customers and Websites use servers to store the Web pages and handle the incoming requests. Typical server operating systems are Linux, FreeBSD, Solaris, and
the Windows Server family.
1.4.3 Personal Computer Operating Systems
The next category is the personal computer operating system. Modern ones all
support multiprogramming, often with dozens of programs started up at boot time,
and multiprocessor architectures. Their job is to provide good support to a single
user. They are widely used for word processing, spreadsheets, games, and Internet
access. Common examples are Windows 11, macOS, Linux, and FreeBSD. Personal computer operating systems are so widely known that probably little introduction is needed. In fact, many people are not even aware that other kinds exist.
1.4.4 Smartphone and Handheld Computer Operating Systems
Continuing on down to smaller and smaller systems, we come to tablets (like
Apple’s iPad), smartphones and other handheld computers. A handheld computer,
originally known as a PDA (Personal Digital Assistant), is a small computer that
can be held in your hand during operation. Smartphones and tablets are the bestknown examples. As we have already seen, this market is currently dominated by
Google’s Android and Apple’s iOS. Most of these devices boast multicore CPUs,
GPS, cameras and other sensors, copious amounts of memory, and sophisticated
operating systems. Moreover, all of them have more third-party applications (apps)
than you can shake a (USB) stick at. Google has over 3 million Android apps in
the Play Store and Apple has over 2 million in the App Store.
1.4.5 The Internet of Things and Embedded Operating Systems
The IOT (Internet of Things) comprises all the billions of physical objects
with sensors and actuators that are increasingly connected to the network, such as
fridges, thermostats, security camera’s motion sensors, and so on. All of these
38
INTRODUCTION
CHAP. 1
devices contain small computers and most of them run small operating systems. In
addition, we may have even more embedded systems controlling devices that are
not connected to a network at all. Examples include traditional microwave ovens
and washing machines. Such systems do not accept user-installed software, so the
main property which distinguishes such embedded systems from the computers we
discussed earlier is the certainty that no untrusted software will ever run on it. Few
microwave ovens allow you to download and run new applications—all the software is in ROM. However, even that is changing. Some high-end cameras have
their own app stores that allow users to install custom apps for in-camera editing,
multiple exposures, different focus algorithms, different image compression algorithms, and more.
Nevertheless, for most embedded systems there is no need for protection
between applications, leading to design simplification. Conversely, such operating
systems may provide more support for functionality such as real-time scheduling
or low-power networking which are important in many embedded systems. Systems such as Embedded Linux, QNX, and VxWorks are popular in this domain.
For severely resource-constrained devices, the operating system should be able to
run in just a few kilobytes. For instance, RIOT, an open source operating system
for IoT devices, can run in less 10 KB and support systems that range from 8-bit
microcontrollers to general-purpose 32-bit CPUs. The TinyOS operating similarly
offers a very small footprint, making it popular in sensor nodes.
1.4.6 Real-Time Operating Systems
Real-time systems are characterized by having time as a key parameter. For
example, in industrial process-control systems, real-time computers have to collect
data about the production process and use it to control machines in the factory.
Often there are hard deadlines that must be met. For example, if a car is moving
down an assembly line, certain actions must take place at certain instants of time.
If, for example, a welding robot welds too early or too late, the car will be ruined.
If the action absolutely must occur at a certain moment (or within a certain range),
we have a hard real-time system. Many of these are found in industrial process
control, avionics, military, and similar application areas. These systems must provide absolute guarantees that a certain action will occur by a certain time.
A soft real-time system is one where missing an occasional deadline, while
not desirable, is acceptable and does not cause any permanent damage. Digital
audio or multimedia systems fall in this category. Smartphones are also soft realtime systems.
Since meeting deadlines is crucial in (hard) real-time systems, sometimes the
operating system is simply a library linked in with the application programs, with
everything tightly coupled and no protection between parts of the system. An
example of this type of real-time system is eCos.
SEC. 1.4
THE OPERATING SYSTEM ZOO
39
We should emphasize that the categories of IoT, embedded, real-time and even
handheld systems overlap considerably. Many of them have at least some soft realtime aspects. The embedded and real-time systems run only software put in by the
system designers; users cannot add their own software, which makes protection
easier.
1.4.7 Smart Card Operating Systems
The smallest operating systems run on smart cards, which are credit-card-sized
devices containing a CPU. They have very severe processing power and memory
constraints. Some are powered by contacts in the reader into which they are inserted, while contactless smart cards are inductively powered (which greatly limits
what they can do.) Some of them can handle only a single function, such as electronic payments, but others can handle multiple functions. Often these are proprietary systems.
Some smart cards are Java oriented. This means that the ROM on the smart
card holds an interpreter for the Java Virtual Machine (JVM). Java applets (small
programs) are downloaded to the card and are interpreted by the JVM interpreter.
Some of these cards can handle multiple Java applets at the same time, leading to
multiprogramming and the need to schedule them. Resource management and protection also become an issue when two or more applets are present at the same
time. These issues must be handled by the (usually extremely primitive) operating
system present on the card.
1.5 OPERATING SYSTEM CONCEPTS
Most operating systems provide certain basic concepts and abstractions such as
processes, address spaces, and files that are central to understanding them. In the
following sections, we will look at some of these basic concepts ever so briefly, as
an introduction. We will come back to each of them in great detail later in this
book. To illustrate these concepts we will, from time to time, use examples, generally drawn from UNIX. Similar examples typically exist in other systems as well,
however, and we will study some of them later.
1.5.1 Processes
A key concept in all operating systems is the process. A process is basically a
program in execution. Associated with each process is its address space, a list of
memory locations from 0 to some maximum, which the process can read and write.
The address space contains the executable program, the program’s data, and its
stack. Also associated with each process is a set of resources, commonly including
registers (including the program counter and stack pointer), a list of open files,
40
INTRODUCTION
CHAP. 1
outstanding alarms, lists of related processes, and all the other information needed
to run the program. A process is fundamentally a container that holds all the information needed to run a program.
We will come back to the process concept in much more detail in Chap. 2. For
the time being, the easiest way to get a good intuitive feel for a process is to think
about a multiprogramming system. The user may have started a video editing program and instructed it to convert a 2-hour video to a certain format (something that
can take hours) and then gone off to surf the Web. Meanwhile, a background process that wakes up periodically to check for incoming email may have started running. Thus, we have (at least) three active processes: the video editor, the Web
browser, and the email receiver. Periodically, the operating system decides to stop
running one process and start running another, perhaps because the first one has
used up more than its share of CPU time in the past second or two.
When a process is suspended temporarily like this, it must later be restarted in
exactly the same state it had when it was stopped. This means that all information
about the process must be explicitly saved somewhere during the suspension. For
example, the process may have several files open for reading at once. Associated
with each of these files is a pointer giving the current position (i.e., the number of
the byte or record to be read next). When a process is temporarily suspended, all
these pointers must be saved so that a read call executed after the process is restarted will read the proper data. In many operating systems, all the information about
each process, other than the contents of its own address space, is stored in an operating system table called the process table, which is an array of structures, one for
each process currently in existence.
Thus, a (suspended) process consists of its address space, usually called the
core image (in honor of the magnetic core memories used in days of yore), and its
process table entry, which contains the contents of its registers and many other
items needed to restart the process later.
The key process-management system calls are those dealing with the creation
and termination of processes. Consider a typical example. A process called the
command interpreter or (i.e., shell) reads commands from a terminal. The user
has just typed a command requesting that a program be compiled. The shell must
now create a new process that will run the compiler. When that process has finished the compilation, it executes a system call to terminate itself.
If a process can create one or more other processes (referred to as child processes) and these processes in turn can create child processes, we quickly arrive at
the process tree structure of Fig. 1-13. Related processes that are cooperating to
get some job done often need to communicate with one another and synchronize
their activities. This communication is called interprocess communication, and
will be addressed in detail in Chap. 2.
Other process system calls are available to request more memory (or release
unused memory that is not needed anymore), wait for a child process to terminate,
and overlay its program with a different one.
SEC. 1.5
OPERATING SYSTEM CONCEPTS
41
A
B
D
E
C
F
Figure 1-13. A process tree. Process A created two child processes, B and C.
Process B created three child processes, D, E, and F.
Occasionally, there is a need to convey information to a running process that is
not sitting around waiting for this information. For example, a process that is communicating with another process on a different computer does so by sending messages to the remote process over a computer network. To guard against the possibility that a message or its reply is lost, the sender may request that its own operating system notify it after a specified number of seconds, so that it can retransmit
the message if no acknowledgement has been received yet. After setting this timer,
the program may continue doing other work.
When the specified number of seconds has elapsed, the operating system sends
an alarm signal to the process. The signal causes the process to temporarily suspend whatever it was doing, save its registers on the stack, and start running a special signal-handling procedure, for example, to retransmit a presumably lost message. When the signal handler is done, the running process is restarted in the state
it was in just before the signal. Signals are the software analog of hardware interrupts and can be generated by a variety of causes in addition to timers expiring.
Many traps detected by hardware, such as executing an illegal instruction or using
an invalid address, are also converted into signals to the guilty process.
Each person authorized to use a system is assigned a UID (User IDentification) by the system administrator. Every process started has the UID of the person
who started it. On UNIX, a child process has the same UID as its parent. Users can
be members of groups, each of which has a GID (Group IDentification).
One UID, called the superuser or root (in UNIX), or Administrator (in Windows), has special power and may override many of the protection rules. In large
installations, only the system administrator knows the superuser password, but
many of the ordinary users (especially students) devote considerable effort seeking
flaws in the system that allow them to become superuser without the password.
We will study processes and interprocess communication in Chap. 2.
1.5.2 Address Spaces
Every computer has some main memory that it uses to hold executing programs. In a very simple operating system, only one program at a time is in memory. To run a second program, the first one has to be removed and the second one
placed in memory. This is known as swapping.
42
INTRODUCTION
CHAP. 1
More sophisticated operating systems allow multiple programs to be in memory at the same time. To keep them from interfering with one another (and with the
operating system), some kind of protection mechanism is needed. While the hardware must provide this mechanism, it is the operating system that controls it.
The above viewpoint is concerned with managing and protecting the computer’s main memory. A different, but equally important, memory-related issue is
managing the address space of the processes. Normally, each process has some set
of addresses it can use, typically running from 0 up to some maximum. In the simplest case, the maximum amount of address space a process has is less than the
main memory. In this way, a process can fill up its address space and there will be
enough room in main memory to hold it all.
However, on many computers addresses are 32 or 64 bits, giving an address
space of 232 or 264 bytes, respectively. What happens if a process has more address
space than the computer has main memory and the process wants to use it all? In
the first computers, such a process was just out of luck. Nowadays, a technique
called virtual memory exists, as mentioned earlier, in which the operating system
keeps part of the address space in main memory and part on SSD or disk and shuttles pieces back and forth between them as needed. In essence, the operating system creates the abstraction of an address space as the set of addresses a process
may reference. The address space is decoupled from the machine’s physical memory and may be either larger or smaller than the physical memory. Management of
address spaces and physical memory forms an important part of what an operating
system does, so all of Chap. 3 is devoted to this topic.
1.5.3 Files
Another key concept supported by virtually all operating systems is the file
system. As noted before, a major function of the operating system is to hide the
peculiarities of the SSDs, disks, and other I/O devices and present the programmer
with a nice, clean abstract model of device-independent files. System calls are
obviously needed to create files, remove files, read files, and write files. Before a
file can be read, it must be located on the storage device and opened, and after
being read it should be closed, so calls are provided to do these things.
To provide a place to keep files, most PC operating systems have the concept
of a directory, sometimes called a folder or a map, as a way of grouping files
together. A student, for example, might have one directory for each course she is
taking (for the programs needed for that course), another directory her email, and
still another directory for her home page on the Web. System calls are then needed
to create and remove directories. Calls are also provided to put an existing file in a
directory and to remove a file from a directory. Directory entries may be either
files or other directories, giving rise to a hierarchy—the file system—as shown in
Fig. 1-14. Just like many other innovations in operating systems, hierarchical file
systems were pioneered by Multics.
SEC. 1.5
43
OPERATING SYSTEM CONCEPTS
Root directory
Students
Robbert
Matty
Faculty
Leo
Prof.Brown
Courses
CS101
Papers
CS105
Prof.Green
Grants
Prof.White
Committees
SOSP
COST-11
Files
Figure 1-14. A file system for a university department.
The process and file hierarchies both are organized as trees, but the similarity
stops there. Process hierarchies usually are not very deep (more than five levels is
unusual), whereas file hierarchies are commonly six, seven, or even more levels
deep. Process hierarchies are typically more short-lived than directory hierarchies
which may exist for years. Ownership and protection also differ for processes and
files. Typically, only a parent process may control or even access a child process,
but mechanisms nearly always exist to allow files and directories to be read by a
wider group than just the owner.
Every file within the directory hierarchy can be specified by giving its path
name from the top of the directory hierarchy, the root directory. Such absolute
path names consist of the list of directories that must be traversed from the root
directory to get to the file, with slashes separating the components. In Fig. 1-14,
the path for file CS101 is /Faculty/Prof.Brown/Courses/CS101. The leading slash
indicates that the path is absolute, that is, starting at the root directory. As an aside,
in Windows, the backslash (\) character is used as the separator instead of the slash
(/) character (for historical reasons), so the file path given above would be written
as \Faculty\Prof.Brown\Courses\CS101. Throughout this book, we will generally
use the UNIX convention for paths.
At every instant, each process has a current working directory, in which path
names not beginning with a slash are looked for. For example, in Fig. 1-14, if
44
INTRODUCTION
CHAP. 1
/Faculty/Prof.Brown were the working directory, use of the path Courses/CS101
would yield the same file as the absolute path name given above. Processes can
change their working directory by issuing a system call specifying the new working directory.
Before a file can be read or written, it must be opened, at which time the permissions are checked. If the access is permitted, the system returns a small integer
called a file descriptor to use in subsequent operations. If the access is prohibited,
an error code is returned.
Another important concept in UNIX is the mounted file system. Most desktop
computers and notebooks have one or more regular USB ports into which USB
memory sticks (really, flash drives) can be plugged, or USB-C ports which can be
used to connect external SSDs and hard disks. Some computers have optical drives
for Blu-ray disks and you can ask old-timers for war stories about DVDs, CDROMs, and floppy disks. To provide an elegant way to deal with these removable
media UNIX allows the file system on these separate storage devices to be attached
to the main tree. Consider the situation of Fig. 1-15(a). Before the mount call, the
root file system, on the hard disk, and a second file system, on USB drive, are separate and unrelated. The USB drive may even be formatted with, say, FAT-32 and
the hard disk with, say, ext4.
USB drive
Root
a
c
b
x
d
y
a
c
(a)
d
b
x
y
(b)
Figure 1-15. (a) Before mounting, the files on the USB drive are not accessible.
(b) After mounting, they are part of the file hierarchy.
Unfortunately, if the current directory of the shell is in the hard disk file system, the file system on the USB drive cannot be used, because there is no way to
specify path names on it. UNIX does not allow path names to be prefixed by a
drive name or number; that would be precisely the kind of device dependence that
operating systems ought to eliminate. Instead, the mount system call allows the file
system on the USB drive to be attached to the root file system wherever the program wants it to be. In Fig. 1-15(b) the file system on the USB drive has been
mounted on directory b, thus allowing access to files /b/x and /b/y. If directory b
had contained any files, they would not be accessible while the USB drive was
mounted, since /b would now refer to the root directory of the USB drive. (Not
being able to access these files is not as serious as it at first seems: file systems are
SEC. 1.5
OPERATING SYSTEM CONCEPTS
45
nearly always mounted on empty directories.) If a system contains multiple hard
disks, they can all be mounted into a single tree as well.
Another important concept in UNIX is the special file. Special files are provided in order to make I/O devices look like files. That way, they can be read and
written using the same system calls as are used for reading and writing files. Two
kinds of special files exist: block special files and character special files. Block
special files are used to model devices that consist of a collection of randomly
addressable blocks, such as SSDs and disks. By opening a block special file and
reading, say, block 4, a program can directly access the fourth block on the device,
without regard to the structure of the file system contained on it. Similarly, character special files are used to model printers, keyboards, mice, and other devices that
accept or output a character stream. By convention, the special files are kept in the
/dev directory. For example, /dev/lp might be the printer (once called the line printer).
The last feature we will discuss in this overview relates to both processes and
files: pipes. A pipe is a sort of pseudofile that can be used to connect two processes, as shown in Fig. 1-16. If processes A and B wish to talk using a pipe, they
must set it up in advance. When process A wants to send data to process B, it writes
on the pipe as though it were an output file. In fact, the implementation of a pipe is
very much like that of a file. Process B can read the data by reading from the pipe
as though it were an input file. Thus, communication between processes in UNIX
looks very much like ordinary file reads and writes. Stronger yet, the only way a
process can discover that the output file it is writing on is not really a file, but a
pipe, is by making a special system call. File systems are very important. We will
have much more to say about them in Chap. 4 and also in Chaps. 10 and 11.
Process
Process
Pipe
A
B
Figure 1-16. Two processes connected by a pipe.
1.5.4 Input/Output
All computers have physical devices for acquiring input and producing output.
After all, what good would a computer be if the users could not tell it what to do
and could not get the results after it did the work requested? Many kinds of input
and output devices exist, including keyboards, monitors, printers, and so on. It is
up to the operating system to manage these devices.
Consequently, every operating system has an I/O subsystem for managing its
I/O devices. Some of the I/O software is device independent, that is, it applies to
many or all I/O devices equally well. Other parts of it, such as device drivers, are
specific to particular I/O devices. In Chap. 5, we will have a look at I/O software.
46
INTRODUCTION
CHAP. 1
1.5.5 Protection
Computers contain large amounts of information that users often want to protect and keep confidential. This information may include email, business plans, tax
returns, and much more. It is up to the operating system to manage the system
security so that files, for example, are accessible only to authorized users.
As a simple example, just to get an idea of how security can work, consider
UNIX. Files in UNIX are protected by assigning each one a 9-bit binary protection code. The protection code consists of three 3-bit fields, one for the owner, one
for other members of the owner’s group (users are divided into groups by the system administrator), and one for everyone else. Each field has a bit for read access,
a bit for write access, and a bit for execute access. These 3 bits are known as the
rwx bits. For example, the protection code rwxr-x--x means that the owner can
read, write, or execute the file, other group members can read or execute (but not
write) the file, and everyone else can execute (but not read or write) the file. For a
directory, x indicates search permission. A dash means that the corresponding permission is absent.
In addition to file protection, there are many other security issues. Protecting
the system from unwanted intruders, both human and nonhuman (e.g., viruses), is
one of them. We will look at various security issues in Chap. 9.
1.5.6 The Shell
The operating system is the code that carries out the system calls. Editors,
compilers, assemblers, linkers, utility programs, and command interpreters definitely are not part of the operating system, even though they are important and useful. At the risk of confusing things somewhat, in this section we will look briefly
at the UNIX command interpreter, the shell. Although it is not part of the operating system, it makes heavy use of many operating system features and thus serves
as a good example of how the system calls are used. It is also the main interface
between a user sitting at his terminal and the operating system, unless the user is
using a graphical user interface. Many shells exist, including sh, csh, ksh, bash, and
zsh. All of them support the functionality described below, which derives from the
original shell (sh).
When any user logs in, a shell is started up. The shell has the terminal as standard input and standard output. It starts out by typing the prompt, a character
such as a dollar sign, which tells the user that the shell is waiting to accept a command. If the user now types
date
for example, the shell creates a child process and runs the date program as the
child. While the child process is running, the shell waits for it to finish. When the
child finishes, the shell types the prompt again and tries to read the next input line.
SEC. 1.5
OPERATING SYSTEM CONCEPTS
47
The user can specify that standard output be redirected to a file, for example,
date >file
Similarly, standard input can be redirected, as in
sor t <file1 >file2
which invokes the sort program with input taken from file1 and output sent to file2.
The output of one program can be used as the input for another program by
connecting them with a pipe. Thus
cat file1 file2 file3 | sort >/dev/lp
invokes the cat program to concatenate three files and send the output to sort to
arrange all the lines in alphabetical order. The output of sort is redirected to the file
/dev/lp, typically the printer.
If a user puts an ampersand after a command, the shell does not wait for it to
complete. Instead it just gives a prompt immediately. Consequently,
cat file1 file2 file3 | sort >/dev/lp &
starts up the sort as a background job, allowing the user to continue working normally while the sort is going on. The shell has a number of other interesting features, which we do not have space to discuss here. Most books on UNIX discuss
the shell at some length (e.g., Kochan and Wood, 2016; and Shotts, 2019).
Most personal computers these days use a GUI. In fact, the GUI is just a program running on top of the operating system, like a shell. In Linux systems, this
fact is made obvious because the user has a choice of multiple GUIs: Gnome, KDE
or even none at all (using a terminal window on X11). In Windows, replacing the
standard GUI desktop is not typically done.
1.5.7 Ontogeny Recapitulates Phylogeny
After Charles Darwin’s book On the Origin of the Species was published, the
German zoologist Ernst Haeckel stated that ‘‘ontogeny recapitulates phylogeny.’’
By this he meant that the development of an embryo (ontogeny) repeats (i.e., recapitulates) the evolution of the species (phylogeny). In other words, after fertilization, a human egg goes through stages of being a fish, a pig, and so on before turning into a human baby. Modern biologists regard this as a gross simplification, but
it still has a kernel of truth in it.
Something vaguely analogous has happened in the computer industry. Each
new species (mainframe, minicomputer, personal computer, handheld, embedded
computer, smart card, etc.) seems to go through the development that its ancestors
did, both in hardware and in software. We often forget that much of what happens
in the computer business and a lot of other fields is technology driven. The reason
the ancient Romans lacked automobiles is not that they enjoyed walking so much.
48
INTRODUCTION
CHAP. 1
It is because they did not know how to build them. Personal computers exist not
because millions of people have a centuries-old pent-up desire to own a computer,
but because it is now possible to manufacture them cheaply. We often forget how
much technology affects our view of systems and it is worth reflecting on this point
from time to time.
In particular, it frequently happens that a change in technology renders some
idea obsolete and it quickly vanishes. However, another change in technology
could revive it again. This is especially true when the change has to do with the
relative performance of different parts of the system. For instance, when CPUs
became much faster than memories, caches became important to speed up the
‘‘slow’’ memory. If new memory technology someday makes memories much
faster than CPUs, caches will vanish. And if a new CPU technology makes them
faster than memories again, caches will reappear. In biology, extinction is forever,
but in computer science, it is sometimes only for a few years.
As a consequence of this impermanence, in this book we will from time to
time look at ‘‘obsolete’’ concepts, that is, ideas that are not optimal with current
technology. However, changes in the technology may bring back some of the
so-called ‘‘obsolete concepts.’’ For this reason, it is important to understand why a
concept is obsolete and what changes in the environment might bring it back again.
To make this point clearer, let us consider a simple example. Early computers
had hardwired instruction sets. The instructions were executed directly by hardware and could not be changed. Then came microprogramming (first introduced on
a large scale with the IBM 360), in which an underlying interpreter carried out the
‘‘hardware instructions’’ in software. Hardwired execution became obsolete. It
was not flexible enough. Then RISC computers were invented, and microprogramming (i.e., interpreted execution) became obsolete because direct execution was
faster. Now we are seeing the resurgence of microprogramming because it allows
CPUs to be updated in the field (for instance in response to dangerous CPU vulnerabilities such as Spectre, Meltdown, and RIDL). Thus the pendulum has already
swung several cycles between direct execution and interpretation and may yet
swing again in the future.
Large Memories
Let us now examine some historical developments in hardware and how they
have affected software repeatedly. The first mainframes had limited memory. A
fully loaded IBM 7090 or 7094, which played king of the mountain from late 1959
until 1964, had just over 128 KB of memory. It was mostly programmed in assembly language and its operating system was written in assembly language to save
precious memory.
As time went on, compilers for languages like FORTRAN and COBOL got
good enough that assembly language was pronounced dead. But when the first
commercial minicomputer (the PDP-1) was released, it had only 4096 18-bit words
SEC. 1.5
OPERATING SYSTEM CONCEPTS
49
of memory, and assembly language made a surprise comeback. Eventually, minicomputers acquired more memory and high-level languages became prevalent on
them.
When microcomputers hit in the early 1980s, the first ones had 4-KB memories and assembly-language programming rose from the dead. Embedded computers often used the same CPU chips as the microcomputers (8080s, Z80s, and
later 8086s) and were also programmed in assembler initially. Now their descendants, the personal computers, have lots of memory and are programmed in C,
C++, Python, Java, and other high-level languages. Smart cards are undergoing a
similar development, although beyond a certain size, the smart cards often have a
Java interpreter and execute Java programs interpretively, rather than having Java
being compiled to the smart card’s machine language.
Protection Hardware
Early mainframes, like the IBM 7090/7094, had no protection hardware, so
they just ran one program at a time. A buggy program could wipe out the operating system and easily crash the machine. With the introduction of the IBM 360, a
primitive form of hardware protection became available. These machines could
then hold several programs in memory at the same time and have them take turns
running (multiprogramming). Monoprogramming was declared obsolete.
At least until the first minicomputer showed up—without protection hardware—so multiprogramming was not possible. Although the PDP-1 and PDP-8
had no protection hardware, eventually the PDP-11 did, and this feature led to multiprogramming and eventually to UNIX.
When the first microcomputers were built, they used the Intel 8080 CPU chip,
which had no hardware protection, so we were back to monoprogramming—one
program in memory at a time. It was not until the Intel 80286 chip that protection
hardware was added and multiprogramming became possible. Until this day, many
embedded systems have no protection hardware and run just a single program.
That works because the system designers have total control over all the software.
Disks
Early mainframes were largely magnetic-tape based. They would read in a program from tape, compile it, run it, and write the results back to another tape. There
were no disks and no concept of a file system. That began to change when IBM
introduced the first hard disk—the RAMAC (RAndoM ACcess) in 1956. It occupied about 4 square meters of floor space and could store 5 million 7-bit characters, enough for a single medium-resolution digital photo. But with an annual
rental fee of about $35,000, assembling enough of them to store the equivalent of a
roll of film got pricey quite fast. But eventually prices came down and primitive
file systems were developed for the successors of these unwieldy devices.
50
INTRODUCTION
CHAP. 1
Typical of these new developments was the CDC 6600, introduced in 1964 and
for years by far the fastest computer in the world. Users could create so-called
‘‘permanent files’’ by giving them names and hoping that no other user had also
decided that, say, ‘‘data’’ was a suitable name for a file. This was a single-level directory. Eventually, mainframes developed complex hierarchical file systems, perhaps culminating in the MULTICS file system.
As minicomputers came into use, they eventually also had hard disks. The
standard disk on the PDP-11 when it was introduced in 1970 was the RK05 disk,
with a capacity of 2.5 MB, about half of the IBM RAMAC, but it was only about
40 cm in diameter and 5 cm high. But it, too, had a single-level directory initially.
When microcomputers came out, CP/M was initially the dominant operating system, and it, too, supported just one directory on the (floppy) disk. Later minicomputers and microcomputers got hierarchical file systems also.
Virtual Memory
Virtual memory (discussed in Chap. 3) gives the ability to run programs larger
than the machine’s physical memory by rapidly moving pieces back and forth
between RAM and stable storage (SSD or disk). It underwent a similar development, first appearing on mainframes, then moving to the minis and the micros. Virtual memory also allowed having a program dynamically link in a library at run
time instead of having it compiled in. MULTICS was again the first system to allow this. Eventually, the idea propagated down the line and is now widely used on
most UNIX and Windows systems.
In all these developments, we see ideas invented in one context and later
thrown out when the context changes (assembly-language programming, monoprogramming, single-level directories, etc.) only to reappear in a different context
often a decade later. For this reason, in this book we will sometimes look at ideas
and algorithms that may seem dated on today’s high-end PCs, but which may soon
come back on embedded computers, smart watches, or smart cards.
1.6 SYSTEM CALLS
We have seen that operating systems have two main functions: providing
abstractions to user programs and managing the computer’s resources. For the most
part, the interaction between user programs and the operating system deals with the
former; for example, creating, writing, reading, and deleting files. The resource-management part is largely transparent to the users and done automatically. Thus,
the interface between user programs and the operating system is primarily about
dealing with the abstractions. To really understand what operating systems do, we
must examine this interface closely. The system calls available in the interface vary
from one operating system to another, but the underlying concepts are similar.
SEC. 1.6
SYSTEM CALLS
51
We are thus forced to make a choice between (1) vague generalities (‘‘operating systems have system calls for reading files’’) and (2) some specific system
(‘‘UNIX has a read system call with three parameters: one to specify the file, one
to tell where the data are to be put, and one to tell how many bytes to read’’).
We have chosen the latter approach. It’s more work that way, but it gives more
insight into what operating systems really do. Although this discussion specifically
refers to POSIX (International Standard 9945-1), hence also to UNIX, System V,
BSD, Linux, MINIX 3, and so on, most other modern operating systems have system calls that perform the same functions, even if the details differ. Since the actual
mechanics of issuing a system call are highly machine dependent and often must
be expressed in assembly code, a procedure library is provided to make it possible
to make system calls from C programs and often from other languages as well.
It is useful to keep the following in mind. Any single-CPU computer can
execute only one instruction at a time. If a process is running a user program in
user mode and needs a system service, such as reading data from a file, it has to
execute a trap instruction to transfer control to the operating system. The operating
system then figures out what the calling process wants by inspecting the parameters. Then it carries out the system call and returns control to the instruction following the system call. In a sense, making a system call is like making a special
kind of procedure call—only system calls enter the kernel and procedure calls do
not.
To make the system-call mechanism clearer, let us take a quick look at the read
system call. As mentioned above, it has three parameters: the first one specifying
the file, the second one pointing to the buffer, and the third one giving the number
of bytes to read. Like nearly all system calls, it is invoked from C programs by calling a library procedure with the same name as the system call: read. A call from a
C program might look like this:
count = read(fd, buffer, nbytes);
The system call (and the library procedure) returns the number of bytes actually
read in count. This value is normally the same as nbytes, but may be smaller, if,
for example, end-of-file is encountered while reading.
If the system call cannot be carried out owing to an invalid parameter or a disk
error, count is set to <1, and the error number is put in a global variable, errno.
Programs should always check the results of a system call to see if an error
occurred.
System calls are performed in a series of steps. To make this concept clearer,
let us examine the read call discussed above. In preparation for calling the read
library procedure, which actually makes the read system call, the calling program
first prepares the parameters, for instance by storing them in a set of registers that
by convention are used for parameters. For instance, on x86-64 CPUs, Linux,
FreeBSD, Solaris, and macOS use the System V AMD64 ABI calling convention,
which means that the first six parameters are passed in registers RDI, RSI, RDX,
52
INTRODUCTION
CHAP. 1
RCX, R8, and R9. If there are more than six arguments, the remainder will be
pushed onto the stack. As we have only three arguments for read library procedure, this is shown as steps 1–3 in Fig. 1-17.
Address
0xFFFFFFFF
Return to caller
Trap to the kernel
register
reg.
5 Put code for read in RAX
10
4
User space
Library
procedure
read
(Next
instruction)
11
Increment
SP
Call read
nbytes
in RDX register
Push
fd
3 Put
Push
&buffer
buffer
in RSI register
2 Put
Push
1 Put
fdnbytes
in RDI register
9
6
Kernel space
(Operating system)
Dispatch
User program
calling read
7
8
Sys call
handler
0
Figure 1-17. The 10 steps in making the system call read(fd, buffer, nbytes).
The first and third parameters are passed by value, but the second parameter is
a reference, meaning that the address of the buffer is passed, not the contents of the
buffer. Then comes the actual call to the library procedure (step 4). This instruction is the normal procedure-call instruction used to call all procedures.
The library procedure, written in assembly language, typically puts the system-call number in a place where the operating system expects it, such as the RAX
register (step 5). Then it executes a trap instruction (such the X86-64 SYSCALL
instruction) to switch from user mode to kernel mode and start execution at a fixed
address within the kernel (step 6). The trap instruction is actually fairly similar to
the procedure-call instruction in the sense that the instruction following it is taken
from a distant location and the return address is saved on the stack for use later.
Nevertheless, the trap instruction also differs from the procedure-call instruction in two fundamental ways. First, as a side effect, it switches into kernel mode.
The procedure call instruction does not change the mode. Second, rather than giving a relative or absolute address where the procedure is located, the trap instruction cannot jump to an arbitrary address. Depending on the architecture, either it
SEC. 1.6
SYSTEM CALLS
53
jumps to a single fixed location (this is the case for the x86-4 SYSCALL instruction)
or there is an 8-bit field in the instruction giving the index into a table in memory
containing jump addresses, or equivalent.
The kernel code that starts following the trap examines the system-call number
in the RAX register and then dispatches to the correct system-call handler, usually
via a table of pointers to system-call handlers indexed on system-call number (step
7). At that point, the system-call handler runs (step 8). Once it has completed its
work, control may be returned to the user-space library procedure at the instruction
following the trap instruction (step 9). This procedure then returns to the user program in the usual way procedure calls return (step 10), which then continues with
the next instruction in the program (step 11).
In step 9 above, we said ‘‘may be returned to the user-space library procedure’’
for good reason. The system call may block the caller, preventing it from continuing. For example, if it is trying to read from the keyboard and nothing has been
typed yet, the caller has to be blocked. In this case, the operating system will look
around to see if some other process can be run next. Later, when the desired input
is available, this process will get the attention of the system and run steps 9 and 10.
In the following sections, we will examine some of the most heavily used
POSIX system calls, or more specifically, the library procedures that make those
system calls. POSIX has about 100 procedure calls. Some of the most important
ones are listed in Fig. 1-18, grouped for convenience in four categories. In the text,
we will briefly examine each call to see what it does.
To a large extent, the services offered by these calls determine most of what
the operating system has to do, since the resource management on personal computers is minimal (at least compared to big machines with multiple users). The
services include things like creating and terminating processes, creating, deleting,
reading, and writing files, managing directories, and performing input and output.
As an aside, it is worth pointing out that the mapping of POSIX procedure
calls onto system calls is not one-to-one. The POSIX standard specifies a number
of procedures that a conformant system must supply, but it does not specify whether they are system calls, library calls, or something else. If a procedure can be carried out without invoking a system call (i.e., without trapping to the kernel), it will
usually be done in user space for reasons of performance. However, most of the
POSIX procedures do invoke system calls, usually with one procedure mapping
directly onto one system call. In a few cases, especially where several required
procedures are only minor variations of one another, one system call handles more
than one library call.
1.6.1 System Calls for Process Management
The first group of calls in Fig. 1-18 deals with process management. Fork is a
good place to start the discussion. Fork is the only way to create a new process in
POSIX. It creates an exact duplicate of the original process, including all the file
54
INTRODUCTION
CHAP. 1
Process management
Call
Description
pid = fork( )
Create a child process identical to the parent
pid = waitpid(pid, &statloc, options)
Wait for a child to terminate
s = execve(name, argv, environp)
Replace a process’ core image
exit(status)
Terminate process execution and return status
File management
Call
Description
fd = open(file, how, ...)
Open a file for reading, writing, or both
s = close(fd)
Close an open file
n = read(fd, buffer, nbytes)
Read data from a file into a buffer
n = write(fd, buffer, nbytes)
Write data from a buffer into a file
position = lseek(fd, offset, whence)
Move the file pointer
s = stat(name, &buf)
Get a file’s status information
Directory- and file-system management
Call
Description
s = mkdir(name, mode)
Create a new directory
s = rmdir(name)
Remove an empty directory
s = link(name1, name2)
Create a new entry, name2, pointing to name1
s = unlink(name)
Remove a directory entry
s = mount(special, name, flag)
Mount a file system
s = umount(special)
Unmount a file system
Miscellaneous
Call
Description
s = chdir(dirname)
Change the working directory
s = chmod(name, mode)
Change a file’s protection bits
s = kill(pid, signal)
Send a signal to a process
seconds = time(&seconds)
Get the elapsed time since Jan. 1, 1970
Figure 1-18. Some of the major POSIX system calls. The return code s is <1 if
an error has occurred. The return codes are as follows: pid is a process id, fd is a
file descriptor, n is a byte count, position is an offset within the file, and seconds
is the elapsed time. The parameters are explained in the text.
descriptors, registers—everything. After the fork, the original process and the copy
(the parent and child) go their separate ways. All the variables have identical values at the time of the fork, but since the parent’s data are copied to create the child,
subsequent changes in one of them do not affect the other one. In fact, the memory
of the child may be shared copy-on-write with the parent. This means that parent
SEC. 1.6
SYSTEM CALLS
55
and child share a single physical copy of the memory until one of the two modifies
a value at a location in memory—in which case the operating system makes a copy
of the small chunk of memory containing that location. Doing so minimizes the
amount of memory that needs to be copied a priori, as much can remain shared.
Moreover, part of the memory, for instance, the program text does not change at, so
it can always be shared between parent and child. The fork call returns a value,
which is zero in the child and equal to the child’s PID (Process IDentifier) in the
parent. Using the returned PID, the two processes can see which one is the parent
process and which one is the child process.
In most cases, after a fork, the child will need to execute different code from
the parent. Consider the case of the shell. It reads a command from the terminal,
forks off a child process, waits for the child to execute the command, and then
reads the next command when the child terminates. To wait for the child to finish,
the parent executes a waitpid system call, which just waits until the child terminates
(any child if more than one exists). Waitpid can wait for a specific child, or for any
old child by setting the first parameter to <1. When waitpid completes, the address
pointed to by the second parameter, statloc, will be set to the child process’ exit
status (normal or abnormal termination and exit value). Various options are also
provided, specified by the third parameter. For example, returning immediately if
no child has already exited.
Now consider how fork is used by the shell. When a command is typed, the
shell forks off a new process. This child process must execute the user command.
It does this by using the execve system call, which causes its entire core image to
be replaced by the file named in its first parameter. A highly simplified shell illustrating the use of fork, waitpid, and execve is shown in Fig. 1-19.
#define TRUE 1
while (TRUE) {
type prompt( );
read command(command, parameters);
if (fork( ) != 0) {
/* Parent code. */
waitpid(<1, &status, 0);
} else {
/* Child code. */
execve(command, parameters, 0);
}
/* repeat forever */
/* display prompt on the screen */
/* read input from terminal */
/* fork off child process */
/* wait for child to exit */
/* execute command */
}
Figure 1-19. A stripped-down shell. Throughout this book, TRUE is assumed to
be defined as 1.
In the most general case, execve has three parameters: the name of the file to
be executed, a pointer to the argument array, and a pointer to the environment
56
INTRODUCTION
CHAP. 1
array. These will be described shortly. Various library routines, including execl,
execv, execle, and execve, are provided to allow the parameters to be omitted or
specified in various ways. Throughout this book we will use the name exec to
represent the system call invoked by all of these.
Let us consider the case of a command such as
cp file1 file2
used to copy file1 to file2. After the shell has forked, the child process locates and
executes the file cp and passes to it the names of the source and target files.
The main program of cp (and main program of most other C programs) contains the declaration
main(argc, argv, envp)
where argc is a count of the number of items on the command line, including the
program name. For the example above, argc is 3.
The second parameter, argv, is a pointer to an array. Element i of that array is a
pointer to the ith string on the command line. In our example, argv[0] would point
to the string ‘‘cp’’, argv[1] would point to the string ‘‘file1’’, and argv[2] would
point to the string ‘‘file2’’.
The third parameter of main, envp, is a pointer to the environment, an array of
strings containing assignments of the form name = value used to pass information
such as the terminal type and home directory name to programs. There are library
procedures that programs can call to get the environment variables, which are often
used to customize how a user wants to perform certain tasks (e.g., the default printer to use). In Fig. 1-19, no environment is passed to the child, so the third parameter of execve is a zero.
If exec seems complicated, do not despair; it is (semantically) the most complex of all the POSIX system calls. All the other ones are much simpler. As an
example of a simple one, consider exit, which processes should use when they are
finished executing. It has one parameter, the exit status (0 to 255), which is
returned to the parent via statloc in the waitpid system call.
Processes in UNIX have their memory divided up into three segments: the text
segment (i.e., the program code), the data segment (i.e., the variables), and the
stack segment. The data segment grows upward and the stack grows downward,
as shown in Fig. 1-20. Between them is a gap of unused address space. The stack
grows into the gap automatically, as needed, but expansion of the data segment is
done explicitly by using a system call, brk, which specifies the new address where
the data segment is to end. This call, however, is not defined by the POSIX standard, since programmers are encouraged to use the malloc library procedure for
dynamically allocating storage, and the underlying implementation of malloc was
not thought to be a suitable subject for standardization since few programmers use
it directly and it is doubtful that anyone even notices that brk is not in POSIX. (In
SEC. 1.6
SYSTEM CALLS
57
most systems, there are other memory areas also, for instance those create with the
mmap system call, which creates a new virtual memory areas, but we will get to
those later.)
Stack
Address (hex)
FFFF
Gap
Data
Text
0000
Figure 1-20. Processes have three segments: text, data, and stack.
1.6.2 System Calls for File Management
Many system calls relate to the file system. In this section, we will look at
calls that operate on individual files; in the next one we will examine those that
involve directories or the file system as a whole.
To read or write a file, it must first be opened. This call specifies the file name
to be opened, either as an absolute path name or relative to the working directory,
as well as a code of O RDONLY, O WRONLY, or O RDWR, meaning open for
reading, writing, or both. To create a new file, the O CREAT parameter is used.
The file descriptor returned can then be used for reading or writing. Afterward, the
file can be closed by close, which makes the file descriptor available for reuse on a
subsequent open.
The most heavily used calls are undoubtedly read and write. We saw read earlier. Write has the same parameters.
Although most programs read and write files sequentially, for some applications programs need to be able to access any part of a file at random. Associated
with each file is a pointer that indicates the current position in the file. When reading (writing) sequentially, it normally points to the next byte to be read (written).
The lseek call changes the value of the position pointer, so that subsequent calls to
read or write can begin anywhere in the file.
Lseek has three parameters: the first is the file descriptor for the file, the second is a file position, and the third tells whether the file position is relative to the
beginning of the file, the current position, or the end of the file. The value returned
by lseek is the absolute position in the file (in bytes) after changing the pointer.
For each file, UNIX keeps track of the file mode (regular file, special file,
directory, and so on), size, time of last modification, and other information. Programs can ask to see this information via the stat system call. The first parameter
58
INTRODUCTION
CHAP. 1
specifies the file to be inspected; the second one is a pointer to a structure where
the information is to be put. The fstat calls does the same thing for an open file.
1.6.3 System Calls for Directory Management
In this section, we will look at some system calls that relate more to directories
or the file system as a whole, rather than just to one specific file as in the previous
section. The first two calls, mkdir and rmdir, create and remove empty directories,
respectively. The next call is link. Its purpose is to allow the same file to appear
under two or more names, often in different directories. A typical use is to allow
several members of the same programming team to share a common file, with each
of them having the file appear in his own directory, possibly under different names.
Sharing a file is not the same as giving every team member a private copy; having
a shared file means that changes that any member of the team makes are instantly
visible to the other members—there is only one file. When copies are made of a
file, subsequent changes made to one copy do not affect the others.
To see how link works, consider the situation of Fig. 1-21(a). Here are two
users, ast and jim, each having his own directory with some files. If ast now
executes a program containing the system call
link("/usr/jim/memo", "/usr/ast/note");
the file memo in jim’s directory is now entered into ast’s directory under the name
note. Thereafter, /usr/jim/memo and /usr/ast/note refer to the same file. As an
aside, whether user directories are kept in /usr, /user, /home, or somewhere else is
simply a decision made by the local system administrator.
/usr/ast
/usr/jim
16 mail
81 games
40 test
31
70
59
38
bin
memo
f.c.
prog1
/usr/ast
16
81
40
70
/usr/jim
mail
games
test
note
(a)
31
70
59
38
bin
memo
f.c.
prog1
(b)
Figure 1-21. (a) Two directories before linking /usr/jim/memo to ast’s directory.
(b) The same directories after linking.
Understanding how link works will probably make clearer what it does. Every
file in UNIX has a unique number, its i-number, that identifies it. This i-number is
an index into a table of i-nodes, one per file, telling who owns the file, where its
disk blocks are, and so on†. A directory is simply a file containing a set of (i-number, ASCII name) pairs. In the first versions of UNIX, each directory entry was 16
†Most people still call them disk blocks, even if they reside on SSD.
SEC. 1.6
59
SYSTEM CALLS
bytes—2 bytes for the i-number and 14 bytes for the name. Now a more complicated structure is needed to support long file names, but conceptually a directory is
still a set of (i-number, ASCII name) pairs. In Fig. 1-21, mail has i-number 16, and
so on. What link does is simply create a brand new directory entry with a (possibly
new) name, using the i-number of an existing file. In Fig. 1-21(b), two entries have
the same i-number (70) and thus refer to the same file. If either one is later
removed, using the unlink system call, the other one remains. If both are removed,
UNIX sees that no entries to the file exist (a field in the i-node keeps track of the
number of directory entries pointing to the file), so the file is removed from the
SSD or disk and its blocks are returned to the free block pool.
As we have mentioned earlier, the mount system call allows two file systems to
be merged into one. A common situation is to have the root file system, containing
the binary (executable) versions of the common commands and other heavily used
files, on an SSD / hard disk (sub)partition and user files on another (sub)partition.
Further, the user can then insert a USB disk with files to be read.
By executing the mount system call, the USB file system can be attached to the
root file system, as shown in Fig. 1-22. A typical statement in C to mount is
mount("/dev/sdb0", "/mnt", 0);
where the first parameter is the name of a block special file for USB drive 0, the
second parameter is the place in the tree where it is to be mounted, and the third
parameter tells whether the file system is to be mounted read-write or read-only.
bin
dev
lib
(a)
mnt
usr
bin
dev
lib
usr
(b)
Figure 1-22. (a) File system before the mount. (b) File system after the mount.
After the mount call, a file on drive 0 can be accessed by just using its path
from the root directory or the working directory, without regard to which drive it is
on. In fact, second, third, and fourth drives can also be mounted anywhere in the
tree. The mount call makes it possible to integrate removable media into a single
integrated file hierarchy, without having to worry about which device a file is on.
Although this example involves USB drives, portions of hard disks (often called
partitions or minor devices) can also be mounted this way, as well as external
hard disks and SSDs. When a file system is no longer needed, it can be unmounted
with the umount system call. After that, it is no longer accessible. Of course, if it
is needed later on, it can be mounted again.
60
INTRODUCTION
CHAP. 1
1.6.4 Miscellaneous System Calls
A variety of other system calls exist as well. We will look at just four of them
here. The chdir call changes the current working directory. After the call
chdir("/usr/ast/test");
an open on the file xyz will open /usr/ast/test/xyz. The concept of a working directory eliminates the need for typing (long) absolute path names all the time.
In UNIX every file has a mode used for protection. The mode includes the
read-write-execute bits for the owner, group, and others. The chmod system call
makes it possible to change the mode of a file. For example, to make a file readonly by everyone except the owner, one could execute
chmod("file", 0644);
The kill system call is the way users and user processes send signals. If a process is prepared to catch a particular signal, then when it arrives, a signal handler is
run. If the process is not prepared to handle a signal, then its arrival kills the process (hence the name of the call).
POSIX defines a number of procedures for dealing with time. For example,
time just returns the current time in seconds, with 0 corresponding to Jan. 1, 1970
at midnight (just as the day was starting, not ending). On computers using 32-bit
words, the maximum value time can return is 232 < 1 seconds (assuming an unsigned integer is used). This value corresponds to a little over 136 years. Thus in
the year 2106, 32-bit UNIX systems will go berserk, not unlike the famous Y2K
problem that would have wreaked havoc with the world’s computers in 2000, were
it not for the massive effort the IT industry put into fixing the problem. If you currently have a 32-bit UNIX system, you are advised to trade it in for a 64-bit one
sometime before the year 2106.
1.6.5 The Windows API
So far we have focused primarily on UNIX. Now it is time to look briefly at
Windows. Windows and UNIX differ in a fundamental way in their respective programming models. A UNIX program consists of code that does something or
other, making system calls to have certain services performed. In contrast, a Windows program is normally event driven. The main program waits for some event to
happen, then calls a procedure to handle it. Typical events are keys being struck,
the mouse being moved, a mouse button being pushed, or a USB drive inserted or
removed from the computer. Handlers are then called to process the event, update
the screen, and update the internal program state. All in all, this leads to a somewhat different style of programming than with UNIX, but since the focus of this
book is on operating system function and structure, these different programming
models will not concern us much more.
SEC. 1.6
SYSTEM CALLS
61
Of course, Windows also has system calls. With UNIX, there is almost a oneto-one relationship between the system calls (e.g., read) and the library procedures
(e.g., read) used to invoke the system calls. In other words, for each system call,
there is roughly one library procedure that is called to invoke it, as indicated in
Fig. 1-17. Furthermore, POSIX has only on the order of 100 procedure calls.
With Windows, the situation is radically different. To start with, the library
calls and the actual system calls are highly decoupled. Microsoft has defined a set
of procedures called the WinAPI, Win32 API, or Win64 API (Application Programming Interface) that programmers are expected to use to get operating system services. This interface is (partially) supported on all versions of Windows
since Windows 95. By decoupling the API interface that programmer’s use from
the actual system calls, Microsoft retains the ability to change the actual system
calls in time (even from release to release) without invalidating existing programs.
What actually constitutes Win32 is also slightly ambiguous because recent versions of Windows have many new calls that were not previously available. In this
section, Win32 means the interface supported by all versions of Windows. Win32
provides compatibility among versions of Windows. Win64 is largely Win32 with
bigger pointers so we will focus on Win32 here.
The number of Win32 API calls is extremely large, numbering in the thousands. Furthermore, while many of them do invoke system calls, a substantial number are carried out entirely in user space. As a consequence, with Windows it is
impossible to see what is a system call (i.e., performed by the kernel) and what is
simply a user-space library call. In fact, what is a system call in one version of
Windows may be done in user space in a different version, and vice versa. When
we discuss the Windows system calls in this book, we will use the Win32 procedures (where appropriate) since Microsoft guarantees that these will be stable over
time. But it is worth remembering that not all of them are true system calls (i.e.,
traps to the kernel).
The Win32 API has a huge number of calls for managing windows, geometric
figures, text, fonts, scrollbars, dialog boxes, menus, and other features of the GUI.
To the extent that the graphics subsystem runs in the kernel (true on some versions
of Windows but not on all), these are system calls; otherwise they are just library
calls. Should we discuss these calls in this book or not? Since they are not really
related to the function of an operating system, we have decided not to, even though
they may be carried out by the kernel. Readers interested in the Win32 API should
consult one of the many books on the subject (e.g., Yosifovich, 2020).
Even introducing all the Win32 API calls here is out of the question, so we will
restrict ourselves to those calls that roughly correspond to the functionality of the
UNIX calls listed in Fig. 1-18. These are listed in Fig. 1-23.
Let us now briefly go through the list of Fig. 1-23. CreateProcess creates a
new process. It does the combined work of fork and execve in UNIX. It has many
parameters specifying the properties of the newly created process. Windows does
not have a process hierarchy like UNIX does, so there is no concept of a parent
62
INTRODUCTION
UNIX
Win32
CHAP. 1
Description
fork
CreateProcess
waitpid
WaitForSingleObject Can wait for a process to exit
Create a new process
execve
(none)
CreateProcess = fork + execve
exit
ExitProcess
Terminate execution
open
CreateFile
Create a file or open an existing file
close
CloseHandle
Close a file
read
ReadFile
Read data from a file
write
WriteFile
Write data to a file
lseek
SetFilePointer
Move the file pointer
stat
GetFileAttributesEx
Get various file attributes
mkdir
CreateDirectory
Create a new directory
rmdir
RemoveDirector y
Remove an empty directory
link
(none)
Win32 does not support links
unlink
DeleteFile
Destroy an existing file
mount
(none)
Win32 does not support mount
umount
(none)
Win32 does not support mount, so no umount
chdir
SetCurrentDirectory
Change the current working directory
chmod
(none)
Win32 does not support security (although NT does)
kill
(none)
Win32 does not support signals
time
GetLocalTime
Get the current time
Figure 1-23. The Win32 API calls that roughly correspond to the UNIX calls of
Fig. 1-18. It is worth emphasizing that Windows has a very large number of other system calls, most of which do not correspond to anything in UNIX.
process and a child process. After a process is created, the creator and createe are
equals. WaitForSingleObject is used to wait for an event. Many possible events can
be waited for. If the parameter specifies a process, then the caller waits for the
specified process to exit, which is done using ExitProcess.
The next six calls operate on files and are functionally similar to their UNIX
counterparts although they differ in the parameters and details. Still, files can be
opened, closed, read, and written pretty much as in UNIX. The SetFilePointer and
GetFileAttributesEx calls set the file position and get some of the file attributes.
Windows has directories and they are created with CreateDirectory and
RemoveDirectory API calls, respectively. There is also a notion of a current directory, set by SetCurrentDirectory. The current time of day is acquired using GetLocalTime.
The Win32 interface does not have links to files, mounted file systems, security, or signals, so the calls corresponding to the UNIX ones do not exist. Of course,
Win32 has a huge number of other calls that UNIX does not have, especially for
SEC. 1.6
SYSTEM CALLS
63
managing the GUI. For instance, Windows 11 has an elaborate security system
and also supports file links.
One last note about Win32 is perhaps worth making. Win32 is not a terribly
uniform or consistent interface. The main culprit here was the need to be backward compatible with the previous 16-bit interface used in Windows 3.x.
1.7 OPERATING SYSTEM STRUCTURE
Now that we have seen what operating systems look like on the outside (i.e.,
the programmer’s interface), it is time to take a look inside. In the following sections, we will examine six different structures that have been tried, in order to get
some idea of the spectrum of possibilities. These are by no means exhaustive, but
they give an idea of some designs that have been tried in practice. The six designs
we will discuss here are monolithic systems, layered systems, microkernels, client-server systems, virtual machines, and exo- and unikernels.
1.7.1 Monolithic Systems
By far the most common organization, the monolithic approach is to run the
entire operating system as a single program in kernel mode. The operating system
is written as a collection of procedures, linked together into a single large
executable binary program. When this technique is used, each procedure in the system is free to call any other one, if the latter provides some useful computation that
the former needs. Being able to call any procedure you want is very efficient, but
having thousands of procedures that can call each other without restriction may
also lead to a system that is unwieldy and difficult to understand. Also, a crash in
any of these procedures will take down the entire operating system.
To construct the actual object program of the operating system when this approach is used, one first compiles all the individual procedures (or the files containing the procedures) and then binds them all together into a single executable
file using the system linker. In terms of information hiding, there is essentially
none—every procedure is visible to every other procedure (as opposed to a structure containing modules or packages, in which much of the information is hidden
away inside modules, and only the officially designated entry points can be called
from outside the module).
Even in monolithic systems, however, it is possible to have some structure. The
services (system calls) provided by the operating system are requested by putting
the parameters in a well-defined place (e.g., on the stack) and then executing a trap
instruction. This instruction switches the machine from user mode to kernel mode
and transfers control to the operating system, shown as step 6 in Fig. 1-17. The
operating system then fetches the parameters and determines which system call is
to be carried out. After that, it indexes into a table that contains in slot k a pointer
to the procedure that carries out system call k (step 7 in Fig. 1-17).
64
INTRODUCTION
CHAP. 1
This organization suggests a basic structure for the operating system:
1. A main program that invokes the requested service procedure.
2. A set of service procedures that carry out the system calls.
3. A set of utility procedures that help the service procedures.
In this model, for each system call there is one service procedure that takes care of
it and executes it. The utility procedures do things that are needed by several service procedures, such as fetching data from user programs. This division of the
procedures into three layers is shown in Fig. 1-24.
Main
procedure
Service
procedures
Utility
procedures
Figure 1-24. A simple structuring model for a monolithic system.
In addition to the core operating system that is loaded when the computer is
booted, many operating systems support loadable extensions, such as I/O device
drivers and file systems. These components are loaded on demand. In UNIX they
are called shared libraries. In Windows they are called DLLs (Dynamic-Link
Libraries). They have file extension .dll and the C:\Windows\system32 directory
on Windows systems has well over 1000 of them.
1.7.2 Layered Systems
A generalization of the approach of Fig. 1-24 is to organize the operating system as a hierarchy of layers, each one constructed upon the one below it. The first
system constructed in this way was the THE system built at the Technische
Hogeschool Eindhoven in the Netherlands by E. W. Dijkstra (1968) and his students. The THE system was a simple batch system for a Dutch computer, the Electrologica X8, which had 32K of 27-bit words (bits were expensive back then).
The system had six layers, as shown in Fig. 1-25. Layer 0 dealt with allocation
of the processor, switching between processes when interrupts occurred or timers
expired. Above layer 0, the system consisted of sequential processes, each of
SEC. 1.7
OPERATING SYSTEM STRUCTURE
65
which could be programmed without having to worry about the fact that multiple
processes were running on a single processor. In other words, layer 0 provided the
basic multiprogramming of the CPU.
Layer
Function
5
The operator
4
User programs
3
Input/output management
2
Operator-process communication
1
Memory and drum management
0
Processor allocation and multiprogramming
Figure 1-25. Structure of the THE operating system.
Layer 1 did the memory management. It allocated space for processes in main
memory and on a 512K word drum used for holding parts of processes (pages) for
which there was no room in main memory. Above layer 1, processes did not have
to worry about whether they were in memory or on the drum; the layer 1 software
took care of making sure pages were brought into memory at the moment they
were needed and removed when they were not needed.
Layer 2 handled communication between each process and the operator console (that is, the user). On top of this layer, each process effectively had its own
operator console. Layer 3 took care of managing the I/O devices and buffering the
information streams to and from them. Above layer 3 each process could deal with
abstract I/O devices with nice properties, instead of real devices with many peculiarities. Layer 4 was where the user programs were found. They did not have to
worry about process, memory, console, or I/O management. The system operator
process was located in layer 5.
A further generalization of the layering concept was present in the MULTICS
system. Instead of layers, MULTICS was described as having a series of concentric
rings, with the inner ones being more privileged than the outer ones (which is
effectively the same thing). When a procedure in an outer ring wanted to call a
procedure in an inner ring, it had to make the equivalent of a system call, that is, a
TRAP instruction whose parameters were carefully checked for validity before the
call was allowed to proceed. Although the entire operating system was part of the
address space of each user process in MULTICS, the hardware made it possible to
designate individual procedures (memory segments, actually) as protected against
reading, writing, or executing.
Whereas the THE layering scheme was really only a design aid, because all the
parts of the system were ultimately linked together into a single executable program, in MULTICS, the ring mechanism was very much present at run time and
enforced by the hardware. The advantage of the ring mechanism is that it can easily be extended to structure user subsystems. For example, a professor could write a
66
INTRODUCTION
CHAP. 1
program to test and grade student programs and run this program in ring n, with
the student programs running in ring n + 1 so that they could not change their
grades.
1.7.3 Microkernels
With the layered approach, the designers have a choice where to draw the kernel-user boundary. Traditionally, all the layers went in the kernel, but that is not
necessary. In fact, a strong case can be made for putting as little as possible in kernel mode because bugs in the kernel can bring down the system instantly. In contrast, user processes have less power so that a bug there may not be fatal.
Various researchers have repeatedly studied the number of bugs per 1000 lines
of code (e.g., Basilli and Perricone, 1984; and Ostrand and Weyuker, 2002). Bug
density depends on module size, module age, and more, but a ballpark figure for
serious industrial systems is between two and ten bugs per thousand lines of code.
This means that a monolithic operating system of five million lines of code is likely to contain between 10,000 and 50,000 kernel bugs. Not all of these are fatal, of
course, since some bugs may be things like a minor misspelling in an error message is rarely needed.
The basic idea behind the microkernel design is to achieve high reliability by
splitting the operating system up into small, well-defined modules, only one of
which—the microkernel—runs in kernel mode and the rest run as relatively powerless ordinary user processes. In particular, by running each device driver and file
system as a separate user process, a bug in one of these can crash that component,
but cannot crash the entire system. Thus, a bug in the audio driver will cause the
sound to be garbled or stop, but will not crash the computer. In contrast, in a
monolithic system with all the drivers in the kernel, a buggy audio driver can easily
reference an invalid memory address and bring the system to a grinding halt
instantly.
Many microkernels have been implemented and deployed for decades (Haertig
et al., 1997; Heiser et al., 2006; Herder et al., 2006; Hildebrand, 1992; Kirsch et
al., 2005; Liedtke, 1993, 1995, 1996; Pike et al., 1992; and Zuberi et al., 1999).
With the exception of macOS, which is based on the Mach microkernel (Accetta et
al., 1986), common desktop operating systems do not use microkernels. However,
they are dominant in real-time, industrial, avionics, and military applications that
are mission critical and have very high reliability requirements. A few of the better-known microkernels include Integrity, K42, L4, PikeOS, QNX, Symbian, and
MINIX 3. We now give a brief overview of MINIX 3, which has taken the idea of
modularity to the limit, breaking most of the operating system up into a number of
independent user-mode processes. MINIX 3 is a POSIX-conformant, open source
system freely available at www.minix3.org (Giuffrida et al., 2012; Giuffrida et al.,
2013; Herder et al., 2006; Herder et al., 2009; and Hruby et al., 2013). Intel
adopted MINIX 3 for its management engine in virtually all its CPUs.
SEC. 1.7
67
OPERATING SYSTEM STRUCTURE
The MINIX 3 microkernel is only about 15,000 lines of C and some 1400 lines
of assembler for very low-level functions such as catching interrupts and switching
processes. The C code manages and schedules processes, handles interprocess
communication (by passing messages between processes), and offers a set of about
40 kernel calls to allow the rest of the operating system to do its work. These calls
perform functions like hooking handlers to interrupts, moving data between
address spaces, and installing memory maps for new processes. The process structure of MINIX 3 is shown in Fig. 1-26, with the kernel call handlers labeled Sys.
The device driver for the clock is also in the kernel because the scheduler interacts
closely with it. The other device drivers run as separate user processes.
Process
Make
Shell
User
mode
FS
Disk
Proc.
TTY
Other
...
Reinc.
Netw
...
Print
Microkernel handles interrupts, processes,
scheduling, interprocess communication
User programs
Other
...
Clock
Servers
Other
Drivers
Sys
Figure 1-26. Simplified structure of the MINIX system.
Outside the kernel, the system is structured as three layers of processes all running in user mode. The lowest layer contains the device drivers. Since they run in
user mode, they do not have physical access to the I/O port space and cannot issue
I/O commands directly. Instead, to program an I/O device, the driver builds a structure telling which values to write to which I/O ports and makes a kernel call telling
the kernel to do the write. This approach means that the kernel can check to see
that the driver is writing (or reading) from I/O it is authorized to use. Consequently
(and unlike a monolithic design), a buggy audio driver cannot accidentally write on
the SSD or disk.
Above the drivers is another user-mode layer containing the servers, which do
most of the work of the operating system. One or more file servers manage the file
system(s), the process manager creates, destroys, and manages processes, and so
on. User programs obtain operating system services by sending short messages to
the servers asking for the POSIX system calls. For example, a process needing to
do a read sends a message to one of the file servers telling it what to read.
One interesting server is the reincarnation server, whose job is to check if the
other servers and drivers are functioning correctly. In the event that a faulty one is
68
INTRODUCTION
CHAP. 1
detected, it is automatically replaced without any user intervention. In this way,
the system is self-healing and can achieve high reliability.
The system has many restrictions limiting the power of each process. As mentioned, drivers can touch only authorized I/O ports, but access to kernel calls is also
controlled on a per-process basis, as is the ability to send messages to other processes. Processes can also grant limited permission for other processes to have the
kernel access their address spaces. As an example, a file system can grant permission for the disk driver to let the kernel put a newly read-in disk block at a specific
address within the file system’s address space. The sum total of all these restrictions is that each driver and server has exactly the power to do its work and nothing
more, thus greatly limiting the damage a buggy component can do. Restricting
what a component can do to exactly that what it needs to do its work is known as
the POLA (Principle of Least Authority) an important design principle for building secure systems. We will discuss other such principles in Chap. 9.
An idea somewhat related to having a minimal kernel is to put the mechanism
for doing something in the kernel but not the policy. To make this point better,
consider the scheduling of processes. A relatively simple scheduling algorithm is
to assign a numerical priority to every process and then have the kernel run the
highest-priority process that is runnable. The mechanism—in the kernel—is to
look for the highest-priority process and run it. The policy—assigning priorities to
processes—can be done by user-mode processes. In this way, policy and mechanism can be decoupled and the kernel can be made smaller.
1.7.4 Client-Server Model
A slight variation of the microkernel idea is to distinguish two classes of processes, the servers, each of which provides some service, and the clients, which use
these services. This model is known as the client-server model. The essence is the
presence of client processes and server processes.
Communication between clients and servers is often by message passing. To
obtain a service, a client process constructs a message saying what it wants and
sends it to the appropriate service. The service then does the work and sends back
the answer. If the client and server happen to run on the same machine, certain
optimizations are possible, but conceptually, we are still talking about message
passing here.
A generalization of this idea is to have the clients and servers run on different
computers, connected by a local or wide-area network, as depicted in Fig. 1-27.
Since clients communicate with servers by sending messages, the clients need not
know whether the messages are handled locally on their own machines, or whether
they are sent across a network to servers on a remote. machine. As far as the client is concerned, the same thing happens in both cases: requests are sent and
replies come back. Thus, the client-server model is an abstraction that can be used
for a single machine or for a network of machines.
SEC. 1.7
69
OPERATING SYSTEM STRUCTURE
Machine 1
Machine 2
Machine 3
Machine 4
Client
File server
Process server
Terminal server
Kernel
Kernel
Kernel
Kernel
Network
Message from
client to server
Figure 1-27. The client-server model over a network.
Increasingly many systems involve users at their home PCs as clients and large
machines elsewhere running as servers. In fact, much of the Web operates this
way. A PC sends a request for a Web page to the server and the Web page comes
back. This is a typical use of the client-server model in a network.
1.7.5 Virtual Machines
The initial releases of OS/360 were strictly batch systems. Nevertheless, many
360 users wanted to be able to work interactively at a terminal, so various groups,
both inside and outside IBM, decided to write timesharing systems for it. The
official IBM timesharing system, TSS/360, was delivered late, and when it finally
arrived it was so big and slow that few sites converted to it. It was eventually abandoned after its development had consumed some $50 million (Graham, 1970). But
a group at IBM’s Scientific Center in Cambridge, Massachusetts, produced a radically different system that IBM eventually accepted as a product. A linear descendant of it, called z/VM, is now widely used on IBM’s current mainframes, the
zSeries, which are heavily used in large corporate data centers, for example, as
e-commerce servers that handle hundreds or thousands of transactions per second
and use databases whose sizes run to millions of gigabytes.
VM/370
This system, originally called CP/CMS and later renamed VM/370 (Seawright
and MacKinnon, 1979), was based on an astute observation: a timesharing system
provides (1) multiprogramming and (2) an extended machine with a more convenient interface than the bare hardware. The essence of VM/370 is to completely
separate these two functions.
The heart of the system, called a virtual machine monitor, runs on the bare
hardware and does the multiprogramming, providing not one, but several virtual
machines to the next layer up, as shown in Fig. 1-28. However, unlike all other
operating systems, these virtual machines are not extended machines, with files
70
INTRODUCTION
CHAP. 1
and other nice features. Instead, they are exact copies of the bare hardware, including kernel/user mode, I/O, interrupts, and everything else the real machine has.
Virtual 370s
System calls here
I/O instructions here
CMS
Trap here
CMS
CMS
Trap here
VM/370
370 Bare hardware
Figure 1-28. The structure of VM/370 with CMS.
Because each virtual machine is identical to the true hardware, each one can
run any operating system that will run directly on the bare hardware. Different virtual machines can, and frequently do, run different operating systems. On the original IBM VM/370 system, some ran OS/360 or one of the other large batch or transaction-processing operating systems, while others ran a single-user, interactive
system called CMS (Conversational Monitor System) for interactive timesharing
users. The latter was popular with programmers.
When a CMS program executed a system call, the call was trapped to the operating system in its own virtual machine, not to VM/370, just as it would be were it
running on a real machine instead of a virtual one. CMS then issued the normal
hardware I/O instructions for reading its virtual disk or whatever was needed to
carry out the call. These I/O instructions were trapped by VM/370, which then performed them as part of its simulation of the real hardware. By completely separating the functions of multiprogramming and providing an extended machine, each
of the pieces could be much simpler, more flexible, and much easier to maintain.
In its modern incarnation, z/VM is usually used to run multiple complete operating systems rather than stripped-down single-user systems like CMS. For example, the zSeries is capable of running one or more Linux virtual machines along
with traditional IBM operating systems.
Virtual Machines Rediscovered
While IBM has had a virtual-machine product available for four decades, and a
few other companies, including Oracle and Hewlett-Packard, have recently added
virtual-machine support to their high-end enterprise servers, the idea of virtualization has largely been ignored in the PC world until recently. But in the past
decades, a combination of new needs, new software, and new technologies have
combined to make it a hot topic.
First the needs. Many companies have traditionally run their mail servers, Web
servers, FTP servers, and other servers on separate computers, sometimes with
SEC. 1.7
71
OPERATING SYSTEM STRUCTURE
different operating systems. They see virtualization as a way to run them all on the
same machine without having a crash of one server bring down the rest.
Virtualization is also popular in the Web hosting world. Without virtualization,
Web hosting customers are forced to choose between shared hosting (which just
gives them a login account on a Web server, but no control over the server software) and dedicated hosting (which gives them their own machine, which is very
flexible but not cost effective for small to medium Websites). When a Web hosting
company offers virtual machines for rent, a single physical machine can run many
virtual machines, each of which appears to be a complete machine. Customers who
rent a virtual machine can run whatever operating system and software they want
to, but at a fraction of the cost of a dedicated server (because the same physical
machine supports many virtual machines at the same time).
Another use of virtualization is for end users who want to be able to run two or
more operating systems at the same time, say Windows and Linux, because some
of their favorite application packages run on one and some run on the other. This
situation is illustrated in Fig. 1-29(a), where the term ‘‘virtual machine monitor’’
has been renamed type 1 hypervisor, which is commonly used nowadays because
‘‘virtual machine monitor’’ requires more keystrokes than people are prepared to
put up with now. Note that many authors use the terms interchangeably though.
Guest OS process
Excel Word Mplayer Apollon
Windows
Linux
Guest OS process
Host OS
process
...
Kernel
module
Guest OS
Guest OS
Machine simulator
Type 2 hypervisor
Type 1 hypervisor
Host operating system
Host operating system
(a)
(b)
(c)
Figure 1-29. (a) A type 1 hypervisor. (b) A pure type 2 hypervisor. (c) A practical type 2 hypervisor.
While no one disputes the attractiveness of virtual machines today, the problem
then was implementation. In order to run virtual machine software on a computer,
its CPU must be virtualizable (Popek and Goldberg, 1974). In a nutshell, here is
the problem. When an operating system running on a virtual machine (in user
mode) executes a privileged instruction, such as modifying the PSW or doing I/O,
it is essential that the hardware trap to the virtual-machine monitor so the instruction can be emulated in software. On some CPUs—notably the Pentium, its predecessors, and its clones—attempts to execute privileged instructions in user mode
are just ignored. This property made it impossible to have virtual machines on this
hardware, which explains the lack of interest in the x86 world. Of course, there
72
INTRODUCTION
CHAP. 1
were interpreters for the Pentium, such as Bochs, that ran on the Pentium, but with
a performance loss of one to two orders of magnitude, they were not useful for serious work.
This situation changed as a result of several academic research projects in the
1990s and early years of this millennium, notably Disco at Stanford (Bugnion et
al., 1997) and Xen at Cambridge University (Barham et al., 2003). These research
papers led to several commercial products (e.g., VMware Workstation and Xen)
and a revival of interest in virtual machines. Besides VMware and Xen, popular
hypervisors today include KVM (for the Linux kernel), VirtualBox (by Oracle),
and Hyper-V (by Microsoft).
Some of the early research projects improved the performance over interpreters
like Bochs by translating blocks of code on the fly, storing them in an internal
cache, and then reusing them if they were executed again. This improved the performance considerably, and led to what we will call machine simulators, as
shown in Fig. 1-29(b). However, although this technique, known as binary translation, helped improve matters, the resulting systems, while good enough to publish papers about in academic conferences, were still not fast enough to use in
commercial environments where performance matters a lot.
The next step in improving performance was to add a kernel module to do
some of the heavy lifting, as shown in Fig. 1-29(c). In practice now, all commercially available hypervisors, such as VMware Workstation, use this hybrid strategy
(and have many other improvements as well). They are called type 2 hypervisors
by everyone, so we will (somewhat grudgingly) go along and use this name in the
rest of this book, even though we would prefer to called them type 1.7 hypervisors
to reflect the fact that they are not entirely user-mode programs. In Chap. 7, we
will describe in detail how VMware Workstation works and what the various
pieces do.
In practice, the real distinction between a type 1 hypervisor and a type 2 hypervisor is that a type 2 makes uses of a host operating system and its file system to
create processes, store files, and so on. A type 1 hypervisor has no underlying support and must perform all these functions itself.
After a type 2 hypervisor is started, it reads the installation image file for the
chosen guest operating system and installs the guest OS on a virtual disk, which
is just a big file in the host operating system’s file system. Type 1 hypervisors cannot do this because there is no host operating system to store files on. They must
manage their own storage on a raw disk partition.
When the guest operating system is booted, it does the same thing it does on
the actual hardware, typically starting up some background processes and then a
GUI. To the user, the guest operating system behaves the same way it does when
running on the bare metal even though that is not the case here.
A different approach to handling control instructions is to modify the operating
system to remove them. This approach is not true virtualization. Instead it is
called paravirtualization. We will discuss virtualization in Chap. 7.
SEC. 1.7
OPERATING SYSTEM STRUCTURE
73
The Java Virtual Machine
Another area where virtual machines are used, but in a somewhat different
way, is for running Java programs. When Sun Microsystems invented the Java programming language, it also invented a virtual machine (i.e., a computer architecture) called the JVM (Java Virtual Machine). Sun no longer exists today
(because Oracle bought the company), but Java is still with us. The Java compiler
produces code for JVM, which then typically is executed by a software JVM interpreter. The advantage of this approach is that the JVM code can be shipped over
the Internet to any computer that has a JVM interpreter and run there. If the compiler had produced SPARC or x86 binary programs, for example, they could not
have been shipped and run anywhere as easily. (Of course, Sun could have produced a compiler that produced SPARC binaries and then distributed a SPARC
interpreter, but JVM is a much simpler architecture to interpret.) Another advantage of using JVM is that if the interpreter is implemented properly, which is not
completely trivial, incoming JVM programs can be checked for safety and then
executed in a protected environment so they cannot steal data or do any damage.
Containers
Besides full virtualization, we can also run multiple instances of an operating
system on a single machine at the same time by having the operating system itself
support different systems, or containers. Containers are provided by the host
operating system such as Windows or Linux and mostly run just the user mode
portion of an operating system. Each container shares the host operating system
kernel and typically the binaries and libraries in a read-only fashion. This way, a
Linux host can support many Linux containers. Since a container does not contain
a full operating system, it can be extremely lightweight.
Of course, there are downsides to containers also. First, it is not possible to run
a container with a completely different operating system from that of the host.
Also, unlike virtual machines, there is no strict resource partitioning. The container
may be restricted in what it may access on SSD or disk and how much CPU time it
gets, but all containers still share the resources in the underlying host operating
system. Phrased differently, containers are process-level isolated. This means that a
container that messes with the stability of the underlying kernel will also affect
other containers.
1.7.6 Exokernels and Unikernels
Rather than cloning the actual machine, as is done with virtual machines,
another strategy is partitioning it, in other words, giving each user a subset of the
resources. Thus one virtual machine might get disk blocks 0 to 1023, the next one
might get blocks 1024 to 2047, and so on.
74
INTRODUCTION
CHAP. 1
At the bottom layer, running in kernel mode, is a program called the exokernel
(Engler et al., 1995). Its job is to allocate resources to virtual machines and then
check attempts to use them to make sure no machine is trying to use somebody
else’s resources. Each user-level virtual machine can run its own operating system,
as on VM/370 and the Pentium virtual 8086s, except that each one is restricted to
using only the resources it has asked for and been allocated.
The advantage of the exokernel scheme is that it saves a layer of mapping. In
the other designs, each virtual machine thinks it has its own disk or SSD, with
blocks running from 0 to some maximum, so the virtual machine monitor must
maintain tables to remap disk block addresses (and all other resources). With the
exokernel, this remapping is not needed. The exokernel need only keep track of
which virtual machine has been assigned which resource. This method still has the
advantage of separating the multiprogramming (in the exokernel) from the user
operating system code (in user space), but with less overhead, since all the exokernel has to do is keep the virtual machines out of each other’s hair.
The operating system functions were linked with the applications in the virtual
machine in the form of a LibOS (Library Operating System), that needed only
the functionality for the application(s) running in the user-level virtual machine.
This idea, like so many others, was forgotten for a few decades, only to be
rediscovered in recent years, in the form of Unikernels, minimal LibOS-based systems that contain just enough functionality to support a single application (such as
a Web server) on a virtual machine. Unikernels have the potential to be highly
efficient as protection between the operating system (LibOS) and application is not
needed: since there is only one application on the virtual machine, all code can run
in kernel mode.
1.8 THE WORLD ACCORDING TO C
Operating systems are normally large C (or sometimes C++) programs consisting of many pieces written by many programmers. The environment used for
developing operating systems is very different from what individuals (such as students) are used to when writing small Java programs. This section is an attempt to
give a very brief introduction to the world of writing an operating system for smalltime Java or Python programmers.
1.8.1 The C Language
This is not a guide to C, but a short summary of some of the key differences
between C and languages like Python and especially Java. Java is based on C, so
there are many similarities between the two. Python is somewhat different, but still
fairly similar. For convenience, we focus on Java. Java, Python, and C are all
imperative languages with data types, variables, and control statements, for example. The primitive data types in C are integers (including short and long ones),
SEC. 1.8
THE WORLD ACCORDING TO C
75
characters, and floating-point numbers. Composite data types can be constructed
using arrays, structures, and unions. The control statements in C are similar to
those in Java, including if, switch, for, and while statements. Functions and parameters are roughly the same in both languages.
One feature C has that Java and Python do not is explicit pointers. A pointer
is a variable that points to (i.e., contains the address of) a variable or data structure.
Consider the statements
char c1, c2, *p;
c1 = ’c’;
p = &c1;
c2 = *p;
which declare c1 and c2 to be character variables and p to be a variable that points
to (i.e., contains the address of) a character. The first assignment stores the ASCII
code for the character ‘‘c’’ in the variable c1. The second one assigns the address
of c1 to the pointer variable p. The third one assigns the contents of the variable
pointed to by p to the variable c2, so after these statements are executed, c2 also
contains the ASCII code for ‘‘c’’. In theory, pointers are typed, so you are not supposed to assign the address of a floating-point number to a character pointer, but in
practice compilers accept such assignments, albeit sometimes with a warning.
Pointers are a very powerful construct, but also a great source of errors when used
carelessly.
Some things that C does not have include built-in strings, threads, packages,
classes, objects, type safety, and garbage collection. The last one is a show stopper
for operating systems. All storage in C is either static or explicitly allocated and
released by the programmer, usually with the library functions malloc and free. It
is the latter property—total programmer control over memory—along with explicit
pointers that makes C attractive for writing operating systems. Operating systems
are basically real-time systems to some extent, even general-purpose ones. When
an interrupt occurs, the operating system may have only a few microseconds to
perform some action or lose critical information. Having the garbage collector kick
in at an arbitrary moment is intolerable.
1.8.2 Header Files
An operating system project generally consists of some number of directories,
each containing many .c files containing the code for some part of the system,
along with some .h header files that contain declarations and definitions used by
one or more code files. Header files can also include simple macros, such as
#define BUFFER SIZE 4096
which allow the programmer to name constants, so that when BUFFER SIZE is
used in the code, it is replaced during compilation by the number 4096. Good C
76
INTRODUCTION
CHAP. 1
programming practice is to name every constant except 0, 1, and <1, and sometimes even them. Macros can have parameters, such as
#define max(a, b) (a > b ? a : b)
which allows the programmer to write
i = max(j, k+1)
and get
i = (j > k+1 ? j : k+1)
to store the larger of j and k+1 in i. Headers can also contain conditional compilation, for example
#ifdef X86
intel int ack();
#endif
which compiles into a call to the function intel int ack if the macro X86 is defined
and nothing otherwise. Conditional compilation is heavily used to isolate architecture-dependent code so that certain code is inserted only when the system is compiled on the X86, other code is inserted only when the system is compiled on a
SPARC, and so on. A .c file can bodily include zero or more header files using the
#include directive. There are also many header files that are common to nearly
every .c and are stored in a central directory.
1.8.3 Large Programming Projects
To build the operating system, each .c is compiled into an object file by the C
compiler. Object files, which have the suffix .o, contain binary instructions for the
target machine. They will later be directly executed by the CPU. There is nothing
like Java byte code or Python byte code in the C world.
The first pass of the C compiler is called the C preprocessor. As it reads each
.c file, every time it hits a #include directive, it goes and gets the header file named
in it and processes it, expanding macros, handling conditional compilation (and
certain other things) and passing the results to the next pass of the compiler as if
they were physically included.
Since operating systems are very large (five million lines of code is not
unusual), having to recompile the entire thing every time one file is changed would
be unbearable. On the other hand, changing a key header file that is included in
thousands of other files does require recompiling those files. Keeping track of
which object files depend on which header files is completely unmanageable without help.
Fortunately, computers are very good at precisely this sort of thing. On UNIX
systems, there is a program called make (with numerous variants such as gmake,
SEC. 1.8
77
THE WORLD ACCORDING TO C
pmake, etc.) that reads the Makefile, which tells it which files are dependent on
which other files. What make does is see which object files are needed to build the
operating system binary and for each one, check to see if any of the files it depends
on (the code and headers) have been modified subsequent to the last time the
object file was created. If so, that object file has to be recompiled. When make has
determined which .c files have to recompiled, it then invokes the C compiler to
recompile them, thus reducing the number of compilations to the bare minimum.
In large projects, creating the Makefile is error prone, so there are tools that do it
automatically.
Once all the .o files are ready, they are passed to a program called the linker to
combine all of them into a single executable binary file. Any library functions called are also included at this point, interfunction references are resolved, and
machine addresses are relocated as need be. When the linker is finished, the result
is an executable program, traditionally called a.out on UNIX systems. The various
components of this process are illustrated in Fig. 1-30 for a program with three C
files and two header files. Although we have been discussing operating system
development here, all of this applies to developing any large program.
defs.h
mac.h
main.c
help.c
other.c
C
preprocesor
C
compiler
main.o
libc.a
help.o
other.o
linker
Executable
binary program
a.out
Figure 1-30. The process of compiling C and header files to make an executable.
78
INTRODUCTION
CHAP. 1
1.8.4 The Model of Run Time
Once the operating system binary has been linked, the computer can be
rebooted and the new operating system started. Once running, it may dynamically
load pieces that were not statically included in the binary such as device drivers
and file systems. At run time, the operating system may consist of multiple segments, for the text (the program code), the data, and the stack. The text segment is
normally immutable, not changing during execution. The data segment starts out
at a certain size and initialized with certain values, but it can change and grow as
need be. The stack is initially empty but grows and shrinks as functions are called
and returned from. Often the text segment is placed near the bottom of memory,
the data segment just above it, with the ability to grow upward, and the stack segment at a high virtual address, with the ability to grow downward, but different
systems work differently.
In all cases, the operating system code is directly executed by the hardware,
with no interpreter and no just-in-time compilation, as is normal with Java.
1.9 RESEARCH ON OPERATING SYSTEMS
Computer science is a rapidly advancing field and it is hard to predict where it
is going. Researchers at universities and industrial research labs are constantly
thinking up new ideas, some of which go nowhere but some of which become the
cornerstone of future products and have massive impact on the industry and users.
Telling which is which turns out to be easier to do in hindsight than in real time.
Separating the wheat from the chaff is especially difficult because it often takes 20
to 30 years from idea to impact.
For example, when President Dwight Eisenhower set up the Dept. of Defense’s
Advanced Research Projects Agency (ARPA) in 1958, he was trying to keep the
Army from killing the Navy and the Air Force over the Pentagon’s research budget. He was not trying to invent the Internet. But one of the things ARPA did was
fund some university research on the then-obscure concept of packet switching,
which led to the first experimental packet-switched network, the ARPANET. It
went live in 1969. Before long, other ARPA-funded research networks were connected to the ARPANET, and the Internet was born. The Internet was then happily
used by academic researchers for sending email to each other for 20 years. In the
early 1990s, Tim Berners-Lee invented the World Wide Web at the CERN research
lab in Geneva and Marc Andreesen wrote a graphical browser for it at the University of Illinois. All of a sudden, the Internet was full of twittering teenagers. President Eisenhower is probably rolling over in his grave.
Research in operating systems has also led to dramatic changes in practical
systems. As we discussed earlier, the first commercial computer systems were all
batch systems, up until M.I.T. invented general-purpose timesharing in the early
SEC. 1.9
RESEARCH ON OPERATING SYSTEMS
79
1960s. Computers were all text-based until Doug Engelbart invented the mouse
and the graphical user interface at Stanford Research Institute in the late 1960s.
Who knows what will come next?
In this section, and in comparable sections throughout the book, we will take a
brief look at some of the research in operating systems that has taken place during
the past 5–10 years, just to give a flavor of what might be on the horizon. This
introduction is certainly not comprehensive. It is based largely on papers that have
been published in the top research conferences because these ideas have at least
survived a rigorous peer review process in order to get published. Note that in computer science—in contrast to other scientific fields—most research is published in
conferences, not in journals. Most of the papers cited in the research sections were
published by either ACM, the IEEE Computer Society, or USENIX and are available over the Internet to (student) members of these organizations. For more information about these organizations and their digital libraries, see
ACM
IEEE Computer Society
USENIX
http://www.acm.org
http://www.computer.org
http://www.usenix.org
All operating systems researchers realize that current operating systems are
massive, inflexible, unreliable, insecure, and loaded with bugs, certain ones more
than others (names withheld to protect the guilty). Consequently, there is a lot of
research on how to build better ones. Work has recently been published about bugs
and debugging (Kasikci et al., 2017; Pina et al., 2019; Li et al., 2019), crashes and
recovery (Chen et al., 2017; and Bhat et al., 2021), energy management (Petrucci
and Loques, 2012; Shen et al., 2013; and Li et al., 2020), file and storage systems
(Zhang et al., 2013a; Chen et al., 2017; Maneas et al., 2020; Ji et al., 2021; and
Miller et al., 2021), high-performance I/O (Rizzo, 2012; Li et al., 2013a; and Li et
al., 2017); hyperthreading and multithreading (Li et al., 2019), dynamic updates
(Pina et al., 2019), managing GPUs (Volos et al., 2018), memory management
(Jantz et al., 2013; and Jeong et al., 2013), embedded systems (Levy et al., 2017),
operating system correctness and reliability (Klein et al., 2009; and Chen et al.,
2017), operating system reliability (Chen et al., 2017; Chajed et al., 2019; and Zou
et al., 2019), security (Oliverio et al., 2017; Konoth et al., 2018; Osterlund et al.,
2019; Duta et al. 2021), virtualization and containers (Tack Lim et al., 2017;
Manco et al., 2017; and Tarasov et al., 2013) among many other topics.
1.10 OUTLINE OF THE REST OF THIS BOOK
We have now completed our introduction and bird’s-eye view of the operating
system. It is time to get down to the details. As mentioned, from the programmer’s point of view, the primary purpose of an operating system is to provide some
80
INTRODUCTION
CHAP. 1
key abstractions, the most important of which are processes and threads, address
spaces, and files. Accordingly the next three chapters are devoted to these topics.
Chapter 2 is about processes and threads. It discusses their properties and how
they communicate with one another. It also gives a number of detailed examples
of how interprocess communication works and how to avoid some of the pitfalls.
In Chap. 3 we study address spaces and their adjunct, memory management.
The important topic of virtual memory will be examined, together with paging.
Then, in Chap. 4, we come to the topic of file systems. To a considerable
extent, what the user sees is the file system. We will look at both the file-system
interface and the file-system implementation.
Input/Output is covered in Chap. 5. We will cover the concept of device
(in)dependence using examples such as storage devices, keyboards, and displays.
Chapter 6 is about deadlocks, including ways to prevent or avoid them.
At this point, we will have completed our study of the basic principles of single-CPU operating systems. However, there is more to say, especially about
advanced topics. In Chap. 7, we examine virtualization. We discuss both the principles, and some of the existing virtualization solutions in detail. Another
advanced topic is multiprocessor systems, including multicores, parallel computers, and distributed systems. These subjects are covered in Chap. 8. Another
important subject is operating system security, which we cover in Chap 9.
Next we have some case studies of real operating systems. These are UNIX,
Linux, and Android (Chap. 10), and Windows 11 (Chap. 11). The text concludes
with some wisdom and thoughts about operating system design in Chap. 12.
1.11 METRIC UNITS
To avoid any confusion, it is worth stating explicitly that in this book, as in
computer science in general, metric units are used instead of traditional English
units (the furlong-stone-fortnight system). The principal metric prefixes are listed
in Fig. 1-31. The prefixes are typically abbreviated by their first letters, with the
units greater than 1 capitalized. Thus a 1-TB database occupies 1012 bytes of storage and a 100-psec (or 100-ps) clock ticks every 10<10 seconds. Since milli and
micro both begin with the letter ‘‘m,’’ a choice had to be made. Normally, ‘‘m’’ is
for milli and ‘‘µ’’ (the Greek letter mu) is for micro.
It is also worth pointing out that, in common industry practice, the units for
measuring memory sizes have slightly different meanings. There kilo means 210
(1024) rather than 103 (1000) because memories are always a power of two. Thus a
1-KB memory contains 1024 bytes, not 1000 bytes. Similarly, a 1-MB memory
contains 220 (1,048,576) bytes and a 1-GB memory contains 230 (1,073,741,824)
bytes. However, a 1-Kbps communication line transmits 1000 bits per second and a
1-Gbps LAN runs at 1,000,000,000 bits/sec because these speeds are not powers of
two. Unfortunately, many people tend to mix up these two systems, especially for
SEC. 1.11
Exp.
81
METRIC UNITS
Prefix
Exp.
10<3
0.001
milli
103
1,000
10<6
0.000001
micro
106
1,000,000
Mega
10<9
0.000000001
nano
109
1,000,000,000
Giga
10<12
0.000000000001
pico
1012
1,000,000,000,000
Tera
10<15
0.000000000000001
femto
1015
1,000,000,000,000,000
Peta
10<18
0.000000000000000001
atto
1018
1,000,000,000,000,000,000
Exa
0.000000000000000000001
zepto
1021
1,000,000,000,000,000,000,000
Zetta
0.000000000000000000000001
yocto
1024
1,000,000,000,000,000,000,000,000
Yotta
10
<21
10<24
Explicit
Explicit
Prefix
Kilo
Figure 1-31. The principal metric prefixes.
SSD or disk sizes. To avoid ambiguity, in this book, we will use the symbols KB,
MB, and GB for 210 , 220 , and 230 bytes, respectively, and the symbols Kbps, Mbps,
and Gbps for 103 , 106 , and 109 bits/sec, respectively.
1.12 SUMMARY
Operating systems can be viewed from two viewpoints: resource managers and
extended machines. In the resource-manager view, the operating system’s job is to
manage the different parts of the system efficiently. In the extended-machine view,
the system provides the users with abstractions that are more convenient to use
than the actual machine. These include processes, address spaces, and files.
Operating systems have a long history, starting from the days when they
replaced the operator, to modern multiprogramming systems. Highlights include
early batch systems, multiprogramming systems, and personal computer systems.
Since operating systems interact closely with the hardware, some knowledge
of computer hardware is useful to understanding them. Computers are built up of
processors, memories, and I/O devices. These parts are connected by buses.
Operating systems can be structured as monolithic, layered, microkernel/client-server, virtual machine, or exokernel/unikernel systems. Regardless, the basic
concepts on which they are built are processes, memory management, I/O management, the file system, and security. The main interface of an operating system is
the set of system calls that it can handle. These tell us what it really does.
PROBLEMS
1. What are the two main functions of an operating system?
2. What is multiprogramming?
82
INTRODUCTION
CHAP. 1
3. In Sec. 1.4, nine different types of operating systems are described. Give a list of possible applications for each of these systems (at least one for each of the operating systems types).
4. To use cache memory, main memory is divided into cache lines, typically 32 or 64
bytes long. An entire cache line is cached at once. What is the advantage of caching an
entire line instead of a single byte or word at a time?
5. What is spooling? Do you think that advanced personal computers will have spooling
as a standard feature in the future?
6. On early computers, every byte of data read or written was handled by the CPU (i.e.,
there was no DMA). What implications does this have for multiprogramming?
7. Why was timesharing not widespread on second-generation computers?
8. Instructions related to accessing I/O devices are typically privileged instructions, that
is, they can be executed in kernel mode but not in user mode. Give a reason why these
instructions are privileged.
9. One reason GUIs were initially slow to be adopted was the cost of the hardware needed to support them. How much video RAM is needed to support a 25-line × 80-row
character monochrome text screen? How much for a 1024 × 768-pixel 24-bit color bitmap? What was the cost of this RAM at 1980 prices ($5/KB)? How much is it now?
10. There are several design goals in building an operating system, for example, resource
utilization, timeliness, robustness, and so on. Give an example of two design goals that
may contradict one another.
11. What is the difference between kernel and user mode? Explain how having two distinct
modes aids in designing an operating system.
12. A 255-GB disk has 65,536 cylinders with 255 sectors per track and 512 bytes per sector. How many platters and heads does this disk have? Assuming an average cylinder
seek time of 11 msec, average rotational delay of 7 msec. and reading rate of 100
MB/sec, calculate the average time it will take to read 100 KB from one sector.
13. Consider a system that has two CPUs, each CPU having two threads (hyperthreading).
Suppose three programs, P0, P1, and P2, are started with run times of 5, 10 and 20
msec, respectively. How long will it take to complete the execution of these programs?
Assume that all three programs are 100% CPU bound, do not block during execution,
and do not change CPUs once assigned.
14. List some differences between personal computer operating systems and mainframe
operating systems.
15. A computer has a pipeline with four stages. Each stage takes the same time to do its
work, namely, 1 nsec. How many instructions per second can this machine execute?
16. Consider a computer system that has cache memory, main memory (RAM) and disk,
and an operating system that uses virtual memory. It takes 2 nsec to access a word
from the cache, 10 nsec to access a word from the RAM, and 10 msec to access a word
from the disk. If the cache hit rate is 95% and main memory hit rate (after a cache
miss) is 99%, what is the average time to access a word?
CHAP. 1
PROBLEMS
83
17. When a user program makes a system call to read or write a disk file, it provides an
indication of which file it wants, a pointer to the data buffer, and the count. Control is
then transferred to the operating system, which calls the appropriate driver. Suppose
that the driver starts the disk and terminates until an interrupt occurs. In the case of
reading from the disk, obviously the caller will have to be blocked (because there are
no data for it). What about the case of writing to the disk? Need the caller be blocked
awaiting completion of the disk transfer?
18. What is the key difference between a trap and an interrupt?
19. Is there any reason why you might want to mount a file system on a nonempty directory? If so, what is it?
20. What is the purpose of a system call in an operating system?
21. Give one reason why mounting file systems is a better design option than prefixing
path names with a drive name or number. Explain why file systems are almost always
mounted on empty directories.
22. For each of the following system calls, give a condition that causes it to fail: open,
close, and lseek.
23. What type of multiplexing (time, space, or both) can be used for sharing the following
resources: CPU, memory, SSD/disk, network card, printer, keyboard, and display?
24. Can the
count = write(fd, buffer, nbytes);
call return any value in count other than nbytes? If so, why?
25. A file whose file descriptor is fd contains the following sequence of bytes: 2, 7, 1, 8, 2,
8, 1, 8, 2, 8, 4. The following system calls are made:
lseek(fd, 3, SEEK SET);
read(fd, &buffer, 4);
where the lseek call makes a seek to byte 3 of the file. What does buffer contain after
the read has completed?
26. Suppose that a 10-MB file is stored on a disk on the same track (track 50) in consecutive sectors. The disk arm is currently situated over track number 100. How long will
it take to retrieve this file from the disk? Assume that it takes about 1 msec to move the
arm from one cylinder to the next and about 5 msec for the sector where the beginning
of the file is stored to rotate under the head. Also, assume that reading occurs at a rate
of 100 MB/s.
27. What is the essential difference between a block special file and a character special
file?
28. In the example given in Fig. 1-17, the library procedure is called read and the system
call itself is called read. Is it essential that both of these have the same name? If not,
which one is more important?
29. The client-server model is popular in distributed systems. Can it also be used in a single-computer system?
84
INTRODUCTION
CHAP. 1
30. To a programmer, a system call looks like any other call to a library procedure. Is it
important that a programmer know which library procedures result in system calls?
Under what circumstances and why?
31. Figure 1-23 shows that a number of UNIX system calls have no Win32 API equivalents. For each of the calls listed as having no Win32 equivalent, what are the consequences for a programmer of converting a UNIX program to run under Windows?
32. A portable operating system is one that can be ported from one system architecture to
another without any modification. Explain why it is infeasible to build an operating
system that is completely portable. Describe two high-level layers that you will have in
designing an operating system that is highly portable.
33. Explain how separation of policy and mechanism aids in building microkernel-based
operating systems.
34. Virtual machines have become very popular for a variety of reasons. Nevertheless,
they have some downsides. Name one.
35. Here are some questions for practicing unit conversions:
(a) How long is a microyear in seconds?
(b) Micrometers are often called microns. How long is a gigamicron?
(c) How many bytes are there in a 1-TB memory?
(d) The mass of the earth is 6000 yottagrams. What is that in grams?
36. Write a shell that is similar to Fig. 1-19 but contains enough code that it actually works
so you can test it. You might also add some features such as redirection of input and
output, pipes, and background jobs.
37. If you have a personal UNIX-like system (Linux, MINIX 3, FreeBSD, etc.) available
that you can safely crash and reboot, write a shell script that attempts to create an
unlimited number of child processes and observe what happens. Before running the
experiment, type sync to the shell to flush the file system buffers to disk to avoid ruining the file system. You can also do the experiment safely in a virtual machine. Note:
Do not try this on a shared system without first getting permission from the system
administrator. The consequences will be instantly obvious so you are likely to be
caught and sanctions may follow.
38. Examine and try to interpret the contents of a UNIX-like or Windows directory with a
tool like the UNIX od program. (Hint: How you do this will depend upon what the OS
allows. One trick that may work is to create a directory on a USB stick with one operating system and then read the raw device data using a different operating system that
allows such access.)
2
PROCESSES AND THREADS
We are now about to embark on a detailed study of how operating systems are
designed and constructed. The most central concept in any operating system is the
process: an abstraction of a running program. Everything else hinges on this concept, and the operating system designer (and student) should have a thorough
understanding of what a process is as early as possible.
Processes are one of the oldest and most important abstractions that operating
systems provide. They support the ability to have (pseudo) concurrent operation
even when there is only one CPU available. They turn a single CPU into multiple
virtual CPUs. When there are four or eight or more CPUs (cores) available, there
could be dozens or hundreds of processes running. Without the process abstraction, modern computing could not exist. In this chapter, we will go into considerable detail about processes and their first cousins, threads.
2.1 PROCESSES
All modern computers often do several things at the same time. People used to
working with computers may not be fully aware of this fact, so a few examples
may make the point clearer. First consider a Web server. Requests come in from all
over asking for Web pages. When a request comes in, the server checks to see if
the page needed is in the cache. If it is, it is sent back; if it is not, a disk request is
started to fetch it. However, from the CPU’s perspective, disk requests take
85
86
PROCESSES AND THREADS
CHAP. 2
eternity. While waiting for a disk request to complete, many more requests may
come in. If there are multiple disks present, some or all of the newer ones may be
fired off to other disks long before the first request is satisfied. Clearly some way is
needed to model and control this concurrency. Processes (and especially threads)
can help here.
Now consider a user PC. When the system is booted, many processes are
secretly started, often unknown to the user. For example, a process may be started
up to wait for incoming email. Another process may run on behalf of the antivirus
program to check periodically if any new virus definitions are available. In addition, explicit user processes may be running, printing files and backing up the
user’s photos on a USB stick, all while the user is surfing the Web. All this activity
has to be managed, and a multiprogramming system supporting multiple processes
comes in very handy here. Even simple computing devices, such as smartphones
and tablets, can support multiple processes.
In any multiprogramming system, each CPU switches from process to process
quickly, running each for tens or maybe hundreds of milliseconds. While strictly
speaking, at any one instant each CPU is running only one process, in the course of
1 second it may work on several of them, giving the illusion of parallelism. Sometimes people speak of pseudoparallelism in this context, to contrast it with the
true hardware parallelism of multiprocessor systems (which have two or more
CPUs sharing the same physical memory). Keeping track of multiple, parallel
activities is hard for people to do. Therefore, operating system designers over the
years have evolved a conceptual model (sequential processes) that makes parallelism easier to deal with. That model, its uses, and some of its consequences form
the subject of this chapter.
2.1.1 The Process Model
In this model, all the runnable software on the computer, sometimes including
the operating system, is organized into a number of sequential processes, or just
processes for short. A process is just an instance of an executing program, including the current values of the program counter, registers, and variables. Conceptually, each process has its own virtual CPU. In reality, of course, each real
CPU switches back and forth from process to process, but to understand the system, it is much easier to think about a collection of processes running in (pseudo)
parallel than to try to keep track of how each CPU switches from program to program. Switching rapidly back and forth like this is known as multiprogramming,
as we saw in Chap. 1.
In Fig. 2-1(a), we see a computer multiprogramming four programs in memory. In Fig. 2-1(b) we see four processes, each with its own flow of control (i.e., its
own logical program counter), and each one running independently of the other
ones. Of course, there is only one physical program counter, so when each process
runs, its logical program counter is loaded into the real program counter. When it is
SEC. 2.1
87
PROCESSES
finished (for the time being), the physical program counter is saved in the process’
stored logical program counter in memory. In Fig. 2-1(c) we see that, viewed over
a long enough time interval, all the processes have made progress, but at any given
instant only one process is actually running.
One program counter
Four program counters
Process
switch
Process
A
B
C
A
B
C
D
D
C
B
A
D
(a)
Time
(b)
(c)
Figure 2-1. (a) Multiprogramming four programs. (b) Conceptual model of four
independent, sequential processes. (c) Only one program is active at once.
In this chapter, we will assume there is only one CPU. Often that assumption
does not hold, since new chips are often multicore, with two, four, or more cores.
We will look at multicore chips and multiprocessors in general in Chap. 8, but for
the time being, it is simpler just to think of one CPU at a time. So when we say that
a CPU can really run only one process at a time, if there are two cores (or CPUs)
each of them can run only one process at a time.
With the CPU switching back and forth among multiple processes, the rate at
which a process performs its computation will not be uniform and probably not
even reproducible if the same processes are run again. Thus, processes must not be
programmed with built-in assumptions about their speed. Consider, for example,
an audio process that plays music to accompany a high-quality video run by another core. Because the audio should start a little later than the video, it signals the
video server to start playing, and then runs an idle loop 10,000 times before playing back the audio. All goes well, if the loop is a reliable timer, but if the CPU decides to switch to another process during the idle loop, the audio process may not
run again until the corresponding video frames have already come and gone, and
the video and audio will be annoyingly out of sync. When a process has critical
real-time requirements like this, that is, particular events must occur within a specified number of milliseconds, special measures must be taken to ensure that they do
occur. Normally, however, most processes are not affected by the underlying multiprogramming of the CPU or the relative speeds of different processes.
The difference between a process and a program is subtle, but absolutely crucial. An analogy may help you here. Consider a culinary-minded computer scientist who is baking a birthday cake for his young daughter. He has a birthday cake
recipe and a kitchen well stocked with all the input: flour, eggs, sugar, extract of
88
PROCESSES AND THREADS
CHAP. 2
vanilla, and so on. In this analogy, the recipe is the program, that is, an algorithm
expressed in some suitable notation, the computer scientist is the processor (CPU),
and the cake ingredients are the input data. The process is the activity consisting of
our baker reading the recipe, fetching the ingredients, and baking the cake.
Now imagine that the computer scientist’s son comes running in screaming his
head off, saying that he has been stung by a bee. The computer scientist records
where he was in the recipe (the state of the current process is saved), gets out a
first-aid book, and begins following the directions in it. Here we see the processor
being switched from one process (baking) to a higher-priority process (administering medical care), each having a different program (recipe versus first aid book).
When the bee sting has been taken care of, the computer scientist goes back to his
cake, continuing at the point where he left off.
The key idea here is that a process is an activity of some kind. It has a program, input, output, and a state. A single processor may be shared among several
processes, with some scheduling algorithm being accustomed to determine when to
stop work on one process and service a different one. In contrast, a program is
something that may be stored on disk, not doing anything.
It is worth noting that if a program is running twice, it counts as two processes.
For example, it is often possible to start a word processor twice or print two files at
the same time if two printers are available. The fact that two processes happen to
be running the same program does not matter; they are distinct processes. The
operating system may be able to share the code between them so only one copy is
in memory, but that is a technical detail that does not change the conceptual situation of two processes running.
2.1.2 Process Creation
Operating systems need some way to create processes. In very simple systems,
or in systems designed for running only a single application (e.g., the controller in
a microwave oven), it may be possible to have all the processes that will ever be
needed be present when the system comes up. In general-purpose systems, however, some way is needed to create and terminate processes as needed during operation. We will now look at some of the issues.
Four principal events cause processes to be created:
1. System initialization
2. Execution of a process-creation system call by a running process
3. A user request to create a new process
4. Initiation of a batch job
When an operating system is booted, typically numerous processes are created.
Some of these processes are foreground processes, that is, processes that interact
SEC. 2.1
PROCESSES
89
with (human) users and perform work for them. Others run in the background and
are not associated with particular users, but instead have some specific function.
For example, one background process may be designed to accept incoming email,
sleeping most of the day but suddenly springing to life when email arrives. Another
background process may be designed to accept incoming requests for Web pages
hosted on that machine, waking up when a request arrives to service the request.
Processes that stay in the background to handle some activity such as email, Web
pages, news, printing, and so on are called daemons. Large systems commonly
have dozens of them. In UNIX†, the ps program can be used to list the running
processes. In Windows, the task manager can be used.
In addition to the processes created at boot time, new processes can be created
afterward as well. Often a running process will issue system calls to create one or
more new processes to help it do its job. Creating new processes is particularly useful when the work to be done can easily be formulated in terms of several related,
but otherwise independent interacting processes. For example, if a large amount of
data are being fetched over a network for subsequent processing, it may be convenient to create one process to fetch the data and put them in a shared buffer while
a second process removes the data items and processes them. On a multiprocessor,
allowing each process to run on a different CPU may also make the job go faster.
In interactive systems, users can start a program by typing a command or (double) clicking on an icon. Taking either of these actions starts a new process and
runs the selected program in it. In command-based UNIX systems running the X
Window System, the new process takes over the window in which it was started. In
Windows, when a process is started it does not have a window, but it can create one
(or more) and most do. In both systems, users may have multiple windows open at
once, each running some process. Using the mouse, the user can select a window
and interact with the process, for example, providing input when needed.
The last situation in which processes are created applies only to batch systems
found on large mainframes. Think of inventory management at the end of a day at
a chain of stores—calculating what to order, analyzing per-store product popularity, etc. Here users can submit batch jobs to the system (possibly remotely). When
the operating system decides it has the resources to run another job, it creates a
new process and runs the next job from the input queue in it.
Technically, in all these cases, a new process is created by having an existing
process execute a process creation system call. That process may be a running user
process, a system process invoked from the keyboard or mouse, or a batch-manager process. What that process does is execute a system call to create the new
process. This system call tells the operating system to create a new process and
indicates, directly or indirectly, which program to run in it. To get the ball rolling,
the very first process is hard-crafted when the system is booted.
† In this chapter, UNIX should be interpreted as including almost all POSIX-based systems, including
Linux, FreeBSD, MacOS, Solaris, etc., and to some extent, Android and iOS as well.
90
PROCESSES AND THREADS
CHAP. 2
In UNIX, there is only one system call to create a new process: fork. This call
creates an exact clone of the calling process. After the fork, the two processes, the
parent and the child, have the same memory image, the same environment strings,
and the same open files. That is all there is. Usually, the child process then
executes execve or a similar system call to change its memory image and run a
new program. For example, when a user types a command, say, sort, to the shell,
the shell forks off a child process and the child executes sort. The reason for this
two-step process is to allow the child to manipulate its file descriptors after the fork
but before the execve in order to accomplish redirection of standard input, standard
output, and standard error.
In Windows, in contrast, a single Win32 function call, CreateProcess, handles
both process creation and loading the correct program into the new process. This
call has 10 parameters, which include the program to be executed, the command-line parameters to feed that program, various security attributes, bits that
control whether open files are inherited, priority information, a specification of the
window to be created for the process (if any), and a pointer to a structure in which
information about the newly created process is returned to the caller. In addition to
CreateProcess, Win32 has about 100 other functions for managing and synchronizing processes and related topics.
In both UNIX and Windows systems, after a process is created, the parent and
child have their own distinct address spaces. If either process changes a word in its
address space, the change is not visible to the other process. In traditional UNIX,
the child’s initial address space is a copy of the parent’s, but there are definitely
two distinct address spaces involved; no writable memory is shared. Some UNIX
implementations share the program text between the two since that cannot be modified. Alternatively, the child may share all of the parent’s memory, but in that case
the memory is shared copy-on-write, which means that whenever either of the two
wants to modify part of the memory, that chunk of memory is explicitly copied
first to make sure the modification occurs in a private memory area. Again, no
writable memory is shared. It is, however, possible for a newly created process to
share some of its creator’s other resources, such as open files. In Windows, the parent’s and child’s address spaces are different from the start.
2.1.3 Process Termination
After a process has been created, it starts running and does whatever its job is.
However, nothing lasts forever, not even processes. Sooner or later the new process will terminate, usually due to one of the following conditions:
1. Normal exit (voluntary)
2. Error exit (voluntary)
3. Fatal error (involuntary)
4. Killed by another process (involuntary)
SEC. 2.1
PROCESSES
91
Most processes terminate because they have done their work. When a compiler
has compiled the program given to it, the compiler executes a system call to tell the
operating system that it is finished. This call is exit in UNIX and ExitProcess in
Windows. Screen-oriented programs also support voluntary termination. Word
processors, Internet browsers, and similar programs always have an icon or menu
item that the user can click to tell the process to remove any temporary files it has
open and then terminate.
The second reason for termination is that the process discovers a fatal error.
For example, if a user types the command
cc foo.c
to compile the program foo.c and no such file exists, the compiler simply announces this fact and exits. Screen-oriented interactive processes generally do not exit
when given bad parameters. Instead they pop up a dialog box and ask the user to
try again.
The third reason for termination is an error caused by the process, often due to
a program bug. Examples include executing an illegal instruction, referencing nonexistent memory, or dividing by zero. In some systems (e.g., UNIX), a process can
tell the operating system that it wishes to handle certain errors itself, in which case
the process is signaled (interrupted) instead of terminated when one of the errors
occurs.
The fourth reason a process might terminate is that the process executes a system call telling the operating system to kill some other process. In UNIX this call
is kill. The corresponding Win32 function is TerminateProcess. In both cases, the
killer must have the necessary authorization to do in the killee. In some systems,
when a process terminates, either voluntarily or otherwise, all processes it created
are immediately killed as well. Neither UNIX nor Windows works this way, however.
2.1.4 Process Hierarchies
In some systems, when a process creates another process, the parent process
and child process continue to be associated in certain ways. The child process can
itself create more processes, forming a process hierarchy. Note that unlike plants
and animals that use sexual reproduction, a process has only one parent (but zero,
one, two, or more children). So a process is more like a hydra than like, say, a cow.
In UNIX, a process and all of its children and further descendants together
form a process group. When a user sends a signal from the keyboard (e.g., by
pressing CTRL-C), the signal is delivered to all members of the process group currently associated with the keyboard (usually all active processes that were created
in the current window). Individually, each process can catch the signal, ignore the
signal, or take the default action, which is to be killed by the signal.
92
PROCESSES AND THREADS
CHAP. 2
As another example of where the process hierarchy plays a key role, let us look
at how UNIX initializes itself when it is started, just after the computer is booted.
A special process, called init, is present in the boot image. When it starts running,
it reads a file telling how many terminals there are. Then it forks off a new process
per terminal. These processes wait for someone to log in. If a login is successful,
the login process executes a shell to accept commands. These commands may start
up more processes, and so forth. Thus, all the processes in the whole system
belong to a single tree, with init at the root.
In contrast, Windows has no concept of a process hierarchy. All processes are
equal. The only hint of a process hierarchy is that when a process is created, the
parent is given a special token (called a handle) that it can use to control the child.
However, it is free to pass this token to some other process, thus invalidating the
hierarchy. Processes in UNIX cannot disinherit their children.
2.1.5 Process States
Although each process is an independent entity, with its own program counter
and internal state, processes often need to interact with other processes. One process may generate some output that another process uses as input. In the command
cat chapter1 chapter2 chapter3 | grep tree
the first process, running cat, concatenates and outputs three files. The second
process, running grep, selects all lines containing the word ‘‘tree.’’ Depending on
the relative speeds of the two processes (which depends on both the relative complexity of the programs and how much CPU time each one has had), it may happen
that grep is ready to run, but there is no input waiting for it. It must then block
until some input is available.
When a process blocks, it does so because logically it cannot continue, typically because it is waiting for input that is not yet available. It is also possible for a
process that is conceptually ready and able to run to be stopped because the operating system has decided to allocate the CPU to another process for a while. These
two conditions are completely different. In the first case, the suspension is inherent
in the problem (you cannot process the user’s command line until it has been
typed). In the second case, it is a technicality of the system (not enough CPUs to
give each process its own private processor). In Fig. 2-2 we see a state diagram
showing the three states a process may be in:
1. Running (actually using the CPU at that instant).
2. Ready (runnable; temporarily stopped to let another process run).
3. Blocked (unable to run until some external event happens).
Logically, the first two states are similar. In both cases the process is willing to run,
only in the second one, there is temporarily no CPU available for it. The third state
SEC. 2.1
PROCESSES
93
is fundamentally different from the first two in that the process cannot run, even if
the CPU is idle and has nothing else to do.
Running
1
Blocked
3
2
4
Ready
1. Process blocks for input
2. Scheduler picks another process
3. Scheduler picks this process
4. Input becomes available
Figure 2-2. A process can be in running, blocked, or ready state. Transitions between these states are as shown.
Four transitions are possible among these three states, as shown. Transition 1
occurs when the operating system discovers that a process cannot continue right
now. In some systems the process can execute a system call, such as pause, to get
into blocked state. In other systems, including UNIX, when a process reads from a
pipe or special file (e.g., a terminal) and there is no input available, the process is
automatically blocked.
Transitions 2 and 3 are caused by the process scheduler, a part of the operating
system, without the process even knowing about them. Transition 2 occurs when
the scheduler decides that the running process has run long enough, and it is time
to let another process have some CPU time. Transition 3 occurs when all the other
processes have had their fair share and it is time for the first process to get the CPU
to run again. The subject of scheduling, that is, deciding which process should run
when and for how long, is an important one; we will look at it later in this chapter.
Many algorithms have been devised to try to balance the competing demands of
efficiency for the system as a whole and fairness to individual processes. We will
study some of them later in this chapter.
Transition 4 occurs when the external event for which a process was waiting
(such as the arrival of some input) happens. If no other process is running at that
instant, transition 3 will be triggered and the process will start running. Otherwise
it may have to wait in ready state for a little while until the CPU is available and its
turn comes.
Using the process model, it becomes much easier to think about what is going
on inside the system. Some of the processes run programs that carry out commands
typed in by a user. Other processes are part of the system and handle tasks such as
carrying out requests for file services or managing the details of running a disk or a
tape drive. When a disk interrupt occurs, the system makes a decision to stop running the current process and run the disk process, which was blocked waiting for
that interrupt. Thus, instead of thinking about interrupts, we can think about user
processes, disk processes, terminal processes, and so on, which block when they
are waiting for something to happen. When the disk has been read or the character
typed, the process waiting for it is unblocked and is eligible to run again.
94
PROCESSES AND THREADS
CHAP. 2
This view gives rise to the model shown in Fig. 2-3. Here the lowest level of
the operating system is the scheduler, with a variety of processes on top of it. All
the interrupt handling and details of actually starting and stopping processes are
hidden away in what is here called the scheduler, which is actually not much code.
The rest of the operating system is nicely structured in process form. Few real systems are as nicely structured as this, however.
Processes
0
1
n–2 n–1
Scheduler
Figure 2-3. The lowest layer of a process-structured operating system handles
interrupts and scheduling. Above that layer are sequential processes.
2.1.6 Implementation of Processes
To implement the process model, the operating system maintains a table (an
array of structures), called the process table, with one entry per process. (Some
authors call these entries process control blocks.) This entry contains important
information about the process’ state, including its program counter, stack pointer,
memory allocation, the status of its open files, its accounting and scheduling information, and everything else about the process that must be saved when the process
is switched from running to ready or blocked state so that it can be restarted later
as if it had never been stopped.
Figure 2-4 shows some of the key fields in a typical system. The fields in the
first column relate to process management. The other two relate to memory management and file management, respectively. It should be noted that precisely which
fields the process table has is highly system dependent, but this figure gives a general idea of the kinds of information needed.
Now that we have looked at the process table, it is possible to explain a little
more about how the illusion of multiple sequential processes is maintained on one
(or each) CPU and also explain interrupts in more detail than we were able to do in
Chap. 1. Associated with each I/O class is a location (typically at a fixed location
near the bottom of memory) called the interrupt vector. It contains the address of
the ISR (Interrupt Service Routine). Suppose that user process 3 is running
when a disk interrupt happens. User process 3’s program counter, program status
word, and sometimes one or more registers are pushed onto the (current) stack by
the interrupt hardware. The computer then jumps to the address in the interrupt
vector. That is all the hardware does. From here on, it is up to the ISR in software.
SEC. 2.1
95
PROCESSES
Process management
Registers
Program counter
Program status word
Stack pointer
Process state
Priority
Scheduling parameters
Process ID
Parent process
Process group
Signals
Time when process started
CPU time used
Children’s CPU time
Time of next alarm
Memory management
Pointer to text segment info
Pointer to data segment info
Pointer to stack segment info
File management
Root directory
Working directory
File descriptors
User ID
Group ID
Figure 2-4. Some of the fields of a typical process-table entry.
All interrupts start by saving the registers, often in the process table entry for
the current process. Then the information pushed onto the stack by the interrupt is
removed and the stack pointer is set to point to a temporary stack used by the process handler. Actions such as saving the registers and setting the stack pointer cannot even be expressed in high-level languages such as C, so they are performed by
a small assembly-language routine, usually the same one for all interrupts since the
work of saving the registers is identical, no matter the cause of the interrupt.
When this routine is finished, it calls a C procedure to do the rest of the work
for this specific interrupt type. (We assume the operating system is written in C,
the usual choice for all operating systems used in production.) When it has done
its job, possibly making some process now ready, the scheduler is called to see
what to run next. After that, control is passed back to the assembly-language code
to load up the registers and memory map for the now-current process and start it
running. Interrupt handling and scheduling are summarized in Fig. 2-5. It is worth
noting that the details vary somewhat from system to system.
A process may be interrupted thousands of times during its execution, but the
key idea is that after each interrupt the interrupted process returns to precisely the
same state it was in before the interrupt occurred.
2.1.7 Modeling Multiprogramming
When multiprogramming is used, the CPU utilization can be improved.
Crudely put, if the average process computes only 20% of the time it is sitting in
memory, then with five processes in memory at once the CPU should be busy all
the time. This model is unrealistically optimistic, however, since it tacitly assumes
that all five processes will never be waiting for I/O at the same time.
96
PROCESSES AND THREADS
CHAP. 2
1. Hardware stacks program counter, etc.
2. Hardware loads new program counter from interrupt vector.
3. Assembly-language procedure saves registers.
4. Assembly-language procedure sets up new stack.
5. C interrupt service runs (typically reads and buffers input).
6. Scheduler decides which process is to run next.
7. C procedure returns to the assembly code.
8. Assembly-language procedure starts up new current process.
Figure 2-5. Skeleton of what the lowest level of the operating system does when
an interrupt occurs. The details may vary between operating systems.
A better model is to look at CPU usage from a probabilistic viewpoint. Suppose that a process spends a fraction p of its time waiting for I/O to complete. With
n processes in memory at once, the probability that all n processes are waiting for
I/O (in which case the CPU will be idle) is p n . The CPU utilization is then given
by the formula
CPU utilization = 1 < p n
CPU utilization (in percent)
Figure 2-6 shows, for different values of p (or ‘‘I/O wait’’), the CPU utilization as a
function of n, which is called the degree of multiprogramming.
20% I/O wait
100
50% I/O wait
80
60
80% I/O wait
40
20
0
1
2
3
4
5
6
7
8
Degree of multiprogramming
9
10
Figure 2-6. CPU utilization as a function of the number of processes in memory.
From the figure it is clear that if processes spend 80% of their time waiting for
I/O, at least 10 processes must be in memory at once to get the CPU waste below
10%. When you realize that an interactive process waiting for a user to type something at a terminal (or click on an icon) is in I/O wait state, it should be clear that
I/O wait times of 80% and more are not unusual. But even on servers, processes
doing a lot of disk I/O will often have this percentage or more.
SEC. 2.1
PROCESSES
97
For the sake of accuracy, it should be pointed out that the probabilistic model
just described is only an approximation. It implicitly assumes that all n processes
are independent, meaning that it is quite acceptable for a system with five processes in memory to have three running and two waiting. But with a single CPU, we
cannot have three processes running at once, so a process becoming ready while
the CPU is busy will have to wait. Thus, the processes are not independent. A more
accurate model can be constructed using queueing theory, but the point we are
making—multiprogramming lets processes use the CPU when it would otherwise
become idle—is, of course, still valid, even if the true curves of Fig. 2-6 are slightly different from those shown in the figure.
Even though the model of Fig. 2-6 is fairly simple-minded, it can nevertheless
be used to make specific, although approximate, predictions about CPU performance. Suppose, for example, that a computer has 8 GB of memory, with the
operating system and its tables taking up 2 GB and each user program also taking
up 2 GB. These sizes allow three user programs to be in memory at once. With an
80% average I/O wait, we have a CPU utilization (ignoring operating system overhead) of 1 < 0. 83 or about 49%. Adding another 8 GB of memory allows the system to go from three-way multiprogramming to seven-way multiprogramming,
thus raising the CPU utilization to 79%. In other words, the additional 8 GB will
raise the throughput by 30%.
Adding yet another 8 GB would increase CPU utilization only from 79% to
91%, thus raising the throughput by only another 12%. Using this model, the computer’s owner might decide that the first addition was a good investment but that
the second was not.
2.2 THREADS
In traditional operating systems, each process has an address space and a single
thread of control. In fact, that is almost the definition of a process. Nevertheless, in
many situations, it is useful to have multiple threads of control in the same address
space running in quasi-parallel, as though they were (almost) separate processes
(except for the shared address space). In the following sections, we will discuss
threads and their implications. Later we will look at an alternative solution.
2.2.1 Thread Usage
Why would anyone want to have a kind of process within a process? It turns
out there are several reasons for having these miniprocesses, called threads. Let
us now examine some of them. The main reason for having threads is that in many
applications, multiple activities are going on at once. Some of these may block
from time to time. By decomposing such an application into multiple sequential
threads that run in quasi-parallel, the programming model becomes simpler.
98
PROCESSES AND THREADS
CHAP. 2
We have seen this argument once before. It is precisely the argument for having processes. Instead of thinking about interrupts, timers, and context switches,
we can think about parallel processes. Only now with threads we add a new element: the ability for the parallel entities to share an address space and all of its data
among themselves. This ability is essential for certain applications, which is why
having multiple processes (with their separate address spaces) will not work.
A second argument for having threads is that since they are lighter weight than
processes, they are easier (i.e., faster) to create and destroy. In many systems,
creating a thread goes 10–100 times faster than creating a process. When the number of threads needed changes dynamically and rapidly, this property is useful to
have.
A third reason for having threads is also a performance argument. Threads
yield no performance gain when all of them are CPU bound, but when there is substantial computing and also substantial I/O, having threads allows these activities
to overlap, thus speeding up the application.
Finally, threads are useful on systems with multiple CPUs, where real parallelism is possible. We will come back to this issue in Chap. 8.
It is easiest to see why threads are useful by looking at some concrete examples. As a first example, consider a word processor. Word processors usually display the document being created on the screen formatted exactly as it will appear
on the printed page. In particular, all the line breaks and page breaks are in their
correct and final positions, so that the user can inspect them and change the document if need be (e.g., to eliminate widows and orphans—incomplete top and bottom lines on a page, which are considered esthetically unpleasing).
Suppose that the user is writing a book. From the author’s point of view, it is
easiest to keep the entire book as a single file to make it easier to search for topics,
perform global substitutions, and so on. Alternatively, each chapter might be a separate file. However, having every section and subsection as a separate file is a real
nuisance when global changes have to be made to the entire book, since then hundreds of files have to be individually edited, one at a time. For example, if proposed standard xxxx is approved just before the book goes to press, all occurrences of
‘‘Draft Standard xxxx’’ have to be changed to ‘‘Standard xxxx’’ at the last minute.
If the entire book is one file, typically a single command can do all the substitutions. In contrast, if the book is spread over 300 files, each one must be edited separately.
Now consider what happens when the user suddenly deletes one sentence from
page 1 of an 800-page book. After checking the changed page for correctness, she
now wants to make another change on page 600 and types in a command telling
the word processor to go to that page (possibly by searching for a phrase occurring
only there). The word processor is now forced to reformat the entire book up to
page 600 on the spot because it does not know what the first line of page 600 will
be until it has processed all the previous pages. There may be a substantial delay
before page 600 can be displayed, leading to an unhappy user.
SEC. 2.2
99
THREADS
Threads can help here. Suppose that the word processor is written as a twothreaded program. One thread interacts with the user and the other handles reformatting in the background. As soon as the sentence is deleted from page 1, the
interactive thread tells the reformatting thread to reformat the whole book. Meanwhile, the interactive thread continues to listen to the keyboard and mouse and
responds to simple commands like scrolling page 1 while the other thread is computing madly in the background. With a little luck, the reformatting will be completed before the user asks to see page 600, so it can be displayed instantly.
While we are at it, why not add a third thread? Many word processors have a
feature of automatically saving the entire file to disk every few minutes to protect
the user against losing a day’s work in the event of a program crash, system crash,
or power failure. The third thread can handle the disk backups without interfering
with the other two. The situation with three threads is shown in Fig. 2-7.
Four score and seven
years ago, our fathers
brought forth upon this
continent a new nation:
conceived in liberty,
and dedicated to the
proposition that all
men are created equal.
Now we are engaged
in a great civil war
testing whether that
nation, or any nation
so conceived and so
dedicated, can long
endure. We are met on
a great battlefield of
that war.
We have come to
dedicate a portion of
that field as a final
resting place for those
who here gave their
lives that this nation
might live. It is
altogether fitting and
proper that we should
do this.
But, in a larger sense,
we cannot dedicate, we
cannot consecrate we
cannot hallow this
ground. The brave
men, living and dead,
who struggled here
have consecrated it, far
above our poor power
to add or detract. The
world will little note,
nor long remember,
what we say here, but
it can never forget
what they did here.
It is for us the living,
rather, to be dedicated
here to the unfinished
work which they who
fought here have thus
far so nobly advanced.
It is rather for us to be
here dedicated to the
great task remaining
before us, that from
these honored dead we
take increased devotion
to that cause for which
they gave the last full
measure of devotion,
that we here highly
resolve that these dead
shall not have died in
vain that this nation,
under God, shall have
a new birth of freedom
and that government of
the people by the
people, for the people
Kernel
Disk
Keyboard
Figure 2-7. A word processor with three threads.
If the program were single-threaded, then whenever a disk backup started,
commands from the keyboard and mouse would be ignored until the backup was
finished. The user would surely perceive this as sluggish performance. Alternatively, keyboard and mouse events could interrupt the disk backup, allowing good
performance but leading to a complex interrupt-driven programming model. With
three threads, the programming model is much simpler. The first thread just interacts with the user. The second thread reformats the document when told to. The
third thread writes the contents of RAM to disk periodically.
It should be clear that having three separate processes would not work here
because all three threads need to operate on the document. By having three threads
instead of three processes, they share a common memory and thus all have access
to the document being edited. With three processes this would be impossible.
100
PROCESSES AND THREADS
CHAP. 2
An analogous situation exists with many other interactive programs. For example, an electronic spreadsheet is a program that allows a user to maintain a matrix,
some of whose elements are data provided by the user. Other elements are computed based on the input data using potentially complex formulas. A spreadsheet
that computes the predicted annual profit of a substantial company might have hundreds of pages and thousands of complex formulas based on hundreds of input
variables. When a user changes one input variable, many cells may have to be
recomputed. By having a background thread do the recomputation, the interactive
thread can allow the user to make additional changes while the computation is
going on. Similarly, a third thread can handle periodic backups to disk on its own.
Now consider yet another example of where threads can be useful: a server for
a Website. Requests for pages come in and the requested page is sent back to the
client. At most Websites, some pages are more commonly accessed than other
pages. For example, Samsung’s home page is accessed far more than a page deep
in the tree containing the detailed technical specifications of any smartphone model. Web servers use this fact to improve performance by maintaining a collection
of heavily used pages in main memory to eliminate the need to go to disk to get
them. Such a collection is called a cache and is used in many other contexts as
well. We saw CPU caches in Chap. 1, for example.
One way to organize the Web server is shown in Fig. 2-8(a). Here one thread,
the dispatcher, reads incoming requests for work from the network. After examining the request, it chooses an idle (i.e., blocked) worker thread and hands it the
request, possibly by writing a pointer to the message into a special word associated
with each thread. The dispatcher then wakes up the sleeping worker, moving it
from blocked state to ready state.
Web server process
Dispatcher thread
Worker thread
User
space
Web page cache
Kernel
Kernel
space
Network
connection
Figure 2-8. A multithreaded Web server.
When the worker wakes up, it checks to see if the request can be satisfied from
the Web page cache, to which all threads have access. If not, it starts a read
SEC. 2.2
THREADS
101
operation to get the page from the disk and blocks until the disk operation completes. When the thread blocks on the disk operation, another thread is chosen to
run, possibly the dispatcher, in order to acquire more work, or possibly another
worker that is now ready to run.
This model allows the server to be written as a collection of sequential threads.
The dispatcher’s program consists of an infinite loop for getting a work request and
handing it off to a worker. Each worker’s code consists of an infinite loop consisting of accepting a request from the dispatcher and checking the Web cache to see if
the page is present. If so, it is returned to the client, and the worker blocks waiting
for a new request. If not, it gets the page from the disk, returns it to the client, and
blocks waiting for a new request.
A rough outline of the code is given in Fig. 2-9. Here, as in the rest of this
book, TRUE is assumed to be the constant 1. Also, buf and page are structures
appropriate for holding a work request and a Web page, respectively.
while (TRUE) {
get next request(&buf);
handoff work(&buf);
}
(a)
while (TRUE) {
wait for work(&buf)
look for page in cache(&buf, &page);
if (page not in cache(&page))
read page from disk(&buf, &page);
return page(&page);
}
(b)
Figure 2-9. A rough outline of the code for Fig. 2-8. (a) Dispatcher thread.
(b) Worker thread.
Consider how the Web server could be written in the absence of threads. One
possibility is to have it operate as a single thread. The main loop of the Web server
gets a request, examines it, and carries it out to completion before getting the next
one. While waiting for the disk, the server is idle and does not process any other
incoming requests. If the Web server is running on a dedicated machine, as is commonly the case, the CPU is simply idle while the Web server is waiting for the
disk. The net result is that many fewer requests/sec can be processed. Thus, threads
gain considerable performance, but each thread is programmed sequentially, in the
usual way. We will look at an alternative, event-driven, approach later.
A third example where threads are useful is in applications that must process
very large amounts of data. The normal approach is to read in a block of data,
process it, and then write it out again. The problem here is that if only blocking
system calls are available, the process blocks while data are coming in and data are
going out. Having the CPU go idle when there is lots of computing to do is clearly
wasteful and should be avoided if possible.
Threads offer a solution. The process could be structured with an input thread,
a processing thread, and an output thread. The input thread reads data into an input
buffer. The processing thread takes data out of the input buffer, processes them,
102
PROCESSES AND THREADS
CHAP. 2
and puts the results in an output buffer. The output buffer writes these results back
to disk. In this way, input, output, and processing can all be going on at the same
time. Of course, this model works only if a system call blocks only the calling
thread, not the entire process.
2.2.2 The Classical Thread Model
Now that we have seen why threads might be useful and how they can be used,
let us investigate the idea a bit more closely. The process model is based on two
independent concepts: resource grouping and execution. Sometimes it is useful to
separate them; this is where threads come in. First we will look at the classical
thread model; after that we will examine the Linux thread model, which blurs the
line between processes and threads.
One way of looking at a process is that it is a convenient way to group together
related resources. A process has an address space that contains program text and
data, as well as other resources. These resources may include open files, child
processes, pending alarms, signal handlers, accounting information, and more. By
putting them together in the form of a process, they can be managed more easily.
The other concept a process has is a thread of execution, usually shortened to
just thread. The thread has a program counter associated with it that keeps track
of which instruction to execute next. It has registers, which hold its current working variables. It also has a stack, which contains the thread’s execution history, one
frame for each procedure called but not yet returned from. Although a thread must
execute in some process, the thread and its process are different concepts and can
be treated separately. Processes are used to group resources together; threads are
the entities scheduled for execution on the CPU.
What threads add to the process model is to allow multiple executions to take
place in the same process environment (and address space), to a large degree independent of one another. Having multiple threads running in parallel in one process
is analogous to having multiple processes running in parallel in one computer. In
the former case, the threads share an address space and other resources. In the latter case, processes share physical memory, disks, printers, and other resources.
Because threads have some of the properties of processes, they are sometimes called lightweight processes. The term multithreading is also used to describe the
situation of allowing multiple threads in the same process. As we saw in Chap. 1,
some CPUs have direct hardware support for multithreading and allow a thread
switch to happen on a nanosecond time scale.
In Fig. 2-10(a) we see three traditional processes. Each process has its own
address space and a single thread of control. In contrast, in Fig. 2-10(b) we see a
single process with three threads of control. Although in both cases we have three
threads, in Fig. 2-10(a) each of them operates in a different address space, whereas
in Fig. 2-10(b) all three of them share the same address space.
SEC. 2.2
Process 1
103
THREADS
Process 2
Process 3
Process
User
space
Thread
Kernel
space
Thread
Kernel
Kernel
(a)
(b)
Figure 2-10. (a) Three processes each with one thread. (b) One process with
three threads.
When a multithreaded process is run on a single-CPU system, the threads take
turns running. In Fig. 2-1, we saw how multiprogramming of processes works. By
switching back and forth among multiple processes, the system gives the illusion
of separate sequential processes running in parallel. Multithreading works the same
way. The CPU switches rapidly back and forth among the threads, providing the
illusion that the threads are running in parallel, albeit on a slower CPU. With three
compute-bound threads in a process, the threads would appear to be running in parallel, each one on a CPU with one-third the speed of the real CPU.
The ifferent threads in a process are not as independent as different processes.
All threads have exactly the same address space, which means that they also share
the same global variables. Since every thread can access every memory address
within the process’ address space, one thread can read, write, or even wipe out another thread’s stack. There is no protection between threads because (1) it is impossible, and (2) it should not be necessary. Unlike different processes, which may
be from different users and which may be mutually hostile to one another, a process is always owned by a single user, who has presumably created multiple threads
so that they can cooperate, not fight with each other. In addition to sharing an address space, all the threads can share the same set of open files, child processes,
signals, alarms, and so forth, as shown in Fig. 2-11. Thus, the organization of
Fig. 2-10(a) would be used when the three processes are essentially unrelated,
whereas Fig. 2-10(b) would be appropriate when the three threads are actually part
of the same job and are actively and closely cooperating with each other.
The items in the first column are process properties, not thread properties. For
example, if one thread opens a file, that file is visible to the other threads in the
process and they can read and write it. This is logical, since the process is the unit
of resource management, not the thread. If each thread had its own address space,
open files, pending alarms, and so on, it would be a separate process. What we are
104
PROCESSES AND THREADS
Per-process items
Address space
Global variables
Open files
Child processes
Pending alarms
Signals and signal handlers
Accounting information
CHAP. 2
Per-thread items
Program counter
Registers
Stack
State
Figure 2-11. The first column lists some items shared by all threads in a process.
The second one lists some items private to each thread.
trying to achieve with the thread concept is the ability for multiple threads of
execution to share a set of resources so that they can work together closely to perform some task.
Like a traditional process (i.e., a process with only one thread), a thread can be
in any one of several states: running, blocked, ready, or terminated. A running
thread currently has the CPU and is active. In contrast, a blocked thread is waiting
for some event to unblock it. For example, when a thread performs a system call to
read from the keyboard, it is blocked until input is typed. A thread can block waiting for some external event to happen or for some other thread to unblock it. A
ready thread is scheduled to run and will as soon as its turn comes up. The transitions between thread states are the same as those between process states and are
illustrated in Fig. 2-2.
It is important to realize that each thread has its own stack, as illustrated in
Fig. 2-12. Each thread’s stack contains one frame for each procedure called but
not yet returned from. This frame contains the procedure’s local variables and the
return address to use when the procedure call has finished. For example, if procedure X calls procedure Y and Y calls procedure Z, then while Z is executing, the
frames for X, Y, and Z will all be on the stack. Each thread will generally call different procedures and thus have a different execution history. This is why each
thread needs its own stack.
When multithreading is present, processes usually start with a single thread
present. This thread has the ability to create new threads by calling a library procedure such as thread create. A parameter to thread create specifies the name of a
procedure for the new thread to run. It is not necessary (or even possible) to specify
anything about the new thread’s address space, since it automatically runs in the
address space of the creating thread. Sometimes threads are hierarchical, with a
parent-child relationship, but often no such relationship exists, with all threads
being equal. With or without a hierarchical relationship, the creating thread is
usually returned a thread identifier that names the new thread.
When a thread has finished its work, it can exit by calling a library procedure,
say, thread exit. It then vanishes and is no longer schedulable. In some thread
SEC. 2.2
105
THREADS
Thread 2
Thread 1
Thread 3
Process
Thread 3's stack
Thread 1's
stack
Kernel
Figure 2-12. Each thread has its own stack.
systems, one thread can wait for a (specific) thread to exit by calling a procedure,
for example, thread join. This procedure blocks the calling thread until a (specific) thread has exited. In this regard, thread creation and termination is very much
like process creation and termination, with approximately the same options as well.
Another common thread call is thread yield, which allows a thread to voluntarily give up the CPU to let another thread run. Such a call is important because
there is no clock interrupt to actually enforce multiprogramming as there is with
processes. Thus, it is important for threads to be polite and voluntarily surrender
the CPU from time to time to give other threads a chance to run. Other calls allow
one thread to wait for another thread to finish some work, for a thread to announce
that it has finished some work, and so on.
While threads are often useful, they also introduce a number of serious complications into the programming model. To start with, consider the effects of the
UNIX fork system call. If the parent process has multiple threads, should the child
also have them? If not, the process may not function properly, since all of them
may be essential. However, if the child process gets as many threads as the parent,
what happens if a thread in the parent was blocked on a read call, say, from the
keyboard? Are two threads now blocked on the keyboard, one in the parent and
one in the child? When a line is typed, do both threads get a copy of it? Only the
parent? Only the child? The same problem exists with open network connections.
The designers of the operating system have to make choices clear and carefully
define the semantics so users will understand the behavior of the threads.
We will look at some of these issues and observe that the solutions are often
pragmatic. For instance, on a system such as Linux, a fork of a multithreaded process will create only a single thread in the child. However, using Posix threads a
program can use the pthread atfork() call to register fork handlers (procedures that
are called when a fork occurs), so it can start additional threads and do whatever is
106
PROCESSES AND THREADS
CHAP. 2
needed to get going properly again. Even so, it is important to note that many of
these issues are design choices and different systems may opt for different solutions. For now, the most important thing to remember is that the relation between
threads and fork can be quite complex.
Another class of problems is related to the fact that threads share many data
structures. What happens if one thread closes a file while another one is still reading from it? Suppose one thread notices that there is too little memory and starts
allocating more memory. Partway through, a thread switch occurs, and the new
thread also notices that there is too little memory and also starts allocating more
memory. Memory will probably be allocated twice. These problems can be solved
with some effort, but careful thought and design are needed to make multithreaded
programs work correctly.
2.2.3 POSIX Threads
To make it possible to write portable threaded programs, IEEE has defined a
standard for threads in IEEE standard 1003.1c. The threads package it defines is
called Pthreads. Most UNIX systems support it. The standard defines over 60
function calls, which is far too many to go over here. Instead, we will just describe
a few of the major ones to give an idea of how it works. The calls we will describe
below are listed in Fig. 2-13.
Thread call
Description
pthread create
Create a new thread
pthread exit
Terminate the calling thread
pthread join
Wait for a specific thread to exit
pthread yield
Release the CPU to let another thread run
pthread attr init
Create and initialize a thread’s attribute structure
pthread attr destroy
Remove a thread’s attribute structure
Figure 2-13. Some of the Pthreads function calls.
All Pthreads threads have certain properties. Each one has an identifier, a set of
registers (including the program counter), and a set of attributes, which are stored
in an attribute structure. The attributes include the stack size, scheduling parameters, and other items needed to use the thread.
A new thread is created using the pthread create call. The thread identifier of
the newly created thread is returned as the function value. This call is intentionally
very much like the fork system call (except with parameters), with the thread identifier playing the role of the PID, mostly for identifying threads referenced in other
calls. When a thread has finished the work it has been assigned, it can terminate by
calling pthread exit. This call stops the thread and releases its stack.
Often a thread has a need to wait for another thread to finish its work and exit
before it can continue. The thread that is waiting calls pthread join to wait for a
SEC. 2.2
THREADS
107
specific other thread to terminate. The thread identifier of the thread to wait for is
given as a parameter.
Sometimes it happens that a thread is not logically blocked, but feels that it has
run long enough and wants to give another thread a chance to run. It can accomplish this goal by calling pthread yield. There is no such call for processes because the assumption there is that processes are fiercely competitive and each
wants all the CPU time it can get (although a very public-spirited process could
call sleep to yield the CPU briefly). However, since the threads of a process are
working together and their code is invariably written by the same programmer,
sometimes the programmer wants them to give each other another chance.
The next two thread calls deal with attributes. Pthread attr init creates the
attribute structure associated with a thread and initializes it to the default values.
These values (such as the priority) can be changed by manipulating fields in the
attribute structure.
Finally, pthread attr destroy removes a thread’s attribute structure, freeing up
its memory. It does not affect threads using it; they continue to exist.
To get a better feel for how Pthreads works, consider the simple example of
Fig. 2-14. Here the main program loops NUMBER OF THREADS times, creating
a new thread on each iteration, after announcing its intention. If the thread creation
fails, it prints an error message and then exits. After creating all the threads, the
main program exits.
When a thread is created, it prints a one-line message announcing itself, then it
exits. The order in which the various messages are interleaved is nondeterminate
and may vary on consecutive runs of the program.
The Pthreads calls described above are not the only ones. We will examine
some of the others after we have discussed process and thread synchronization.
2.2.4 Implementing Threads in User Space
There are two main places to implement threads: user space and the kernel.
The choice is a bit controversial, and a hybrid implementation is also possible. We
will now describe these methods, along with their advantages and disadvantages.
The first method is to put the threads package entirely in user space. The kernel knows nothing about them. As far as the kernel is concerned, it is managing
ordinary, single-threaded processes. The first, and most obvious, advantage is that
a user-level threads package can be implemented on an operating system that does
not support threads. All operating systems used to fall into this category, and even
now some still do. With this approach, threads are implemented by a library.
All of these implementations have the same general structure, illustrated in
Fig. 2-15(a). The threads run on top of a run-time system, which is a collection of
procedures that manage threads. We have seen four of these already: pthread create, pthread exit, pthread join, and pthread yield, but usually there are more.
108
PROCESSES AND THREADS
CHAP. 2
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
#define NUMBER OF THREADS
10
void *print hello world(void *tid)
{
/* This function prints the thread’s identifier and then exits. */
printf("Hello World. Greetings from thread %d\n", tid);
pthread exit(NULL);
}
int main(int argc, char *argv[])
{
/* The main program creates 10 threads and then exits. */
pthread t threads[NUMBER OF THREADS];
int status, i;
for(i=0; i < NUMBER OF THREADS; i++) {
printf("Main here. Creating thread %d\n", i);
status = pthread create(&threads[i], NULL, print hello world, (void *)i);
if (status != 0) {
printf("Oops. pthread create returned error code %d\n", status);
exit(-1);
}
}
exit(NULL);
}
Figure 2-14. An example program using threads.
When threads are managed in user space, each process needs its own private
thread table to keep track of the threads in that process. This table is analogous to
the kernel’s process table, except that it keeps track only of the per-thread properties, such as each thread’s program counter, stack pointer, registers, state, and so
forth. The thread table is managed by the run-time system. When a thread is moved
to the ready or blocked state, the information needed to restart it is stored in the
thread table, exactly the same way as the kernel stores information about processes
in the process table.
When a thread does something that may cause it to become blocked locally, for
example, waiting for another thread in its process to complete some work, it calls a
run-time system procedure. This procedure checks to see if the thread must be put
into blocked state. If so, it stores the thread’s registers (i.e., its own) in the thread
table, looks in the table for a ready thread to run, and reloads the machine registers
with the new thread’s saved values. As soon as the stack pointer and program
SEC. 2.2
109
THREADS
Thread
Process
Process
Thread
User
space
Kernel
space
Kernel
Kernel
Run-time
system
Thread
table
Process
table
Process
table
Thread
table
Figure 2-15. (a) A user-level threads package. (b) A threads package managed
by the kernel.
counter have been switched, the new thread comes to life again automatically. If
the machine happens to have an instruction to store all the registers and another
one to load them all, the entire thread switch can be done in just a handful of
instructions. Doing thread switching like this is at least an order of magnitude—
maybe more—faster than trapping to the kernel and is a strong argument in favor
of user-level threads packages.
In addition, when a thread is finished running for the moment, for example,
when it calls thread yield, the code of thread yield saves the thread’s information
in the thread table and then calls the thread scheduler to pick another thread to run.
The procedure that saves the thread’s state and the scheduler are just local procedures, so invoking them is much more efficient than making a kernel call. There is
no need for a trap, a context switch, flushing of caches, and so on. This makes
thread scheduling very fast.
User-level threads also have other advantages. They allow each process to have
its own customized scheduling algorithm. For some applications, for example,
those with a garbage-collector thread, not having to worry about a thread being
stopped at an inconvenient moment is a plus. They also scale better, since kernel
threads invariably require some table space and stack space in the kernel, which
can be a problem if there is a very large number of threads.
Despite their better performance, user-level threads packages have some major
problems. First among these is the problem of how blocking system calls are
implemented. Suppose that a thread reads from the keyboard before any keys have
been hit. Letting the thread actually make the system call is unacceptable, since
this will stop all the threads. One of the main goals of having threads in the first
place was to allow each thread to use blocking calls, but to prevent one blocked
110
PROCESSES AND THREADS
CHAP. 2
thread from affecting the others. With blocking system calls, it is hard to see how
this goal can be achieved readily.
The system calls could all be changed to be nonblocking (e.g., a read on the
keyboard would just return 0 bytes if no characters were already buffered), but
requiring changes to the operating system is unattractive. Besides, one argument
for user-level threads was precisely that they could run with existing operating systems. In addition, changing the semantics of read will require changes to many
user programs.
Another alternative is available in the event that it is possible to tell in advance
if a call will block. In most versions of UNIX, a system call, select, exists, which
allows the caller to tell whether a prospective read will block. When this call is
present, the library procedure read can be replaced with a new one that first does a
select call and then does the read call only if it is safe (i.e., will not block). If the
read call will block, the call is not made. Instead, another thread is run. The next
time the run-time system gets control, it can check again to see if the read is now
safe. This approach requires rewriting parts of the system call library, and is inefficient and inelegant, but there is little choice. The code placed around the system
call to do the checking is called a wrapper. (As we shall see, many operating systems have even more efficient mechanisms for asynchronous I/O, such as epoll on
Linux and kqueue on FreeBSD).
Somewhat analogous to the problem of blocking system calls is the problem of
page faults. We will study these in Chap. 3. For the moment, suffice it to say that
computers can be set up in such a way that not all of the program is in main memory at once. If the program calls or jumps to an instruction that is not in memory, a
page fault occurs and the operating system will go and get the missing instruction
(and its neighbors) from disk. This is called a page fault. The process is blocked
while the necessary instruction is being located and read in. If a thread causes a
page fault, the kernel, unaware of even the existence of threads, naturally blocks
the entire process until the disk I/O is complete, even though other threads might
be runnable.
Another problem with user-level thread packages is that if a thread starts running, no other thread in that process will ever run unless the first thread voluntarily
gives up the CPU. Within a single process, there are no clock interrupts, making it
impossible to schedule processes in round-robin fashion (taking turns). Unless a
thread exits the run-time system of its own free will, the scheduler will never run.
One possible solution to the problem of threads running forever is to have the
run-time system request a clock signal (interrupt) once a second to give it control,
but this, too, is crude and messy to program. Periodic clock interrupts at a higher
frequency are not always possible, and even if they are, the total overhead may be
substantial. Furthermore, a thread might also need a clock interrupt, interfering
with the run-time system’s use of the clock.
Another, and really the most devastating, argument against user-level threads is
that programmers typically want threads precisely in applications where threads
SEC. 2.2
THREADS
111
block often, as, for example, in a multithreaded Web server. These threads are constantly making system calls. Once a trap has occurred to the kernel to carry out the
system call, it is hardly any more work for the kernel to switch threads if the old
one has blocked, and having the kernel do this eliminates the need for constantly
making select system calls to see if read system calls are safe. For applications that
are CPU bound and rarely block, what is the point of having threads at all? No one
would seriously propose computing the first n prime numbers or playing chess
using threads because there is nothing to be gained by doing it that way.
2.2.5 Implementing Threads in the Kernel
Now let us consider having the kernel know about and manage the threads. As
shown in Fig. 2-15(b), there is now no need for a run-time system or thread table in
each process. Instead, the kernel has a thread table that keeps track of all the
threads in the system. When a thread wants to create a new thread or destroy an
existing thread, it makes a kernel call, which then does the creation or destruction
by updating the kernel thread table.
The kernel’s thread table holds each thread’s registers, state, and other information. The information is the same as with user-level threads, but now kept in the
kernel instead of in user space (inside the run-time system). This information is a
subset of the information that traditional kernels maintain about their singlethreaded processes, that is, the process state. In addition, the kernel also maintains
the traditional process table to keep track of processes.
All calls that might block a thread are implemented as system calls, at considerably greater cost than a call to a run-time system procedure. When a thread
blocks, the kernel may choose to run either another thread from the same process
(if one is ready) or a thread from a different process. With user-level threads, the
run-time system keeps running threads from its own process until the kernel takes
the CPU away from it (or there are no ready threads left to run).
Due to the relatively greater cost of creating and destroying threads in the kernel, some systems take an environmentally correct approach and recycle their
threads. When a thread is destroyed, it is marked as not runnable, but its kernel
data structures are not otherwise affected. Later, when a new thread must be created, an old thread is reactivated, saving some overhead. Thread recycling is also
possible for user-level threads, but since the thread-management overhead is much
smaller, there is less incentive to do this.
Kernel threads do not require any new, nonblocking system calls. In addition,
if one thread in a process causes a page fault, the kernel can easily check to see if
the process has any other runnable threads, and if so, run one of them while waiting for the required page to be brought in from the disk. Their main disadvantage is
that the cost of a system call is substantial, so if thread operations (creation, termination, etc.) are common, much more overhead will be incurred.
112
PROCESSES AND THREADS
CHAP. 2
While kernel threads solve some problems, they do not solve all problems. For
example, we still have to think about what happens when a multithreaded process
forks. Does the new process have as many threads as the old one did, or does it
have just one? In many cases, the best choice depends on what the process is planning to do next. If it is going to call exec to start a new program, probably one
thread is the correct choice, but if it continues to execute, reproducing all the
threads is probably best.
Another issue with threads is signals. Remember that signals are sent to processes, not to threads, at least in the classical model. When a signal comes in, which
thread should handle it? Possibly threads could register their interest in certain signals, so when a signal came in it would be given to the thread that said it wants it.
On Linux, for instance, a signal may be handled by any thread and the lucky winner is selected by the operating system, but we can simply block the signal on all
threads except one. If two or more threads register for the same signal, the operating system picks a thread (say, at random) and lets it handle the signal. Anyway,
these are only some of the problems threads introduce, and there are more. Unless
the programmer is very careful, it is easy to make mistakes.
2.2.6 Hybrid Implementations
Various ways have been investigated to try to combine the advantages of userlevel threads with kernel-level threads. One way is use kernel-level threads and
then multiplex user-level threads onto some or all of them, as shown in Fig. 2-16.
When this approach is used, the programmer can determine how many kernel
threads to use and how many user-level threads to multiplex on each one. This
model gives the ultimate flexibility.
Multiple user threads
on a kernel thread
User
space
Kernel
Kernel thread
Kernel
space
Figure 2-16. Multiplexing user-level threads onto kernel-level threads.
With this approach, the kernel is aware of only the kernel-level threads and
thus schedules those. Some of those threads may have multiple user-level threads
SEC. 2.2
113
THREADS
multiplexed on top of them. These user-level threads are created, destroyed, and
scheduled just like user-level threads in a process that runs on an operating system
without multithreading capability. In this model, each kernel-level thread has some
set of user-level threads that take turns using it.
2.2.7 Making Single-Threaded Code Multithreaded
Many existing programs were written for single-threaded processes. Converting these to multithreading is much trickier than it may at first appear. Below we
will examine just a few of the pitfalls.
As a start, the code of a thread normally consists of multiple procedures, just
like a process. These may have local variables, global variables, and parameters.
Local variables and parameters do not cause any trouble, but variables that are global to a thread but not global to the entire program are a problem. These are variables that are global in the sense that many procedures within the thread use them
(as they might use any global variable), but other threads should logically leave
them alone.
As an example, consider the errno variable maintained by UNIX. When a
process (or a thread) makes a system call that fails, the error code is put into errno.
In Fig. 2-17, thread 1 executes the system call access to find out if it has permission to access a certain file. The operating system returns the answer in the global
variable errno. After control has returned to thread 1, but before it has a chance to
read errno, the scheduler decides that thread 1 has had enough CPU time for the
moment and switches to thread 2. Thread 2 executes an open call that fails, which
causes errno to be overwritten and thread 1’s access code to be lost forever. When
thread 1 starts up later, it will read the wrong value and behave incorrectly.
Time
Thread 1
Thread 2
Access (errno set)
Open (errno overwritten)
Errno inspected
Figure 2-17. Conflicts between threads over the use of a global variable.
Various solutions to this problem are possible. One is to prohibit global variables altogether. However worthy this ideal may be, it conflicts with much existing
114
PROCESSES AND THREADS
CHAP. 2
software. Another is to assign each thread its own private global variables, as
shown in Fig. 2-18. In this way, each thread has its own private copy of errno and
other global variables, so conflicts are avoided. In effect, this decision creates a
new scoping level, variables visible to all the procedures of a thread (but not to
other threads), in addition to the existing scoping levels of variables visible only to
one procedure and variables visible everywhere in the program.
Thread 1's
code
Thread 2's
code
Thread 1's
stack
Thread 2's
stack
Thread 1's
globals
Thread 2's
globals
Figure 2-18. Threads can have private global variables.
Accessing the private global variables is a bit tricky, however, since most programming languages have a way of expressing local variables and global variables,
but not intermediate forms. It is possible to allocate a chunk of memory for the globals and pass it to each procedure in the thread as an extra parameter. While hardly
an elegant solution, it works.
Alternatively, new library procedures can be introduced to create, set, and read
these threadwide global variables. The first call might look like this:
create global("bufptr");
It allocates storage for a pointer called bufptr on the heap or in a special storage
area reserved for the calling thread. No matter where the storage is allocated, only
the calling thread has access to the global variable. If another thread creates a global variable with the same name, it gets a different storage location that does not
conflict with the existing one.
Two calls are needed to access global variables: one for writing them and the
other for reading them. For writing, something like
set global("bufptr", &buf);
will do. It stores the value of a pointer in the storage location previously created by
the call to create global. To read a global variable, the call might look like
SEC. 2.2
THREADS
115
bufptr = read global("bufptr");
It returns the address stored in the global variable, so its data can be accessed.
The next problem in turning a single-threaded program into a multithreaded
one is that many library procedures are not reentrant. That is, they were not
designed to have a second call made to any given procedure while a previous call
has not yet finished. For example, sending a message over the network may well be
programmed to assemble the message in a fixed buffer within the library, then to
trap to the kernel to send it. What happens if one thread has assembled its message
in the buffer, then a clock interrupt forces a switch to a second thread that
immediately overwrites the buffer with its own message?
Similarly, memory-allocation procedures such as malloc in UNIX, maintain
crucial tables about memory usage, for example, a linked list of available chunks
of memory. While malloc is busy updating these lists, they may temporarily be in
an inconsistent state, with pointers that point nowhere. If a thread switch happens
to occur while the tables are inconsistent and a new call comes in from a different
thread, an invalid pointer may be used, leading to a program crash. Fixing all these
problems effectively means rewriting the entire library. Doing so is a nontrivial
activity with a real possibility of introducing subtle errors.
A different solution is to provide each procedure with a wrapper that sets a bit
to mark the library as being in use. Any attempt for another thread to use a library
procedure while a previous call has not yet completed is blocked. Although this
approach can be made to work, it greatly eliminates potential parallelism.
Next, consider signals. Some signals are logically thread specific; others are
not. For example, if a thread calls alarm, it makes sense for the resulting signal to
go to the thread that made the call. However, when threads are implemented entirely in user space, the kernel does not even know about threads and can hardly direct
the signal to the right one. An additional complication occurs if a process may only
have one alarm pending at a time and several threads call alarm independently.
Other signals, such as keyboard interrupt, are not thread specific. Who should
catch them? One designated thread? All the threads? Furthermore, what happens
if one thread changes the signal handlers without telling other threads about it?
And what happens if one thread wants to catch a particular signal (say, the user hitting CTRL-C), and another thread wants this signal to terminate the process? This
situation can arise if one or more threads run standard library procedures and others are user-written. Clearly, these wishes are incompatible. In general, signals are
difficult enough to manage even in a single-threaded environment. Going to a multithreaded environment does not make them any easier to handle.
One last problem introduced by threads is stack management. In many systems, when a process’ stack overflows, the kernel just provides that process with
more stack automatically. When a process has multiple threads, it must also have
multiple stacks. If the kernel is not aware of all these stacks, it cannot grow them
automatically upon stack fault. In fact, it may not even realize that a memory fault
is related to the growth of some thread’s stack.
116
PROCESSES AND THREADS
CHAP. 2
These problems are certainly not insurmountable, but they do show that just
introducing threads into an existing system without a fairly substantial system
redesign is not going to work at all. The semantics of system calls may have to be
redefined and libraries rewritten, at the very least. And all of these things must be
done in such a way as to remain backward compatible with existing programs for
the limiting case of a process with only one thread. For additional information
about threads, see Cook (2008) and Rodrigues et al. (2010).
2.3 EVENT-DRIVEN SERVERS
In the previous section, we have seen two possible designs for a Web server: a
fast multithreaded one and a slow single-threaded one. Suppose that threads are not
available or not desirable but the system designers find the performance loss due to
single threading, as described so far, unacceptable. If nonblocking versions of system calls, such as read, are available, a third approach is possible. When a request
comes in, the one and only thread examines it. If it can be satisfied from the cache,
fine, but if not, a nonblocking disk operation is started.
The server records the state of the current request in a table and then goes and
gets the next event. The next event may either be a request for new work or a reply
from the disk about a previous operation. If it is new work, that work is started. If
it is a reply from the disk, the relevant information is fetched from the table and the
reply processed. With nonblocking disk I/O, a reply probably will have to take the
form of a signal or interrupt.
In this design, the ‘‘sequential process’’ model that we had in the first two
cases is lost. The state of the computation must be explicitly saved and restored in
the table every time the server switches from working on one request to another. In
effect, we are simulating the threads and their stacks the hard way. A design like
this, in which each computation has a saved state, and there exists some set of
events that can occur to change the state, is called a finite-state machine. This
concept is widely used throughout computer science.
In fact, it is very popular in high-throughput servers where even threads are
considered too expensive and instead an event-driven programming paradigm is
used. By implementing the server as a finite state machine that responds to events
(e.g., the availability of data on a socket) and interacting with the operating system
using non-blocking (or asynchronous) system calls, the implementation can be
very efficient. Every event leads to a burst of activity, but it never blocks.
Fig. 2-19 shows a pseudo-code example of an event-driven thank-you server
(the server thanks every client that sends it a message) that uses the select call to
monitor multiple network connections (line 17). The select determines which file
descriptors are ready for receiving or sending data and, looping over them, receives
all the messages it can and then tries to send thank-you messages on all corresponding connections that are ready to receive data. In case the server could not
SEC. 2.3
EVENT-DRIVEN SERVERS
117
0. /* Preliminaries:
1.
svrSock : the main server socket, bound to TCP port 12345
2.
toSend : database to track what data we still have to send to the client
3.
- toSend.put (fd, msg) will register that we need to send msg on fd
4.
- toSend.get (fd) returns the string we need to send msg on fd
5.
- toSend.destroy (fd) removes all information about fd from toSend */
6.
7. inFds
= { svrSock }
/* file descriptors to watch for incoming data */
8. outFds
={}
/* file descriptors to watch to see if sending is possible */
9. exceptFds = { }
/* file descriptors to watch for exception conditions (not used) */
10.
/* buffer in which to receive messages */
11. char msgBuf [MAX MSG SIZE]
12. char *thankYouMsg = "Thank you!" /* reply to send back */
13.
14. while (TRUE)
15. {
16.
/* block until some file descriptors are ready to be used */
17.
rdyIns, rdyOuts, rdyExcepts = select (inFds, outFds, exceptFds, NO TIMEOUT)
18.
19.
for (fd in rdyIns) /* iterate over all the connections that have something for us */
20.
{
21.
if (fd == svrSock)
/* a new connection from a client */
22.
{
23.
newSock = accept (svrSock)
/* create new socket for client */
24.
inFds = inFds F { newSock } /* must monitor it also */
25.
}
26.
else
27.
{
/* receive the message from the client */
28.
n = receive (fd, msgBuf, MAX MSG SIZE)
29.
printf ("Received: %s.0, msgBuf)
30.
31.
toSend.put (fd, thankYouMsg) /* must still send thankYouMsg on fd */
/* so must monitor this fd */
32.
outFds = outFds F { fd }
33.
}
34.
}
35.
for (fd in rdyOuts) /* iterate over all the connections that we can now thank */
36.
{
37
msg = toSend.get (fd)
/* see what we need to send on this connection */
38
n = send (fd, msg, strlen(msg))
39
if (n < strlen (thankYouMsg)
40
{
41
toSend.put (fd, msg+n) /* remaining characters to send next time*/
42
} else
43.
{
44.
toSend.destroy (fd)
45.
outFds = outFds \ { fd } /* we have thanked this one already */
46
}
47.
}
47. }
Figure 2-19. An event-driven thank-you server (pseudo code).
118
PROCESSES AND THREADS
CHAP. 2
send the full thank you message, it remembers which bytes it still needs to send, so
it can try again later, when there is more space available. We simplified the program to keep it relatively short, by means of pseudo code and not worrying about
errors or connections closing. Nevertheless, it illustrates that a single-threaded
event-driven server can handle many clients concurrently.
Most popular operating systems offer special, highly optimized event notification interfaces for asynchronous I/O that are much more efficient than select.
Well-known examples include the epoll system call on Linux, and the similar
kqueue interface on FreeBSD. Windows and Solaris have slightly different solutions. They all allow the server to monitor many network connections at once without blocking on any of them. Because of this Web servers such as nginx can comfortably handle ten thousand concurrent connections. This is no trivial feat and
even has its own name: the C10k problem.
Single-Threaded Versus Multi-Threaded Versus Event-driven Servers
Finally, let us compare the three different ways to build a server. It should now
be clear what threads have to offer. They make it possible to retain the idea of
sequential processes that make blocking calls (e.g., for disk I/O) and still achieve
parallelism. Blocking system calls make programming easier, and parallelism
improves performance. The single-threaded server retains the simplicity of blocking system calls but gives up performance.
The third approach, event-driven programming, also achieves high performance through parallelism but uses nonblocking calls and interrupts to do so. It
is considered harder to program. These models are summarized in Fig. 2-20.
Model
Characteristics
Threads
Parallelism, blocking system calls
Single-threaded process
No parallelism, blocking system calls
Finite-state machine/event-driven
Parallelism, nonblocking system calls, interrupts
Figure 2-20. Three ways to construct a server.
These three approaches to handle requests from a client apply not only to user
programs, but also to the kernel itself where concurrency is just as important for
performance. In fact, this is a good moment to point out that this book introduces
many operating system concepts with an emphasis on what they mean for user programs, but of course the operating system itself uses these concepts internally also
(and some are even more relevant to the operating system than to user programs).
Thus, the operating system kernel itself may consist of multithreaded or eventdriven software. For instance, the Linux kernel on modern Intel CPUs is a multithreaded operating system kernel. In contrast, MINIX 3 consists of many servers
implemented following the model of finite state machine and events.
SEC. 2.4
SYNCHRONIZATION AND INTERPROCESS COMMUNICATION
119
2.4 SYNCHRONIZATION AND INTERPROCESS COMMUNICATION
Processes frequently need to synchronize and communicate with other processes. For example, in a shell pipeline, the output of the first process must be passed to the second process, and so on down the line. Thus, there is a need for communication between processes, preferably in a well-structured way not using interrupts. In the following sections, we will look at some of the issues related to this
IPC (InterProcess Communication).
Very briefly, there are three issues here. The first was alluded to above: how
one process can pass information to another. The second has to do with making
sure two or more processes or threads do not get in each other’s way, for example,
two threads in an airline reservation system each trying to grab the last seat on a
plane for different customers. The third concerns proper sequencing when dependencies are present: if thread A produces data and thread B prints them, B has to
wait until A has produced some data before starting to print. We will examine all
three of these issues starting in the next section.
It is also important to mention that two of these issues apply to threads as well
as to processes with shared memory. The first one—passing information—is clearly easier for threads since they share a common address space by nature. However,
the other two—keeping out of each other’s hair and proper sequencing—are complicated for threads also. Below we will discuss the problems in the context of
processes, but please keep in mind that the same problems and solutions apply to
threads.
2.4.1 Race Conditions
In some operating systems, processes that are working together may share
some common storage that each one can read and write. The shared storage may be
in main memory (possibly in a kernel data structure) or it may be a shared file; the
location of the shared memory does not change the nature of the communication or
the problems that arise. To see how interprocess communication works in practice,
let us now consider a simple but common example: a print spooler. When a process
wants to print a file, it enters the file name in a special spooler directory. Another
process, the printer daemon, periodically checks to see if there are any files to be
printed, and if there are, it prints them and then removes their names from the
directory.
Imagine that our spooler directory has a very large number of slots, numbered
0, 1, 2, ..., each one capable of holding a file name. Also imagine that there are two
shared variables, out, which points to the next file to be printed, and in, which
points to the next free slot in the directory. These two variables might well be kept
in a two-word file available to all processes. At a certain instant, slots 0 –3 are
empty (the files have already been printed) and slots 4–6 are full (with the names
of files queued for printing). More or less simultaneously, processes A and B
decide they want to queue a file for printing. This situation is shown in Fig. 2-21.
120
PROCESSES AND THREADS
CHAP. 2
Spooler
directory
4
Process A
abc
5
prog.c
6
prog.n
7
out = 4
in = 7
Process B
Figure 2-21. Two processes want to access shared memory at the same time.
In jurisdictions where Murphy’s law† is applicable, the following could easily
happen. Process A reads in and stores the value, 7, in a local variable called
next free slot. Just then a clock interrupt occurs and the CPU decides that process A has run long enough, so it switches to process B. Process B also reads in and
also gets a 7. It, too, stores it in its local variable next free slot. At this instant,
both processes think that the next available slot is 7.
Process B now continues to run. It stores the name of its file in slot 7 and
updates in to be an 8. Then it goes off and does other things.
Eventually, process A runs again, starting from the place it left off. It looks at
next free slot, finds a 7 there, and writes its file name in slot 7, erasing the name
that process B just put there. Then it computes next free slot + 1, which is 8, and
sets in to 8. The spooler directory is now internally consistent, so the printer daemon will not notice anything wrong, but process B will never receive any output.
User B will hang around the printer for years, wistfully hoping for output that
never comes. Situations like this, where two or more processes are reading or writing some shared data and the final result depends on who runs precisely when, are
called race conditions. Debugging programs containing race conditions is no fun
at all. The results of most test runs are fine, but once in a blue moon something
weird and unexplained happens. Unfortunately, with increasing parallelism due to
increasing numbers of cores, race conditions are becoming more common.
2.4.2 Critical Regions
How do we avoid race conditions? The key to preventing trouble here and in
many other situations involving shared memory, shared files, and shared everything
else is to find some way to prohibit more than one process from reading and
† If something can go wrong, it will.
SEC. 2.4 SYNCHRONIZATION AND INTERPROCESS COMMUNICATION
121
writing the shared data at the same time. Put in other words, what we need is
mutual exclusion, that is, some way of making sure that if one process is using a
shared variable or file, the other processes will be excluded from doing the same
thing. The difficulty above occurred because process B started using one of the
shared variables before process A was finished with it. The choice of appropriate
operations for achieving mutual exclusion is a major design issue in any operating
system, and a subject that we will examine in great detail in the following sections.
The problem of avoiding race conditions can also be formulated in an abstract
way. Part of the time, a process is busy doing internal computations and other
things that do not lead to race conditions. However, sometimes a process has to
access shared memory or files, or do other critical things that can lead to races.
That part of the program where the shared memory is accessed is called the critical
region or critical section. If we could arrange matters such that no two processes
were ever in their critical regions at the same time, we could avoid races.
Although this requirement avoids race conditions, it is not sufficient for having
parallel processes cooperate correctly and efficiently using shared data. We need
four conditions to hold to have a good solution:
1. No two processes may be simultaneously inside their critical regions.
2. No assumptions may be made about speeds or the number of CPUs.
3. No process running outside its critical region may block any process.
4. No process should have to wait forever to enter its critical region.
In an abstract sense, the behavior that we want is shown in Fig. 2-22. Here
process A enters its critical region at time T 1 . A little later, at time T 2 process B
attempts to enter its critical region but fails because another process is already in
its critical region and we allow only one at a time. Consequently, B is temporarily
suspended until time T 3 when A leaves its critical region, allowing B to enter
immediately. Eventually B leaves (at T 4 ) and we are back to the original situation
with no processes in their critical regions.
2.4.3 Mutual Exclusion with Busy Waiting
In this section, we will examine various proposals for achieving mutual exclusion, so that while one process is busy updating shared memory in its critical
region, no other process will enter its critical region and cause trouble.
Disabling Interrupts
On a single-processor system, the simplest solution is to have each process disable all interrupts just after entering its critical region and re-enable them just
before leaving it. With interrupts disabled, no clock interrupts can occur. The CPU
122
PROCESSES AND THREADS
A enters critical region
CHAP. 2
A leaves critical region
Process A
B leaves
critical region
B enters
critical region
B attempts to
enter critical
region
Process B
B blocked
T1
T2
T3
T4
Time
Figure 2-22. Mutual exclusion using critical regions.
is only switched from process to process as a result of clock interrupt or other
interrupts, after all, and with interrupts turned off the CPU will not be switched to
another process. Thus, once a process has disabled interrupts, it can examine and
update the shared memory without fear that any other process will intervene and
make a mess of things.
This approach is generally unattractive because it is unwise to give user processes the power to turn off interrupts. What if one of them did it, and never turned
them on again? That could be the end of the system. Furthermore, if the system is
a multiprocessor (with two or more CPUs), disabling interrupts affects only the
CPU that executed the disable instruction. The other ones will continue running
and can access the shared memory.
On the other hand, it is frequently convenient for the kernel itself to disable
interrupts for a few instructions while it is updating variables or especially critical
lists. If an interrupt occurs while the list of ready processes, for example, is in an
inconsistent state, race conditions could occur. The conclusion is: disabling interrupts is often a useful technique within the operating system itself but is not appropriate as a general mutual exclusion mechanism for user processes. The kernel
should not disable interrupts for more than a few instructions lest it miss interrupts.
The possibility of achieving mutual exclusion by disabling interrupts—even
within the kernel—is becoming less every day due to the increasing number of
multicore chips even in low-end PCs. Two cores are already common, 4 are present
in many machines, and 8, 16, or 32 are not far behind. In a multicore (i.e., multiprocessor system), disabling the interrupts of one CPU does not prevent other
CPUs from interfering with operations the first CPU is performing. Consequently,
more sophisticated schemes are needed.
SEC. 2.4 SYNCHRONIZATION AND INTERPROCESS COMMUNICATION
123
Lock Variables
As a second attempt, let us look for a software solution. Consider having a single, shared (lock) variable, initially 0. When a process wants to enter its critical
region, it first tests the lock. If the lock is 0, the process sets it to 1 and enters the
critical region. If the lock is already 1, the process just waits until it becomes 0.
Thus, a 0 means that no process is in its critical region, and a 1 means that some
process is in its critical region.
Unfortunately, this idea contains exactly the same fatal flaw that we saw in the
spooler directory. Suppose that one process reads the lock and sees that it is 0.
Before it can set the lock to 1, another process is scheduled, runs, and sets the lock
to 1. When the first process runs again, it will also set the lock to 1, and two processes will be in their critical regions at the same time.
Now you might think that we could get around this problem by first reading
out the lock value, then checking it again just before storing into it, but that really
does not help. The race now occurs if the second process modifies the lock just
after the first process has finished its second check.
Strict Alternation
A third approach to the mutual exclusion problem is shown in Fig. 2-23. This
program fragment, like nearly all the others in this book, is written in C. C was
chosen here because real operating systems are virtually always written in C (or
occasionally C++), but hardly ever in languages like Java, Python, or Haskell. C is
powerful, efficient, and predictable, characteristics critical for writing operating
systems. Java, for example, is not predictable because it might run out of storage at
a critical moment and need to invoke the garbage collector to reclaim memory at a
most inopportune time. This cannot happen in C because there is no garbage collection in C. A quantitative comparison of C, C++, Java, and four other languages
is given by Prechelt (2000).
while (TRUE) {
while (turn != 0) { }
critical region( );
turn = 1;
noncritical region( );
}
(a)
/* loop */
while (TRUE) {
while (turn != 1) { }
critical region( );
turn = 0;
noncritical region( );
}
(b)
/* loop */
Figure 2-23. A proposed solution to the critical-region problem. (a) Process 0.
(b) Process 1. In both cases, be sure to note the semicolons terminating the while
statements.
In Fig. 2-23, the integer variable turn, initially 0, keeps track of whose turn it is
to enter the critical region and examine or update the shared memory. Initially,
124
PROCESSES AND THREADS
CHAP. 2
process 0 inspects turn, finds it to be 0, and enters its critical region. Process 1 also
finds it to be 0 and therefore sits in a tight loop continually testing turn to see when
it becomes 1. Continuously testing a variable until some value appears is called
busy waiting. It should usually be avoided, since it wastes CPU time. Only when
there is a reasonable expectation that the wait will be short is busy waiting used. A
lock that uses busy waiting is called a spin lock.
When process 0 leaves the critical region, it sets turn to 1, to allow process 1 to
enter its critical region. Suppose that process 1 finishes its critical region quickly,
so that both processes are in their noncritical regions, with turn set to 0. Now process 0 executes its whole loop quickly, exiting its critical region and setting turn to
1. At this point, turn is 1 and both processes are in their noncritical regions.
Suddenly, process 0 finishes its noncritical region and goes back to the top of
its loop. Unfortunately, it is not permitted to enter its critical region now, because
turn is 1 and process 1 is busy with its noncritical region. It hangs in its while loop
until process 1 sets turn to 0. Put differently, taking turns is not a good idea when
one of the processes is much slower than the other.
This situation violates condition 3 set out above: process 0 is being blocked by
a process not in its critical region. Going back to the spooler directory discussed
above, if we now associate the critical region with reading and writing the spooler
directory, process 0 would not be allowed to print another file because process 1
was doing something else.
In fact, this solution requires that the two processes strictly alternate in entering their critical regions, for example, in spooling files. Neither one would be permitted to spool two in a row. While this algorithm does avoid all races, it is not
really a serious candidate as a solution because it violates condition 3.
Peterson’s Solution
By combining the idea of taking turns with the idea of lock variables and warning variables, a Dutch mathematician, T. Dekker, was the first one to devise a software solution to the mutual exclusion problem that does not require strict alternation. For a discussion of Dekker’s algorithm, see Dijkstra (1965).
In 1981, G. L. Peterson discovered a much simpler way to achieve mutual exclusion, thus rendering Dekker’s solution effectively obsolete. Peterson’s algorithm
is shown in Fig. 2-24. This algorithm consists of two procedures written in ANSI
C, which means that function prototypes should be supplied for all the functions
defined and used. However, to save space, we will not show prototypes here or
later.
Before using the shared variables (i.e., before entering its critical region), each
process calls enter region with its own process number, 0 or 1, as parameter. This
call will cause it to wait, if need be, until it is safe to enter. After it has finished
with the shared variables, the process calls leave region to indicate that it is done
and to allow the other process to enter, if it so desires.
SEC. 2.4 SYNCHRONIZATION AND INTERPROCESS COMMUNICATION
#define FALSE 0
#define TRUE 1
#define N
2
/* number of processes */
int turn;
int interested[N];
/* whose turn is it? */
/* all values initially 0 (FALSE) */
void enter region(int process);
{
int other;
/* process is 0 or 1 */
}
125
/* number of the other process */
other = 1 < process;
/* the opposite of process */
interested[process] = TRUE;
/* show that you are interested */
turn = process;
/* set flag */
while (turn == process && interested[other] == TRUE) /* null statement */ ;
void leave region(int process)
{
interested[process] = FALSE;
}
/* process: who is leaving */
/* indicate departure from critical region */
Figure 2-24. Peterson’s solution for achieving mutual exclusion.
Let us see how this solution works. Initially neither process is in its critical
region. Now process 0 calls enter region. It indicates its interest by setting its
array element and sets turn to 0. Since process 1 is not interested, enter region
returns immediately. If process 1 now makes a call to enter region, it will hang
there until interested[0] goes to FALSE, an event that happens only when process 0
calls leave region to exit the critical region.
Now consider the case that both processes call enter region almost simultaneously. Both will store their process number in turn. Whichever store is done last
is the one that counts; the first one is overwritten and lost. Suppose that process 1
stores last, so turn is 1. When both processes come to the while statement, process
0 executes it zero times and enters its critical region. Process 1 loops and does not
enter its critical region until process 0 exits its critical region.
The TSL Instruction
Now let us look at a proposal that requires a little help from the hardware.
Some computers, especially those designed with multiple processors in mind, have
an instruction like
TSL RX,LOCK
(Test and Set Lock) that works as follows. It reads the contents of the memory
word lock into register RX and then stores a nonzero value at the memory address
126
PROCESSES AND THREADS
CHAP. 2
lock. The operations of reading the word and storing into it are guaranteed to be
indivisible—no other processor can access the memory word until the instruction is
finished. The CPU executing the TSL instruction locks the memory bus to prohibit
other CPUs from accessing memory until it is done.
It is important to note that locking the memory bus is very different from disabling interrupts. Disabling interrupts then performing a read on a memory word
followed by a write does not prevent a second processor on the bus from accessing
the word between the read and the write. In fact, disabling interrupts on processor
1 has no effect at all on processor 2. The only way to keep processor 2 out of the
memory until processor 1 is finished is to lock the bus, which requires a special
hardware facility (basically, a bus line asserting that the bus is locked and not available to processors other than the one that locked it).
To use the TSL instruction, we will use a shared variable, lock, to coordinate
access to shared memory. When lock is 0, any process may set it to 1 using the TSL
instruction and then read or write the shared memory. When it is done, the process
sets lock back to 0 using an ordinary move instruction.
How can this instruction be used to prevent two processes from simultaneously
entering their critical regions? The solution is given in Fig. 2-25. It shows a
four-instruction subroutine in a fictitious (but typical) assembly language. The first
instruction copies the old value of lock to the register and then sets lock to 1. Then
the old value is compared with 0. If it is nonzero, the lock was already set, so the
program just goes back to the beginning and tests it again. Sooner or later, it will
become 0 (when the process currently in its critical region is done with its critical
region), and the subroutine returns, with the lock set. Clearing the lock is very simple. The program just stores a 0 in lock. No special synchronization instructions
are needed.
enter region:
TSL REGISTER,LOCK
CMP REGISTER,#0
JNE enter region
RET
| copy lock to register and set lock to 1
| was lock zero?
| if it was not zero, lock was set, so loop
| return to caller; critical region entered
leave region:
MOVE LOCK,#0
RET
| store a 0 in lock
| return to caller
Figure 2-25. Entering and leaving a critical region using the TSL instruction.
One solution to the critical-region problem is now easy. Before entering its
critical region, a process calls enter region, which does busy waiting until the lock
is free; then it acquires the lock and returns. After leaving the critical region the
process calls leave region, which stores a 0 in lock. As with all solutions based on
SEC. 2.4 SYNCHRONIZATION AND INTERPROCESS COMMUNICATION
127
critical regions, the processes must call enter region and leave region at the correct times for the method to work. If one process cheats, the mutual exclusion will
fail. In other words, critical regions work only if the processes cooperate.
An alternative instruction to TSL is XCHG, which exchanges the contents of two
locations atomically, for example, a register and a memory word. The code is
shown in Fig. 2-26, and, as can be seen, is essentially the same as the solution with
TSL. All Intel x86 CPUs use XCHG instruction for low-level synchronization.
enter region:
MOVE REGISTER,#1
XCHG REGISTER,LOCK
CMP REGISTER,#0
JNE enter region
RET
| put a 1 in the register
| swap contents of register and lock variable
| was lock zero?
| if it was non zero, lock was set, so loop
| return to caller; critical region entered
leave region:
MOVE LOCK,#0
RET
| store a 0 in lock
| return to caller
Figure 2-26. Entering and leaving a critical region using the XCHG instruction.
2.4.4 Sleep and Wakeup
Both Peterson’s solution and the solutions using TSL or XCHG are correct, but
both have the defect of requiring busy waiting. In essence, what these solutions do
is this: when a process wants to enter its critical region, it checks to see if the entry
is allowed. If it is not, the process just sits in a tight loop burning the CPU while
waiting until it is.
Not only does this approach waste CPU time, but it can also have unexpected
effects. Consider a computer with two processes, H, with high priority, and L, with
low priority. The scheduling rules are such that H is run whenever it is in ready
state. At a certain moment, with L in its critical region, H becomes ready to run
(e.g., an I/O operation completes). H now begins busy waiting, but since L is never
scheduled while H is running, L never gets the chance to leave its critical region, so
H loops forever. This situation is sometimes referred to as a variant of the priority
inversion problem.
Now let us look at some interprocess communication primitives that block
instead of wasting CPU time when they are not allowed to enter their critical
regions. One of the simplest is the pair sleep and wakeup. Sleep is a system call
that causes the caller to block, that is, be suspended until another process wakes it
up. The wakeup call has one parameter, the process to be awakened. Alternatively,
both sleep and wakeup each have one parameter, a memory address used to match
up sleeps with wakeups.
128
PROCESSES AND THREADS
CHAP. 2
The Producer-Consumer Problem
As an example of how these primitives can be used, let us consider the producer-consumer problem (also known as the bounded-buffer problem). Two
processes share a common, fixed-size buffer. One of them, the producer, puts information into the buffer, and the other one, the consumer, takes it out. (It is also possible to generalize the problem to have m producers and n consumers, but we will
consider only the case of one producer and one consumer because this assumption
simplifies the solutions.)
Trouble arises when the producer wants to put a new item in the buffer, but it is
already full. The solution is for the producer to go to sleep, to be awakened when
the consumer has removed one or more items. Similarly, if the consumer wants to
remove an item from the buffer and sees that the buffer is empty, it goes to sleep
until the producer puts something in the buffer and wakes it up.
This approach sounds simple enough, but it leads to the same kinds of race
conditions we saw earlier with the spooler directory. To keep track of the number
of items in the buffer, we will need a variable, count. If the maximum number of
items the buffer can hold is N, the producer’s code will first test to see if count is N.
If it is, the producer will go to sleep; if it is not, the producer will add an item and
increment count.
The consumer’s code is similar: first test count to see if it is 0. If it is, go to
sleep; if it is nonzero, remove an item and decrement the counter. Each of the processes also tests to see if the other should be awakened, and if so, wakes it up. The
code for both producer and consumer is shown in Fig. 2-27.
To express system calls such as sleep and wakeup in C, we will show them as
calls to library routines. They are not part of the standard C library but presumably
would be made available on any system that actually had these system calls. The
procedures insert item and remove item, which are not shown, handle the bookkeeping of putting items into the buffer and taking items out of the buffer.
Now let us get back to the race condition. It can occur because access to count
is unconstrained. As a consequence, the following situation could possibly occur.
The buffer is empty and the consumer has just read count to see if it is 0. At that
instant, the scheduler decides to stop running the consumer temporarily and start
running the producer. The producer inserts an item in the buffer, increments count,
and notices that it is now 1. Reasoning that count was just 0, and thus the consumer
must be sleeping, the producer calls wakeup to wake the consumer up.
Unfortunately, the consumer is not yet logically asleep, so the wakeup signal is
lost. When the consumer next runs, it will test the value of count it previously read,
find it to be 0, and go to sleep. Sooner or later the producer will fill up the buffer
and also go to sleep. Both will sleep forever.
The essence of the problem here is that a wakeup sent to a process that is not
(yet) sleeping is lost. If it were not lost, everything would work. A quick fix is to
modify the rules to add a wakeup waiting bit to the picture. When a wakeup is
SEC. 2.4 SYNCHRONIZATION AND INTERPROCESS COMMUNICATION
#define N 100
int count = 0;
129
/* number of slots in the buffer */
/* number of items in the buffer */
void producer(void)
{
int item;
while (TRUE) {
item = produce item( );
if (count == N) sleep( );
inser t item(item);
count = count + 1;
if (count == 1) wakeup(consumer);
}
/* repeat forever */
/* generate next item */
/* if buffer is full, go to sleep */
/* put item in buffer */
/* increment count of items in buffer */
/* was buffer empty? */
}
void consumer(void)
{
int item;
while (TRUE) {
if (count == 0) sleep( );
item = remove item( );
count = count < 1;
if (count == N < 1) wakeup(producer);
consume item(item);
}
/* repeat forever */
/* if buffer is empty, got to sleep */
/* take item from buffer */
/* decrement count of items in buffer */
/* was buffer full? */
/* print item */
}
Figure 2-27. The producer-consumer problem with a fatal race condition.
sent to a process that is still awake, this bit is set. Later, when the process tries to
go to sleep, if the wakeup waiting bit is on, it will be turned off, but the process
will stay awake. The wakeup waiting bit is a piggy bank for storing wakeup signals. The consumer clears the wakeup waiting bit in every iteration of the loop.
While the wakeup waiting bit saves the day in this simple example, it is easy to
construct examples with three or more processes in which one wakeup waiting bit
is insufficient. We could make another patch and add a second wakeup waiting bit,
or maybe 32 or 64 of them, but in principle the problem is still there.
2.4.5 Semaphores
This was the situation in 1965, when E. W. Dijkstra (1965) suggested using an
integer variable to count the number of wakeups saved for future use. In his initial
proposal, a new variable type, which he called a semaphore, was introduced. A
130
PROCESSES AND THREADS
CHAP. 2
semaphore could have the value 0, indicating that no wakeups were saved, or some
positive value if one or more wakeups were pending.
Dijkstra proposed having two operations on semaphores, now usually called
down and up (generalizations of sleep and wakeup, respectively). The down operation on a semaphore checks to see if the value is greater than 0. If so, it decrements the value (i.e., uses up one stored wakeup) and just continues. If the value is
0, the process is put to sleep without completing the down for the moment. Checking the value, changing it, and possibly going to sleep, are all done as a single, indivisible atomic action. It is guaranteed that once a semaphore operation has
started, no other process can access the semaphore until the operation has completed or blocked. This atomicity is absolutely essential to solving synchronization
problems and avoiding race conditions. Atomic actions, in which a group of related
operations are either all performed without interruption or not performed at all, are
extremely important in many other areas of computer science as well.
The up operation increments the value of the semaphore addressed. If one (or
more) processes were sleeping on that semaphore, unable to complete an earlier
down operation, one of them is chosen by the system (e.g., at random) and is allowed to complete its down. After an up on a semaphore with processes sleeping
on it, the semaphore will still have the value of 0. However, there will be one fewer process sleeping on it. The operation of incrementing the semaphore and waking
up one process is also indivisible. No process ever blocks doing an up, just as no
process ever blocks doing a wakeup in the earlier model.
As an aside, in Dijkstra’s original paper, he used the names P and V instead of
down and up, respectively. Since these have no mnemonic significance to people
who do not speak Dutch and only marginal significance to those people who do—
Proberen (try) and Verhogen (raise, make higher)—we will use the terms down and
up instead. These were first introduced in the Algol 68 programming language.
Solving the Producer-Consumer Problem Using Semaphores
Semaphores solve the lost-wakeup problem, as shown in Fig. 2-28. To make
them work correctly, it is essential that they be implemented in an indivisible way.
The normal way is to implement up and down as system calls, with the operating
system briefly disabling all interrupts while it is testing the semaphore, updating it,
and putting the process to sleep, if necessary. As all of these actions take only a
few instructions, no harm is done in disabling interrupts. If multiple CPUs are
being used, each semaphore should be protected by a lock variable, with the TSL or
XCHG instructions used to make sure that only one CPU at a time examines the
semaphore.
Be sure you understand that using TSL or XCHG to prevent several CPUs from
accessing the semaphore at the same time is quite different from the producer or
consumer busy waiting for the other to empty or fill the buffer. The semaphore
operation will take only a few nanoseconds, whereas the producer or consumer
might take arbitrarily long.
SEC. 2.4 SYNCHRONIZATION AND INTERPROCESS COMMUNICATION
#define N 100
typedef int semaphore;
semaphore mutex = 1;
semaphore empty = N;
semaphore full = 0;
131
/* number of slots in the buffer */
/* semaphores are a special kind of int */
/* controls access to critical region */
/* counts empty buffer slots */
/* counts full buffer slots */
void producer(void)
{
int item;
while (TRUE) {
item = produce item( );
down(&empty);
down(&mutex);
inser t item(item);
up(&mutex);
up(&full);
}
/* TRUE is the constant 1 */
/* generate something to put in buffer */
/* decrement empty count */
/* enter critical region */
/* put new item in buffer */
/* leave critical region */
/* increment count of full slots */
}
void consumer(void)
{
int item;
while (TRUE) {
down(&full);
down(&mutex);
item = remove item( );
up(&mutex);
up(&empty);
consume item(item);
}
/* infinite loop */
/* decrement full count */
/* enter critical region */
/* take item from buffer */
/* leave critical region */
/* increment count of empty slots */
/* do something with the item */
}
Figure 2-28. The producer-consumer problem using semaphores.
This solution uses three semaphores: one called full for counting the number of
slots that are full, one called empty for counting the number of slots that are empty,
and one called mutex to make sure the producer and consumer do not access the
buffer at the same time. Full is initially 0, empty is initially equal to the number of
slots in the buffer, and mutex is initially 1. Semaphores that are initialized to 1 and
used by two or more processes to ensure that only one of them can enter its critical
region at the same time are called binary semaphores. If each process does a
down just before entering its critical region and an up just after leaving it, mutual
exclusion is guaranteed.
Now that we have a good interprocess communication and synchronization
primitive at our disposal, let us go back and again look at the interrupt sequence of
132
PROCESSES AND THREADS
CHAP. 2
Fig. 2-5. In a system using semaphores, the natural way to hide interrupts is to
have a semaphore, initially set to 0, associated with each I/O device. Just after
starting an I/O device, the managing process does a down on the associated semaphore, thus blocking immediately. When the interrupt comes in, the interrupt handler then does an up on the associated semaphore, which makes the relevant process
ready to run again. In this model, step 5 in Fig. 2-5 consists of doing an up on the
device’s semaphore, so that in step 6 the scheduler will be able to run the device
manager. Of course, if several processes are now ready, the scheduler may choose
to run an even more important process next. We will look at some of the algorithms
used for scheduling later on in this chapter.
In the example of Fig. 2-28, we have actually used semaphores in two different
ways. This difference is important enough to make explicit. The mutex semaphore
is used for mutual exclusion. It is designed to guarantee that only one process at a
time will be reading or writing the buffer and the associated variables. This mutual
exclusion is required to prevent chaos. We will study mutual exclusion and how to
achieve it in the next section.
The other use of semaphores is for synchronization. The full and empty semaphores are needed to guarantee that certain event sequences do or do not occur. In
this case, they ensure that the producer stops running when the buffer is full, and
that the consumer stops running when it is empty. This use is different from mutual
exclusion.
The Readers and Writers Problem
The producers-consumers problem is useful for modeling two processes (or
threads) that exchange blocks of data while sharing a buffer. Another famous problem is the readers and writers problem (Courtois et al., 1971), which models access
to a database. Imagine, for example, an airline reservation system, with many competing processes wishing to read and write it. It is acceptable to have multiple processes reading the database at the same time, but if one process is updating (writing)
the database, no other processes may have access to the database, not even readers.
The question is how do you program the readers and the writers? One solution is
shown in Fig. 2-29.
In this solution, the first reader to get access to the database does a down on the
semaphore db. Subsequent readers merely increment a counter, rc. As readers
leave, they decrement the counter, and the last to leave does an up on the semaphore, allowing a blocked writer, if there is one, to get in.
The solution presented here implicitly contains a subtle decision worth noting.
Suppose that while a reader is using the database, another reader comes along.
Since having two readers at the same time is not a problem, the second reader is
admitted. Additional readers can also be admitted if they come along.
Now suppose a writer shows up. The writer may not be admitted to the database, since writers must have exclusive access, so the writer must be suspended.
SEC. 2.4 SYNCHRONIZATION AND INTERPROCESS COMMUNICATION
typedef int semaphore;
semaphore mutex = 1;
semaphore db = 1;
int rc = 0;
/* use your imagination */
/* controls access to rc */
/* controls access to the database */
/* # of processes reading or wanting to */
void reader(void)
{
while (TRUE) {
down(&mutex);
rc = rc + 1;
if (rc == 1) down(&db);
up(&mutex);
read data base( );
down(&mutex);
rc = rc < 1;
if (rc == 0) up(&db);
up(&mutex);
use data read( );
}
}
/* repeat forever */
/* get exclusive access to rc */
/* one reader more now */
/* if this is the first reader ... */
/* release exclusive access to rc */
/* access the data */
/* get exclusive access to rc */
/* one reader fewer now */
/* if this is the last reader ... */
/* release exclusive access to rc */
/* noncritical region */
void writer(void)
{
while (TRUE) {
think up data( );
down(&db);
write data base( );
up(&db);
}
}
/* repeat forever */
/* noncritical region */
/* get exclusive access */
/* update the data */
/* release exclusive access */
133
Figure 2-29. A solution to the readers and writers problem.
Later, additional readers show up. As long as at least one reader is still active, subsequent readers are admitted. As a consequence of this strategy, as long as there is
a steady supply of readers, they will all get in as soon as they arrive. The writer
will be kept suspended until no reader is present. If a new reader arrives, say, every
2 sec, and each reader takes 5 sec to do its work, the writer will never get in. Obviously, this is not a satisfactory situation.
To avoid this situation, the program could be written slightly differently: when
a reader arrives and a writer is waiting, the reader is suspended behind the writer
instead of being admitted immediately. In this way, a writer has to wait for readers
that were active when it arrived to finish but does not have to wait for readers that
came along after it. The disadvantage of this solution is that it achieves less concurrency and thus lower performance. Courtois et al. present a solution that gives
priority to writers. For details, we refer you to the paper.
134
PROCESSES AND THREADS
CHAP. 2
2.4.6 Mutexes
When the semaphore’s ability to count is not needed, a simplified version of
the semaphore, called a mutex, is sometimes used. Mutexes are good only for managing mutual exclusion to some shared resource or piece of code. They are easy
and efficient to implement, which makes them especially useful in thread packages
that are implemented entirely in user space.
A mutex is a shared variable that can be in one of two states: unlocked or
locked. Consequently, only 1 bit is required to represent it, but in practice an integer often is used, with 0 meaning unlocked and all other values meaning locked.
Two procedures are used with mutexes. When a thread (or process) needs access
to a critical region, it calls mutex lock. If the mutex is currently unlocked (meaning that the critical region is available), the call succeeds and the calling thread is
free to enter the critical region.
On the other hand, if the mutex is already locked, the calling thread is blocked
until the thread in the critical region is finished and calls mutex unlock. If multiple threads are blocked on the mutex, one of them is chosen at random and allowed
to acquire the lock.
Because mutexes are so simple, they can easily be implemented in user space
provided that a TSL or XCHG instruction is available. The code for mutex lock and
mutex unlock for use with a user-level threads package are shown in Fig. 2-30.
The solution with XCHG is essentially the same.
mutex lock:
TSL REGISTER,MUTEX
CMP REGISTER,#0
JZE ok
CALL thread yield
JMP mutex lock
ok:
RET
| copy mutex to register and set mutex to 1
| was mutex zero?
| if it was zero, mutex was unlocked, so return
| mutex is busy; schedule another thread
| tr y again
| return to caller; critical region entered
mutex unlock:
MOVE MUTEX,#0
RET
| store a 0 in mutex
| return to caller
Figure 2-30. Implementation of mutex lock and mutex unlock.
The code of mutex lock is similar to the code of enter region of Fig. 2-25 but
with a crucial difference. When enter region fails to enter the critical region, it
keeps testing the lock repeatedly (busy waiting). Eventually, the clock runs out and
some other process is scheduled to run. Sooner or later the process holding the lock
gets to run and releases it.
With (user) threads, the situation is different because there is no clock that
stops threads that have run too long. Consequently, a thread that tries to acquire a
lock by busy waiting will loop forever and never acquire the lock because it never
allows any other thread to run and release the lock.
SEC. 2.4 SYNCHRONIZATION AND INTERPROCESS COMMUNICATION
135
That is where the difference between enter region and mutex lock comes in.
When the later fails to acquire a lock, it calls thread yield to give up the CPU to
another thread. Consequently there is no busy waiting. When the thread runs the
next time, it tests the lock again.
Since thread yield is just a call to the thread scheduler in user space, it is very
fast. As a consequence, neither mutex lock nor mutex unlock requires any kernel
calls. Using them, user-level threads can synchronize entirely in user space using
procedures that require only a handful of instructions.
The mutex system that we have described above is a bare-bones set of calls.
With all software, there is always a demand for more features, and synchronization
primitives are no exception. For example, sometimes a thread package offers a call
mutex trylock that either acquires the lock or returns a code for failure, but does
not block. This call gives the thread the flexibility to decide what to do next if there
are alternatives to just waiting.
There is a subtle issue that up until now we have glossed over but which is
worth making explicit. With a user-space threads package, there is no problem with
multiple threads having access to the same mutex, since all the threads operate in a
common address space. However, with most of the earlier solutions, such as Peterson’s algorithm and semaphores, there is an unspoken assumption that multiple
processes have access to at least some shared memory, perhaps only one word, but
something. If processes have disjoint address spaces, as we have consistently said,
how can they share the turn variable in Peterson’s algorithm, or semaphores or a
common buffer?
There are two answers. First, some of the shared data structures, such as the
semaphores, can be stored in the kernel and accessed only by means of system
calls. This approach eliminates the problem. Second, most modern operating systems (including UNIX and Windows) offer a way for processes to share some portion of their address space with other processes. In this way, buffers and other data
structures can be shared. In the worst case, if nothing else is possible, a shared file
can be used.
If two or more processes share most or all of their address spaces, the distinction between processes and threads becomes somewhat blurred but is nevertheless present. Two processes that share a common address space still have different
open files, alarm timers, and other per-process properties, whereas the threads
within a single process share them. And it is always true that multiple processes
sharing a common address space never have the efficiency of user-level threads
since the kernel is deeply involved in their management.
Futexes
With increasing parallelism, efficient synchronization and locking is very
important for performance. Spin locks (and mutexes implemented by busy waiting
in general) are fast if the wait is short, but waste CPU cycles if not. If there is much
136
PROCESSES AND THREADS
CHAP. 2
contention, it is therefore more efficient to block the process and let the kernel
unblock it only when the lock is free. Unfortunately, this has the inverse problem:
it works well under heavy contention, but continuously switching to the kernel is
expensive if there is very little contention to begin with. To make matters worse, it
may not be easy to predict the amount of lock contention. One nice solution that
tries to combine the best of both worlds is the futex, or ‘‘fast user space mutex.’’
A futex is a feature of Linux that implements basic locking (much like a
mutex) but avoids dropping into the kernel unless it really has to. Since switching
to the kernel and back is quite expensive, doing so improves performance considerably. While we focus our discussion on mutex-style locking, futexes are very versatile and used to implement a range of synchronization primitives, from mutexes
to condition variables. They are also a very low-level feature of the kernel that
most users will never use directly—instead they are wrapped by standard libraries
that offer higher-level primitives. It is only when you lift the hood that you see the
futex mechanism powering many different kinds of synchronization.
A futex is a construct supported by the kernel to let user space processes synchronize on shared events. It consists of two parts: a kernel service and a user library. The kernel service provides a ‘‘wait queue’’ that allows multiple processes to
wait on a lock. They will not run, unless the kernel explicitly unblocks them. For a
process to be put on the wait queue requires an (expensive) system call. If possible,
it should be avoided. In the absence of any contention, therefore, the futex works
entirely in user space. Specifically, the processes or threads share a common lock
variable—a fancy name for an integer in shared memory that serves as the lock.
Suppose we have multithreaded program and the lock is initially—1 which we
assume to mean that the lock is free. A thread may grab the lock by performing an
atomic ‘‘decrement and test’’ (atomic functions in Linux consist of inline assembly
wrapped in C functions and are defined in header files). Next, the thread inspects
the result to see whether or not the lock was free. If it was not in the locked state,
all is well and our thread has successfully grabbed the lock.
However, if the lock is held by another thread, our thread has to wait. In that
case, the futex library does not spin, but uses a system call to put the thread on the
wait queue in the kernel. Hopefully, the cost of the switch to the kernel is now justified, because the thread was blocked anyway. When a thread is done with the
lock, it releases the lock with an atomic ‘‘increment and test’’ and checks the result
to see if any processes are still blocked on the kernel wait queue. If so, it will let
the kernel know that it may wake up (unblock) one or more of these processes. In
other words, if there is no contention, the kernel is not involved at all.
Mutexes in Pthreads
Pthreads provides a number of functions for synchronizing threads. The basic
mechanism uses a mutex variable, which can be locked or unlocked, to guard each
critical region. The implementation of a mutex varies from operating system to
SEC. 2.4 SYNCHRONIZATION AND INTERPROCESS COMMUNICATION
137
operating system, but on Linux it is built on top of futexes. A thread wishing to enter a critical region first tries to lock the associated mutex. If the mutex is
unlocked, the thread can enter immediately and the lock is atomically set, preventing other threads from entering. If the mutex is already locked, the calling thread is
blocked until it is unlocked. If multiple threads are waiting on the same mutex,
when it is unlocked, only one of them is allowed to continue and relock it. These
locks are not mandatory. It is up to the programmer to make sure threads use them
correctly.
The major calls relating to mutexes are shown in Fig. 2-31. As you might
expect, mutexes can be created and destroyed. The calls for performing these operations are pthread mutex init and pthread mutex destroy, respectively. They can
also be locked—by pthread mutex lock—which tries to acquire the lock and
blocks if is already locked. There is also an option for trying to lock a mutex and
failing with an error code instead of blocking if it is already blocked. This call is
pthread mutex trylock. This call allows a thread to effectively do busy waiting if
that is ever needed. Finally, pthread mutex unlock unlocks a mutex and releases
exactly one thread if one or more are waiting on it. Mutexes can also have attributes, but these are used only for specialized purposes.
Thread call
Description
pthread mutex init
Create a mutex
pthread mutex destroy
Destroy an existing mutex
pthread mutex lock
Acquire a lock or block
pthread mutex trylock
Acquire a lock or fail
pthread mutex unlock
Release a lock
Figure 2-31. Some of the Pthreads’ calls relating to mutexes.
In addition to mutexes, Pthreads offers a second synchronization mechanism,
condition variables, discussed later. Mutexes are good for allowing or blocking
access to a critical region. Condition variables allow threads to block due to some
condition not being met. Almost always the two methods are used together. Let us
now look at the interaction of threads, mutexes, and condition variables in a bit
more detail.
As a simple example, consider the producer-consumer scenario again: one
thread puts things in a buffer and another one takes them out. If the producer discovers that there are no more free slots available in the buffer, it has to block until
one becomes available. Mutexes make it possible to do the check atomically without interference from other threads, but having discovered that the buffer is full, the
producer needs a way to block and be awakened later. This is what condition variables allow.
The most important calls related to condition variables are shown in Fig. 2-32.
As you would probably expect, there are calls to create and destroy condition
138
PROCESSES AND THREADS
CHAP. 2
variables. They can have attributes and there are various calls for managing the
attributes (not shown). The most important operations on condition variables are
pthread cond wait and pthread cond signal. The former blocks the calling
thread until some other thread signals it (using the latter call). The reasons for
blocking and waiting are not part of the waiting and signaling protocol, of course.
The blocking thread often is waiting for the signaling thread to do some work, release some resource, or perform some other activity. Only then can the blocking
thread continue. The condition variables allow this waiting and blocking to be done
atomically. The pthread cond broadcast call is used when there are multiple
threads potentially all blocked and waiting for the same signal.
Thread call
Description
pthread cond init
Create a condition variable
pthread cond destroy
Destroy a condition variable
pthread cond wait
Block waiting for a signal
pthread cond signal
Signal another thread and wake it up
pthread cond broadcast
Signal multiple threads and wake all of them
Figure 2-32. Some of the Pthreads calls relating to condition variables.
Condition variables and mutexes are always used together. The pattern is for
one thread to lock a mutex, then wait on a conditional variable when it cannot get
what it needs. Eventually another thread will signal it and it can continue. The
pthread cond wait call atomically unlocks the mutex it is holding. Then, upon
successful return, the mutex shall have been locked again and owned by the calling
thread. For this reason, the mutex is one of the parameters.
It is also worth noting that condition variables (unlike semaphores) have no
memory. If a signal is sent to a condition variable on which no thread is waiting,
the signal is lost. Programmers have to be careful not to lose signals.
As an example of how mutexes and condition variables are used, Fig. 2-33
shows a very simple producer-consumer problem with a single item buffer. When
the producer has filled the buffer, it must wait until the consumer empties it before
producing the next item. Similarly, when the consumer has removed an item, it
must wait until the producer has produced another one. While very simple, this
example illustrates the basic mechanisms. The statement that puts a thread to sleep
should always check the condition to make sure it is satisfied before continuing, as
the thread might have been awakened due to a UNIX signal or some other reason.
2.4.7 Monitors
With semaphores and mutexes interprocess communication looks easy, right?
Forget it. Look closely at the order of the downs before inserting or removing items
from the buffer in Fig. 2-28. Suppose that the two downs in the producer’s code
SEC. 2.4 SYNCHRONIZATION AND INTERPROCESS COMMUNICATION
#include <stdio.h>
#include <pthread.h>
#define MAX 1000000000
pthread mutex t the mutex;
pthread cond t condc, condp;
int buffer = 0;
void *producer(void *ptr)
{
int i;
139
/* how many numbers to produce */
/* used for signaling */
/* buffer used between producer and consumer */
/* produce data */
for (i= 1; i <= MAX; i++) {
pthread mutex lock(&the mutex); /* get exclusive access to buffer */
while (buffer != 0) pthread cond wait(&condp, &the mutex);
buffer = i;
/* put item in buffer */
/* wake up consumer */
pthread cond signal(&condc);
pthread mutex unlock(&the mutex); /* release access to buffer */
}
pthread exit(0);
}
void *consumer(void *ptr)
{
int i;
/* consume data */
for (i = 1; i <= MAX; i++) {
pthread mutex lock(&the mutex); /* get exclusive access to buffer */
while (buffer ==0 ) pthread cond wait(&condc, &the mutex);
buffer = 0;
/* take item from buffer (not shown) and reinitialize */
/* wake up producer */
pthread cond signal(&condp);
pthread mutex unlock(&the mutex); /* release access to buffer */
}
pthread exit(0);
}
int main(int argc, char **argv)
{
pthread t pro, con;
pthread mutex init(&the mutex, 0);
pthread cond init(&condc, 0);
pthread cond init(&condp, 0);
pthread create(&con, 0, consumer, 0);
pthread create(&pro, 0, producer, 0);
pthread join(pro, 0);
pthread join(con, 0);
pthread cond destroy(&condc);
pthread cond destroy(&condp);
pthread mutex destroy(&the mutex);
}
Figure 2-33. Using threads to solve the producer-consumer problem.
140
PROCESSES AND THREADS
CHAP. 2
were reversed in order, so mutex was decremented before empty instead of after it.
If the buffer were completely full, the producer would block, with mutex set to 0.
Consequently, the next time the consumer tried to access the buffer, it would do a
down on mutex, now 0, and block too. Both processes would stay blocked forever
and no more work would ever be done. This unfortunate situation is called a deadlock. We will study deadlocks in detail in Chap. 6.
This problem is pointed out to show how careful you must be when using semaphores. One subtle error and everything comes to a grinding halt. It is like programming in assembly language, only worse, because the errors are race conditions, deadlocks, and other forms of unpredictable and irreproducible behavior.
To make it easier to write correct programs, Brinch Hansen (1973) and Hoare
(1974) proposed a higher-level synchronization primitive called a monitor. Their
proposals differed slightly, as described below. A monitor is a collection of procedures, variables, and data structures that are all grouped together in a special kind
of module or package. Processes may call the procedures in a monitor whenever
they want to, but they cannot directly access the monitor’s internal data structures
from procedures declared outside the monitor. Figure 2-34 illustrates a monitor
written in an imaginary language, Pidgin Pascal. C cannot be used here because
monitors are a language concept and C does not have them.
monitor example
integer i;
condition c;
procedure producer( );
.
.
.
end;
procedure consumer( );
.
.
.
end;
end monitor;
Figure 2-34. A monitor.
Monitors have an important property that makes them useful for achieving
mutual exclusion: only one process can be active in a monitor at any instant. Monitors are a programming-language construct, so the compiler knows they are special
and can handle calls to monitor procedures differently from other procedure calls.
Typically, when a process calls a monitor procedure, the first few instructions of
the procedure will check to see if any other process is currently active within the
monitor. If so, the calling process will be suspended until the other process has left
the monitor. If no other process is using the monitor, the calling process may enter.
SEC. 2.4 SYNCHRONIZATION AND INTERPROCESS COMMUNICATION
141
It is up to the compiler to implement mutual exclusion on monitor entries, but
a common way is to use a mutex or a binary semaphore. Because the compiler, not
the programmer, is arranging for the mutual exclusion, it is much less likely that
something will go wrong. In any event, the person writing the monitor does not
have to be aware of how the compiler arranges for mutual exclusion. It is sufficient
to know that by turning all the critical regions into monitor procedures, no two
processes will ever execute their critical regions at the same time.
Although monitors provide an easy way to achieve mutual exclusion, as we
have seen above, that is not enough. We also need a way for processes to block
when they cannot proceed. In the producer-consumer problem, it is easy enough to
put all the tests for buffer-full and buffer-empty in monitor procedures, but how
should the producer block when it finds the buffer full?
The solution lies again in the introduction of condition variables, along with
two operations on them, wait and signal. When a monitor procedure discovers that
it cannot continue (e.g., the producer finds the buffer full), it does a wait on some
condition variable, say, full. This action causes the calling process to block. It also
allows another process that had been previously prohibited from entering the monitor to enter now. We saw condition variables and these operations in the context of
Pthreads earlier.
This other process, for example, the consumer, can wake up its sleeping partner by doing a signal on the condition variable that its partner is waiting on. To
avoid having two active processes in the monitor at the same time, we need a rule
telling what happens after a signal. Hoare proposed letting the newly awakened
process run, suspending the other one. Brinch Hansen proposed finessing the problem by requiring that a process doing a signal must exit the monitor immediately.
In other words, a signal statement may appear only as the final statement in a monitor procedure. We will use Brinch Hansen’s proposal because it is conceptually
simpler and is also easier to implement. If a signal is done on a condition variable
on which several processes are waiting, only one of them, determined by the system scheduler, is revived.
As an aside, there is also a third solution, not proposed by either Hoare or
Brinch Hansen. This is to let the signaler continue to run and allow the waiting
process to start running only after the signaler has exited the monitor.
Condition variables are not counters. They do not accumulate signals for later
use the way semaphores do. Thus, if a condition variable is signaled with no one
waiting on it, the signal is lost forever. In other words, the wait must come before
the signal. This rule makes the implementation much simpler. In practice, it is not
a problem because it is easy to keep track of the state of each process with variables, if need be. A process that might otherwise do a signal can see that this operation is not necessary by looking at the variables.
A skeleton of the producer-consumer problem with monitors is given in
Fig. 2-35 in Pidgin Pascal. The advantage of using Pidgin Pascal here is that it is
pure and simple and follows the Hoare/Brinch Hansen model exactly.
142
PROCESSES AND THREADS
CHAP. 2
monitor ProducerConsumer
condition full, empty;
integer count;
procedure insert(item: integer);
begin
if count = N then wait(full);
insert item(item);
count := count + 1;
if count = 1 then signal(empty)
end;
function remove: integer;
begin
if count = 0 then wait(empty);
remove = remove item;
count := count < 1;
if count = N < 1 then signal(full)
end;
count := 0;
end monitor;
procedure producer;
begin
while true do
begin
item = produce item;
ProducerConsumer.insert(item)
end
end;
procedure consumer;
begin
while true do
begin
item = ProducerConsumer.remove;
consume item(item)
end
end;
Figure 2-35. An outline of the producer-consumer problem with monitors. Only
one monitor procedure at a time is active. The buffer has N slots.
You may be thinking that the operations wait and signal look similar to sleep
and wakeup, which we saw earlier had fatal race conditions. Well, they are very
similar, but with one crucial difference: sleep and wakeup failed because while one
process was trying to go to sleep, the other one was trying to wake it up. With
monitors, that cannot happen. The automatic mutual exclusion on monitor procedures guarantees that if, say, the producer inside a monitor procedure discovers that
the buffer is full, it will be able to complete the wait operation without having to
SEC. 2.4 SYNCHRONIZATION AND INTERPROCESS COMMUNICATION
143
worry about the possibility that the scheduler may switch to the consumer just
before the wait completes. The consumer will not even be let into the monitor at all
until the wait is finished and the producer has been marked as no longer runnable.
Although Pidgin Pascal is an imaginary language, some real programming languages also support monitors, although not always in the form designed by Hoare
and Brinch Hansen. One such language is Java. Java is an object-oriented language
that supports user-level threads and also allows methods (procedures) to be
grouped together into classes. By adding the keyword synchronized to a method
declaration, Java guarantees that once any thread has started executing that method,
no other thread will be allowed to start executing any other synchronized method
of that object. Without synchronized, there are no guarantees about interleaving.
A solution to the producer-consumer problem using monitors in Java is given
in Fig. 2-36. Our solution has four classes. The outer class, ProducerConsumer,
creates and starts two threads, p and c. The second and third classes, producer and
consumer, respectively, contain the code for the producer and consumer. Finally,
the class our monitor, is the monitor. It contains two synchronized threads that are
used for actually inserting items into the shared buffer and taking them out. Unlike
the previous examples, here we have the full code of insert and remove.
The producer and consumer threads are functionally identical to their counterparts in all our previous examples. The producer has an infinite loop generating
data and putting it into the common buffer. The consumer has an equally infinite
loop taking data out of the common buffer and doing some fun thing with it.
The interesting part of this program is the class our monitor, which holds the
buffer, the administration variables, and two synchronized methods. When the producer is active inside insert, it knows for sure that the consumer cannot be active
inside remove, making it safe to update the variables and the buffer without fear of
race conditions. The variable count keeps track of how many items are in the buffer. It can take on any value from 0 through and including N < 1. The variable lo is
the index of the buffer slot where the next item is to be fetched. Similarly, hi is the
index of the buffer slot where the next item is to be placed. It is permitted that
lo = hi, which means that either 0 items or N items are in the buffer. The value of
count tells which case holds.
Synchronized methods in Java differ from classical monitors in an essential
way: Java does not have condition variables built in. Instead, it offers two procedures, wait and notify, which are the equivalent of sleep and wakeup except that
when they are used inside synchronized methods, they are not subject to race conditions. In theory, the method wait can be interrupted, which is what the code surrounding it is all about. Java requires that the exception handling be made explicit.
For our purposes, just imagine that go to sleep is the way to go to sleep.
By making the mutual exclusion of critical regions automatic, monitors make
parallel programming much less error prone than using semaphores. Nevertheless,
they too have some drawbacks. It is not for nothing that our two examples of monitors were in Pidgin Pascal instead of C, as are the other examples in this book.
144
PROCESSES AND THREADS
public class ProducerConsumer {
static final int N = 100;
// constant giving the buffer size
static producer p = new producer( ); // instantiate a new producer thread
static consumer c = new consumer( ); // instantiate a new consumer thread
// instantiate a new monitor
static our monitor mon = new our monitor( );
public static void main(String args[ ]) {
p.star t( );
// star t the producer thread
c.star t( );
// star t the consumer thread
}
static class producer extends Thread {
public void run( ) { // run method contains the thread code
int item;
while (true) {
// producer loop
item = produce item( );
mon.inser t(item);
}
}
// actually produce
private int produce item( ) { ... }
}
static class consumer extends Thread {
public void run( ) { run method contains the thread code
int item;
while (true) {
// consumer loop
item = mon.remove( );
consume item (item);
}
}
private void consume item(int item) { ... } // actually consume
}
static class our monitor { // this is a monitor
private int buffer[ ] = new int[N];
private int count = 0, lo = 0, hi = 0; // counters and indices
public synchronized void insert(int val) {
if (count == N) go to sleep( ); // if the buffer is full, go to sleep
buffer [hi] = val; // inser t an item into the buffer
hi = (hi + 1) % N;
// slot to place next item in
count = count + 1;
// one more item in the buffer now
if (count == 1) notify( );
// if consumer was sleeping, wake it up
}
public synchronized int remove( ) {
int val;
if (count == 0) go to sleep( ); // if the buffer is empty, go to sleep
val = buffer [lo]; // fetch an item from the buffer
lo = (lo + 1) % N;
// slot to fetch next item from
count = count < 1;
// one few items in the buffer
if (count == N < 1) notify( ); // if producer was sleeping, wake it up
return val;
}
private void go to sleep( ) { try{wait( );} catch(InterruptedException exc) {};}
}
}
Fig. 2-36. A solution to the producer-consumer problem in Java.
CHAP. 2
SEC. 2.4 SYNCHRONIZATION AND INTERPROCESS COMMUNICATION
145
As we said earlier, monitors are a programming-language concept. The compiler must recognize them and arrange for the mutual exclusion somehow or other. C,
Pascal, and most other languages do not have monitors, so it is unreasonable to
expect their compilers to enforce any mutual exclusion rules. In fact, how could the
compiler even know which procedures were in monitors and which were not?
These same languages do not have semaphores either, but adding semaphores
is easy: all you need to do is add two short assembly-code routines to the library to
issue the up and down system calls. The compilers do not even have to know that
they exist. Of course, the operating systems have to know about the semaphores,
but at least if you have a semaphore-based operating system, you can still write the
user programs for it in C or C++ (or even assembly language if you are masochistic enough). With monitors, you need a language that has them built in.
Another problem with monitors, and also with semaphores, is that they were
designed for solving the mutual exclusion problem on one or more CPUs that all
have access to a common memory. By putting the semaphores in the shared memory and protecting them with TSL or XCHG instructions, we can avoid races. When
we move to a distributed system consisting of multiple CPUs, each with its own
private memory and connected by a local area network, these primitives become
inapplicable. The conclusion is that semaphores are too low level and monitors are
not usable except in a few programming languages. Also, none of the primitives
allow information exchange between machines. Something else is needed.
2.4.8 Message Passing
That something else is message passing. This method of interprocess communication uses two primitives, send and receive, which, like semaphores and unlike
monitors, are system calls rather than language constructs. As such, they can easily
be put into library procedures, such as
send(destination, &message);
and
receive(source, &message);
The former call sends a message to a given destination and the latter one receives a
message from a given source (or from ANY, if the receiver does not care). If no
message is available, the receiver can block until one arrives. Alternatively, it can
return immediately with an error code.
Design Issues for Message-Passing Systems
Message-passing systems have many problems and design issues that do not
arise with semaphores or with monitors, especially if the communicating processes
are on different machines connected by a network. For example, messages can be
146
PROCESSES AND THREADS
CHAP. 2
lost by the network. To guard against lost messages, the sender and receiver can
agree that as soon as a message has been received, the receiver will send back a
special acknowledgement message. If the sender has not received the acknowledgement within a certain time interval, it retransmits the message.
Now consider what happens if the message is received correctly, but the acknowledgement back to the sender is lost. The sender will retransmit the message,
so the receiver will get it twice. It is essential that the receiver be able to distinguish a new message from the retransmission of an old one. This problem is solved
by putting consecutive sequence numbers in each original message. If the receiver
gets a message bearing the same sequence number as the previous message, it
knows that the message is a duplicate that can be ignored. Successfully communicating in the face of unreliable message passing is a major part of the study of
computer networks. For more information, see Tanenbaum et al. (2020).
Message systems also have to deal with the question of how processes are
named, so that the process specified in a send or receive call is unambiguous.
Authentication is also an issue in message systems: how can the client tell that it
is communicating with the real file server, and not with an imposter?
At the other end of the spectrum, there are also design issues that are important
when the sender and receiver are on the same machine. One of these is performance. Copying messages from one process to another is always slower than
doing a semaphore operation or entering a monitor. Much work has gone into making message passing efficient.
The Producer-Consumer Problem with Message Passing
Now let us see how the producer-consumer problem can be solved with message passing and no shared memory. A solution is given in Fig. 2-37. We assume
that all messages are the same size and that messages sent but not yet received are
buffered automatically by the operating system. In this solution, a total of N messages is used, analogous to the N slots in a shared-memory buffer. The consumer
starts out by sending N empty messages to the producer. Whenever the producer
has an item to give to the consumer, it takes an empty message and sends back a
full one. In this way, the total number of messages in the system remains constant
in time, so they can be stored in a given amount of memory known in advance.
If the producer works faster than the consumer, all the messages will end up
full, waiting for the consumer; the producer will be blocked, waiting for an empty
to come back. If the consumer works faster, then the reverse happens: all the messages will be empties waiting for the producer to fill them up; the consumer will be
blocked, waiting for a full message.
Many variants are possible with message passing. For starters, let us look at
how messages are addressed. One way is to assign each process a unique address
and have messages be addressed to processes. A different way is to invent a new
data structure, called a mailbox. A mailbox is a place to buffer a certain number
SEC. 2.4 SYNCHRONIZATION AND INTERPROCESS COMMUNICATION
#define N 100
/* number of slots in the buffer */
void producer(void)
{
int item;
message m;
/* message buffer */
while (TRUE) {
item = produce item( );
receive(consumer, &m);
build message(&m, item);
send(consumer, &m);
}
147
/* generate something to put in buffer */
/* wait for an empty to arrive */
/* construct a message to send */
/* send item to consumer */
}
void consumer(void)
{
int item, i;
message m;
for (i = 0; i < N; i++) send(producer, &m); /* send N empties */
while (TRUE) {
receive(producer, &m);
/* get message containing item */
/* extract item from message */
item = extract item(&m);
send(producer, &m);
/* send back empty reply */
/* do something with the item */
consume item(item);
}
}
Figure 2-37. The producer-consumer problem with N messages.
of messages, typically specified when the mailbox is created. When mailboxes are
used, the address parameters in the send and receive calls are mailboxes, not processes. When a process tries to send to a mailbox that is full, it is suspended until a
message is removed from that mailbox, making room for a new one.
For the producer-consumer problem, both the producer and consumer would
create mailboxes large enough to hold N messages. The producer would send messages containing actual data to the consumer’s mailbox, and the consumer would
send empty messages to the producer’s mailbox. When mailboxes are used, the
buffering mechanism is clear: the destination mailbox holds messages that have
been sent to the destination process but have not yet been accepted.
The other extreme from having mailboxes is to eliminate all buffering. When
this approach is taken, if the send is done before the receive, the sending process is
blocked until the receive happens, at which time the message can be copied directly from the sender to the receiver, with no buffering. Similarly, if the receive call is
done first, the receiver is blocked until a send happens. This strategy is often
148
PROCESSES AND THREADS
CHAP. 2
known as a rendezvous. It is easier to implement than a buffered message scheme
but is less flexible since the sender and receiver are forced to run in lockstep.
Message passing is commonly used in parallel programming systems. One
well-known message-passing system, for example, is MPI (Message-Passing
Interface). It is widely used for scientific computing. For more information about
it, see for example Gropp et al. (1994) and Snir et al. (1996).
2.4.9 Barriers
Our last synchronization mechanism is intended for groups of processes rather
than two-process producer-consumer type situations. Some applications are divided into phases and have the rule that no process may proceed into the next phase
until all processes are ready to proceed to the next phase. This behavior may be
achieved by placing a barrier at the end of each phase. When a process reaches
the barrier, it is blocked until all processes have reached the barrier. This allows
groups of processes to synchronize. Barrier operation is illustrated in Fig. 2-38.
Barrier
B
C
A
B
B
C
Barrier
Process
A
Barrier
A
D
D
Time
(a)
C
D
Time
Time
(b)
(c)
Figure 2-38. Use of a barrier. (a) Processes approaching a barrier. (b) All processes but one blocked at the barrier. (c) When the last process arrives at the barrier, all of them are let through.
In Fig. 2-38(a) we see four processes approaching a barrier. What this means is
that they are just computing and have not reached the end of the current phase yet.
After a while, the first process finishes all the computing required of it during the
first phase. It then executes the barrier primitive, generally by calling a library procedure. The process is then suspended. A little later, a second and then a third
process finish the first phase and also execute the barrier primitive. This situation is
illustrated in Fig. 2-38(b). Finally, when the last process, C, hits the barrier, all the
processes are released, as shown in Fig. 2-38(c).
As an example of a problem requiring barriers, consider a common relaxation
problem in physics or engineering. There is typically a matrix that contains some
SEC. 2.4 SYNCHRONIZATION AND INTERPROCESS COMMUNICATION
149
initial values. The values might represent temperatures at various points on a sheet
of metal. The idea might be to calculate how long it takes for the effect of a flame
placed at one corner to propagate throughout the sheet.
Starting with the current values, a transformation is applied to the matrix to get
the second version of the matrix, for example, by applying the laws of
thermodynamics to see what all the temperatures are 6T later. Then the process is
repeated over and over, giving the temperatures at the sample points as a function
of time as the sheet heats up. The algorithm produces a sequence of matrices over
time, each one for a given point in time.
Now imagine that the matrix is very large (for example, 1 million by 1 million), so that parallel processes are needed (possibly on a multiprocessor) to speed
up the calculation. Different processes work on different parts of the matrix, calculating the new matrix elements from the old ones according to the laws of physics.
However, no process may start on iteration n + 1 until iteration n is complete, that
is, until all processes have finished their current work. The way to achieve this goal
is to program each process to execute a barrier operation after it has finished its
part of the current iteration. When all of them are done, the new matrix (the input
to the next iteration) will be finished, and all processes will be simultaneously
released to start the next iteration.
It is worth mentioning that special low-level barriers are popular also to synchronize memory operations. Such barriers, unimaginatively called memory barriers or memory fences, enforce an order to guarantee that all memory operations
(to read or write memory) started before the barrier instruction will also finish
before the memory operations issued after the barrier. They are important because
modern CPUs execute instructions out of order and that may cause problems. For
instance, if instruction 2 does not depend on the result of instruction 1, the CPU
may start executing it ahead of time. After all, modern processors are superscalar
and have many execution units to perform calculations and memory accesses in
parallel. In fact, if instruction 1 takes a long time, instruction 2 may even complete
before it, and the CPU may then start executing instruction 3. Now consider the
situation where one thread waits on another using busy waiting:
THREAD 1:
while (turn != 1) { } /* loop */
printf ("%d\n", x);
THREAD 2:
x = 100;
turn = 1;
If turn == 0 initially and all instructions execute in order, the program will print the
value 100. However, if the instructions in Thread 2 execute out of order, turn will
be updated before x and the printed value could be some older value of x. Similarly, the instructions of Thread 1 may be reordered, making it read x before performing the check in the line above it. The solution in both cases is to wedge a barrier instruction in between the two lines.
Incidentally, memory barriers often play an important role in the mitigation of
a nasty class of CPU vulnerabilities that are commonly referred to as transient
150
PROCESSES AND THREADS
CHAP. 2
execution vulnerabilities. Here, attackers can exploit the fact that CPUs execute
instructions out of order. Between the first disclosure of the Meltdown and Spectre
issues in 2018, many such vulnerabilities have come to light. Since they generally
impact the operating system also, we will briefly look at transient execution attacks
in Chap. 9.
2.4.10 Priority Inversion
Earlier in this chapter, we mentioned the priority inversion problem, a truly
classic problem that was known already in the 1970s. Now let us look at it in more
detail.
A famous example of priority inversion occurred on Mars in 1997. In an
impressive engineering effort, NASA had managed to land a little robot rover on
the red planet, destined to send a wealth of interesting information back to earth.
Except there was a problem. Pathfinder’s radio transmissions stopped sending data
constantly, requiring system resets to get it going again. It turned out that three
threads were getting in each other’s hair. Pathfinder used a form of shared memory,
which was called the ‘‘information bus,’’ for passing information between its different components. A low-priority thread used the bus periodically to pass on the
meteorological data (a kind of Mars weather report) it had gathered. Meanwhile, a
high priority thread for information bus management would also periodically
access it. To prevent both threads from accessing the shared memory at the same
time, its access was controlled by a mutex in the rover’s software. A third,
medium-priority thread was responsible for communications and did not need the
mutex at all.
The priority inversion occurred when the low-priority thread for meteorological data gathering had been preempted by the medium priority communications
thread, while holding the mutex. After some time, the high-priority thread needed
to run but immediately blocked as it could not grab the mutex. The long-running
medium-priority thread kept executing, as if it had higher priority than the information bus thread.
There are different ways to solve the priority inversion problem. The simplest
one is to disable all interrupts while in the critical region. As mentioned earlier,
this is not desirable for user programs: What if they forget to enable them again?
Another solution, known as priority ceiling is to associate a priority with the
mutex itself and assign that to the process holding it. As long as no process that
needs to grab the mutex has a higher priority than the ceiling priority, inversion is
no longer possible.
A third way is priority inheritance. Here, the low-priority task holding the
mutex will temporarily inherit the priority of the high-priority task trying to obtain
it. Again, no medium priority task will be able to preempt the task holding the
mutex. This was the technique eventually used to fix the Mars Pathfinder problems.
SEC. 2.4 SYNCHRONIZATION AND INTERPROCESS COMMUNICATION
151
Finally, operating systems such as Microsoft Windows employ random boosting, essentially rolling the dice every now and then and giving random mutex-holding threads a high priority until they exit the critical region.
2.4.11 Avoiding Locks: Read-Copy-Update
The fastest locks are no locks at all. And no locks also means no risk of priority inversion. The question is whether we can allow for concurrent read and write
accesses to shared data structures without locking. In the general case, the answer
is clearly no. Imagine thread A sorting an array of numbers, while thread B is calculating the average. Because A moves the values back and forth across the array,
B may encounter some values multiple times and others not at all. The result could
be anything, but it would almost certainly be wrong.
In some cases, however, we can allow a writer to update a data structure even
though other processes are still using it. The trick is to ensure that each reader
either reads the old version of the data, or the new one, but not some weird combination of old and new. As an illustration, consider the tree shown in Fig. 2-39.
Readers traverse the tree from the root to its leaves. In the top half of the figure, a new node X is added. To do so, we make the node ‘‘just right’’ before making it visible in the tree: we initialize all values in node X, including its child pointers. Then, with one atomic write, we make X a child of A. No reader will ever read
an inconsistent version. In the bottom half of the figure, we subsequently remove B
and D. First, we make A’s left child pointer point to C. All readers that were in A
will continue with node C and never see B or D. In other words, they will see only
the new version. Likewise, all readers currently in B or D will continue following
the original data structure pointers and see the old version. All is well, and we
never need to lock anything. The main reason that the removal of B and D works
without locking the data structure, is that RCU (Read-Copy-Update) decouples
the removal and reclamation phases of the update.
Of course, there is a problem. As long as we are not sure that there are no more
readers of B or D, we cannot really free them. But how long should we wait? One
minute? Ten? We have to wait until the last reader has left these nodes. RCU carefully determines the maximum time a reader may hold a reference to the data structure. After that period, it can safely reclaim the memory. Specifically, readers
access the data structure in what is known as a read-side critical section which
may contain any code, as long as it does not block or sleep. In that case, we know
the maximum time we need to wait. Specifically, we define a grace period as any
time period in which we know that each thread to be outside the read-side critical
section at least once. All will be well if we wait for a duration that is at least equal
to the grace period before reclaiming. As the code in a read-side critical section is
not allowed to block or sleep, a simple criterion is to wait until all the threads have
executed a context switch.
152
PROCESSES AND THREADS
CHAP. 2
RCU data structures are not so common in user processes, but quite popular in
operating system kernels for data structures that are accessed by multiple threads
and require high efficiency. The Linux kernel has thousands of uses of its RCU
API, spread across most of its subsystems. The network stack, the file system, drivers, and memory management all use RCU for concurrent reading and writing.
Adding a node:
A
A
B
C
X
B
D
E
(a) Original tree.
C
A
D
E
(b) Initialize node X and
connect E to X. Any readers
in A and E are not affected.
X
B
C
D
E
(c) When X is completely initialized,
connect X to A. Readers currently
in E will have read the old version,
while readers in A will pick up the
new version of the tree.
Removing nodes:
A
X
B
C
A
D
E
(d) Decouple B from A. Note
that there may still be readers
in B. All readers in B will see
the old version of the tree,
while all readers currently
in A will see the new version.
X
B
C
A
D
E
(e) Wait until we are sure
that all readers have left B
and C. These nodes cannot
be accessed any more.
X
C
E
(f) Now we can safely
remove B and D
Figure 2-39. Read-Copy-Update: inserting a node in the tree and then removing
a branch—all without locks.
2.5 SCHEDULING
When a computer is multiprogrammed, it frequently has multiple processes or
threads competing for the CPU at the same time. This situation occurs whenever
two or more of them are simultaneously in the ready state. If only one CPU is
available, a choice has to be made which process to run next. The part of the operating system that makes the choice is called the scheduler, and the algorithm it
uses is called the scheduling algorithm. These topics form the subject matter of
the following sections.
SEC. 2.5
SCHEDULING
153
Many of the same issues that apply to process scheduling also apply to thread
scheduling, although some are different. When the kernel manages threads, scheduling is usually done per thread, with little or no regard to which process the thread
belongs. Initially we will focus on scheduling issues that apply to both processes
and threads. Later on we will explicitly look at thread scheduling and some of the
unique issues it raises. We will deal with multicore chips in Chap. 8.
2.5.1 Introduction to Scheduling
Back in the old days of batch systems with input in the form of card images on
a magnetic tape, the scheduling algorithm was simple: just run the next job on the
tape. With multiprogramming, the scheduling algorithm became more complex
because there were usually multiple users waiting for service. Some mainframes
still combine batch and timesharing service, requiring the scheduler to decide
whether a batch job or an interactive user at a terminal should go next. (As an
aside, a batch job may be a request to run multiple programs in succession, but for
this section, we will just assume it is a request to run a single program.) Because
CPU time is a scarce resource on these machines, a good scheduler can make a big
difference in perceived performance and user satisfaction. Consequently, a lot of
work has gone into devising clever and efficient scheduling algorithms.
With the advent of personal computers, the situation changed in two ways.
First, most of the time there is only one active process. A user entering a document
on a word processor is unlikely to be simultaneously compiling a program in the
background. When the user types a command to the word processor, the scheduler
does not have to do much work to figure out which process to run—the word processor is the only candidate.
Second, computers have become so much faster over the years that the CPU is
rarely a scarce resource any more. Most programs for personal computers are limited by the rate at which the user can present input (by typing or clicking), not by
the rate the CPU can process it. Even compilations, a major sink of CPU cycles in
the past, take just a few seconds in most cases nowadays. Even when two programs
are actually running at once, such as a word processor and a spreadsheet, it hardly
matters which goes first since the user is probably waiting for both of them to finish (except that they generally complete their tasks so quickly that the user will not
be doing much waiting much anyway). As a consequence, scheduling does not
matter much on simple PCs. Of course, there are applications that practically eat
the CPU alive. For instance, rendering one hour of high-resolution video while
tweaking the colors in each of the 107,892 frames (in NTSC) or 90,000 frames (in
PAL) requires serious computing power. However, similar applications are the
exception rather than the rule.
When we turn to networked servers, the situation changes appreciably. Here
multiple processes often do compete for the CPU, so scheduling matters again. For
example, when the CPU has to choose between running a process that gathers the
154
PROCESSES AND THREADS
CHAP. 2
daily statistics and one that serves user requests, the users will be a lot happier if
the latter gets first crack at the CPU.
The ‘‘abundance of resources’’ argument also does not hold on IoT devices and
sensor nodes, and perhaps not even on smartphones. Even if the CPUs on phones
have become more powerful and the memory more plentiful, battery lifetime has
not. Since battery lifetime is one of the most important constraints on all these
devices, some schedulers try to optimize the power consumption.
In addition to picking the right process to run, the scheduler also has to worry
about making efficient use of the CPU because process switching is expensive. To
start with, a switch from user mode to kernel mode must occur. Then the state of
the current process must be saved, including storing its registers in the process table so they can be reloaded later. In some systems, the memory map (e.g., memory
reference bits in the page table) must be saved as well. This is called a context
switch, although people sometimes also use this term to refer to the full process
switch. Next a new process must be selected by running the scheduling algorithm.
After that, the memory management unit (MMU) must be reloaded with the memory map of the new process. Finally, the new process must be started. In addition to
all that, the process switch may invalidate the memory cache and related tables,
forcing it to be dynamically reloaded from the main memory twice (upon entering
the kernel and upon leaving it). All in all, doing too many process switches per second can chew up a substantial amount of CPU time, so caution is advised.
Process Behavior
Nearly all processes alternate bursts of computing with (disk or network) I/O
requests, as shown in Fig. 2-40. Often, the CPU runs for a while without stopping,
then a system call is made to read from a file or write to a file. When the system
call completes, the CPU computes again until it needs more data or has to write
more data, and so on. Note that some I/O activities count as computing. For example, when the CPU copies bits to a video RAM to update the screen, it is computing, not doing I/O, because the CPU is in use. I/O in this sense is when a process
enters the blocked state waiting for an external device to complete its work.
The important thing to notice about Fig. 2-40 is that some processes, such as
the one in Fig. 2-40(a), spend most of their time computing, while other processes,
such as the one shown in Fig. 2-40(b), spend most of their time waiting for I/O.
The former are called compute-bound or CPU-bound; the latter are called I/Obound. Compute-bound processes typically have long CPU bursts and thus infrequent I/O waits, whereas I/O-bound processes have short CPU bursts and thus frequent I/O waits. Note that the key factor is the length of the CPU burst, not the
length of the I/O burst. I/O-bound processes are I/O bound because they do not
compute much between I/O requests, not because they have especially long I/O
requests. It takes the same time to issue the hardware request to read a disk block
no matter how much or how little time it takes to process the data after they arrive.
SEC. 2.5
155
SCHEDULING
(a)
Long CPU burst
Waiting for I/O
Short CPU burst
(b)
Time
Figure 2-40. Bursts of CPU usage alternate with periods of waiting for I/O.
(a) A CPU-bound process. (b) An I/O-bound process.
It is worth noting that if CPUs get faster, processes tend to get more I/Obound. This effect occurs because CPUs are improving faster than hard disks. As a
consequence, the scheduling of I/O-bound processes could become a more important subject in the future. The basic idea here is that if an I/O-bound process wants
to run, it should get a chance quickly so that it can issue its disk request and keep
the disk busy. As we saw in Fig. 2-6, when processes are I/O bound, it takes quite a
few of them to keep the CPU fully occupied.
On the other hand, CPUs do not seem to be getting much faster these days
because making them go faster produces too much heat. Hard disks are not getting
any faster either, but SSDs are replacing hard disks in desktop and notebook computers. Then again, in large data centers, hard disks are still widely used due to
their lower cost per bit. The consequence of all this is that scheduling depends a lot
on the context and an algorithm that works well on a notebook may not work well
in a data center. And in 10 years, everything may be different.
When to Schedule
A key issue related to scheduling is when to make scheduling decisions. It
turns out that there are a variety of situations in which scheduling is needed. First,
when a new process is created, a decision needs to be made whether to run the parent process or the child process. Since both processes are in ready state, it is a normal scheduling decision and can go either way, that is, the scheduler can legitimately choose to run either the parent or the child next.
Second, a scheduling decision must be made when a process exits. That process can no longer run (since it no longer exists), so some other process must be
chosen from the set of ready processes. If no process is ready, a system-supplied
idle process is normally run.
156
PROCESSES AND THREADS
CHAP. 2
Third, when a process blocks on I/O, on a semaphore, or for some other reason, another process has to be selected to run. Sometimes the reason for blocking
may play a role in the choice. For example, if A is an important process and it is
waiting for B to exit its critical region, letting B run next will allow it to exit its
critical region and thus let A continue. The trouble, however, is that the scheduler
generally does not have the necessary information to take this dependency into
account.
Fourth, when an I/O interrupt occurs, a scheduling decision may be made. If
the interrupt came from an I/O device that has now completed its work, some process that was blocked waiting for the I/O may now be ready to run. It is up to the
scheduler to decide whether to run the newly ready process, the process that was
running at the time of the interrupt, or some third process.
If a hardware clock provides periodic interrupts at 50 or 60 Hz (or possibly at
some other—potentially higher—frequency), a scheduling decision can be made at
each clock interrupt or at every kth clock interrupt. Scheduling algorithms can be
divided into two categories with respect to how they deal with clock interrupts. A
nonpreemptive scheduling algorithm picks a process to run and then just lets it
run until it blocks (either on I/O or waiting for another process) or voluntarily
releases the CPU. Even if it runs for many hours, it will not be forcibly suspended.
In effect, no scheduling decisions are made during clock interrupts. After clockinterrupt processing has been finished, the process that was running before the
interrupt is resumed, unless a higher-priority process was waiting for a now-satisfied timeout.
In contrast, a preemptive scheduling algorithm picks a process and lets it run
for a maximum of some fixed time. If it is still running at the end of the time interval, it is suspended and the scheduler picks another process to run (if one is available). Doing preemptive scheduling requires having a clock interrupt occur at the
end of the time interval to give control of the CPU back to the scheduler. If no
clock is available, nonpreemptive scheduling is the only option.
Preemption is not just relevant for applications, but also for operating system
kernels, especially monolithic ones. Nowadays many of them are preemptive. If
they were not, a poorly implemented driver or a very slow system call could hog
the CPU. Instead, in a preemptive kernel the scheduler can force the long running
driver or system call to context switch.
Categories of Scheduling Algorithms
Not surprisingly, in different environments different scheduling algorithms are
needed. This situation arises because different application areas (and different
kinds of operating systems) have different goals. In other words, what the scheduler should optimize for is not the same in all systems. Three environments worth
distinguishing are:
SEC. 2.5
SCHEDULING
157
1. Batch
2. Interactive
3. Real time
Batch systems are still in widespread use in the business world for doing payroll,
inventory, accounts receivable, accounts payable, interest calculation (at banks),
claims processing (at insurance companies), and other periodic tasks. In batch systems, there are no users impatiently waiting at their terminals for a quick response
to a short request. Consequently, nonpreemptive algorithms, or preemptive algorithms with long time periods for each process, are often acceptable. This approach
reduces process switches and thus improves performance. The batch algorithms are
actually fairly general and often applicable to other situations as well, which makes
them worth studying, even for people not involved in corporate mainframe computing.
In an environment with interactive users, preemption is essential to keep one
process from hogging the CPU and denying service to the others. Even if no process intentionally ran forever, one process might shut out all the others indefinitely
due to a program bug. Preemption is needed to prevent this behavior. Servers also
fall into this category, since they normally serve multiple (remote) users, all of
whom are in a big hurry. Computer users are always in a big hurry.
In systems with real-time constraints, preemption is, oddly enough, sometimes
not needed because the processes know that they may not run for long periods of
time and usually do their work and block quickly. The difference with interactive
systems is that real-time systems run only programs that are intended to further the
application at hand. Interactive systems are general purpose and may run arbitrary
programs that are not cooperative and even possibly malicious.
Scheduling Algorithm Goals
In order to design a scheduling algorithm, it is necessary to have some idea of
what a good algorithm should do. Some goals depend on the environment (batch,
interactive, or real time), but some are desirable in all cases. Some goals are listed
in Fig. 2-41. We will discuss these in turn below.
Under all circumstances, fairness is important. Comparable processes should
get comparable service. Giving one process much more CPU time than an equivalent one is not fair. Of course, different categories of processes may be treated differently. Think of safety control and doing the payroll on a nuclear reactor’s computer.
Somewhat related to fairness is actually enforcing the system’s policies. If the
local policy is that safety control processes get to run whenever they want to, even
if it means the payroll is 30 sec late, the scheduler has to make sure this policy is
enforced. That may take some extra effort.
158
PROCESSES AND THREADS
CHAP. 2
All systems
Fairness—giving each process a fair share of the CPU
Policy enforcement—seeing that stated policy is carried out
Balance—keeping all parts of the system busy
Batch systems
Throughput—maximize jobs per hour
Turnaround time—minimize time between submission and termination
CPU utilization—keep the CPU busy all the time
Interactive systems
Response time—respond to requests quickly
Propor tionality—meet users’ expectations
Real-time systems
Meeting deadlines—avoid losing data
Predictability—avoid quality degradation in multimedia systems
Figure 2-41. Some goals of the scheduling algorithm under different circumstances.
Another general goal is keeping all parts of the system busy when possible. If
the CPU and all the I/O devices can be kept running all the time, more work gets
done per second than if some of the components are idle. In a batch system, for
example, the scheduler has control of which jobs are brought into memory to run.
Having some CPU-bound processes and some I/O-bound processes in memory
together is a better idea than first loading and running all the CPU-bound jobs and
then, when they are finished, loading and running all the I/O-bound jobs. If the latter strategy is used, when the CPU-bound processes are running, they will fight for
the CPU and the disk will be idle. Later, when the I/O-bound jobs come in, they
will fight for the disk and the CPU will be idle. Better to keep the whole system
running at once by a careful mix of processes.
The managers of large data centers that run many batch jobs typically look at
three metrics to see how well their systems are performing: throughput, turnaround
time, and CPU utilization. Throughput is the number of jobs per hour that the
system completes. All things considered, finishing 50 jobs per hour is better than
finishing 40 jobs per hour. Turnaround time is the statistically average time from
the moment that a batch job is submitted until the moment it is completed. It measures how long the average user has to wait for the output. Here the rule is: Small is
Beautiful.
A scheduling algorithm that tries to maximize throughput may not necessarily
minimize turnaround time. For example, given a mix of short jobs and long jobs, a
scheduler that always ran short jobs and never ran long jobs might achieve an
SEC. 2.5
SCHEDULING
159
excellent throughput (many short jobs per hour) but at the expense of a terrible
turnaround time for the long jobs. If short jobs kept arriving at a fairly steady rate,
the long jobs might never run, making the mean turnaround time infinite while
achieving a high throughput.
CPU utilization is often used as a metric on batch systems. Actually though, it
is not a good metric. What really matters is how many jobs per hour come out of
the system (throughput) and how long it takes to get a job back (turnaround time).
Using CPU utilization as a metric is like rating cars based on how many times per
hour the engine turns over. However, knowing when the CPU utilization is almost
100% is useful for knowing when it is time to get more computing power.
For interactive systems, different goals apply. The most important one is to
minimize response time, that is, the time between issuing a command and getting
the result. On a personal computer where a background process is running (for
example, reading and storing email from the network), a user request to start a program or open a file should take precedence over the background work. Having all
interactive requests go first will be perceived as good service.
A somewhat related issue is what might be called proportionality. Users have
an inherent (but often incorrect) idea of how long things should take. When a
request that the user perceives as complex takes a long time, users accept that, but
when a request that is perceived as simple takes a long time, users get irritated. For
example, if clicking on an icon that starts uploading a 5-GB video to a cloud server
takes 60 sec, the user will probably accept that as a fact of life because he does not
expect the upload to take 5 sec. He knows it will take time.
On the other hand, when a user clicks on the icon that breaks the connection to
the cloud server after the video has been uploaded, he has different expectations. If
it has not completed after 30 sec, the user will probably be swearing a blue streak,
and after 60 sec he will be foaming at the mouth. This behavior is due to the common user perception that sending a lot of data is supposed to take a lot longer than
just breaking the connection. In some cases (such as this one), the scheduler cannot
do anything about the response time, but in other cases it can, especially when the
delay is due to a poor choice of process order.
Real-time systems have different properties than interactive systems, and thus
different scheduling goals. They are characterized by having deadlines that must or
at least should be met. For example, if a computer is controlling a device that produces data at a regular rate, failure to run the data-collection process on time may
result in lost data. Thus, the foremost need in a real-time system is meeting all (or
most) deadlines.
In some real-time systems, especially those involving multimedia, predictability is important. Missing an occasional deadline is not fatal, but if the audio process runs too erratically, the sound quality will deteriorate rapidly. Video is also an
issue, but the ear is much more sensitive to jitter than the eye. To avoid this problem, process scheduling must be highly predictable and regular. We will study
batch and interactive scheduling algorithms in this chapter.
160
PROCESSES AND THREADS
CHAP. 2
2.5.2 Scheduling in Batch Systems
It is now time to turn from general scheduling issues to specific scheduling
algorithms. In this section, we will look at algorithms used in batch systems. In the
following ones, we will examine interactive and real-time systems. It is worth
pointing out that some algorithms are used in both batch and interactive systems.
We will study these later.
First-Come, First-Served
Probably the simplest of all scheduling algorithms ever devised is nonpreemptive first-come, first-served. With this algorithm, processes are assigned the CPU
in the order they request it. Basically, there is a single queue of ready processes.
When the first job enters the system from the outside in the morning, it is started
immediately and allowed to run as long as it wants to. It is not interrupted because
it has run too long. As other jobs come in, they are put onto the end of the queue.
When the running process blocks, the first process on the queue is run next. When
a blocked process becomes ready, like a newly arrived job, it is put on the end of
the queue, behind all waiting processes.
The great strength of this algorithm is that it is easy to understand and equally
easy to program. It is also fair in the same sense that allocating scarce concert tickets or brand-new iPhones to people who are willing to stand on line starting at 2
A.M. is fair. With this algorithm, a single linked list keeps track of all ready processes. Picking a process to run just requires removing one from the front of the
queue. Adding a new job or unblocked process just requires attaching it to the end
of the queue. What could be simpler to understand and implement?
Unfortunately, first-come, first-served also has a powerful disadvantage. Suppose there is one compute-bound process that runs for 1 sec at a time and many
I/O-bound processes that use little CPU time but each have to perform 1000 disk
reads to complete. The compute-bound process runs for 1 sec, then it reads a disk
block. All the I/O processes now run and start disk reads. When the computebound process gets its disk block, it runs for another 1 sec, followed by all the I/Obound processes in quick succession.
The net result is that each I/O-bound process gets to read 1 block per second
and will take 1000 sec to finish. With a scheduling algorithm that preempted the
compute-bound process every 10 msec, the I/O-bound processes would finish in 10
sec instead of 1000 sec, without slowing down the compute-bound process much.
Shortest Job First
Now let us look at another nonpreemptive batch algorithm that assumes the run
times are known in advance. In an insurance company, for example, people can
predict quite accurately how long it will take to run a batch of 1000 claims, since
SEC. 2.5
161
SCHEDULING
similar work is done every day. When several equally important jobs are sitting in
the input queue waiting to be started, the scheduler picks the shortest job first.
Look at Fig. 2-42. Here we find four jobs A, B, C, and D with run times of 8, 4, 4,
and 4 minutes, respectively. By running them in that order, the turnaround time for
A is 8 minutes, for B is 12 minutes, for C is 16 minutes, and for D is 20 minutes for
an average of 14 minutes.
8
4
4
4
4
4
4
8
A
B
C
D
B
C
D
A
(a)
(b)
Figure 2-42. An example of shortest-job-first scheduling. (a) Running four jobs
in the original order. (b) Running them in shortest-job-first order.
Now let us consider running these four jobs using shortest job first, as shown
in Fig. 2-42(b). The turnaround times are now 4, 8, 12, and 20 minutes for an average of 11 minutes. Shortest job first is provably optimal. Consider the case of four
jobs, with execution times of a, b, c, and d, respectively. The first job finishes at
time a, the second at time a + b, and so on. The mean turnaround time is
(4a + 3b + 2c + d)/4. It is clear that a contributes more to the average than the
other times, so it should be the shortest job, with b next, then c, and finally d as the
longest since it affects only its own turnaround time. The same argument applies
equally well to any number of jobs.
It is worth pointing out that shortest job first is optimal only when all the jobs
are available simultaneously. As a counterexample, consider five jobs, A through E,
with run times of 2, 4, 1, 1, and 1, respectively. Their arrival times are 0, 0, 3, 3,
and 3. Initially, only A or B can be chosen, since the other three jobs have not arrived yet. Using shortest job first, we will run the jobs in the order A, B, C, D, E, for
an average wait of 4.6. However, running them in the order B, C, D, E, A has an
average wait of 4.4.
Shortest Remaining Time Next
A preemptive version of shortest job first is shortest remaining time next.
With this algorithm, the scheduler always chooses the process whose remaining
run time is the shortest. Again here, the run time has to be known in advance.
When a new job arrives, its total time is compared to the current process’ remaining time. If the new job needs less time to finish than the current process, the current process is suspended and the new job started. This scheme allows new short
jobs to get good service.
162
PROCESSES AND THREADS
CHAP. 2
2.5.3 Scheduling in Interactive Systems
We will now look at some algorithms that can be used in interactive systems.
These are common on personal computers, servers, and other kinds of systems as
well.
Round-Robin Scheduling
One of the oldest, simplest, fairest, and most widely used algorithms is round
robin. Each process is assigned a time interval, called its quantum, during which
it is allowed to run. If the process is still running at the end of the quantum, the
CPU is preempted and given to another process. If the process has blocked or finished before the quantum has elapsed, the CPU switching is done when the process
blocks, of course. Round robin is easy to implement. All the scheduler needs to do
is maintain a list of runnable processes, as shown in Fig. 2-43(a). When the process uses up its quantum, it is put on the end of the list, as shown in Fig. 2-43(b).
Current
process
B
Next
process
F
D
(a)
Current
process
G
A
F
D
G
A
B
(b)
Figure 2-43. Round-robin scheduling. (a) The list of runnable processes.
(b) The list of runnable processes after B uses up its quantum.
The only really interesting issue with round robin is the length of the quantum.
Switching from one process to another requires a certain amount of time for doing
the administration—saving and loading registers and memory maps, updating various tables and lists, flushing and reloading the memory cache, and so on. Suppose
that this context switch takes 1 msec, including switching memory maps, flushing
and reloading the cache, etc. Also suppose that the quantum is set at 4 msec. With
these parameters, after doing 4 msec of useful work, the CPU will have to spend
(i.e., waste) 1 msec on process switching. Thus, 20% of the CPU time will be
thrown away on administrative overhead. Clearly, this is too much.
To improve the CPU efficiency, we could set the quantum to, say, 100 msec.
Now the wasted time is only 1%. But consider what happens on a server system if
50 requests come in within a very short time interval and with widely varying CPU
requirements. Fifty processes will be put on the list of runnable processes. If the
CPU is idle, the first one will start immediately, the second one may not start until
100 msec later, and so on. The unlucky last one may have to wait 5 sec before getting a chance, assuming all the others use their full quanta. Most users will perceive a 5-sec response to a short command as sluggish. This situation is especially
SEC. 2.5
SCHEDULING
163
bad if some of the requests near the end of the queue required only a few milliseconds of CPU time. With a short quantum, they would have gotten better service.
Another factor is that if the quantum is set longer than the mean CPU burst,
preemption will not happen very often. Instead, most processes will perform a
blocking operation before the quantum runs out, causing a process switch. Eliminating preemption improves performance because process switches then happen
only when they are logically necessary, that is, when a process blocks and cannot
continue.
The conclusion can be formulated as follows: setting the quantum too short
causes too many process switches and lowers the CPU efficiency, but setting it too
long may cause poor response to short interactive requests. A quantum around
20–50 msec is often a reasonable compromise.
Priority Scheduling
Round-robin scheduling makes the implicit assumption that all processes are
equally important. Frequently, the people who own and operate multiuser computers have quite different ideas on that subject. At a university, for example, the
pecking order may be the president first, the faculty deans next, then professors,
secretaries, janitors, and finally students. The need to take external factors into account leads to priority scheduling. The basic idea is straightforward: each process is assigned a priority, and the runnable process with the highest priority is
allowed to run.
Even on a PC with a single owner, there may be multiple processes, some of
them more important than others. For example, a daemon process sending electronic mail in the background should be assigned a lower priority than a process
displaying a video film on the screen in real time.
To prevent high-priority processes from running indefinitely, the scheduler
may decrease the priority of the currently running process at each clock tick (i.e.,
at each clock interrupt). If this action causes its priority to drop below that of the
next highest process, a process switch occurs. Alternatively, each process may be
assigned a maximum time quantum that it is allowed to run. When this quantum is
used up, the next-highest-priority process is given a chance to run. After a process
has been punished long enough, its priority needs to be raised by some algorithm,
to let it run again. Otherwise all processes will eventually end up at 0.
Priorities can be assigned to processes statically or dynamically. On a military
computer, processes started by generals might begin at priority 100, processes
started by colonels at 90, majors at 80, captains at 70, lieutenants at 60, and so on
down the totem pole. Alternatively, at a commercial data center, high-priority jobs
might cost $100 an hour, medium priority $75 an hour, and low priority $50 an
hour. The UNIX system has a command, nice, which allows a user to voluntarily
reduce the priority of his process, in order to be nice to the other users. Not surprisingly, nobody ever uses it.
164
PROCESSES AND THREADS
CHAP. 2
Priorities can also be assigned dynamically by the system to achieve certain
system goals. For example, some processes are highly I/O bound and spend most
of their time waiting for I/O to complete. Whenever such a process wants the CPU,
it should be given the CPU immediately, to let it start its next I/O request, which
can then proceed in parallel with another process actually computing. Making the
I/O-bound process wait a long time for the CPU will just mean having it around
occupying memory for an unnecessarily long time. A simple algorithm for giving
good service to I/O-bound processes is to set the priority to 1/ f , where f is the fraction of the last quantum that a process used. A process that used only 1 msec of its
50-msec quantum would get priority 50, while a process that ran 25 msec before
blocking would get priority 2, and a process that used the whole quantum would
get priority 1.
It is often convenient to group processes into priority classes and use priority
scheduling among the classes but round-robin scheduling within each class. Figure
2-44 shows a system with four priority classes. The scheduling algorithm is as follows: as long as there are runnable processes in priority class 4, just run each one
for one quantum, round-robin fashion, and never bother with lower-priority classes.
If priority class 4 is empty, then run the class 3 processes round robin. If classes 4
and 3 are both empty, then run class 2 round robin, and so on. If priorities are not
adjusted occasionally, lower-priority classes may all starve to death.
Queue
headers
Priority 4
Runnable processes
(Highest priority)
Priority 3
Priority 2
Priority 1
(Lowest priority)
Figure 2-44. A scheduling algorithm with four priority classes.
Multiple Queues
One of the earliest priority schedulers was in CTSS, the M.I.T. Compatible
TimeSharing System that ran on the IBM 7094 (Corbató et al., 1962). CTSS had
the problem that process switching was slow because the 7094 could hold only one
process in memory. Each switch meant swapping the current process to disk and
reading in a new one from disk. The CTSS designers quickly realized that it was
more efficient to give CPU-bound processes a large quantum once in a while, rather than giving them small quanta frequently (to reduce swapping). On the other
hand, giving all processes a large quantum would mean poor response time, as we
SEC. 2.5
SCHEDULING
165
have already seen. Their solution was to set up priority classes. Processes in the
highest class were run for one quantum. Processes in the next-highest class were
run for two quanta. Processes in the next one were run for four quanta, etc. Whenever a process used up all the quanta allocated to it, it was moved down one class.
Thus, the processes in the highest class would run more frequently and with high
priority, but for a shorter time—ideal for interactive processes.
As an example, consider a process that needed to compute continuously for
100 quanta. It would initially be given one quantum, then swapped out. Next time
it would get two quanta before being swapped out. On succeeding runs it would get
4, 8, 16, 32, and 64 quanta, although it would have used only 37 of the final 64
quanta to complete its work. Only 7 swaps would be needed (including the initial
load) instead of 100 with a pure round-robin algorithm. Furthermore, as the process sank deeper and deeper into the priority queues, it would be run less and less
frequently, saving the CPU for short, interactive processes.
The following policy was adopted to avoid punishing forever a process that
needed to run for a long time when it first started but became interactive later.
Whenever a carriage return (Enter key) was typed at a terminal, the process
belonging to that terminal was moved to the highest-priority class, on the assumption that it was about to become interactive. One fine day, some user with a heavily
CPU-bound process discovered that just sitting at the terminal and typing carriage
returns at random every few seconds did wonders for his response time. He told all
his friends. They told all their friends. Moral of the story: getting it right in practice is much harder than getting it right in principle.
Shortest Process Next
Because shortest job first always produces the minimum average response time
for batch systems, it would be nice if it could be used for interactive processes as
well. To a certain extent, it can be. Interactive processes generally follow the pattern of wait for command, execute command, wait for command, execute command, etc. If we regard the execution of each command as a separate ‘‘job,’’ then
we can minimize overall response time by running the shortest one first. The problem is figuring out which of the currently runnable processes is the shortest one.
One approach is to make estimates based on past behavior and run the process
with the shortest estimated running time. Suppose that the estimated time per command for some process is T 0 . Now suppose its next run is measured to be T 1 . We
could update our estimate by taking a weighted sum of these two numbers, that is,
aT 0 + (1 < a)T 1 . Through the choice of a we can decide to have the estimation
process forget old runs quickly, or remember them for a long time. With a = 1/2,
we get successive estimates of
T 0 , T 0 /2 + T 1 /2, T 0 /4 + T 1 /4 + T 2 /2, T 0 /8 + T 1 /8 + T 2 /4 + T 3 /2
After three new runs, the weight of T 0 in the new estimate has dropped to 1/8.
166
PROCESSES AND THREADS
CHAP. 2
The technique of estimating the next value in a series by taking the weighted
average of the current measured value and the previous estimate is sometimes called aging. It is applicable to many situations where a prediction must be made
based on previous values. Aging is especially easy to implement when a = 1/2. All
that is needed is to add the new value to the current estimate and divide the sum by
2 (by shifting it right 1 bit).
Guaranteed Scheduling
A completely different approach to scheduling is to make real promises to the
users about performance and then live up to those promises. One promise that is
realistic to make and easy to live up to is this: If n users are logged in while you are
working, you will receive about 1/n of the CPU power. Similarly, on a single-user
system with n processes running, all things being equal, each one should get 1/n of
the CPU cycles. That seems fair enough.
To make good on this promise, the system must keep track of how much CPU
each process has had since its creation. It then computes the amount of CPU each
one is entitled to, namely the time since creation divided by n. Since the amount of
CPU time each process has actually had is also known, it is fairly straightforward
to compute the ratio of actual CPU time consumed to CPU time entitled. A ratio of
0.5 means that a process has only had half of what it should have had, and a ratio
of 2.0 means that a process has had twice as much as it was entitled to. The algorithm is then to run the process with the lowest ratio until its ratio has moved above
that of its closest competitor. Then that one is chosen to run next.
A variant of such a scheduling regime is used in Linux’ CFS (Completely
Fair Scheduling) algorithm which keeps track of the ‘‘spent execution time’’ for
processes in an efficient red-black tree. The left-most node in the tree corresponds
to the process with the least spent execution time. The scheduler indexes the tree
by execution time and selects the left-mode node to run. When the process stops
running (either because it used up its time slot, or it was blocked or interrupted),
the scheduler reinserts it in the tree based on its new spent execution time.
Lottery Scheduling
While making promises to the users and then living up to them is a fine idea, it
is difficult to implement. However, another algorithm can be used to give similarly
predictable results with a much simpler implementation. It is called lottery scheduling (Waldspurger and Weihl, 1994).
The basic idea is to give processes lottery tickets for various system resources,
such as CPU time. Whenever a scheduling decision has to be made, a lottery ticket
is chosen at random, and the process holding that ticket gets the resource. When
applied to CPU scheduling, the system might hold a lottery 50 times a second, with
each winner getting 20 msec of CPU time as a prize.
SEC. 2.5
SCHEDULING
167
To paraphrase George Orwell: ‘‘All processes are equal, but some processes
are more equal.’’ More important processes can be given extra tickets, to increase
their odds of winning. If there are 100 tickets outstanding, and one process holds
20 of them, it will have a 20% chance of winning each lottery. In the long run, it
will get about 20% of the CPU. In contrast to a priority scheduler, where it is very
hard to state what having a priority of 40 actually means, here the rule is clear: a
process holding a fraction f of the tickets will get about a fraction f of the resource
in question.
Lottery scheduling has several interesting properties. For example, if a new
process shows up and is granted some tickets, at the very next lottery it will have a
chance of winning in proportion to the number of tickets it holds. In other words,
lottery scheduling is highly responsive.
Cooperating processes may exchange tickets if they wish. For example, when a
client process sends a message to a server process and then blocks, it may give all
of its tickets to the server, to increase the chance of the server running next. When
the server is finished, it returns the tickets so that the client can run again. In fact,
in the absence of clients, servers need no tickets at all.
Lottery scheduling can be used to solve problems that are difficult to handle
with other methods. One example is a video server in which several processes are
feeding video streams to their clients, but at different frame rates. Suppose that the
processes need frames at 10, 20, and 25 frames/sec. By allocating these processes
10, 20, and 25 tickets, respectively, they will automatically divide the CPU in
approximately the correct proportion, that is, 10:20:25.
Fair-Share Scheduling
So far we have assumed that each process is scheduled on its own, without
regard to who its owner is. As a result, if user 1 starts up nine processes and user 2
starts up one process, with round robin or equal priorities, user 1 will get 90% of
the CPU and user 2 only 10% of it.
To prevent this situation, some systems take into account which user owns a
process before scheduling it. In this model, each user is allocated some fraction of
the CPU and the scheduler picks processes in such a way as to enforce it. Thus, if
two users have each been promised 50% of the CPU, they will each get that, no
matter how many processes they have in existence.
As an example, consider a system with two users, each of which has been
promised 50% of the CPU. User 1 has four processes, A, B, C, and D, and user 2
has only one process, E. If round-robin scheduling is used, a possible scheduling
sequence that meets all the constraints is this one:
A E B E C E D E A E B E C E D E ...
On the other hand, if user 1 is entitled to twice as much CPU time as user 2, we
might get:
168
PROCESSES AND THREADS
CHAP. 2
A B E C D E A B E C D E ...
Numerous other possibilities exist, of course, and can be exploited, depending on
what the notion of fairness is.
2.5.4 Scheduling in Real-Time Systems
A real-time system is one in which time plays an essential role. Typically, one
or more physical devices external to the computer generate stimuli, and the computer must react appropriately to them within a fixed amount of time. For example,
the computer in a compact disc player gets the bits as they come off the drive and
must convert them into music within a very tight time interval. If the calculation
takes too long, the music will sound peculiar. Other real-time systems are patient
monitoring in a hospital intensive-care unit, the autopilot in an aircraft, and robot
control in an automated factory. In all these cases, having the right answer but having it too late is often just as bad as not having it at all.
Real-time systems are generally categorized as hard real time, meaning there
are absolute deadlines that must be met—or else!—and soft real time, meaning
that missing an occasional deadline is undesirable, but nevertheless tolerable. In
both cases, real-time behavior is achieved by dividing the program into a number
of processes, each of whose behavior is predictable and known in advance. These
processes are generally short lived and can run to completion in well under a second. When an external event is detected, it is the job of the scheduler to schedule
the processes in such a way that all deadlines are met.
The events that a real-time system may have to respond to can be further categorized as periodic (meaning they occur at regular intervals) or aperiodic (meaning they occur unpredictably). A system may have to respond to multiple periodicevent streams. Depending on how much time each event requires for processing,
handling all of them may not even be possible. For example, if there are m periodic
events and event i occurs with period P i and requires C i sec of CPU time to handle
each event, then the load can be handled only if
m
Ci
Y )1
i=1 P i
A real-time system that meets this criterion is said to be schedulable. This means
it can actually be implemented. A process that fails to meet this test cannot be
scheduled because the total amount of CPU time the processes want collectively is
more than the CPU can deliver.
As an example, consider a soft real-time system with three periodic events,
with periods of 100, 200, and 500 msec, respectively. If these events require 50, 30,
and 100 msec of CPU time per event, respectively, the system is schedulable
because 0. 5 + 0. 15 + 0. 2 < 1. If a fourth event with a period of 1 sec is added, the
system will remain schedulable as long as this event does not need more than 150
msec of CPU time per event. Implicit in this calculation is the assumption that the
context-switching overhead is so small that it can be ignored.
SEC. 2.5
SCHEDULING
169
Real-time scheduling algorithms can be static or dynamic. The former make
their scheduling decisions before the system starts running. The latter make their
scheduling decisions at run time, after execution has started. Static scheduling
works only when there is perfect information available in advance about the work
to be done and the deadlines that have to be met. Dynamic scheduling algorithms
do not have these restrictions.
2.5.5 Policy Versus Mechanism
Up until now, we have tacitly assumed that all the processes in the system belong to different users and are thus competing for the CPU. While this is often
true, sometimes it happens that one process has many children running under its
control. For example, a database-management-system process may have many
children. Each child might be working on a different request, or each might have
some specific function to perform (query parsing, disk access, etc.). It is entirely
possible that the main process has an excellent idea of which of its children are the
most important (or time critical) and which the least. Unfortunately, none of the
schedulers discussed earlier accept any input from user processes about scheduling
decisions. As a result, the scheduler rarely makes the best choice.
The solution to this problem is to separate the scheduling mechanism from
the scheduling policy, a long-established principle (Levin et al., 1975). What this
means is that the scheduling algorithm is parameterized in some way, but the
parameters can be filled in by user processes. Let us consider the database example
once again. Suppose that the kernel uses a priority-scheduling algorithm but provides a system call by which a process can set (and change) the priorities of its
children. In this way, the parent can control how its children are scheduled, even
though it itself does not do the scheduling. Here the mechanism is in the kernel but
policy is set by a user process. Policy-mechanism separation is a key idea.
2.5.6 Thread Scheduling
When several processes each have multiple threads, we have two levels of parallelism present: processes and threads. Scheduling in such systems differs substantially depending on whether user-level threads or kernel-level threads (or both)
are supported.
Let us consider user-level threads first. Since the kernel is not aware of the
existence of threads, it operates as it always does, picking a process, say, A, and
giving A control for its quantum. The thread scheduler inside A decides which
thread to run, say A1. Since there are no clock interrupts to multiprogram threads,
this thread may continue running as long as it wants to. If it uses up the process’
entire quantum, the kernel will select another process to run.
170
PROCESSES AND THREADS
CHAP. 2
When the process A finally runs again, thread A1 will resume running. It will
continue to consume all of A’s time until it is finished. However, its antisocial
behavior will not affect other processes. They will get whatever the scheduler considers their appropriate share, no matter what is going on inside process A.
Now consider the case that A’s threads have relatively little work to do per
CPU burst, for example, 5 msec of work within a 50-msec quantum. Consequently,
each one runs for a little while, then yields the CPU back to the thread scheduler.
This might lead to the sequence A1, A2, A3, A1, A2, A3, A1, A2, A3, A1, before the
kernel switches to process B. This situation is illustrated in Fig. 2-45(a).
Process A
Process B
Process A
Process B
Order in which
threads run
2. Run-time
system
picks a
thread
1
2
1
3
1. Kernel picks a process
3
2
1. Kernel picks a thread
Possible:
A1, A2, A3, A1, A2, A3
Not possible: A1, B1, A2, B2, A3, B3
Possible:
A1, A2, A3, A1, A2, A3
Also possible: A1, B1, A2, B2, A3, B3
(a)
(b)
Figure 2-45. (a) Possible scheduling of user-level threads with a 50-msec process quantum and threads that run 5 msec per CPU burst. (b) Possible scheduling
of kernel-level threads with the same characteristics as (a).
The scheduling algorithm used by the run-time system can be any of the ones
described above. In practice, round-robin scheduling and priority scheduling are
most common. The only constraint is the absence of a clock to interrupt a thread
that has run too long. Since threads cooperate, this is usually not an issue.
Now consider the situation with kernel-level threads. Here the kernel picks a
particular thread to run. It does not have to take into account which process the
thread belongs to, but it can if it wants to. The thread is given a quantum and is forcibly suspended if it exceeds the quantum. With a 50-msec quantum and threads
that block after 5 msec, the thread order for some period of 30 msec might be A1,
B1, A2, B2, A3, B3, something not possible with these parameters and user-level
threads. This situation is partially depicted in Fig. 2-45(b).
A major difference between user-level threads and kernel-level threads is the
performance. Doing a thread switch with user-level threads takes a handful of
machine instructions. With kernel-level threads it requires a full context switch
changing the memory map and invalidating the cache, which is several orders of
SEC. 2.5
SCHEDULING
171
magnitude slower. On the other hand, with kernel-level threads, having a thread
block on I/O does not suspend the entire process as it does with user-level threads.
Since the kernel knows that switching from a thread in process A to a thread in
process B is more expensive than running a second thread in process A (due to having to change the memory map and having the memory cache spoiled), it can take
this information into account when making a decision. For example, given two
threads that are otherwise equally important, with one of them belonging to the
same process as a thread that just blocked and one belonging to a different process,
preference could be given to the former.
Another important factor is that user-level threads can employ an application-specific thread scheduler. Consider, for example, the Web server of Fig. 2-8.
Suppose that a worker thread has just blocked and the dispatcher thread and two
worker threads are ready. Who should run next? The run-time system, knowing
what all the threads do, can easily pick the dispatcher to run next, so that it can
start another worker running. This strategy maximizes the amount of parallelism in
an environment where workers frequently block on disk I/O. With kernel-level
threads, the kernel would never know what each thread did (although they could be
assigned different priorities). In general, however, application-specific thread
schedulers can tune an application better than the kernel can.
2.6 RESEARCH ON PROCESSES AND THREADS
In Chap. 1, we looked at some of the current research in operating system
structure. In this and subsequent chapters, we will look at more narrowly focused
research, starting with processes. As will become clear in time, some subjects are
much more settled than others. Most of the research tends to be on the new topics,
rather than ones that have been around for decades.
The concept of a process is an example of something that is fairly well settled.
Almost every system has some notion of a process as a container for grouping
together related resources such as an address space, threads, open files, protection
permissions, and so on. Different systems do the grouping slightly differently, but
these are just engineering differences. The basic idea is not very controversial any
more, and there is little new research on the subject of processes.
Threads are a newer idea than processes, but they, too, have been chewed over
quite a bit. Still, the occasional paper about threads appears from time to time, for
example, about core-aware thread management (Qin et al., 2019), or on how well
modern operating systems like Linux scale with many threads and many cores
(Boyd-Wickizer, 2010).
Additionally, there is a lot of work trying to prove things do not break in the
presence of concurrency, for instance in file systems (Chajed et al., 2019; and Zou
et al., 2019) and other services (Setty et al., 2018; and Li et al., 2019). This is
important work as researchers have shown that concurrency bugs are unfortunately
172
PROCESSES AND THREADS
CHAP. 2
extremely common (Li et al., 2019). As we have seen, locking is not just hard, but
also expensive and operating systems have adopted RCU to avoid locking altogether (McKenney et al., 2013).
An active research area deals with recording and replaying a process’ execution (Viennot et al., 2013). Replaying helps developers track down hard-to-find
bugs and security experts to investigate incidents.
Speaking of security, a major event in 2018 was the disclosure of a series of
very serious security vulnerabilities in modern CPUs. They required changes
everywhere: hardware, firmware, operating system, and even applications. For this
chapter, the scheduling implications are especially relevant. For instance, Windows
adopted a scheduling algorithm to prevent code in different security domains from
sharing the same processor core (Microsoft, 2018).
Scheduling (both uniprocessor and multiprocessor) in general is still a topic
near and dear to the heart of some researchers. Some topics being researched
include scheduling in clusters for deep learning (Xiao et al., 2018), scheduling for
microservices (Sriraman, 2018), and schedulability (Yang et al., 2018). All in all,
processes, threads, and scheduling are not hot topics for research as they once
were. The research has moved on to topics like power management, virtualization,
clouds, and security.
2.7 SUMMARY
To hide the effects of interrupts, operating systems provide a conceptual model
consisting of sequential processes running in parallel. Processes can be created and
terminated dynamically. Each process has its own address space.
For some applications, it is useful to have multiple threads of control within a
single process. These threads are scheduled independently and each one has its
own stack, but all the threads in a process share a common address space. Threads
can be implemented in user space or in the kernel.
Alternatively, high-throughput servers may opt for an event-driven model
instead. Here, the server operates as a finite state machine that responds to events
and interacts with the operating system using non-blocking system calls.
Processes can synchronize with one another using synchronization and
interprocess communication primitives, for example, semaphores, monitors, or
messages. These primitives are used to ensure that no two processes are ever in
their critical regions at the same time, a situation that leads to chaos. A process can
be running, runnable, or blocked and can change state when it or another process
executes one of the interprocess communication primitives. Interthread communication is similar.
A great many scheduling algorithms have been studied. Some of these are primarily used for batch systems, such as shortest-job-first scheduling. Others are
common in both batch systems and interactive systems. These algorithms include
SEC. 2.7
SUMMARY
173
round robin, priority scheduling, multilevel queues, guaranteed scheduling, lottery
scheduling, and fair-share scheduling. Some systems make a clean separation
between the scheduling mechanism and the scheduling policy, which allows users
to have control of the scheduling algorithm.
PROBLEMS
1. In Fig. 2-2, three process states are shown. In theory, with three states, there could be
six transitions, two out of each state. However, only four transitions are shown. Are
there any circumstances in which either or both of the missing transitions might occur?
2. Suppose that you were to design an advanced computer architecture that did process
switching in hardware, instead of having interrupts. What information would the CPU
need? Describe how the hardware process switching might work.
3. On all current computers, at least part of the interrupt handlers are written in assembly
language. Why?
4. When an interrupt or a system call transfers control to the operating system, a kernel
stack area separate from the stack of the interrupted process is generally used. Why?
5. A computer system has enough room to hold four programs in its main memory. These
programs are idle waiting for I/O half the time. What fraction of the CPU time is
wasted?
6. A computer has 2 GB of RAM of which the operating system occupies 256 MB. The
processes are all 128 MB (for simplicity) and have the same characteristics. If the goal
is 99% CPU utilization, what is the maximum I/O wait that can be tolerated?
7. Multiple jobs can run in parallel and finish faster than if they had run sequentially.
Suppose that two jobs, each needing 10 minutes of CPU time, start simultaneously.
How long will the last one take to complete if they run sequentially? How long if they
run in parallel? Assume 50% I/O wait.
8. Consider a multiprogrammed system with degree of 5 (i.e., five programs in memory
at the same time). Assume that each process spends 40% of its time waiting for I/O.
What will be the CPU utilization?
9. Explain how a Web browser can utilize the concept of threads to improve performance.
10. Assume that you are trying to download a large 2-GB file from the Internet. The file is
available from a set of mirror servers, each of which can deliver a subset of the file’s
bytes; assume that a given request specifies the starting and ending bytes of the file.
Explain how you might use threads to improve the download time.
11. In the text it was stated that the model of Fig. 2-10(a) was not suited to a file server
using a cache in memory. Why not? Could each process have its own cache?
174
PROCESSES AND THREADS
CHAP. 2
12. In Fig. 2-8, a multithreaded Web server is shown. If the only way to read from a file is
the normal blocking read system call, do you think user-level threads or kernel-level
threads are being used for the Web server? Why?
13. In the text, we described a multithreaded Web server, showing why it is better than a
single-threaded server and a finite-state machine server. Are there any circumstances in
which a single-threaded server might be better? Give an example.
14. In Fig. 2-11 the register set is listed as a per-thread rather than a per-process item.
Why? After all, the machine has only one set of registers.
15. Why would a thread ever voluntarily give up the CPU by calling thread yield? After
all, since there is no periodic clock interrupt, it may never get the CPU back.
16. In this problem, you are to compare reading a file using a single-threaded file server
and a multithreaded server. It takes 15 msec to get a request for work, dispatch it, and
do the rest of the necessary processing, assuming that the data needed are in the block
cache. If a disk operation is needed, as is the case one-third of the time, an additional
75 msec is required, during which time the thread sleeps. How many requests/sec can
the server handle if it is single threaded? If it is multithreaded?
17. What is the biggest advantage of implementing threads in user space? What is the biggest disadvantage?
18. In Fig. 2-14 the thread creations and messages printed by the threads are interleaved at
random. Is there a way to force the order to be strictly thread 1 created, thread 1 prints
message, thread 1 exits, thread 2 created, thread 2 prints message, thread 2 exists, and
so on? If so, how? If not, why not?
19. Suppose that a program has two threads, each executing the get account function,
shown below. Identify a race condition in this code.
int accounts[LIMIT]; int account count = 0;
void *get account(void *tid) {
char *lineptr = NULL;
size t len = 0;
while (account count < LIMIT)
{
// Read user input from terminal and store it in lineptr
getline(&lineptr, &len, stdin);
// Convert user input to integer
// Assume user entered valid integer value
int entered account = atoi(lineptr);
accounts[account count] = entered account;
account count++;
}
CHAP. 2
PROBLEMS
175
// Deallocate memory that was allocated by getline call
free(lineptr);
return NULL; }
20. In the discussion on global variables in threads, we used a procedure create global to
allocate storage for a pointer to the variable, rather than the variable itself. Is this
essential, or could the procedures work with the values themselves just as well?
21. Consider a system in which threads are implemented entirely in user space, with the
run-time system getting a clock interrupt once a second. Suppose that a clock interrupt
occurs exactly while some thread executing in the run-time system is at the point of
blocking or unblocking a thread. What problem might occur? Can you solve it?
22. Suppose that an operating system does not have anything like the select system call to
see in advance if it is safe to read from a file, pipe, or device, but it does allow alarm
clocks (timers) to be set that interrupt blocked system calls. Is it possible to implement
in user space a threads package that will not block all threads when one thread performs a system call that may block? Explain your answer.
23. Does Peterson’s solution to the mutual-exclusion problem shown in Fig. 2-24 work
when process scheduling is preemptive? How about when it is nonpreemptive?
24. Can the priority inversion problem discussed in Sec. 2.3.4 happen with user-level
threads? Why or why not?
25. In Sec. 2.3.4, a situation with a high-priority process, H, and a low-priority process, L,
was described, which led to H looping forever. Does the same problem occur if roundrobin scheduling is used instead of priority scheduling? Discuss.
26. In a system with threads, is there one stack per thread or one stack per process when
user-level threads are used? What about when kernel-level threads are used? Explain.
27. What is a race condition?
28. When a computer is being developed, it is usually first simulated by a program that
runs one instruction at a time. Even multiprocessors are simulated strictly sequentially
like this. Is it possible for a race condition to occur when there are no simultaneous
events? Explain.
29. The producer-consumer problem can be extended to a system with multiple producers
and consumers that write (or read) to (from) one shared buffer. Assume that each producer and consumer runs in its own thread. Will the solution presented in Fig. 2-28,
using semaphores, work for this system?
30. Consider the following solution to the mutual-exclusion problem involving two processes P0 and P1. Assume that the variable turn is initialized to 0. Process P0’s code is
presented below.
/* Other code */
while (turn != 0) { } /* Do nothing and wait. */
Critical Section /* . . . */
turn = 0;
/* Other code */
176
PROCESSES AND THREADS
CHAP. 2
For process P1, replace 0 by 1 in above code. Determine if the solution meets all the
required conditions for a correct mutual-exclusion solution.
31. Show how counting semaphores (i.e., semaphores that can hold an arbitrary value) can
be implemented using only binary semaphores and ordinary machine instructions.
32. If a system has only two processes, does it make sense to use a barrier to synchronize
them? Why or why not?
33. Can two threads in the same process synchronize using a kernel semaphore if the
threads are implemented by the kernel? What if they are implemented in user space?
Assume that no threads in any other processes have access to the semaphore. Discuss
your answers.
34. Suppose that we have a message-passing system using mailboxes. When sending to a
full mailbox or trying to receive from an empty one, a process does not block. Instead,
it gets an error code back. The process responds to the error code by just trying again,
over and over, until it succeeds. Does this scheme lead to race conditions?
35. The CDC 6600 computers could handle up to 10 I/O processes simultaneously using
an interesting form of round-robin scheduling called processor sharing. A process
switch occurred after each instruction, so instruction 1 came from process 1, instruction 2 came from process 2, etc. The process switching was done by special hardware,
and the overhead was zero. If a process needed T sec to complete in the absence of
competition, how much time would it need if processor sharing was used with n processes?
36. Consider the following piece of C code:
void main( ) {
fork( );
fork( );
exit( );
}
How many child processes are created upon execution of this program?
37. Round-robin schedulers normally maintain a list of all runnable processes, with each
process occurring exactly once in the list. What would happen (scheduling-wise) if a
process occurred twice in the list? Can you think of any reason for allowing this?
38. Can a measure of whether a process is likely to be CPU bound or I/O bound be determined by analyzing source code? How can this be determined at run time?
39. In the section ‘‘When to Schedule,’’ it was mentioned that sometimes scheduling could
be improved if an important process could play a role in selecting the next process to
run when it blocks. Give a situation where this could be used and explain how.
40. Explain how time quantum value and context switching time affect each other, in a
round-robin scheduling algorithm.
41. Measurements of a certain system have shown that the average process runs for a time
T before blocking on I/O. A process switch requires a time S, which is effectively
CHAP. 2
PROBLEMS
177
wasted (overhead). For round-robin scheduling with quantum Q, give a formula for
the CPU efficiency for each of the following:
(a) Q = '
(b) Q > T
(c) S < Q < T
(d) Q = S
(e) Q nearly 0
42. Five jobs are waiting to be run. Their expected run times are 9, 6, 3, 5, and X. In what
order should they be run to minimize average response time? (Your answer will
depend on X.)
43. Five batch jobs, A through E, arrive at almost the same time. They have estimated running times of 10, 6, 2, 4, and 8 minutes. Their (externally determined) priorities are 3,
5, 2, 1, and 4, respectively, with 5 being the highest priority. For each of the following
scheduling algorithms, determine the mean process turnaround time. Ignore process
switching overhead.
(a) Round robin.
(b) Priority scheduling.
(c) First-come, first-served (run in order 10, 6, 2, 4, 8).
(d) Shortest job first.
For (a), assume that the system is multiprogrammed, and that each job gets its fair
share of the CPU. For (b) through (d), assume that only one job at a time runs, until it
finishes. All jobs are completely CPU bound.
44. A process running on CTSS needs 30 quanta to complete. How many times must it be
swapped in, including the very first time (before it has run at all)?
45. Can you think of a way to save the CTSS priority system from being fooled by random
carriage returns?
46. Consider a real-time system with two voice calls of periodicity 5 msec each with CPU
time per call of 1 msec, and one video stream of periodicity 33 msec with CPU time
per call of 11 msec. Is this system schedulable? Show how you derived your answer.
47. For the above problem, can another video stream be added and have the system still be
schedulable?
48. The aging algorithm with a = 1/2 is being used to predict run times. The previous four
runs, from oldest to most recent, are 40, 20, 40, and 15 msec. What is the prediction of
the next time?
49. A soft real-time system has four periodic events with periods of 50, 100, 200, and 250
msec each. Suppose that the four events require 35, 20, 10, and x msec of CPU time,
respectively. What is the largest value of x for which the system is schedulable?
50. Explain why two-level scheduling is commonly used. What advantages does it have
over single-level scheduling?
51. A real-time system needs to handle two voice calls that each run every 5 msec and consume 1 msec of CPU time per burst, plus one video at 25 frames/sec, with each frame
178
PROCESSES AND THREADS
CHAP. 2
requiring 20 msec of CPU time. Is this system schedulable? Please explain why or why
not it is schedulable and how you came to that conclusion.
52. Consider a system in which it is desired to separate policy and mechanism for the
scheduling of kernel threads. Propose a means of achieving this goal.
53. The readers and writers problem can be formulated in several ways with regard to
which category of processes can be started when. Carefully describe three different
variations of the problem, each one favoring (or not favoring) some category of processes (e.g., readers or writers). For each variation, specify what happens when a reader
or a writer becomes ready to access the database, and what happens when a process is
finished.
54. Write a shell script that produces a file of sequential numbers by reading the last number in the file, adding 1 to it, and then appending it to the file. Run one instance of the
script in the background and one in the foreground, each accessing the same file. How
long does it take before a race condition manifests itself? What is the critical region?
Modify the script to prevent the race. (Hint: use
ln file file.lock
to lock the data file.)
55. Assume that you have an operating system that provides semaphores. Implement a
message system. Write the procedures for sending and receiving messages.
56. Rewrite the program of Fig. 2-23 to handle more than two processes.
57. Write a producer-consumer problem that uses threads and shares a common buffer.
However, do not use semaphores or any other synchronization primitives to guard the
shared data structures. Just let each thread access them when it wants to. Use sleep and
wakeup to handle the full and empty conditions. See how long it takes for a fatal race
condition to occur. For example, you might have the producer print a number once in a
while. Do not print more than one number every minute because the I/O could affect
the race conditions.
58. A process can be put into a round-robin queue more than once to give it a higher priority. Running multiple instances of a program each working on a different part of a data
pool can have the same effect. First write a program that tests a list of numbers for primality. Then devise a method to allow multiple instances of the program to run at once
in such a way that no two instances of the program will work on the same number. Can
you in fact get through the list faster by running multiple copies of the program? Note
that your results will depend upon what else your computer is doing; on a personal
computer running only instances of this program you would not expect an
improvement, but on a system with other processes, you should be able to grab a bigger share of the CPU this way.
59. Implement a program to count the frequency of words in a text file. The text file is
partitioned into N segments. Each segment is processed by a separate thread that outputs the intermediate frequency count for its segment. The main process waits until all
the threads complete; then it computes the consolidated word-frequency data based on
the individual threads’ output.
3
MEMORY MANAGEMENT
Main memory (RAM) is an important resource that must be very carefully
managed. While the average home computer nowadays has 100,000 times more
memory than the IBM 7094, the largest computer in the world in the early 1960s,
programs are getting bigger faster than memories. To paraphrase Parkinson’s Law,
‘‘Programs expand to fill the memory available to hold them.’’ In this chapter, we
will study how operating systems create abstractions from memory and how they
manage them.
What every programmer would like is a private, infinitely large, infinitely fast
memory that is also nonvolatile, that is, does not lose its contents when the electric
power is switched off. While we are at it, why not make it inexpensive, too? Unfortunately, technology does not provide such memories at present. Maybe you
will discover how to do it.
What is the second choice? Over the years, people discovered the concept of a
memory hierarchy, in which computers have a few megabytes of very fast, expensive, volatile cache memory, a few gigabytes of medium-speed, medium-priced,
volatile main memory, and a few terabytes of slow, cheap, nonvolatile magnetic or
solid-state storage, not to mention removable storage, such as USB sticks. It is the
job of the operating system to abstract this hierarchy into a useful model and then
manage the abstraction.
The part of the operating system that manages (part of) the memory hierarchy
is called the memory manager. Its job is to efficiently manage memory: keep
track of which parts of memory are in use, allocate memory to processes when
they need it, and deallocate it when they are done.
179
180
MEMORY MANAGEMENT
CHAP. 3
In this chapter, we will investigate several different memory management models, ranging from very simple to highly sophisticated. Since managing the lowest
level of cache memory is normally done by the hardware, the focus of this chapter
will be on the programmer’s model of main memory and how it can be managed.
The abstractions for, and the management of, permanent storage—the disk or
SSD—are the subject of the next chapter. We will first look at the simplest possible schemes and then gradually progress to more and more elaborate ones.
3.1 NO MEMORY ABSTRACTION
The simplest memory abstraction is to have no abstraction at all. Early mainframe computers (before 1960), early minicomputers (before 1970), and early personal computers (before 1980) had no memory abstraction. Every program simply
saw the physical memory. When a program executed an instruction like
MOV REGISTER1,1000
the computer just moved the contents of physical memory location 1000 to REGISTER1. Thus, the model of memory presented to the programmer was simply physical memory, a set of addresses from 0 to some maximum, each address corresponding to a cell containing some number of bits, commonly eight.
Under these conditions, it was not possible to have two running programs in
memory at the same time. If the first program wrote a new value to, say, location
2000, this would erase whatever value the second program was storing there. Nothing would work and both programs would crash almost immediately.
Even with the model of memory being just physical memory, several options
are possible. Three variations are shown in Fig. 3-1. The operating system may be
at the very bottom of memory in RAM (Random Access Memory), as shown in
Fig. 3-1(a), or it may be in ROM (Read-Only Memory) at the top of memory, as
shown in Fig. 3-1(b), or the device drivers may be at the top of memory in a ROM
and the rest of the system in RAM down below, as shown in Fig. 3-1(c). The first
model was formerly used on mainframes and minicomputers but is rarely used any
more. The second model is used on some handheld computers and embedded systems. The third model was used by early personal computers (e.g., running MSDOS), where the portion of the system in the ROM is called the BIOS (Basic Input
Output System). Models (a) and (c) have the disadvantage that a bug in the user
program can wipe out the operating system, possibly with disastrous results.
When the system is organized in this way, generally only one process at a time
can be running. As soon as the user types a command, the operating system copies
the requested program from nonvolatile storage to memory and executes it. When
the process finishes, the operating system displays a prompt character and waits for
a user new command. When the operating system receives the command, it loads a
new program into memory, overwriting the first one.
SEC. 3.1
181
NO MEMORY ABSTRACTION
0xFFF …
User
program
Device
drivers in ROM
Operating
system in
ROM
User
program
User
program
Operating
system in
RAM
Operating
system in
RAM
0
(a)
0
(b)
0
(c)
Figure 3-1. Three simple ways of organizing memory with an operating system
and one user process. Other possibilities also exist.
One way to get some parallelism in a system with no memory abstraction is to
program with multiple threads. Since all threads in a process are supposed to see
the same memory image, the fact that they are forced to is not a problem. While
this idea works, it is of limited use since what people often want is unrelated programs to be running at the same time, something the threads abstraction does not
provide. Furthermore, any system that is so primitive as to provide no memory
abstraction is unlikely to provide a threads abstraction.
3.1.1 Running Multiple Programs Without a Memory Abstraction
However, even with no memory abstraction, it is possible to run multiple programs at the same time. What the operating system has to do is save the entire contents of memory to a file on nonvolatile storage, then bring in and run the next program. As long as there is only one program at a time in memory, there are no conflicts. This concept (swapping) will be discussed below.
With the addition of some special hardware, it is possible to run multiple programs concurrently, even without swapping. The early models of the IBM 360
solved the problem as follows. Memory was divided into 2-KB blocks and each
was assigned a 4-bit protection key held in special registers inside the CPU. A
machine with a 1-MB memory needed only 512 of these 4-bit registers for a total
of 256 bytes of key storage. The PSW (Program Status Word) also contained a
4-bit key. The 360 hardware trapped any attempt by a running process to access
memory with a protection code different from the PSW key. Since only the operating system could change the protection keys, user processes were prevented from
interfering with one another and with the operating system itself.
Nevertheless, this solution had a major drawback, depicted in Fig. 3-2. Here
we have two programs, each 16 KB in size, as shown in Fig. 3-2(a) and (b). The
former is shaded to indicate that it has a different memory key than the latter. The
182
MEMORY MANAGEMENT
CHAP. 3
first program starts out by jumping to address 24, which contains a MOV instruction. The second program starts out by jumping to address 28, which contains a
CMP instruction. The instructions that are not relevant to this discussion are not
shown. When the two programs are loaded consecutively in memory starting at
address 0, we have the situation of Fig. 3-2(c). For this example, we assume the
operating system is in high memory and thus not shown.
0
..
.
ADD
MOV
JMP 24
(a)
16380
28
24
20
16
12
8
4
0
0
..
.
CMP
JMP 28
(b)
16380
28
24
20
16
12
8
4
0
0
..
.
32764
CMP
16412
16408
16404
16400
16396
16392
16388
16384
16380
JMP 28
0
..
.
ADD
MOV
JMP 24
28
24
20
16
12
8
4
0
(c)
Figure 3-2. Illustration of the relocation problem. (a) A 16-KB program.
(b) Another 16-KB program. (c) The two programs loaded consecutively
into memory.
After the programs are loaded, they can be run. Since they have different memory keys, neither one can damage the other. But the problem is of a different
nature. When the first program starts, it executes the JMP 24 instruction, which
jumps to the instruction, as expected. This program functions normally.
However, after the first program has run long enough, the operating system
may decide to run the second program, which has been loaded above the first one,
at address 16,384. The first instruction executed is JMP 28, which jumps to the
ADD instruction in the first program, instead of the CMP instruction it is supposed
to jump to. The program will most likely crash in well under 1 sec.
The core problem here is that the two programs both reference absolute physical memory. That is not what we want at all. What we want is that each program
can reference a private set of addresses local to it. We will show how this can be
SEC. 3.1
NO MEMORY ABSTRACTION
183
accomplished shortly. What the IBM 360 did as a stop-gap solution was modify
the second program on the fly as it loaded it into memory using a technique known
as static relocation. It worked like this. When a program was loaded at address
16,384, the constant 16,384 was added to every program address during the load
process (so ‘‘JMP 28’’ became ‘‘JMP 16,412’’, etc.).While this mechanism works
if done right, it is not a very general solution and slows down loading. Furthermore, it requires extra information in all executable programs to indicate which
words contain (relocatable) addresses and which do not. After all, the ‘‘28’’ in
Fig. 3-2(b) has to be relocated but an instruction like
MOV REGISTER1,28
which moves the number 28 to REGISTER1 must not be relocated. The loader needs
some way to tell what is an address and what is a constant.
Finally, as we pointed out in Chap. 1, history tends to repeat itself in the computer world. While direct addressing of physical memory is but a distant memory
(sorry) on mainframes, minicomputers, desktop computers, notebooks, and smartphones, the lack of a memory abstraction is still common in embedded and smart
card systems. Devices such as radios, washing machines, and microwave ovens are
all full of software (in ROM) these days, and in most cases the software addresses
absolute memory. This works because all the programs are known in advance and
users are not free to run their own software on their toaster.
In fact, history loops back upon itself in interesting ways. For instance, modern
Intel x86 processors have advanced forms of memory management and isolation
(as we shall see), far more powerful than the simple combination of protection keys
and static relocation in the IBM 360. Nevertheless, Intel started adding these exact
(and seemingly old-fashioned) protection keys to its CPUs only in 2017, more than
50 years after the first IBM 360 came into use. Now they are touted as an important security-enhancing innovation.
Conversely, where high-end embedded systems (such as smartphones) have
elaborate operating systems, simpler ones do not. In some cases, there is an operating system, but it is just a library that is linked with the application program and
provides system calls for performing I/O and other common tasks. The e-Cos operating system is a common example of an operating system as library.
3.2 A MEMORY ABSTRACTION: ADDRESS SPACES
All in all, exposing physical memory to processes has several major drawbacks. First, if user programs can address every byte of memory, they can easily
trash the operating system, intentionally or by accident, bringing the system to a
grinding halt (unless there is special hardware like the IBM 360’s lock-and-key
scheme). This problem exists even if only one user program (application) is running. Second, with this model, it is difficult to have multiple programs running at
184
MEMORY MANAGEMENT
CHAP. 3
once (taking turns, if there is only one CPU). On personal computers, it is common to have several programs open at once (a word processor, an email program, a
Web browser), one of them having the current focus, but the others being reactivated at the click of a mouse. Since this situation is difficult to achieve when there
is no abstraction from physical memory, something had to be done.
3.2.1 The Notion of an Address Space
Two problems have to be solved to allow multiple applications to be in memory at the same time without interfering with each other: protection and relocation.
We looked at a primitive solution to the former used on the IBM 360: label chunks
of memory with a protection key and compare the key of the executing process to
that of every memory word fetched. However, this approach by itself does not
solve the latter problem, although it can be solved by relocating programs as they
are loaded, but this is a slow and complicated solution.
A better solution is to invent a new abstraction for memory: the address space.
Just as the process concept creates a kind of abstract CPU to run programs, the
address space creates a kind of abstract memory for programs to use. An address
space is the set of addresses that a process can use to address memory. Each process has its own address space, independent of those of other processes (except in
some special circumstances where processes want to share their address spaces).
The concept of an address space is very general and occurs in many contexts.
Consider telephone numbers. In the United States and many other countries, a
local telephone number is usually a 7-digit number. The address space for telephone numbers thus runs from 0,000,000 to 9,999,999, although some numbers,
such as those beginning with 000 are not used. The address space for I/O ports on
the x86 runs from 0 to 16383. IPv4 addresses are 32-bit numbers, so their address
space runs from 0 to 232 < 1 (again, with some reserved numbers).
Address spaces do not have to be numeric. The set of .com Internet domains is
also an address space. This address space consists of all the strings of length 2 to
63 characters that can be made using letters, numbers, and hyphens, followed by
.com. By now you should get the idea. It is fairly simple.
Somewhat harder is how to give each program its own address space, so
address 28 in one program means a different physical location than address 28 in
another program. Below we will discuss a simple way that used to be common but
has fallen into disuse due to the ability to put much more complicated (and better)
schemes on modern CPU chips.
Base and Limit Registers
This simple solution uses a particularly simple version of dynamic relocation.
What it does is map each process’ address space onto a different part of physical
memory in a simple way. The classical solution, which was used on machines
ranging from the CDC 6600 (the world’s first supercomputer) to the Intel 8088 (the
SEC. 3.2
A MEMORY ABSTRACTION: ADDRESS SPACES
185
heart of the original IBM PC), is to equip each CPU with two special hardware
registers, usually called the base and limit registers. When these registers are used,
programs are loaded into consecutive memory locations wherever there is room
and without relocation during loading, as shown in Fig. 3-2(c). When a process is
run, the base register is loaded with the physical address where its program begins
in memory and the limit register is loaded with the length of the program. In
Fig. 3-2(c), the base and limit values that would be loaded into these hardware registers when the first program is run are 0 and 16,384, respectively. The values used
when the second program is run are 16,384 and 32,768, respectively. If a third
16-KB program were loaded directly above the second one and run, the base and
limit registers would be 32,768 and 16,384.
Every time a process references memory, either to fetch an instruction or read
or write a data word, the CPU hardware automatically adds the base value to the
address generated by the process before sending the address out on the memory
bus. Simultaneously, it checks whether the address offered is equal to or greater
than the value in the limit register, in which case a fault is generated and the access
is terminated. Thus, in the case of the first instruction of the second program in
Fig. 3-2(c), the process executes a
JMP 28
instruction, but the hardware treats it as though it were
JMP 16412
so it lands on the CMP instruction as expected. The settings of the base and limit
registers during the execution of the second program of Fig. 3-2(c) are shown in
Fig. 3-3.
Using base and limit registers is an easy way to give each process its own private address space because every memory address generated automatically has the
base-register contents added to it before being sent to memory. In many implementations, the base and limit registers are protected in such a way that only the
operating system can modify them. This was the case on the CDC 6600, but not on
the Intel 8088, which did not even have the limit register. It did have multiple base
registers, allowing program text and data, for example, to be independently relocated, but offered no protection from out-of-range memory references.
A disadvantage of relocation using base and limit registers is the need to perform an addition and a comparison on every memory reference. Comparisons can
be done fast, but additions are slow due to carry-propagation time unless special
addition circuits are used.
3.2.2 Swapping
If the physical memory of the computer is large enough to hold all the processes, the schemes described so far will more or less do. But in practice, the total
amount of RAM needed by all the processes is often much more than can fit in
186
MEMORY MANAGEMENT
CHAP. 3
16384
Limit register
16384
Base register
0
..
.
32764
CMP
16412
16408
16404
16400
16396
16392
16388
16384
16380
JMP 28
0
..
.
ADD
MOV
JMP 24
28
24
20
16
12
8
4
0
(c)
Figure 3-3. Base and limit registers can be used to give each process a separate
address space.
memory. On a typical Windows, MacOS, or Linux system, something like 50–100
processes or more may be started up as soon as the computer is booted. For example, when a Windows application is installed, it often issues commands so that on
subsequent system boots, a process will be started that does nothing except check
for updates to the application. Such a process can easily occupy 5–10 MB of memory. Other background processes check for incoming mail, incoming network connections, and many other things. And all this is before the first user program is
started. Serious user application programs nowadays, like Photoshop, can require
almost a gigabyte just to boot and many gigabytes once they start processing data.
Consequently, keeping all processes in memory all the time requires a huge
amount of memory and cannot be done if there is insufficient memory.
Two general approaches to dealing with memory overload have been developed over the years. The simplest strategy, called swapping of processes, consists
of bringing in each process in its entirety, running it for a while, then putting it
back on nonvolatile storage (disk or SSD). Idle processes are mostly stored on nonvolatile storage, so they do not take up any memory when they are not running
(although some of them wake up periodically to do their work, then go to sleep
again). The other strategy, called virtual memory, allows programs to run even
SEC. 3.2
187
A MEMORY ABSTRACTION: ADDRESS SPACES
when they are only partially in main memory. Below we will study swapping; in
Sec. 3.3 we will examine virtual memory.
The operation of a swapping system is illustrated in Fig. 3-4. Initially, only
process A is in memory. Then processes B and C are created or swapped in from
nonvolatile storage. In Fig. 3-4(d) A is swapped out to nonvolatile storage. Then D
comes in and B goes out. Finally A comes in again. Since A is now at a different
location, addresses contained in it must be relocated, either by software when it is
swapped in or (more likely) by hardware during program execution. For example,
base and limit registers would work fine here.
Time
B
C
C
C
B
B
B
C
C
A
A
A
A
D
D
D
Operating
system
Operating
system
Operating
system
Operating
system
Operating
system
Operating
system
Operating
system
(a)
(b)
(c)
(d)
(e)
(f)
(g)
Figure 3-4. Memory allocation changes as processes come into memory and
leave it. The shaded regions are unused memory.
When swapping creates multiple holes in memory, it is possible to combine
them all into one big one by moving all the processes downward as far as possible.
This technique is known as memory compaction. It is usually not done because it
requires a lot of CPU time. For example, on a 16-GB machine that can copy 8
bytes in 8 nsec, it would take about 16 sec to compact all of memory.
A point that is worth making concerns how much memory should be allocated
for a process when it is created or swapped in. If processes are created with a fixed
size that never changes, then the allocation is simple: the operating system allocates exactly what is needed, no more and no less.
If, however, processes’ data segments can grow, for example, by dynamically
allocating memory from a heap, as in many programming languages, a problem
occurs whenever a process tries to grow. If a hole is adjacent to the process, it can
be allocated and the process allowed to grow into the hole. On the other hand, if
the process is adjacent to another process, the growing process will either have to
be moved to a hole in memory large enough for it, or one or more processes will
have to be swapped out to create a large enough hole. If a process cannot grow in
memory and the swap area on the disk or SSD is full, the process will have to suspended until some space is freed up (or it can be killed).
188
MEMORY MANAGEMENT
CHAP. 3
If it is expected that most processes will grow as they run, it is probably a good
idea to allocate a little extra memory whenever a process is swapped in or moved,
to reduce the overhead associated with moving or swapping processes that no longer fit in their allocated memory. However, when swapping processes to nonvolatile
storage, only the memory actually in use should be swapped; it is wasteful to swap
the extra memory as well. In Fig. 3-5(a), we see a memory configuration in which
space for growth has been allocated to two processes.
B-Stack
Room for growth
Room for growth
B-Data
B
Actually in use
B-Program
A-Stack
Room for growth
Room for growth
A-Data
A
Actually in use
A-Program
Operating
system
Operating
system
(a)
(b)
Figure 3-5. (a) Allocating space for a growing data segment. (b) Allocating
space for a growing stack and a growing data segment.
If processes can have two growing segments—for example, the data segment
being used as a heap for variables that are dynamically allocated and released and a
stack segment for the normal local variables and return addresses—an alternative
arrangement suggests itself, namely that of Fig. 3-5(b). In this figure, we see that
each process illustrated has a stack at the top of its allocated memory that is growing downward, and a data segment just beyond the program text that is growing
upward. The memory between them can be used for either segment. If it runs out,
the process will either have to be moved to a hole with sufficient space, swapped
out of memory until a large enough hole can be created, or killed.
3.2.3 Managing Free Memory
When memory is assigned dynamically, the operating system must manage it.
In general terms, there are two ways to keep track of memory usage: bitmaps and
free lists. In this section and the next one, we will look at these two methods. In
SEC. 3.2
189
A MEMORY ABSTRACTION: ADDRESS SPACES
Chapter 10, we will look at some specific memory allocators used in Linux (like
buddy and slab allocators) in more detail. We will also see in later chapters that
tracking the usage of resources is not specific to memory management. For
instance, file systems also need to keep track of free disk blocks. In fact, keeping
track of what slots are free in an set of resources is common in many programs.
Memory Management with Bitmaps
With a bitmap, memory is divided into allocation units as small as a few words
and as large as several kilobytes. Corresponding to each allocation unit is a bit in
the bitmap, which is 0 if the unit is free and 1 if it is occupied (or vice versa). Figure 3-6(a) shows part of memory and the corresponding bitmap in Fig. 3-6(b).
A
B
C
8
D
16
E
24
(a)
11111000
P
0
5
H
5
3
P
8
6
P 14 4
P 26 3
H 29 3
11111111
11001111
11111000
H 18 2
Hole Starts Length
at 18
2
(b)
P 20 6
X
Process
(c)
Figure 3-6. (a) A part of memory with five processes and three holes. The tick
marks show the memory allocation units. The shaded regions (0 in the bitmap)
are free. (b) The corresponding bitmap. (c) The same information as a list.
The size of the allocation unit is an important design issue. The smaller the
allocation unit, the larger the bitmap. However, even with an allocation unit as
small as 4 bytes, 32 bits of memory will require only 1 bit of the map. A memory
of 32n bits will use n map bits, so the bitmap will take up only 1/32 of memory. If
the allocation unit is chosen large, the bitmap will be smaller, but appreciable
memory may be wasted in the last unit of the process if the process size is not an
exact multiple of the allocation unit.
A bitmap provides a simple way to keep track of memory words in a fixed
amount of memory because the size of the bitmap depends only on the size of
memory and the size of the allocation unit. The main problem is that when it has
been decided to bring a k-unit process into memory, the memory manager must
search the bitmap to find a run of k consecutive 0 bits in the map. Searching a bitmap for a run of a given length is a slow operation (because the run may straddle
word boundaries in the map); this is an argument against bitmaps.
190
MEMORY MANAGEMENT
CHAP. 3
Memory Management with Linked Lists
Another way of keeping track of memory is to maintain a linked list of allocated and free memory segments, where a segment either contains a process or is
an empty hole between two processes. The memory of Fig. 3-6(a) is represented in
Fig. 3-6(c) as a linked list of segments. Each entry in the list specifies a hole (H) or
process (P), the address at which it starts, the length, and a pointer to the next item.
In this example, the segment list is kept sorted by address. Sorting this way has
the advantage that when a process terminates or is swapped out, updating the list is
straightforward. A terminating process normally has two neighbors (except when
it is at the very top or bottom of memory). These may be either processes or holes,
leading to the four combinations shown in Fig. 3-7. In Fig. 3-7(a) updating the list
requires replacing a P by an H. In Fig. 3-7(b) and (c), two entries are coalesced
into one, and the list becomes one entry shorter. In Fig. 3-7(d), three entries are
merged and two items are removed from the list.
Since the process table slot for the terminating process will normally point to
the list entry for the process itself, it may be more convenient to have the list as a
double-linked list, rather than the single-linked list of Fig. 3-6(c). This structure
makes it easier to find the previous entry and to see if a merge is possible.
Before X terminates
(a)
A
X
(b)
A
X
(c)
X
(d)
X
B
B
After X terminates
becomes
A
becomes
A
becomes
B
B
becomes
Figure 3-7. Four neighbor combinations for the terminating process, X.
When the processes and holes are kept on a list sorted by address, several algorithms can be used to allocate memory for a created process (or an existing process
being swapped in from disk or SSD). We assume that the memory manager knows
how much memory to allocate. The simplest algorithm is first fit. The memory
manager scans along the list of segments until it finds a hole that is big enough.
The hole is then broken up into two pieces, one for the process and one for the
unused memory, except in the statistically unlikely case of an exact fit. First fit is a
fast algorithm because it searches as little as possible.
A minor variation of first fit is next fit. It works the same way as first fit,
except that it keeps track of where it is whenever it finds a suitable hole. The next
time it is called to find a hole, it starts searching the list from the place where it left
off last time, instead of always at the beginning, as first fit does. Simulations by
Bays (1977) show that next fit gives slightly worse performance than first fit.
SEC. 3.2
A MEMORY ABSTRACTION: ADDRESS SPACES
191
Another well-known and widely used algorithm is best fit. Best fit searches
the entire list, from beginning to end, and takes the smallest hole that is adequate.
Rather than breaking up a big hole that might be needed later, best fit tries to find a
hole that is close to the actual size needed, to best match the request and the available holes.
As an example of first fit and best fit, consider Fig. 3-6 again. If a block of
size 2 is needed, first fit will allocate the hole at 5, but best fit will allocate the hole
at 18.
Best fit is slower than first fit because it must search the entire list every time it
is called. Somewhat surprisingly, it also results in more wasted memory than first
fit or next fit because it tends to fill up memory with tiny, useless holes. First fit
generates larger holes on the average.
To get around the problem of breaking up nearly exact matches into a process
and a tiny hole, one could think about worst fit, that is, always take the largest
available hole, so that the new hole will be big enough to be useful. Simulation has
shown that worst fit is not a very good idea either.
All four algorithms can be speeded up by maintaining separate lists for processes and holes. In this way, all of them devote their full energy to inspecting
holes, not processes. The inevitable price that is paid for this speedup on allocation
is the additional complexity and slowdown when deallocating memory, since a
freed segment has to be removed from the process list and inserted into the hole
list.
If distinct lists are maintained for processes and holes, the hole list may be kept
sorted on size, to make best fit faster. When best fit searches a list of holes from
smallest to largest, as soon as it finds a hole that fits, it knows that the hole is the
smallest one that will do the job, hence the best fit. No further searching is needed,
as it is with the single-list scheme. With a hole list sorted by size, first fit and best
fit are equally fast, and next fit is pointless.
When the holes are kept on separate lists from the processes, a small optimization is possible. Instead of having a separate set of data structures for maintaining
the hole list, as is done in Fig. 3-6(c), the information can be stored in the holes.
The first word of each hole could be the hole size, and the second word a pointer to
the following entry. The nodes of the list of Fig. 3-6(c), which require three words
and one bit (P/H), are no longer needed.
Yet another allocation algorithm is quick fit, which maintains separate lists for
some of the more common sizes requested. For example, it might have a table with
n entries, in which the first entry is a pointer to the head of a list of 4-KB holes, the
second entry is a pointer to a list of 8-KB holes, the third entry a pointer to 12-KB
holes, and so on. Holes of, say, 21 KB, could be put either on the 20-KB list or on
a special list of odd-sized holes.
With quick fit, finding a hole of the required size is extremely fast, but it has
the same disadvantage as all schemes that sort by hole size, namely, when a process terminates or is swapped out, finding its neighbors to see if a merge with them
192
MEMORY MANAGEMENT
CHAP. 3
is possible is quite expensive. If merging is not done, memory will quickly fragment into a large number of small holes into which no processes fit.
3.3 VIRTUAL MEMORY
While base and limit registers can be used to create the abstraction of address
spaces, there is another problem that has to be solved: managing bloatware. While
memory sizes are increasing rapidly, software sizes are increasing much faster. In
the 1980s, many universities ran a timesharing system with dozens of (more-or-less
satisfied) users running simultaneously on a 4-MB VAX. Now Microsoft recommends having at least 2 GB for 64-bit Windows 10.
As a consequence of these developments, there is a need to run programs that
are too large to fit in memory, and there is certainly a need to have systems that can
support multiple programs running simultaneously, each of which fits in memory
but all of which collectively exceed memory. Swapping is not an attractive option
if your computer is equipped with a hard disk, since a typical SATA disk has a
peak transfer rate of several hundreds of MB/sec, which means it takes seconds to
swap out a 1-GB program and the same to swap in a 1-GB program. While SSDs
are considerably faster, even here the overhead is substantial.
The problem of programs larger than memory has been around since the beginning of computing, albeit in limited areas, such as science and engineering (simulating the creation of the universe or even simulating a new aircraft takes a lot of
memory). A solution adopted in the 1960s was to split programs into little pieces,
called overlays. When a program started, all that was loaded into memory was the
overlay manager, which immediately loaded and ran overlay 0. When it was done,
it would tell the overlay manager to load overlay 1, either above overlay 0 in memory (if there was space for it) or on top of overlay 0 (if there was no space). Some
overlay systems were highly complex, allowing many overlays in memory at once.
The overlays were kept on nonvolatile storage and swapped in and out of memory
by the overlay manager.
Although the actual work of swapping overlays in and out was done by the
operating system, the work of splitting the program into pieces had to be done
manually by the programmer. Splitting large programs up into small, modular
pieces was time consuming, boring, and error prone. Few programmers were good
at this. It did not take long before someone thought of a way to turn the whole job
over to the computer.
The method that was devised (Fotheringham, 1961) has come to be known as
virtual memory. The basic idea behind virtual memory is that each program has
its own address space, which is broken up into chunks called pages. Each page is
a contiguous range of addresses. These pages are mapped onto physical memory,
but not all pages have to be in physical memory at the same time to run the program. When the program references a part of its address space that is in physical
SEC. 3.3
VIRTUAL MEMORY
193
memory, the hardware performs the necessary mapping on the fly. When the program references a part of its address space that is not in physical memory, the operating system is alerted to go get the missing piece and re-execute the instruction
that failed.
In a sense, virtual memory is a generalization of the base-and-limit-register
idea. The 8088 had separate base registers (but no limit registers) for text and data.
With virtual memory, instead of having separate relocation for just the text and
data segments, the entire address space can be mapped onto physical memory in
fairly small units. Different implementations of virtual memory make different
choices with respect to these units. Nowadays most systems use a technique called
paging where the units are fixed-size units of, say, 4 KB. In contrast, an alternative
solution known as segmentation uses entire variable-size segments as units. We
will look at both solutions, but focus on paging, as segmentation is not really used
these days anymore.
Virtual memory works just fine in a multiprogramming system, with bits and
pieces of many programs in memory at once. While a program is waiting for
pieces of itself to be read in, the CPU can be given to another process.
3.3.1 Paging
Most virtual memory systems use a technique called paging, which we will
now describe. On any computer, programs reference a set of memory addresses.
When a program executes an instruction like
MOV REG,1000
it does so to copy the contents of memory address 1000 to REG (assuming the first
operand represents the destination and the second the source). Addresses can be
generated using indexing, base registers, and various other ways.
These program-generated addresses are called virtual addresses and form the
virtual address space. On computers without virtual memory, the virtual address
is put directly onto the memory bus and causes the physical memory word with the
same address to be read or written. When virtual memory is used, the virtual addresses do not go directly to the memory bus. Instead, they go to an MMU (Memory Management Unit) that maps the virtual addresses onto the physical memory
addresses, as illustrated in Fig. 3-8.
A very simple example of how this mapping works is shown in Fig. 3-9. In
this example, we have a computer that generates 16-bit addresses, from 0 up to
64K < 1. These are the virtual addresses. This computer, however, has only 32 KB
of physical memory. So although 64-KB programs can be written, they cannot be
loaded into memory in their entirety and run. A complete copy of a program’s core
image, up to 64 KB, must be present on the disk or SSD, however, so that pieces
can be brought in dynamically as needed.
194
MEMORY MANAGEMENT
CPU
package
CHAP. 3
The CPU sends virtual
addresses to the MMU
CPU
Memory
management
unit
Memory
Disk
controller
Bus
The MMU sends physical
addresses to the memory
Figure 3-8. The position and function of the MMU. Here the MMU is shown as
being a part of the CPU chip because it commonly is nowadays. However, logically it could be a separate chip and was years ago.
The virtual address space consists of fixed-size units called pages. The corresponding units in the physical memory are called page frames. The pages and page
frames are the same size. In this example they are 4 KB, but page sizes from 512
bytes to a gigabyte have been used in real systems. With 64 KB of virtual address
space and 32 KB of physical memory, we get 16 virtual pages and 8 page frames.
Transfers between RAM and nonvolatile storage are always in whole pages. Many
processors support multiple page sizes that can be mixed and matched as the operating system sees fit. For instance, the x86-64 architecture supports 4-KB, 2-MB,
and 1-GB pages, so we could use 4-KB pages for user applications and a single
1-GB page for the kernel. We will see later why it is sometimes better to use a single large page, rather than a large number of small ones.
The notation in Fig. 3-9 is as follows. The range marked 0K–4K means that
the virtual or physical addresses in that page are 0 to 4095. The range 4K–8K
refers to addresses 4096 to 8191, and so on. Each page contains exactly 4096
addresses starting at a multiple of 4096 and ending one shy of a multiple of 4096.
When the program tries to access address 0, for example, using the instruction
MOV REG,0
virtual address 0 is sent to the MMU. The MMU sees that this virtual address falls
in page 0, that is, in the range of 0 to 4095, which according to its mapping is page
frame 2 (8192 to 12287). It thus transforms the address to 8192 and outputs
address 8192 onto the bus. The memory knows nothing at all about the MMU and
just sees a request for reading or writing address 8192, which it honors. Thus, the
MMU has effectively mapped all virtual addresses between 0 and 4095 onto physical addresses 8192 to 12287.
SEC. 3.3
195
VIRTUAL MEMORY
Virtual
address
space
60K–64K
X
56K–60K
X
52K–56K
X
48K–52K
X
Virtual page
44K–48K
7
40K–44K
X
36K–40K
5
32K–36K
X
Physical
memory
address
28K–32K
X
28K–32K
24K–28K
X
24K–28K
20K–24K
3
20K–24K
16K–20K
4
16K–20K
12K–16K
0
12K–16K
8K–12K
6
8K–12K
4K–8K
1
4K–8K
0K–4K
2
0K–4K
Page frame
Figure 3-9. The relation between virtual addresses and physical memory addresses is given by the page table. Every page begins on a multiple of 4096 and ends
4095 addresses higher, so 4K–8K really means 4096–8191 and 8K–12K means
8192–12287.
Similarly, the instruction
MOV REG,8192
is effectively transformed into
MOV REG,24576
because virtual address 8192 (in virtual page 2) is mapped onto 24576 (in physical
page frame 6). As a third example, virtual address 20500 is 20 bytes from the start
of virtual page 5 (virtual addresses 20480 to 24575) and maps onto physical
address 12288 + 20 = 12308.
By itself, this ability to map the 16 virtual pages onto any of the eight page
frames by setting the MMU’s map appropriately does not solve the problem that
the virtual address space is larger than the physical memory. Since we have only
eight physical page frames, only eight of the virtual pages in Fig. 3-9 are mapped
onto physical memory. The others, shown as a cross in the figure, are not mapped.
In the actual hardware, a Present/absent bit keeps track of which pages are physically present in memory.
196
MEMORY MANAGEMENT
CHAP. 3
What happens if the program references an unmapped address, for example, by
using the instruction
MOV REG,32780
which is byte 12 within virtual page 8 (starting at 32768)? The MMU sees that the
page is unmapped (indicated by a cross in the figure) and causes the CPU to trap to
the operating system, called a page fault. The operating system picks a little-used
page frame and writes its contents back to the disk (if it is not already there). It
then fetches (also from the disk) the page that was just referenced into the page
frame just freed, changes the map, and restarts the trapped instruction.
For example, if the operating system decided to evict page frame 1 from memory, it would load virtual page 8 at physical address 4096 and make two changes to
the MMU map. First, it would mark virtual page 1’s entry as unmapped, to trap any
future accesses to virtual addresses between 4096 and 8191. Then it would replace
the cross in virtual page 8’s entry with a 1, so that when the trapped instruction is
reexecuted, it will map virtual address 32780 to physical address 4108 (4096 + 12).
Now let us look inside the MMU to see how it works and why we have chosen
to use a page size that is a power of 2. In Fig. 3-10 we see an example of a virtual
address, 8196 (0010000000000100 in binary), being mapped using the MMU map
of Fig. 3-9. The incoming 16-bit virtual address is split into a 4-bit page number
and a 12-bit offset. With 4 bits for the page number, we can have 16 pages, and
with 12 bits for the offset, we can address all 4096 bytes within a page.
The page number is used as an index into the page table, yielding the number
of the page frame that corresponds to that virtual page. If the Present/absent bit is
0, a trap to the operating system is caused. If the bit is 1, the page frame number
found in the page table is copied to the high-order 3 bits of the output register,
along with the 12-bit offset, which is copied unmodified from the incoming virtual
address. Together they form a 15-bit physical address. The output register is then
put onto the memory bus as the physical memory address.
In our examples, we are using 16-bit addresses to make the text and figures
easier to understand. Modern PCs use 32-bit or 64-bit addresses. In principle, a
computer with 32-bit addresses and 4-KB pages could use exactly the same method discussed above. The page table would need 220 (1,048,576) entries. On a computer with gigabytes of RAM, that is doable. However, 64-bit addresses and 4-KB
pages would require 252 (roughly 4. 5 × 1015 ) entries in the page table, also known
as a ‘‘hell of a lot’’. That is definitely not doable, so other techniques are needed.
We will discuss them shortly.
3.3.2 Page Tables
In a simple implementation, the mapping of virtual addresses onto physical
addresses can be summarized as follows: the virtual address is split into a virtual
page number (high-order bits) and an offset (low-order bits). For example, with a
SEC. 3.3
197
VIRTUAL MEMORY
1 1 0 0 0 0 0 0 0 0 0 0 1 0 0
Page
table
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
0
000
000
000
000
111
000
101
000
000
000
011
100
000
110
001
010
0
0
0
0
1
0
1
0
0
0
1
1
1
1
1
1
Outgoing
physical
address
(24580)
12-bit offset
copied directly
from input
to output
110
Present/
absent bit
Virtual page = 2 is used
as an index into the
page table
0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0
Incoming
virtual
address
(8196)
Figure 3-10. The internal operation of the MMU with 16 4-KB pages.
16-bit address and a 4-KB page size, the upper 4 bits could specify one of the 16
virtual pages and the lower 12 bits would then specify the byte offset (0 to 4095)
within the selected page. However, a split with 3 or 5 or some other number of bits
for the page is also possible. Different splits imply different page sizes.
The virtual page number is used as an index into the page table to find the
entry for that virtual page. From the page table entry, the page frame number (if
any) is found. The page frame number is attached to the high-order end of the offset, replacing the virtual page number, to form a physical address that can be sent
to the memory.
Thus, the purpose of the page table is to map virtual pages onto page frames.
Mathematically speaking, the page table is a function, with the virtual page number as the argument and the physical frame number as result. Using the result of
this function, the virtual page field in a virtual address can be replaced by a page
frame field, thus forming a physical memory address.
198
MEMORY MANAGEMENT
CHAP. 3
In this chapter, we worry only about virtual memory and not full virtualization.
In other words: no virtual machines yet. We will see in Chap. 7 that each virtual
machine requires its own virtual memory and as a result the page table organization becomes much more complicated—involving shadow or nested page tables
and more. Even without such arcane configurations, paging and virtual memory
are fairly sophisticated, as we shall see.
Structure of a Page Table Entry
Let us now turn from the structure of the page tables in the large, to the details
of a single page table entry. The exact layout of an entry in the page table is highly
machine dependent, but the kind of information present is roughly the same from
machine to machine. In Fig. 3-11, we present a sample page table entry. The size
varies from computer to computer, but 64 bits is a common size on today’s general
purpose computers. The most important field is the Page frame number. After all,
the goal of the page mapping is to output this value. If the page size is 4-KB (i.e.,
212 bytes), we only need the most significant 52 bits for the page frame number†,
leaving 12 bits to encode other information about the page. For instance, the
Present/absent bit indicates whether the entry is valid and can be used. If this bit is
0, the virtual page to which the entry belongs is not currently in memory. Accessing a page table entry with this bit set to 0 causes a page fault.
Modified
Caching
disabled Present/absent
Page frame number
Supervisor
Protection
Referenced
Figure 3-11. A typical page table entry.
The Protection bits tell what kinds of access are permitted. In the simplest
form, this field contains 1 bit, with 0 for read/write and 1 for read only. A more
sophisticated arrangement is having 3 bits, one bit each for enabling reading, writing, and executing the page. Somewhat related is the Supervisor bit that indicates
whether the page is accessible only to privileged code, i.e., the operating system
(or supervisor) or also to user programs. Any attempt by a user program to access a
supervisor page will result in a fault.
The Modified and Referenced bits keep track of page usage. When a page is
written to, the hardware automatically sets the Modified bit. This bit is of value
when the operating system decides to reclaim a page frame. If the page in it has
been modified (i.e., is ‘‘dirty’’), it must be written back to nonvolatile storage. If it
† Most 64-bit CPUs use only 48-bit addresses by design, so 36 bits suffice for the page frame number.
SEC. 3.3
VIRTUAL MEMORY
199
has not been modified (i.e., is ‘‘clean’’), it can just be abandoned, since the copy on
disk or SSD is still valid. The bit is sometimes called the dirty bit, since it reflects
the page’s state.
The Referenced bit is set whenever a page is referenced, either for reading or
for writing. Its value is used to help the operating system choose a page to evict
when a page fault occurs. Pages that are not being used are far better candidates
than pages that are, and this bit plays an important role in several of the page replacement algorithms that we will study later in this chapter.
Finally, the last bit allows caching to be disabled for the page. This feature is
important for pages that map onto device registers rather than memory. If the operating system is sitting in a tight loop waiting for some I/O device to respond to a
command it was just given, it is essential that the hardware keep fetching the word
from the device, and not use an old cached copy. With this bit, caching can be turned off. Machines that have a separate I/O space and do not use memory-mapped
I/O do not need this bit.
Note that the disk address (the address of the block on the disk or SSD) used to
hold the page when it is not in memory is not part of the page table. The reason is
simple. The page table holds only that information the hardware needs to translate
a virtual address to a physical address. Information the operating system needs to
handle page faults is kept in software tables inside the operating system. The hardware does not need it.
Before getting into more implementation issues, it is worth pointing out again
that what virtual memory fundamentally does is create a new abstraction—the
address space—which is an abstraction of physical memory, just as a process is an
abstraction of the physical processor (CPU). Virtual memory can be implemented
by breaking the virtual address space up into pages, and mapping each one onto
some page frame of physical memory or having it (temporarily) unmapped. Thus
this section is basically about an abstraction created by the operating system and
how that abstraction is managed.
Also, it may be good to emphasize that all memory accesses made by software
use virtual addresses. This is not just true for the user processes, but also for the
operating system. In other words, the kernel has its own mappings in page tables
also. Whenever a process executes a system call, the operating system page tables
must be used. Because a context switch (which requires swapping page tables) is
not cheap, some systems employ a clever trick and simply map the operating system’s page tables in every user process, but with the Supervisor bit indicating that
these pages can only be accessed by the operating system. Thus, when the user
process tries to access such a page, it will trigger an exception. However, when the
user process performs a system call, there is no need to switch page tables anymore: all the kernel page tables and the user page tables are available for the operating system to. Doing so speeds up the system call. Generally, when the operating
system is mapped into user processes, it is mapped in at the top of the address
space so as not interfere with user programs, which start at or near 0. Sometimes
200
MEMORY MANAGEMENT
CHAP. 3
user programs start at 4K instead of 0 so that references to address 0 (which is
often an error) are trapped.
3.3.3 Speeding Up Paging
We have just seen the basics of virtual memory and paging. It is now time to
go into more detail about possible implementations. In any paging system, two
major issues must be faced:
1. The mapping from virtual address to physical address must be fast.
2. Even if the virtual address space itself is huge, the page table must not
be too large.
The first point is a consequence of the fact that the virtual-to-physical mapping
must be done on every memory reference. All instructions must ultimately come
from memory and many of them reference operands in memory as well. Consequently, it is necessary to make one, two, or sometimes more page table references
per instruction. If an instruction execution takes, say, 1 nsec, the page table lookup
must be done in under 0.2 nsec to avoid having the mapping become a major bottleneck.
The second point follows from the fact that all modern computers use virtual
addresses of at least 32 bits, with 64 bits becoming the norm for desktops and notebooks. Even if a modern processor uses only 48 out of the 64 bits for addressing,
with a 4-KB page size, a 48-bit address space has 64 billion pages. With 64 billion
pages in the virtual address space, the page table must have 64 billion entries of 64
bits each. Most people will agree that using hundreds of gigabytes just to store the
page table is a tad excessive. And remember that each process needs its own page
table (because it has its own virtual address space).
The need for fast page mapping for large address spaces is a very significant
constraint on the way computers are built nowadays. The simplest design (at least
conceptually) is to have a single page table consisting of an array of fast hardware
registers, with one entry for each virtual page, indexed by virtual page number, as
shown in Fig. 3-10. When a process is started up, the operating system loads the
registers with the process’ page table, taken from a copy kept in main memory.
During process execution, no more memory references are needed for the page table. The advantages of this method are that it is straightforward and requires no
memory references during mapping. A disadvantage is that it is unbearably expensive if the page table is large; it is just not practical most of the time. Another one
is that having to load the full page table at every context switch would completely
kill performance.
At the other extreme, the page table can be entirely in main memory. All the
hardware needs then is a single register that points to the start of the page table.
This design allows the virtual-to-physical map to be changed at a context switch by
SEC. 3.3
VIRTUAL MEMORY
201
reloading one register. Of course, it has the disadvantage of requiring one or more
memory references to read page table entries during the execution of each instruction, making it very slow.
Translation Lookaside Buffers
Let us now look at some widely implemented schemes for speeding up paging
and for handling large virtual address spaces, starting with the former. The starting
point of most optimization techniques is that the page table is in memory. Potentially, this design has an enormous impact on performance. Consider, for example,
a 1-byte instruction that copies one register to another. In the absence of paging,
this instruction makes only one memory reference, to fetch the instruction. With
paging, at least one additional memory reference will be needed, to access the page
table. Since execution speed is generally limited by the rate at which the CPU can
get instructions and data out of the memory, having to make two memory references per memory reference reduces performance by half. Under these conditions,
no one would use paging.
Computer designers have known about this problem for years and have come
up with a solution. Their solution is based on the observation that most programs
tend to make a large number of references to a small number of pages, and not the
other way around. Thus only a small fraction of the page table entries are heavily
read; the rest are barely used at all.
The solution that has been devised is to equip computers with a small hardware
device for mapping virtual addresses to physical addresses without going through
the page table. The device, called a TLB (Translation Lookaside Buffer) or
sometimes an associative memory, is illustrated in Fig. 3-12. It is usually inside
the MMU and consists of a small number of entries, eight in this example, but
rarely more than 256. Each entry contains information about one page, including
the virtual page number, a bit that is set when the page is modified, the protection
code (read/write/execute permissions), and the physical page frame in which the
page is located. These fields have a one-to-one correspondence with the fields in
the page table, except for the virtual page number, which is not needed in the page
table. Another bit indicates whether the entry is valid (i.e., in use) or not.
An example that might generate the TLB of Fig. 3-12 is a process in a loop
that spans virtual pages 19, 20, and 21, so that these TLB entries have protection
codes for reading and executing. The main data currently being used (say, an array
being processed) are on pages 129 and 130. Page 140 contains the indices used in
the array calculations. Finally, the stack is on pages 860 and 861.
Let us now see how the TLB functions. When a virtual address is presented to
the MMU for translation, the hardware first checks to see if its virtual page number
is present in the TLB by comparing it to all the entries simultaneously (i.e., in parallel). Doing so requires special hardware, which all MMUs with TLBs have. If a
valid match is found and the access does not violate the protection bits, the page
frame is taken directly from the TLB, without going to the page table in memory.
202
MEMORY MANAGEMENT
Modified
Protection
CHAP. 3
Valid
Virtual page
Page frame
1
140
1
RW
31
1
20
0
R X
38
1
130
1
RW
29
1
129
1
RW
62
1
19
0
R X
50
1
21
0
R X
45
1
860
1
RW
14
1
861
1
RW
75
Figure 3-12. A TLB to speed up paging.
If the virtual page number is present in the TLB but the instruction is trying to
write on a read-only page, a protection fault is generated.
The interesting case is what happens when the virtual page number is not in
the TLB. The MMU detects the miss and does an ordinary page table lookup. It
then evicts one of the entries from the TLB and replaces it with the page table
entry just looked up. Thus if that page is used again soon, the second time it will
result in a TLB hit rather than a miss. When an entry is purged from the TLB, the
modified bit is copied back into the page table entry in memory. The other values
are already there, except the reference bit. When the TLB is loaded from the page
table, all the fields are taken from memory.
If the operating system wants to change the bits in the page table entry (e.g., to
make a read-only page writable), it will do so by modifying it in memory. However, to make sure that the next write to that pages succeeds, it must also flush the
corresponding entry with the old permission bits from the TLB.
Software TLB Management
Up until now, we have assumed that every machine with paged virtual memory
has page tables recognized by the hardware, plus a TLB. In this design, TLB management and handling TLB faults are done entirely by the MMU hardware. Traps
to the operating system occur only when a page is not in memory.
This assumption is true for many CPUs. However, some RISC machines,
including the SPARC, MIPS, and (the now dead) HP PA, provide support for page
management in software. On these machines, the TLB entries are explicitly loaded
by the operating system. When a TLB miss occurs, instead of the MMU going to
the page tables to find and fetch the needed page reference, it just generates a TLB
fault and tosses the problem into the lap of the operating system. The system must
find the page, remove an entry from the TLB, enter the new one, and restart the
instruction that faulted. And, of course, all of this must be done in a handful of
instructions because TLB misses occur much more frequently than page faults.
SEC. 3.3
VIRTUAL MEMORY
203
Be sure you understand why TLB misses are far more common than page
faults. It is an important point. The key is that there are usually thousands of
pages in memory so page faults are rare but TLBs typically hold only 64 entries, so
TLB misses happen all the time. Hardware manufacturers could reduce the number of TLB misses by increasing the size of the TLB, but that is expensive and the
chip area an increased TLB would take would leave less space for other important
features such as caches. Chip design is full of trade-offs.
Surprisingly enough, if the TLB is moderately large (say, 64 entries) to reduce
the miss rate, software management of the TLB turns out to be acceptably efficient.
The main gain here is a much simpler MMU, which frees up area on the chip for
caches and other features that can improve performance.
It is essential to understand the difference between different kinds of misses.
A soft miss occurs when the page referenced is not in the TLB, but is in memory.
All that is needed here is for the TLB to be updated. No disk (or SSD) I/O is needed. Typically a soft miss takes 10–20 machine instructions to handle and can be
completed in a couple of nanoseconds. In contrast, a hard miss occurs when the
page itself is not in memory (and of course, also not in the TLB). An access to the
disk or SSD is required to bring in the page, which can take up to milliseconds,
depending on the nonvolatile storage being used. A hard miss is easily a million
times slower than a soft miss. Looking up the mapping in the page table hierarchy
is known as a page table walk.
Actually, it is worse than that. A miss is not just soft or hard. Some misses are
slightly softer (or slightly harder) than other misses. For instance, suppose the
page walk does not find the page in the process’ page table and the program thus
incurs a page fault. There are three possibilities. First, the page may actually be in
memory, but not in this process’ page table. For instance, the page may have been
brought in from nonvolatile storage by another process. In that case, we do not
need to access the nonvolatile storage again, but merely map the page appropriately
in the page tables. This is a pretty soft miss that is known as a minor page fault.
Second, a major page fault occurs if the page needs to be brought in from nonvolatile storage. Third, it is possible that the program simply accessed an invalid
address and no mapping needs to be added in the TLB at all. In that case, the operating system typically kills the program with a segmentation fault. Only in this
case did the program do something wrong. All other cases are automatically fixed
by the hardware and/or the operating system—at the cost of some performance.
3.3.4 Page Tables for Large Memories
TLBs can be used to speed up virtual-to-physical address translation over the
original page-table-in-memory scheme. But that is not the only problem we have to
tackle. Another problem is how to deal with very large virtual address spaces.
Below we will discuss two ways of dealing with them.
204
MEMORY MANAGEMENT
CHAP. 3
Multilevel Page Tables
As a first approach, consider the use of a multilevel page table. A simple example is shown in Fig. 3-13. In Fig. 3-13(a) we have a 32-bit virtual address that is
partitioned into a 10-bit PT1 field, a 10-bit PT2 field, and a 12-bit Offset field.
Since offsets are 12 bits, pages are 4 KB, and there are a total of 220 of them.
Second-level
page tables
Page
table for
the top
4M of
memory
Top-level
page table
1023
Bits
10
10
PT1
PT2 Offset
(a)
12
6
5
4
3
2
1
0
1023
6
5
4
3
2
1
0
To
pages
(b)
Figure 3-13. (a) A 32-bit address with two page table fields. (b) Two-level page
tables.
The secret to the multilevel page table method is to avoid keeping all the page
tables in memory all the time. In particular, those that are not needed should not
SEC. 3.3
VIRTUAL MEMORY
205
be kept around. Suppose, for example, that a process needs 12 megabytes: the bottom 4 megabytes of memory for program text, the next 4 megabytes for data, and
the top 4 megabytes for the stack. In between the top of the data and the bottom of
the stack is a gigantic hole that is not used.
In Fig. 3-13(b), we see how the two-level page table works. On the left we see
the top-level page table, with 1024 entries, corresponding to the 10-bit PT1 field.
When a virtual address is presented to the MMU, it first extracts the PT1 field and
uses this value as an index into the top-level page table. Each of these 1024 (or 210 )
entries in the top-level page table represents 4M (or 222 bytes) because the entire
4-gigabyte (i.e., 32-bit) virtual address space has been chopped into chunks of
4096 (or 212 ) bytes.
The entry located by indexing into the top-level page table yields the address
or the page frame number of a second-level page table. Entry 0 of the top-level
page table points to the page table for the program text, entry 1 points to the page
table for the data, and entry 1023 points to the page table for the stack. The other
(shaded) entries are not used. The PT2 field is now used as an index into the selected second-level page table to find the page frame number for the page itself.
As an example, consider the 32-bit virtual address 0x00403004 (4,206,596
decimal), which is 4,206,596 - 4MB = 12,292 bytes into the data area. This virtual address corresponds to PT1 = 1, PT2 = 3, and Offset = 4. The MMU first uses
PT1 to index into the top-level page table and obtain entry 1, which corresponds to
addresses 4M to 8M < 1. It then uses PT2 to index into the second-level page table
just found and extract entry 3, corresponding to addresses 12288 to 16383 within
its 4M chunk (i.e., absolute addresses 4,206,592 to 4,210,687). This contains the
page frame number of the page containing virtual address 0x00403004. If that
page is not in memory, the Present/absent bit in the page table entry will have the
value zero, causing a page fault. If the page is present in memory, the page frame
number taken from the second-level page table is combined with the offset (4) to
construct the physical address. This address is put on the bus and sent to memory.
The interesting thing to note about Fig. 3-13 is that although the address space
contains over a million pages, only four page tables are needed: the top-level table,
and the second-level tables for 0 to 4M (for the program text), 4M to 8M (for the
data), and the top 4M (for the stack). The Present/absent bits in the remaining
1021 entries of the top-level page table are set to 0, forcing a page fault if they are
ever accessed. Should this occur, the operating system will notice that the process
is trying to reference memory that it is not supposed to and will take appropriate
action, such as sending it a signal or killing it. In this example, we have chosen
round numbers for the various sizes and have picked PT1 equal to PT2, but in
actual practice other values are also possible, of course.
The two-level page table system of Fig. 3-13 can be expanded to three, four, or
more levels. Additional levels give more flexibility. For instance, Intel’s 32 bit
80386 processor (launched in 1985) was able to address up to 4-GB of memory,
using a two-level page table that consisted of a page directory whose entries
206
MEMORY MANAGEMENT
CHAP. 3
pointed to page tables, which, in turn, pointed to the actual 4-KB page frames.
Both the page directory and the page tables each contained 1024 entries, giving a
total of 210 × 210 × 212 = 232 addressable bytes, as desired.
Ten years later, the Pentium Pro introduced another level: the page directory
pointer table. In addition, it extended each entry in each level of the page table
hierarchy from 32 bits to 64 bits, so that it could address memory above the 4-GB
boundary. As it had only 4 entries in the page directory pointer table, 512 in each
page directory, and 512 in each page table, the total amount of memory it could
address was still limited to a maximum of 4 GB. When proper 64-bit support was
added to the x86 family (originally by AMD), the additional level could have been
called the ‘‘page directory pointer table pointer’’ or something equally horrible.
That would have been perfectly in line with how chip makers tend to name things.
Mercifully, they did not do this. The alternative they cooked up, ‘‘page map level
4,’’ may not be a terribly catchy name either, but at least it is short and a bit clearer.
At any rate, these processors now use all 512 entries in all tables, yielding an
amount of addressable memory of 29 × 29 × 29 × 29 × 212 = 248 bytes. They could
have added another level, but they probably thought that 256 TB would be sufficient for a while.
Turns out they were wrong. Some of the newer processors have support for a
fifth level to extend the size of addresses to 57 bits. With such an address space,
one can address up to 128 petabytes. This is a lot of bytes. It allows huge files to be
mapped in. Of course, the downside of so many levels is that page table walks
become even more expensive.
Inverted Page Tables
An alternative to ever-increasing levels in a paging hierarchy is known as
inverted page tables. They were first used by such processors as the PowerPC,
the UltraSPARC, and the Itanium (sometimes referred to as ‘‘Itanic,’’ as it was not
quite the success Intel had hoped for). It has now gone the way of the Amazon
Fire Phone, Apple Newton, AT&T Picture Phone, Betamax video recorder,
DeLorean car, Ford Edsel, and Windows Vista.
Inverted page tables, however, live on. In this design, there is one entry per
page frame in real memory, rather than one entry per page of virtual address space.
For example, with 64-bit virtual addresses, a 4-KB page size, and 16 GB of RAM,
an inverted page table requires only 4,194,304 entries. The entry keeps track of
which (process, virtual page) is located in the page frame.
Although inverted page tables save lots of space, at least when the virtual
address space is much larger than the physical memory, they have a serious downside: virtual-to-physical translation becomes much, much harder. When process n
references virtual page p, the hardware can no longer find the physical page by
using p as an index into the page table. Instead, it must search the entire inverted
page table for an entry (n, p). Furthermore, this search must be done on every
SEC. 3.3
207
VIRTUAL MEMORY
memory reference, not just on page faults. Searching a 256K table on every memory reference is not the way to make your machine blindingly fast.
The way out of this dilemma is to make use of the TLB. If the TLB can hold
all of the heavily used pages, translation can happen just as fast as with regular
page tables. On a TLB miss, however, the inverted page table has to be searched in
software. One feasible way to accomplish this search is to have a hash table hashed
on the virtual address. All the virtual pages currently in memory that have the same
hash value are chained together, as shown in Fig. 3-14. If the hash table has as
many slots as the machine has physical pages, the average chain will be only one
entry long, greatly speeding up the mapping. Once the page frame number has
been found, the new (virtual, physical) pair is entered into the TLB.
Traditional page
table with an entry
for each of the 252
pages
252 -1
1-GB physical
memory has 218
4-KB page frames
218 -1
0
0
Indexed
by virtual
page
Hash table
218 -1
0
Indexed
by hash on
virtual page
Virtual
page
Page
frame
Figure 3-14. Comparison of a traditional page table with an inverted page table.
Inverted page tables are used on 64-bit machines because even with a very
large page size, the number of page table entries is gigantic. For example, with
4-MB pages and 64-bit virtual addresses, 242 page table entries are needed.
3.4 PAGE REPLACEMENT ALGORITHMS
When a page fault occurs, the operating system has to choose a page to evict
(remove from memory) to make room for the incoming page. If the page to be
removed has been modified while in memory, it must be rewritten to nonvolatile
storage to bring the disk or SSD copy up to date. If, however, the page has not
been changed, for example because it contains the executable code for a program,
text), the disk or SSD copy is already up to date, so no rewrite is needed. The page
to be read in just overwrites the page being evicted.
208
MEMORY MANAGEMENT
CHAP. 3
While it would be possible to pick a random page to evict at each page fault,
system performance is much better if a page that is not heavily used is chosen. If a
heavily used page is removed, it will probably have to be brought back in quickly,
resulting in extra overhead. Much work has been done on the subject of page
replacement algorithms, both theoretical and experimental. Below we will describe
some of the most important ones.
It is worth noting that the problem of ‘‘page replacement’’ occurs in other areas
of computer design as well. For example, most computers have one or more memory caches consisting of recently used 32-byte or 64-byte memory blocks. When
the cache is full, some block has to be chosen for removal. This problem is precisely the same as page replacement except on a shorter time scale (it has to be
done in a few nanoseconds, not tens of microseconds or even milliseconds as with
page replacement). The reason for the shorter time scale is that cache block misses
are satisfied from main memory, which is considerably faster than a magnetic disk
or even an SSD.
A second example is in a Web server. The server can keep a certain number of
heavily used Web pages in its memory cache. However, when the memory cache is
full and a new page is referenced, a decision has to be made which Web page to
evict. The considerations are similar to pages of virtual memory, except that the
Web pages are never modified in the cache, so there is always a fresh copy ‘‘on
nonvolatile storage.’’ In a virtual memory system, pages in main memory may be
either clean or dirty.
In all the page replacement algorithms to be studied below (and others), a certain issue arises: when a page is to be evicted from memory, does it have to be one
of the faulting process’ own pages, or can it be a page belonging to another process? In the former case, we are effectively limiting each process to a fixed number
of pages; in the latter case we are not. Both are possibilities. We will come back
to this point in Sec. 3.5.1.
3.4.1 The Optimal Page Replacement Algorithm
The best possible page replacement algorithm is easy to describe but impossible to actually implement. It goes like this. At the moment that a page fault
occurs, some set of pages is in memory. One of these pages will be referenced on
the very next instruction (the page containing that instruction). Other pages may
not be referenced until 10, 100, 1000, or millions of instructions later. Or maybe
never if the page is page of the initialization phase of the program and has now
completed. Each page can be labeled with the number of instructions that will be
executed before that page is first referenced.
The optimal page replacement algorithm says that the page with the highest
label should be removed. If one page will not be used for 8 million instructions
and another page will not be used for 6 million instructions, removing the former
pushes the page fault that will fetch it back as far into the future as possible. Computers, like people, try to put off unpleasant events for as long as they can.
SEC. 3.4
PAGE REPLACEMENT ALGORITHMS
209
The only problem with this algorithm is that it is unrealizable. At the time of
the page fault, the operating system has no way of knowing when each of the pages
will be referenced next. (We saw a similar situation earlier with the shortest-job-first scheduling algorithm—how can the system tell which job is shortest?)
Still, by running a program on a simulator and keeping track of all page references,
it is possible to implement optimal page replacement on the second run by using
the page-reference information collected during the first run.
In this way, it is possible to compare the performance of realizable algorithms
with the best possible one. If an operating system achieves a performance of, say,
only 1% worse than the optimal algorithm, effort spent in looking for a better algorithm will yield at most a 1% improvement.
To avoid any possible confusion, it should be made clear that this log of page
references refers only to the one program just measured and then with only one
specific input. The page replacement algorithm derived from it is thus specific to
that one program and input data. Although this method is useful for evaluating
page replacement algorithms, it is of no use in practical systems. Below we will
study algorithms that are useful on real systems.
3.4.2 The Not Recently Used Page Replacement Algorithm
In order to allow the operating system to collect useful page usage statistics,
most computers with virtual memory have two status bits, R and M, associated
with each page. R is set whenever the page is referenced (read or written). M is
set when the page is written to (i.e., modified). The bits are contained in each page
table entry, as shown in Fig. 3-11. It is important to realize that these bits must be
updated on every memory reference, so it is essential that they be set by the hardware. Once a bit has been set to 1, it stays 1 until the operating system resets it.
If the hardware does not have these bits, they can be simulated using the operating system’s page fault and clock interrupt mechanisms. When a process is started up, all of its page table entries are marked as not in memory. As soon as any
page is referenced, a page fault will occur. The operating system then sets the R bit
(in its internal tables), changes the page table entry to point to the correct page,
with mode READ ONLY, and restarts the instruction. If the page is subsequently
modified, another page fault will occur, allowing the operating system to set the M
bit and change the page’s mode to READ/WRITE.
The R and M bits can be used to build a simple paging algorithm as follows.
When a process is started up, both page bits for all its pages are set to 0 by the
operating system. Periodically (e.g., on each clock interrupt), the R bit is cleared,
to distinguish pages that have not been referenced recently from those that have
been.
When a page fault occurs, the operating system inspects all the pages and
divides them into four categories based on the current values of their R and M bits:
210
MEMORY MANAGEMENT
CHAP. 3
Class 0: not referenced, not modified.
Class 1: not referenced, modified.
Class 2: referenced, not modified.
Class 3: referenced, modified.
Although class 1 pages seem to be impossible, they occur when a class 3 page has
its R bit cleared by a clock interrupt. Clock interrupts do not clear the M bit
because this information is needed to know whether the page has to be rewritten to
disk later. Clearing R but not M leads to a class 1 page. In other words, a class 1
page is one that was modified long ago and has not been touched since then.
The NRU (Not Recently Used) algorithm removes a page at random from the
lowest-numbered nonempty class. Implicit in this algorithm is the idea that it is
better to remove a modified page that has not been referenced in at least one clock
tick (typically about 20 msec) than a clean page that is in heavy use. The main
attraction of NRU is that it is easy to understand, moderately efficient to implement, and gives a performance that, while certainly not optimal, may be adequate.
3.4.3 The First-In, First-Out (FIFO) Page Replacement Algorithm
Another low-overhead paging algorithm is the FIFO (First-In, First-Out)
algorithm. To illustrate how this works, consider a supermarket that has enough
shelves to display exactly k different products. One day, some company introduces
a new convenience food—instant, freeze-dried, organic yogurt that can be reconstituted in a microwave oven. It is an immediate success, so our finite supermarket
has to get rid of one old product in order to stock it.
One possibility is to find the product that the supermarket has been stocking
the longest (i.e., something it began selling 120 years ago) and get rid of it on the
grounds that no one is interested any more. In effect, the supermarket maintains a
linked list of all the products it currently sells in the order they were introduced.
The new one goes on the back of the list; the one at the front of the list is dropped.
As a page replacement algorithm, the same idea is applicable. The operating
system maintains a list of all pages currently in memory, with the most recent arrival at the tail and the least recent arrival at the head. On a page fault, the page at
the head is removed and the new page added to the tail of the list. When applied to
stores, FIFO might remove mustache wax, but it might also remove flour, salt, or
butter. When applied to computers, the same problem arises: the oldest page may
still be useful. For this reason, FIFO in its pure form is rarely used.
3.4.4 The Second-Chance Page Replacement Algorithm
A simple modification to FIFO that avoids the problem of throwing out a heavily used page is to inspect the R bit of the oldest page. If it is 0, the page is both
old and unused, so it is replaced immediately. If the R bit is 1, the bit is cleared,
the page is put onto the end of the list of pages, and its load time is updated as
though it had just arrived in memory. Then the search continues.
SEC. 3.4
211
PAGE REPLACEMENT ALGORITHMS
The operation of this algorithm, called second chance, is shown in Fig. 3-15.
In Fig. 3-15(a), we see pages A through H kept on a linked list and sorted by the
time they arrived in memory.
Page loaded first
0
3
7
8
12
14
15
18
A
B
C
D
E
F
G
H
Most recently
loaded page
(a)
3
7
8
12
14
15
18
20
B
C
D
E
F
G
H
A
A is treated like a
newly loaded page
(b)
Figure 3-15. Operation of second chance. (a) Pages sorted in FIFO order.
(b) Page list if a page fault occurs at time 20 and A has its R bit set. The numbers
above the pages are their load times.
Suppose that a page fault occurs at time 20. The oldest page is A, which arrived at time 0, when the process started. If A has the R bit cleared, it is evicted
from memory, either by being written to nonvolatile storage (if it is dirty), or just
abandoned (if it is clean). On the other hand, if the R bit is set, A is put onto the
end of the list and its ‘‘load time’’ is reset to the current time (20). The R bit is also
cleared. The search for a suitable page continues with B.
What second chance is looking for is an old page that has not been referenced
in the most recent clock interval. If all the pages have been referenced, second
chance degenerates into pure FIFO. Specifically, imagine that all the pages in
Fig. 3-15(a) have their R bits set. One by one, the operating system moves the
pages to the end of the list, clearing the R bit each time it appends a page to the end
of the list. Eventually, it comes back to page A, which now has its R bit cleared. At
this point, A is evicted. Thus the algorithm always terminates.
3.4.5 The Clock Page Replacement Algorithm
Although second chance is a reasonable algorithm, it is unnecessarily inefficient because it is constantly moving pages around on its list. A better approach
is to keep all the page frames on a circular list in the form of a clock, as shown in
Fig. 3-16. The hand points to the oldest page.
When a page fault occurs, the page being pointed to by the hand is inspected.
If its R bit is 0, the page is evicted, the new page is inserted into the clock in its
place, and the hand is advanced one position. If R is 1, it is cleared and the hand is
advanced to the next page. This process is repeated until a page is found with
R = 0. Not surprisingly, this algorithm is called clock.
212
MEMORY MANAGEMENT
CHAP. 3
A
B
L
K
C
J
D
I
E
H
When a page fault occurs,
the page the hand is
pointing to is inspected.
The action taken depends
on the R bit:
R = 0: Evict the page
R = 1: Clear R and advance hand
F
G
Figure 3-16. The clock page replacement algorithm.
3.4.6 The Least Recently Used (LRU) Page Replacement Algorithm
A good approximation to the optimal algorithm is based on the observation
that pages that have been heavily used in the last few instructions will probably be
heavily used again soon. Conversely, pages that have not been used for ages will
probably remain unused for a long time. This idea suggests a realizable algorithm:
when a page fault occurs, throw out the page that has been unused for the longest
time. This strategy is called LRU (Least Recently Used) paging.
Although LRU is theoretically realizable, it is not cheap by a long shot. To fully implement LRU, it is necessary to maintain a linked list of all pages in memory,
with the most recently used page at the front and the least recently used page at the
rear. The difficulty is that the list must be updated on every memory reference.
Finding a page in the list, deleting it, and then moving it to the front is a time consuming operation, even in hardware (assuming that such hardware could be built).
However, there are other ways to implement LRU with special hardware. Let
us consider the simplest way first. This method requires equipping the hardware
with a 64-bit counter, C, that is automatically incremented after each instruction.
Furthermore, each page table entry must also have a field large enough to contain
the counter. After each memory reference, the current value of C is stored in the
page table entry for the page just referenced. When a page fault occurs, the operating system examines all the counters in the page table to find the lowest one. That
page is the least recently used.
3.4.7 Simulating LRU in Software
Although the previous LRU algorithm is (in principle) realizable, few, if any,
machines have the required hardware. Instead, a solution that can be implemented
in software is needed. One possibility is called the NFU (Not Frequently Used)
SEC. 3.4
PAGE REPLACEMENT ALGORITHMS
213
algorithm. It requires a software counter associated with each page, initially zero.
At each clock interrupt, the operating system scans all the pages in memory. For
each page, the R bit, which is 0 or 1, is added to the counter. The counters roughly
keep track of how often each page has been referenced. When a page fault occurs,
the page with the lowest counter is chosen for replacement.
The main problem with NFU is that it is like an elephant: it never forgets anything. For example, in a multipass compiler, pages that were heavily used during
pass 1 may still have a high count well into later passes. In fact, if pass 1 happens
to have the longest execution time of all the passes, the pages containing the code
for subsequent passes may always have lower counts than the pass-1 pages. Consequently, the operating system will remove useful pages instead of pages no longer
in use.
Fortunately, a small modification to NFU makes it able to simulate LRU quite
well. The modification has two parts. First, the counters are each shifted right 1 bit
before the R bit is added in. Second, the R bit is added to the leftmost rather than
the rightmost bit.
Figure 3-17 illustrates how the modified algorithm, known as aging, works.
Suppose that after the first clock tick, the R bits for pages 0 to 5 have the values 1,
0, 1, 0, 1, and 1, respectively (page 0 is 1, page 1 is 0, page 2 is 1, etc.). In other
words, between tick 0 and tick 1, pages 0, 2, 4, and 5 were referenced, setting their
R bits to 1, while the other ones remained 0. After the six corresponding counters
have been shifted and the R bit inserted at the left, they have the values shown in
Fig. 3-17(a). The four remaining columns show the six counters after the next four
clock ticks.
When a page fault occurs, the page whose counter is the lowest is removed. It
is clear that a page that has not been referenced for, say, four clock ticks will have
four leading zeros in its counter and thus will have a lower value than a counter
that has not been referenced for three clock ticks.
This algorithm differs from LRU in two important ways. Consider pages 3 and
5 in Fig. 3-17(e). Neither has been referenced for two clock ticks; both were referenced in the tick prior to that. According to LRU, if a page must be replaced, we
should choose one of these two. The trouble is, we do not know which of them was
referenced last in the interval between tick 1 and tick 2. By recording only 1 bit
per time interval, we have now lost the ability to distinguish references early in the
clock interval from those occurring later. All we can do is remove page 3, because
page 5 was also referenced two ticks earlier and page 3 was not.
The second difference between LRU and aging is that in aging the counters
have a finite number of bits (8 bits in this example), which limits its past horizon.
Suppose that two pages each have a counter value of 0. All we can do is pick one
of them at random. In reality, it may well be that one of the pages was last referenced nine ticks ago and the other was last referenced 1000 ticks ago. We have no
way of seeing that. In practice, however, 8 bits is generally enough if a clock tick
is around 20 msec. If a page has not been referenced in 160 msec, it probably is
214
MEMORY MANAGEMENT
CHAP. 3
R bits for
pages 0-5,
clock tick 0
R bits for
pages 0-5,
clock tick 1
R bits for
pages 0-5,
clock tick 2
R bits for
pages 0-5,
clock tick 3
R bits for
pages 0-5,
clock tick 4
1 0 1 0 1 1
1 1 0 0 1 0
1 1 0 1 0 1
1 0 0 0 1 0
0 1 1 0 0 0
0
10000000
11000000
11100000
11110000
01111000
1
00000000
10000000
11000000
01100000
10110000
2
10000000
01000000
00100000
00010000
10001000
3
00000000
00000000
10000000
01000000
00100000
4
10000000
11000000
01100000
10110000
01011000
5
10000000
01000000
10100000
01010000
00101000
(a)
(b)
(c)
(d)
(e)
Page
Figure 3-17. The aging algorithm simulates LRU in software. Shown are six
pages for five clock ticks. The five clock ticks are represented by (a) to (e).
not that important. Of course, using a 16-bit, 32-bit, or 64-bit counter provides
more history but the cost is more memory to store it. Usually 8 bits does the job
just fine.
3.4.8 The Working Set Page Replacement Algorithm
In the purest form of paging, processes are started up with none of their pages
in memory. As soon as the CPU tries to fetch the first instruction, it gets a page
fault, causing the operating system to bring in the page containing the first instruction. Other page faults for global variables and the stack usually follow quickly.
After a while, the process has most of the pages it needs and settles down to run
with relatively few page faults. This strategy is called demand paging because
pages are loaded only on demand, not in advance.
Of course, it is easy enough to write a test program that systematically reads all
the pages in a large address space, causing so many page faults that there is not
enough memory to hold them all. Fortunately, most processes do not work this
way. They exhibit a locality of reference, meaning that during any phase of
execution, the process references only a relatively small fraction of its pages. Each
pass of a multipass compiler, for example, references only a fraction of all the
pages, and a different fraction at that.
The set of pages that a process is currently using is its working set (Denning,
1968a; and Denning, 1980). If the entire working set is in memory, the process
SEC. 3.4
PAGE REPLACEMENT ALGORITHMS
215
will run without causing many faults until it moves into another execution phase
(e.g., the next pass of the compiler). If the available memory is too small to hold
the entire working set, the process will cause many page faults and run slowly,
since executing an instruction takes a few nanoseconds and reading in a page from,
say, a disk typically takes 10 msec. At a rate of one or two instructions per 10
msec, it will take ages to finish. A program causing page faults every few instructions is said to be thrashing (Denning, 1968b).
In a multiprogramming system, processes are often moved to disk (i.e., all their
pages are removed from memory) to let others have a turn at the CPU. The question arises of what to do when a process is brought back in again. Technically,
nothing need be done. The process will just cause page faults until its working set
has been loaded. The problem is that having numerous page faults every time a
process is loaded is slow, and it also wastes considerable CPU time, since it takes
the operating system a few milliseconds of CPU time to process a page fault.
Therefore, many paging systems try to keep track of each process’ working set
and make sure that it is in memory before letting the process run. This approach is
called the working set model (Denning, 1970). It is designed to greatly reduce the
page fault rate. Loading the pages before letting processes run is also called
prepaging. Note that the working set changes over time.
It has long been known that programs rarely reference their address space uniformly, but that the references tend to cluster on a small number of pages. A memory reference may fetch an instruction or data, or it may store data. At any instant
of time, t, there exists a set consisting of all the pages used by the k most recent
memory references. This set, w(k, t), is the working set. Because the k > 1 most
recent references must have used all the pages used by the k = 1 most recent references, and possibly others, w(k, t) is a monotonically nondecreasing function of k.
The limit of w(k, t) as k becomes large is finite because a program cannot reference more pages than its address space contains, and few programs will use every
single page. Figure 3-18 depicts the size of the working set as a function of k.
The fact that most programs randomly access a small number of pages, but that
this set changes slowly in time explains the initial rapid rise of the curve and then
the much slower rise for large k. For example, a program that is executing a loop
occupying two pages using data on four pages may reference all six pages every
1000 instructions, but the most recent reference to some other page may be a million instructions earlier, during the initialization phase. Due to this asymptotic
behavior, the contents of the working set are not sensitive to the value of k chosen.
To put it differently, there exists a wide range of k values for which the working set
is unchanged. Because the working set varies slowly with time, it is possible to
make a reasonable guess as to which pages will be needed when the program is
restarted on the basis of its working set when it was last stopped. Prepaging consists of loading these pages before resuming the process.
To implement the working set model, it is necessary for the operating system
to keep track of which pages are in the working set. Having this information also
216
MEMORY MANAGEMENT
CHAP. 3
w(k,t)
k
Figure 3-18. The working set is the set of pages used by the k most recent memory references. The function w(k, t) is the size of the working set at time t.
immediately leads to a possible page replacement algorithm: when a page fault
occurs, find a page not in the working set and evict it. To implement such an algorithm, we need a precise way of determining which pages are in the working set.
By definition, the working set is the set of pages used in the k most recent memory
references (some authors use the k most recent page references, but the choice is
arbitrary). To implement any working set algorithm, some value of k must be chosen in advance. Then, after every memory reference, the set of pages used by the
most recent k memory references is uniquely determined.
Of course, having an operational definition of the working set does not mean
that there is an efficient way to compute it during program execution. One could
imagine a shift register of length k, with every memory reference shifting the register left one position and inserting the most recently referenced page number on the
right. The set of all k page numbers in the shift register would be the working set.
In theory, at a page fault, the contents of the shift register could be read out and
sorted. Duplicate pages could then be removed. The result would be the working
set. However, maintaining the shift register and processing it at a page fault would
both be prohibitively expensive, so this technique is never used.
Instead, various approximations are used. One common approximation is to
drop the idea of counting back k memory references and use execution time
instead. For example, instead of defining the working set as those pages used during the previous 10 million memory references, we can define it as the set of pages
used during the past 100 msec of execution. In practice, such a definition is just as
good and easier to work with. Note that for each process, only its own execution
time counts. Thus if a process starts running at time T and has had 40 msec of
CPU time at real time T + 100 msec, for working set purposes its time is 40 msec.
The amount of CPU time a process has actually used since it started is often called
its current virtual time. With this approximation, the working set of a process is
the set of pages it has referenced during the past o seconds of virtual time.
SEC. 3.4
PAGE REPLACEMENT ALGORITHMS
217
Now let us look at a page replacement algorithm based on the working set. The
basic idea is to find a page that is not in the working set and evict it. In Fig. 3-19.
we see a portion of a page table for some machine. Because only pages located in
memory are considered as candidates for eviction, pages that are absent from
memory are ignored by this algorithm. Each entry contains (at least) two key items
of information: the (approximate) time the page was last used and the R (Referenced) bit. An empty white rectangle symbolizes the other fields not needed for
this algorithm, such as the page frame number, the protection bits, and the M
(Modified) bit.
2204
Information about
one page
R (Referenced) bit
2084
1
2003
1
Time of last use
1980
1
Page referenced
during this tick
1213
0
2014
1
2020
1
2032
1
1620
0
Page not referenced
during this tick
Current virtual time
Scan all pages examining R bit:
if (R == 1)
set time of last use to current virtual time
if (R == 0 and age > o)
remove this page
if (R == 0 and age ) o)
remember the smallest time
Page table
Figure 3-19. The working set algorithm.
The algorithm works as follows. The hardware is assumed to set the R and M
bits, as discussed earlier. Similarly, a periodic clock interrupt is assumed to cause
software to run that clears the Referenced bit on every clock tick. On every page
fault, the page table is scanned to look for a suitable page to evict.
As each entry is processed, the R bit is examined. If it is 1, the current virtual
time is written into the Time of last use field in the page table, indicating that the
page was in use at the time the fault occurred. Since the page has been referenced
during the current clock tick, it is clearly in the working set and is not a candidate
for removal (o is assumed to span multiple clock ticks).
If R is 0, the page has not been referenced during the current clock tick and
may be a candidate for removal. To see whether or not it should be removed, its
age (the current virtual time minus its Time of last use) is computed and compared
to o . If the age is greater than o , the page is no longer in the working set and the
new page replaces it. The scan continues updating the remaining entries.
218
MEMORY MANAGEMENT
CHAP. 3
However, if R is 0 but the age is less than or equal to o , the page is still in the
working set. The page is temporarily spared, but the page with the greatest age
(smallest value of Time of last use) is noted. If the entire table is scanned without
finding a candidate to evict, that means that all pages are in the working set. In
that case, if one or more pages with R = 0 were found, the one with the greatest age
is evicted. In the worst case, all pages have been referenced during the current
clock tick (and thus all have R = 1), so one is chosen at random for removal, preferably a clean page, if one exists.
3.4.9 The WSClock Page Replacement Algorithm
The basic working set algorithm is cumbersome, since the entire page table has
to be scanned at each page fault until a suitable candidate is located. That is a
time-consuming operation. An improved algorithm, which is based on the clock
algorithm but also uses the working set information, is called WSClock (Carr and
Hennessey, 1981). Due to its simplicity of implementation and good performance,
it is widely used in practice.
The data structure needed is a circular list of page frames, as in the clock algorithm, and as shown in Fig. 3-20(a). Initially, this list is empty. When the first page
is loaded, it is added to the list. As more pages are added, they go into the list to
form a ring. Each entry contains the Time of last use field from the basic working
set algorithm, as well as the R bit (shown) and the M bit (not shown).
As with the clock algorithm, at each page fault the page pointed to by the hand
is examined first. If the R bit is set to 1, the page has been used during the current
tick so it is not an ideal candidate to remove. The R bit is then set to 0, the hand
advanced to the next page, and the algorithm repeated for that page. The state after
this sequence of events is shown in Fig. 3-20(b).
Now consider what happens if the page pointed to has R = 0, as shown in
Fig. 3-20(c). If the age is greater than o and the page is clean, it is not in the working set and a valid copy exists on the disk or SSD. The page frame is simply
claimed and the new page put there, as shown in Fig. 3-20(d). On the other hand,
if the page is dirty, it cannot be claimed immediately since no valid copy is present
on nonvolatile storage. To avoid a process switch, the write to nonvolatile storage
is scheduled, but the hand is advanced and the algorithm continues with the next
page. After all, there might be an old, clean page further down the line that can be
used immediately.
In principle, all pages might be scheduled for I/O to nonvolatile storage on one
cycle around the clock. To reduce disk or SSD traffic, a limit might be set, allowing a maximum of n pages to be written back. Once this limit has been reached, no
new writes would be scheduled.
What happens if the hand comes all the way around and back to its starting
point? There are two cases we have to consider:
SEC. 3.4
219
PAGE REPLACEMENT ALGORITHMS
2204 Current virtual time
1620 0
1620 0
2032 1
20 8 4 1
2020 1
2 00 3 1
1980 1
Time of
last use
2020 1
2003 1
1980 1
2014 1
1213 0
2032 1
2084 1
2 20 4 0
1213 0
R bit
(a)
(b)
1620 0
1620 0
2032 1
20 8 4 1
2020 1
2 00 3 1
1980 1
2 20 4 0
2032 1
2084 1
2020 1
2003 1
1980 1
2 20 4 0
1213 0
2204 1
(c)
(d)
New page
Figure 3-20. Operation of the WSClock algorithm. (a) and (b) give an example
of what happens when R = 1. (c) and (d) give an example of R = 0.
1. At least one write has been scheduled.
2. No writes have been scheduled.
In the first case, the hand just keeps moving, looking for a clean page. Since one or
more writes have been scheduled, eventually some write will complete and its page
will be marked as clean. The first clean page encountered is evicted. This page is
220
MEMORY MANAGEMENT
CHAP. 3
not necessarily the first write scheduled because the (disk) driver may reorder
writes in order to optimize performance of nonvolatile storage.
In the second case, all pages are in the working set, otherwise at least one write
would have been scheduled. Lacking additional information, the simplest thing to
do is claim any clean page and use it. The location of a clean page could be kept
track of during the sweep. If no clean pages exist, then the current page is chosen
as the victim and written back to nonvolatile storage.
3.4.10 Summary of Page Replacement Algorithms
We have now looked at a variety of page replacement algorithms. Now we
will briefly summarize them. The list of algorithms discussed is given in Fig. 3-21.
Algorithm
Optimal
Comment
Not implementable, but useful as a benchmark
NRU (Not Recently Used)
Very crude approximation of LRU
FIFO (First-In, First-Out)
Might throw out important pages
Second chance
Big improvement over FIFO
Clock
Realistic
LRU (Least Recently Used)
Excellent, but difficult to implement exactly
NFU (Not Frequently Used)
Fairly crude approximation to LRU
Aging
Efficient algorithm that approximates LRU well
Working set
Somewhat expensive to implement
WSClock
Good efficient algorithm
Figure 3-21. Page replacement algorithms discussed in the text.
The optimal algorithm evicts the page that will be referenced furthest in the
future. Unfortunately, there is no way to determine which page this is, so in practice this algorithm cannot be used. It is useful as a benchmark against which other
algorithms can be measured, however.
The NRU algorithm divides pages into four classes depending on the state of
the R and M bits. A random page from the lowest-numbered class is chosen. This
algorithm is easy to implement, but it is very crude. Better ones exist.
FIFO keeps track of the order in which pages were loaded into memory by
keeping them in a linked list. Removing the oldest page then becomes trivial, but
that page might still be in use, so FIFO is a bad choice.
Second chance is a change to FIFO that checks if a page is in use before removing it. If it is, the page is spared. This modification greatly improves the performance. Clock is simply a different implementation of second chance. It has the
same performance properties, but takes a little less time to execute the algorithm.
LRU is an excellent algorithm, but it cannot be implemented without special
hardware. If this hardware is not available, LRU cannot be used. NFU is a crude
SEC. 3.4
PAGE REPLACEMENT ALGORITHMS
221
attempt to approximate LRU. It is not very good. However, aging is a much better
approximation to LRU and can be implemented efficiently. It is a good choice.
The last two algorithms use the working set. The working set algorithm gives
reasonable performance, but it is somewhat expensive to implement. WSClock is a
variant that not only gives good performance but is also efficient to implement.
All in all, the two best ‘‘pure’’ algorithms are aging and WSClock. They are
based on LRU and the working set, respectively. Both give good paging performance and can be implemented efficiently. In practice, operating systems may
implement their own variants of page replacement algorithms. For instance, Windows combines elements from different (Clock/LRU and working set) policies,
using different strategies for local replacement (evicting pages only from this process) and global replacement (evicting pages from anywhere), and even varies page
replacement based on the underlying hardware. Meanwhile, Linux in 2008 adopted
a split LRU page replacement solution which keeps separate LRU lists for pages
containing file contents and pages containing ‘‘anonymous’’ data (i.e., not backed
by files). The reason is that these pages typically have different usage patterns and
the probability of the anonymous pages getting reused is much higher.
3.5 DESIGN ISSUES FOR PAGING SYSTEMS
In the previous sections, we have explained how paging works and have given
a few of the basic page replacement algorithms. But knowing the bare mechanics
is not enough. To design a system and make it work well, you have to know a lot
more. It is like the difference between knowing how to move the rook, knight,
bishop, and other pieces in chess, and being a good player. In the following sections, we will look at other issues that operating system designers must consider
carefully in order to get good performance from a paging system.
3.5.1 Local versus Global Allocation Policies
In the preceding sections, we have discussed several algorithms for choosing a
page to replace when a fault occurs. A major issue associated with this choice
(which we have carefully swept under the rug until now) is how memory should be
allocated among the competing runnable processes.
Take a look at Fig. 3-22(a). In this figure, three processes, A, B, and C, make
up the set of runnable processes. Suppose A gets a page fault. Should the page
replacement algorithm try to find the least recently used page considering only the
six pages currently allocated to A, or should it consider all the pages in memory?
If it looks only at A’s pages, the page with the lowest age value is A5, so we get the
situation of Fig. 3-22(b).
On the other hand, if the page with the lowest age value is removed without
regard to whose page it is, page B3 will be chosen and we will get the situation of
Fig. 3-22(c). The algorithm of Fig. 3-22(b) is said to be a local page replacement
222
MEMORY MANAGEMENT
A0
A1
A2
A3
A4
A5
B0
B1
B2
B3
B4
B5
B6
C1
C2
C3
(a)
Age
10
7
5
4
6
3
9
4
6
2
5
6
12
3
5
6
CHAP. 3
A0
A1
A2
A3
A4
A6
B0
B1
B2
B3
B4
B5
B6
C1
C2
C3
A0
A1
A2
A3
A4
A5
B0
B1
B2
A6
B4
B5
B6
C1
C2
C3
(b)
(c)
Figure 3-22. Local versus global page replacement. (a) Original configuration.
(b) Local page replacement. (c) Global page replacement.
algorithm, whereas that of Fig. 3-22(c) is said to be a global algorithm. Local algorithms effectively correspond to allocating every process a fixed fraction of the
memory. Global algorithms dynamically allocate page frames among the runnable
processes. Thus the number of page frames assigned to each process varies in time.
In general, global algorithms work better, especially when the working set size
can vary a lot over the lifetime of a process. If a local algorithm is used and the
working set grows, thrashing will result, even if there are a sufficient number of
free page frames. If the working set shrinks, local algorithms waste memory. If a
global algorithm is used, the system must continually decide how many page
frames to assign to each process. One way is to monitor the working set size as
indicated by the aging bits, but this approach does not necessarily prevent thrashing. The working set may change size in milliseconds, whereas the aging bits are a
very crude measure spread over a number of clock ticks.
Another approach is to have an algorithm for allocating page frames to processes. One way is to periodically determine the number of running processes and
allocate each process an equal share. Thus with 12,416 available (i.e., nonoperating
system) page frames and 10 processes, each process gets 1241 frames. The remaining six go into a pool to be used when page faults occur.
Although this method may seem fair, it makes little sense to give equal shares
of the memory to a 10-KB process and a 300-KB process. Instead, pages can be
allocated in proportion to each process’ total size, with a 300-KB process getting
30 times the allotment of a 10-KB process. It is probably wise to give each process
some minimum number, so that it can run no matter how small it is. On some machines, for example, a single two-operand instruction might need as many as six
SEC. 3.5
223
DESIGN ISSUES FOR PAGING SYSTEMS
Page faults/sec
pages because the instruction itself, the source operand, and the destination operand may all straddle page boundaries. With an allocation of only five pages, programs containing such instructions cannot execute at all.
If a global algorithm is used, it may be possible to start each process up with
some number of pages proportional to the process’ size, but the allocation has to be
updated dynamically as the processes run. One way to manage the allocation is to
use the PFF (Page Fault Frequency) algorithm. It tells when to increase or
decrease a process’ page allocation but says nothing about which page to replace
on a fault. It just controls the size of the allocation set.
For a large class of page replacement algorithms, including LRU, it is known
that the fault rate decreases as more pages are assigned, as we discussed above.
This is the assumption behind PFF. This property is illustrated in Fig. 3-23.
A
B
Number of page frames assigned
Figure 3-23. Page fault rate as a function of the number of page frames assigned.
Measuring the page fault rate is straightforward: just count the number of
faults per second, possibly taking a running mean over past seconds as well. One
easy way to do this is to add the number of page faults during the immediately preceding second to the current running mean and divide by two. The dashed line
marked A corresponds to a page fault rate that is unacceptably high, so the faulting
process is given more page frames to reduce the fault rate. The dashed line marked
B corresponds to a page fault rate so low that we can assume the process has too
much memory. In this case, page frames may be taken away from it. Thus, PFF
tries to keep the paging rate for each process within acceptable bounds.
It is important to note that some page replacement algorithms can work with
either a local replacement policy or a global one. For example, FIFO can replace
the oldest page in all of memory (global algorithm) or the oldest page owned by
the current process (local algorithm). Similarly, LRU or some approximation to it
can replace the least recently used page in all of memory (global algorithm) or the
least recently used page owned by the current process (local algorithm). The
choice of local versus global is independent of the algorithm in some cases.
On the other hand, for other page replacement algorithms, only a local strategy
makes sense. In particular, the working set and WSClock algorithms refer to some
224
MEMORY MANAGEMENT
CHAP. 3
specific process and must be applied in that context. There really is no working set
for the machine as a whole, and trying to use the union of all the working sets
would lose the locality property and not work well.
3.5.2 Load Control
Even with the best page replacement algorithm and optimal global allocation
of page frames to processes, it can happen that the system thrashes. In fact, whenever the combined working sets of all processes exceed the capacity of memory,
thrashing can be expected. One symptom of this situation is that the PFF algorithm
indicates that some processes need more memory but no processes need less memory. In this case, there is no way to give more memory to those processes needing
it without hurting some other processes. The only real solution is to temporarily
get rid of some processes.
The simplest solution is a blunt one: kill some processes. Operating systems
often have a special process called OOM (Out of Memory killer) that becomes
active when the system is low on memory. It reviews all running processes and
selects a victim to kill, freeing up its resources to keep the system running. Specifically, the OOM killer will examine all processes and assign them a score to indicate how ‘‘bad’’ it is. For instance, using up a lot of memory will increase a process’ badness score, while important processes (such as root and system processes)
get low scores. Also, the OOM killer will try to minimize the number of processes
to terminate (while still freeing up enough memory). After considering all the
processes, it will kill the process(es) with the highest score(s).
A considerably more friendly way to reduce the number of processes competing for memory is to swap some of them to nonvolatile storage and free up all the
pages they are holding. For example, one process can be swapped to nonvolatile
storage and its page frames divided up among other processes that are thrashing. If
the thrashing stops, the system can run for a while this way. If it does not stop,
another process has to be swapped out, and so on, until the thrashing stops. Thus
even with paging, swapping may still be needed, only now swapping is used to
reduce potential demand for memory, rather than to reclaim pages. Thus paging
and swapping are not mutually contradictory.
Swapping processes out to relieve the load on memory is reminiscent of twolevel scheduling, in which some processes are put on nonvolatile storage and a
short-term scheduler is used to schedule the remaining processes. Clearly, the two
ideas can be combined, with just enough processes swapped out to make the pagefault rate acceptable. Periodically, some processes are brought in from nonvolatile
storage and other ones are swapped out.
However, another factor to consider is the degree of multiprogramming. When
the number of processes in main memory is too low, the CPU may be idle for substantial periods of time. This consideration argues for considering not only process
SEC. 3.5
DESIGN ISSUES FOR PAGING SYSTEMS
225
size and paging rate when deciding which process to swap out, but also its characteristics, such as whether it is CPU bound or I/O bound, and what characteristics
the remaining processes have.
Before we close this section, we should mention that killing and swapping are
not the only options. For instance, another common solution is to reduce the memory usage by compaction and compression. Indeed, reducing a system’s memory
footprint is pretty high up on the priority list for operating system designers anywhere. One clever technique commonly used is known as deduplication or same
page merging. The idea is straightforward: periodically sweep the memory to see
if two pages (possibly in different processes) have the exact same content. If so,
rather than storing that content on two physical page frames, the operating system
removes one of the duplicates and modifies the page table mappings so that there
are now two virtual pages that point to the same frame. The frame is shared copyon-write: as soon as a process tries to write to the page, a fresh copy is made, so
the write does not affect the other page. Some people call this ‘‘dededuplication.’’
While they are not wrong, it is a cruel thing to do to the language of Shakespeare.
3.5.3 Cleaning Policy
Related to the topic of load control is the issue of cleaning. Aging works best
when there is an abundant supply of free page frames that can be claimed as page
faults occur. If every page frame is full, and furthermore modified, before a new
page can be brought in, an old page must first be written to nonvolatile storage. To
ensure a plentiful supply of free page frames, paging systems generally have a
background process, called the paging daemon, that sleeps most of the time but is
awakened periodically to inspect the state of memory. If too few page frames are
free, it begins selecting pages to evict using some page replacement algorithm. If
these pages have been modified since being loaded, they are written to nonvolatile
storage.
In any event, the previous contents of the page are remembered. In the event
one of the evicted pages is needed again before its frame has been overwritten, it
can be reclaimed by removing it from the pool of free page frames. Keeping a supply of page frames around yields better performance than using all of memory and
then trying to find a frame at the moment it is needed. At the very least, the paging
daemon ensures that all the free frames are clean, so they need not be written to
nonvolatile storage in a big hurry when they are required.
One way to implement this cleaning policy is with a two-handed clock. The
front hand is controlled by the paging daemon. When it points to a dirty page, that
page is written back to nonvolatile storage and the front hand is advanced. When it
points to a clean page, it is just advanced. The back hand is used for page replacement, as in the standard clock algorithm. Only now, the probability of the back
hand hitting a clean page is increased due to the work of the paging daemon.
226
MEMORY MANAGEMENT
CHAP. 3
3.5.4 Page Size
The page size is a parameter that can be chosen by the operating system. Even
if the hardware has been designed with, for example, 4096-byte pages, the operating system can easily regard page pairs 0 and 1, 2 and 3, 4 and 5, and so on, as
8-KB pages by always allocating two consecutive 8192-byte page frames for them.
Determining the best page size requires balancing several competing factors.
As a result, there is no overall optimum. To start with, two factors argue for a
small page size. A randomly chosen text, data, or stack segment will not fill an
integral number of pages. On the average, half of the final page will be empty.
The extra space in that page is wasted. This wastage is called internal fragmentation. With n segments in memory and a page size of p bytes, np/2 bytes will be
wasted on internal fragmentation. This reasoning argues for a small page size.
Another argument for a small page size becomes apparent if we think about a
program consisting of eight sequential phases of 4 KB each. With a 32-KB page
size, the program must be allocated 32 KB all the time. With a 16-KB page size, it
needs only 16 KB. With a page size of 4 KB or smaller, it requires only 4 KB at
any instant. In general, a large page size will cause more wasted space to be in
memory than a small page size.
On the other hand, small pages mean that programs will need many pages, and
thus a large page table. A 32-KB program needs only four 8-KB pages, but 64
512-byte pages. Transfers to and from the disk or SSD are generally a page at a
time. If the nonvolatile storage is not an SSD but a magnetic disk, most of the time
will be spent on the seek and rotational delay, so that transferring a small page
takes almost as much time as transferring a large page. It might take 64 × 10 msec
to load 64 512-byte pages, but only 4 × 12 msec to load four 8-KB pages.
Perhaps more importantly, small pages use up much valuable space in the
TLB. Say your program uses 1 MB of memory with a working set of 64 KB. With
4-KB pages, the program would occupy at least 16 entries in the TLB. With 2-MB
pages, a single TLB entry would be sufficient (in theory, it may be that you want to
separate data and instructions). As TLB entries are scarce, and critical for performance, it pays to use large pages wherever possible. To balance all these tradeoffs, operating systems sometimes use different page sizes for different parts of the
system. For instance, large pages for the kernel and smaller ones for user processes. Indeed, some operating systems go out of their way to use large pages, even
moving a process’ memory around to find or create contiguous ranges of memory
suitable for backing by a large page—a feature sometimes referred to as transparent huge pages.
On some machines, the page table must be loaded (by the operating system)
into hardware registers every time the CPU switches from one process to another.
On these machines, having a small page size means that the time required to load
the page registers gets longer as the page size gets smaller. Furthermore, the space
occupied by the page table increases as the page size decreases.
SEC. 3.5
227
DESIGN ISSUES FOR PAGING SYSTEMS
This last point can be analyzed mathematically. Let the average process size be
s bytes and the page size be p bytes. Furthermore, assume that each page entry
requires e bytes. The approximate number of pages needed per process is then s/ p,
occupying se / p bytes of page table space. The wasted memory in the last page of
the process due to internal fragmentation is p/2. The total overhead due to the
page table and the internal fragmentation loss is the sum of these two terms:
overhead = se / p + p/2
The first term (page table size) is large when the page size is small. The second
term (internal fragmentation) is large when the page size is large. The optimum
must lie somewhere in between. By taking the first derivative with respect to p and
equating it to zero, we get the equation
<se / p2 + 1/2 = 0
From this equation, we can derive a formula that gives the optimum page size
(considering only memory wasted in fragmentation and page table size). The
result is:
p=3
2se
}}}
For s = 1MB and e = 8 bytes per page table entry, the optimum page size is 4 KB.
Commercially available computers have used page sizes ranging from 512 bytes to
64 KB. A typical value used to be 1 KB, but nowadays 4 KB is more common.
3.5.5 Separate Instruction and Data Spaces
Most computers have a single address space that holds both programs and data,
as shown in Fig. 3-24(a). If this address space is large enough, everything works
fine. However, if it’s too small, it forces programmers to stand on their heads to fit
everything into the address space.
Single address
space
232
232
I space
D space
Unused page
Data
Data
Program
Program
0
(a)
0
(b)
Figure 3-24. (a) One address space. (b) Separate I and D spaces.
One solution, pioneered on the (16-bit) PDP-11, is to have separate address
spaces for instructions (program text) and data, called I-space and D-space,
228
MEMORY MANAGEMENT
CHAP. 3
respectively, as illustrated in Fig. 3-24(b). Each address space runs from 0 to some
maximum, typically 216 < 1 or 232 < 1. The linker must know when separate Iand D-spaces are being used, because when they are, the data are relocated to virtual address 0 instead of starting after the program.
In a computer with this kind of design, both address spaces can be paged, independently from one another. Each one has its own page table, with its own mapping of virtual pages to physical page frames. When the hardware wants to fetch an
instruction, it knows that it must use I-space and the I-space page table. Similarly,
data must go through the D-space page table. Other than this distinction, having
separate I- and D-spaces does not introduce any special complications for the operating system and it does double the available address space.
While address spaces these days are large, their sizes used to be a serious problem. Even today, though, separate I- and D-spaces are still common. However,
rather than for the normal address spaces, they are now used to divide the L1
cache. After all, in the L1 cache, memory is still plenty scarce. In fact, on some
processors we even find that, under the hood, the TLB is also partitioned in L1 and
L2 and the L1 TLB is further divided in a TLB for instructions and a TLB for data.
3.5.6 Shared Pages
Another design issue is sharing. In a large multiprogramming system, it is
common for several users to be running the same program at the same time. Even a
single user may be running several programs that use the same library. It is clearly
more efficient to share the pages, to avoid having two copies of the same page in
memory at the same time. One problem is that not all pages are sharable. In particular, pages that are read-only, such as program text, can be shared, but for data
pages sharing is more complicated.
If separate I- and D-spaces are supported, it is relatively straightforward to
share programs by having two or more processes use the same page table for their
I-space but different page tables for their D-spaces. Typically in an implementation
that supports sharing in this way, page tables are data structures independent of the
process table. Each process then has two pointers in its process table: one to the Ispace page table and one to the D-space page table, as shown in Fig. 3-25. When
the scheduler chooses a process to run, it uses these pointers to locate the appropriate page tables and sets up the MMU using them. Even without separate I- and Dspaces, processes can share programs (or sometimes, libraries), but the mechanism
is more complicated.
When two or more processes share some code, a problem occurs with the shared pages. Suppose that processes A and B are both running the editor and sharing
its pages. If the scheduler decides to remove A from memory, evicting all its pages
and filling the empty page frames with some other program will cause B to generate a large number of page faults to bring them back in again.
SEC. 3.5
229
DESIGN ISSUES FOR PAGING SYSTEMS
Process
table
Program
Data 1
Data 2
Page tables
Figure 3-25. Two processes sharing the same program sharing its page tables.
Similarly, when A terminates, it is essential to be able to discover that the
pages are still in use so that their nonvolatile storage space will not be freed by
accident. Searching all the page tables to see if a page is shared is usually too
expensive, so special data structures are needed to keep track of shared pages,
especially if the unit of sharing is the individual page (or run of pages), rather than
an entire page table.
Sharing data is trickier than sharing code, but it is not impossible. In particular, in UNIX, after a fork system call, the parent and child are required to share
both program text and data. In a paged system, what is often done is to give each
of these processes its own page table and have both of them point to the same set
of pages. Thus no copying of pages is done at fork time. However, all the data
pages are mapped into both processes as READ ONLY.
As long as both processes just read their data, without modifying it, this situation can continue. As soon as either process updates a memory word, the violation
of the read-only protection causes a trap to the operating system. A copy is then
made of the offending page so that each process now has its own private copy.
Both copies are now set to READ/WRITE, so subsequent writes to either copy
proceed without trapping. This strategy means that those pages that are never modified (including all the program pages) need not be copied. Only the data pages that
are actually modified need to be copied. This approach, called copy on write,
improves performance by reducing copying.
230
MEMORY MANAGEMENT
CHAP. 3
3.5.7 Shared Libraries
Sharing can be done at other granularities than individual pages. If a program
is started up twice, most operating systems will automatically share all the text
pages so that only one copy is in memory. Text pages are always read only, so there
is no problem here. Depending on the operating system, each process may get its
own private copy of the data pages, or they may be shared and marked read only.
If any process modifies a data page, a private copy will be made for it, that is, copy
on write will be applied.
In modern systems, there are many large libraries used by many processes, for
example, multiple I/O and graphics libraries. Statically binding all these libraries to
every executable program on nonvolatile storage would make them even more
bloated than they already are.
Instead, a common technique is to use shared libraries (which are called
DLLs or Dynamic Link Libraries on Windows). To make the idea of a shared
library clear, first consider traditional linking. When a program is linked, one or
more object files and possibly some libraries are named in the command to the
linker, such as the UNIX command
ld *.o –lc –lm
which links all the .o (object) files in the current directory and then scans two
libraries, /usr/lib/libc.a and /usr/lib/libm.a. Any functions called in the object files
but not present there (e.g., printf) are called undefined externals and are sought in
the libraries. If they are found, they are included in the executable binary. Any
functions that they call but are not yet present also become undefined externals.
For example, printf needs write, so if write is not already included, the linker will
look for it and include it when found. When the linker is done, an executable binary file is written to nonvolatile storage containing all the functions needed. Functions present in the libraries but not called are not included. When the program is
loaded into memory and executed, all the functions it needs are there and functions
it does not need are not there.
Now suppose common programs use 20–50 MB worth of graphics and user
interface functions. Statically linking hundreds of programs with all these libraries
would waste a significant amount of space on nonvolatile storage as well as wasting space in RAM when they were loaded since the system would have no way of
knowing it could share them. This is where shared libraries come in. When a program is linked with shared libraries (which are slightly different than static ones),
instead of including the actual function called, the linker includes a small stub routine that binds to the called function at run time. Depending on the system and the
configuration details, shared libraries are loaded either when the program is loaded
or when functions in them are called for the first time. Of course, if another program has already loaded the shared library, there is no need to load it again—that is
the whole point of it. Note that when a shared library is loaded or used, the entire
SEC. 3.5
231
DESIGN ISSUES FOR PAGING SYSTEMS
library is not read into memory in a single blow. It is paged in, page by page, as
needed, so functions that are not called will not be brought into RAM.
In addition to making executable files smaller and also saving space in memory, shared libraries have another important advantage: if a function in a shared
library is updated to remove a bug, it is not necessary to recompile the programs
that call it. The old binaries continue to work. This feature is especially important
for commercial software, where the source code is not distributed to the customer.
For example, if Microsoft finds and fixes a security error in some standard DLL,
Windows Update will download the new DLL and replace the old one, and all programs that use the DLL will automatically use the new version the next time they
are launched.
Shared libraries come with one little problem, however, that has to be solved,
however. The problem is illustrated in Fig. 3-26. Here we see two processes sharing a library of size 20 KB (assuming each box is 4 KB). However, the library is
located at a different address in each process, presumably because the programs
themselves are not the same size. In process 1, the library starts at address 36K; in
process 2 it starts at 12K. Suppose that the first thing the first function in the
library has to do is jump to address 16 in the library. If the library were not shared,
it could be relocated on the fly as it was loaded so that the jump (in process 1)
could be to virtual address 36K + 16. Note that the physical address in the RAM
where the library is located does not matter since all the pages are mapped from
virtual to physical addresses by the MMU hardware.
36K
12K
0
0
Process 1
RAM
Process 2
Figure 3-26. A shared library being used by two processes.
However, since the library is shared, relocation on the fly will not work. After
all, when the first function is called by process 2 (at address 12K), the jump
instruction has to go to 12K + 16, not 36K + 16. This is the little problem. One
way to solve it is to use copy on write and create new pages for each process sharing the library, relocating them on the fly as they are created, but this scheme
defeats the purpose of sharing the library, of course.
232
MEMORY MANAGEMENT
CHAP. 3
A better solution is to compile shared libraries with a special compiler flag telling the compiler not to produce any instructions that use absolute addresses.
Instead only instructions using relative addresses are used. For example, there is
almost always an instruction that says jump forward (or backward) by n bytes (as
opposed to an instruction that gives a specific address to jump to). This instruction
works correctly no matter where the shared library is placed in the virtual address
space. By avoiding absolute addresses, the problem can be solved. Code that uses
only relative offsets is called position-independent code.
3.5.8 Mapped Files
Shared libraries are really a special case of a more general facility called memory-mapped files. The idea here is that a process can issue a system call to map a
file onto a portion of its virtual address space. In most implementations, no pages
are brought in at the time of the mapping, but as pages are touched, they are
demand paged in one page at a time, using the file on nonvolatile storage as the
backing store. When the process exits, or explicitly unmaps the file, all the modified pages are written back to the file on disk or SSD.
Mapped files provide an alternative model for I/O. Instead of doing reads and
writes, the file can be accessed as a big character array in memory. In some situations, programmers find this model more convenient.
If two or more processes map onto the same file at the same time, they can
communicate over shared memory. Writes done by one process to the shared memory are immediately visible when the other one reads from the part of its virtual
address space mapped onto the file. This mechanism thus provides a high-bandwidth channel between processes and is often used as such (even to the extent of
mapping a scratch file). Now it should be clear that if memory-mapped files are
available, shared libraries can use this mechanism.
3.6 IMPLEMENTATION ISSUES
Implementers of virtual memory systems have to make choices among the
major theoretical algorithms, such as second chance versus aging, local versus global page allocation, and demand paging versus prepaging. But they also have to be
aware of a number of practical implementation issues as well. In this section, we
will take a look at a few of the common problems and some solutions.
3.6.1 Operating System Involvement with Paging
There are four times when the operating system has paging-related work to do:
process creation time, process execution time, page fault time, and process termination time. We will now briefly examine each of these to see what has to be done.
SEC. 3.6
IMPLEMENTATION ISSUES
233
When a new process is created in a paging system, the operating system has to
determine how large the program and data will be (initially) and create a page table
for them. Space has to be allocated in memory for the page table and it has to be
initialized. The page table need not be resident when the process is swapped out
but has to be in memory when the process is running. In addition, space has to be
allocated in the swap area on nonvolatile storage so that when a page is swapped
out, it has somewhere to go. The swap area also has to be initialized with program
text and data so that when the new process starts getting page faults, the pages can
be brought in. Some systems page the program text directly from the executable
file, thus saving space on disk or SSD and initialization time. Finally, information
about the page table and swap area on nonvolatile storage must be recorded in the
process table.
When a process is scheduled for execution, the MMU has to be reset for the
new process. In addition, unless the entries in the TLB are explicitly tagged with
an identifier for the processes to which they belong (using so-called tagged TLBs),
the TLB has to be flushed to get rid of traces of the previously executing process.
After all, we do not want to have a memory access in one process to erroneously
touch the page frame of another process. Further, the new process’ page table has
to be made current, usually by copying it or a pointer to it to some hardware register(s). Optionally, some or all of the process’ pages can be brought into memory to
reduce the number of page faults initially (e.g., it is certain that the page pointed to
by the program counter will be needed).
When a page fault occurs, the operating system has to read out hardware registers to determine which virtual address caused the fault. From this information, it
must compute which page is needed and locate that page on nonvolatile storage. It
must then find an available page frame in which to put the new page, evicting some
old page if need be. Then it must read the needed page into the page frame. Finally, it must back up the program counter to have it point to the faulting instruction
and let that instruction execute again.
When a process exits, the operating system must release its page table, its
pages, and the nonvolatile storage space that the pages occupy when they are on
disk or SSD. If some of the pages are shared with other processes, the pages in
memory and on nonvolatile storage can be released only when the last process
using them has terminated.
3.6.2 Page Fault Handling
We are finally in a position to describe in detail what happens on a page fault.
Slightly simplified, the sequence of events is as follows:
1. The hardware traps to the kernel, saving the program counter on the
stack. On most machines, some information about the state of the
current instruction is saved in special CPU registers.
234
MEMORY MANAGEMENT
CHAP. 3
2. An assembly-code interrupt service routine is started to save the registers and other volatile information, to keep the operating system
from destroying it. It then calls the page fault handler.
3. The operating system tries to discover which virtual page is needed.
Often one of the hardware registers contains this information. If not,
the operating system must retrieve the program counter, fetch the
instruction, and parse it in software to figure out what it was doing
when the fault hit.
4. Once the virtual address that caused the fault is known, the system
checks to see if this address is valid and the protection is consistent
with the access. If not, the process is sent a signal or killed. If the
address is valid and no protection fault has occurred, the system
checks to see if a page frame is free. If no frames are free, the page
replacement algorithm is run to select a victim.
5. If the page frame selected is dirty, the page is scheduled for transfer to
nonvolatile storage, and a context switch takes place, suspending the
faulting process and letting another one run until the disk or SSD
transfer has completed. In any event, the frame is marked as busy to
prevent it from being used for another purpose.
6. As soon as the page frame is clean (either immediately or after it is
written to nonvolatile storage), the operating system looks up the disk
address where the needed page is, and schedules a disk or SSD operation to bring it in. While the page is being loaded, the faulting process
is still suspended and another user process is run, if one is available.
7. When the disk or SSD interrupt indicates that the page has arrived,
the page tables are updated to reflect its position, and the frame is
marked as being in the normal state.
8. The faulting instruction is backed up to the state it had when it began
and the program counter is reset to point to that instruction.
9. The faulting process is scheduled, and the operating system returns to
the (assembly-language) routine that called it.
10. This routine reloads the registers and other state information and
returns to user space to continue execution where it left off.
3.6.3 Instruction Backup
So far, we simply said that when a program references a page that is not in
memory, the instruction causing the fault is stopped partway through and a trap to
the operating system occurs. After the operating system has fetched the page needed, it must restart the instruction causing the trap. This is easier said than done.
SEC. 3.6
IMPLEMENTATION ISSUES
235
To see the nature of this problem at its worst, consider a CPU that has instructions with two addresses, such as the Motorola 680x0, widely used in embedded
systems. The instruction
MOV.L #6(A1),2(A0)
is 6 bytes, for example (see Fig. 3-27). In order to restart the instruction, the operating system must determine where the first byte of the instruction is. The value of
the program counter at the time of the trap depends on which operand faulted and
how the CPU’s microcode has been implemented.
MOVE.L #6(A1), 2(A0)
16 Bits
1000
1002
MOVE
6
1004
2
} Opcode
} First operand
} Second operand
Figure 3-27. An instruction causing a page fault.
In Fig. 3-27, we have an instruction starting at address 1000 that makes three
memory references: the instruction word and two offsets for the operands. Depending on which of these three memory references caused the page fault, the program
counter might be 1000, 1002, or 1004 at the time of the fault. It is often impossible
for the operating system to determine unambiguously where the instruction began.
If the program counter is 1002 at the time of the fault, the operating system has no
way of telling whether the word in 1002 is a memory address associated with an
instruction at 1000 (e.g., the address of an operand) or an opcode.
Bad as this problem may be, it could have been worse. Some 680x0 addressing
modes use autoincrementing, which means that a side effect of executing the
instruction is to increment one (or more) registers. Instructions that use autoincrement mode can also fault. Depending on the details of the microcode, the increment may be done before the memory reference, in which case the operating system must decrement the register in software before restarting the instruction. Or,
the autoincrement may be done after the memory reference, in which case it will
not have been done at the time of the trap and must not be undone by the operating
system. Autodecrement mode also exists and causes a similar problem. The precise details of whether autoincrements and autodecrements have or have not been
done before the corresponding memory references may differ from instruction to
instruction and from CPU model to CPU model.
Fortunately, on some machines the CPU designers provide a solution, usually
in the form of a hidden internal register into which the program counter is copied
just before each instruction is executed. These machines may also have a second
register telling which registers have already been autoincremented or autodecremented, and by how much. Given this information, the operating system can
236
MEMORY MANAGEMENT
CHAP. 3
unambiguously undo all the effects of the faulting instruction so that it can be
restarted. If this information is not available, the operating system has to jump
through hoops to figure out what happened and how to repair it. The problem
could have been solved in hardware but that would have made the hardware more
expensive so it was decided to leave it to the software.
3.6.4 Locking Pages in Memory
Although we have not discussed I/O much in this chapter, the fact that a computer has virtual memory does not mean that I/O is absent. Virtual memory and I/O
interact in subtle ways. Consider a process that has just issued a system call to
read from some file or device into a buffer within its address space. While waiting
for the I/O to complete, the process is suspended and another process is allowed to
run. This other process gets a page fault.
If the paging algorithm is global, there is a small, but nonzero, chance that the
page containing the I/O buffer will be chosen to be removed from memory. If an
I/O device is currently in the process of doing a DMA transfer to that page, removing it will cause part of the data to be written in the buffer where they belong, and
part of the data to be written over the just-loaded page. One solution to this problem is to lock pages engaged in I/O in memory so that they will not be removed.
Locking a page is often called pinning it in memory. Another solution is to do all
I/O to kernel buffers and then copy the data to user pages later. However, this
requires an extra copy and thus slows everything down.
3.6.5 Backing Store
In our discussion of page replacement algorithms, we saw how a page is selected for removal. We have not said much about where on nonvolatile storage it is
put when it is paged out. Let us now describe some of the issues related to
disk/SSD management.
The simplest algorithm for allocating page space on nonvolatile storage is to
have a special swap partition on the disk or, even better, on a separate storage device from the file system (to balance the I/O load). UNIX systems traditionally
work like this. This partition does not have a normal file system on it, which eliminates all the overhead of converting offsets in files to block addresses. Instead,
block numbers relative to the start of the partition are used throughout.
When the system is booted, this swap partition is empty and is represented in
memory as a single entry giving its origin and size. In the simplest scheme, when
the first process is started, a chunk of the partition area the size of the first process
is reserved and the remaining area reduced by that amount. As new processes are
started, they are assigned chunks of the swap partition equal in size to their core
images. As they finish, their storage space is freed. The swap partition is managed
as a list of free chunks. Better algorithms will be discussed in Chap. 10.
SEC. 3.6
237
IMPLEMENTATION ISSUES
Associated with each process is the nonvolatile storage address of its swap
area, that is, where on the swap partition its image is kept. This information is kept
in the process table. Calculating the address to write a page to becomes simple:
just add the offset of the page within the virtual address space to the start of the
swap area. However, before a process can start, the swap area must be initialized.
One way is to copy the entire process image to the swap area, so that it can be
brought in as needed. The other is to load the entire process in memory and let it
be paged out as needed.
However, this simple model has a problem: processes can increase in size after
starting. Although the program text is usually fixed, the data area can sometimes
grow, and the stack can always grow. Consequently, it may be better to reserve separate swap areas for the text, data, and stack and allow each of these areas to consist of more than one chunk on the nonvolatile storage.
The other extreme is to allocate nothing in advance and allocate space on nonvolatile storage for each page when it is swapped out and deallocate it when it is
swapped back in. In this way, processes in memory do not tie up any swap space.
The disadvantage is that a disk address is needed in memory to keep track of each
page on nonvolatile storage. In other words, there must be a table per process telling for each page on nonvolatile storage where it is. The two alternatives are
shown in Fig. 3-28.
Main memory
Disk
Main memory
Pages
0
Disk
Pages
3
Swap area
0
3
4
6
Swap area
7
4
6
5
2
1
Page
table
6
6
4
3
4
3
0
0
(a)
7
5
2
1
Page
table
Disk
map
(b)
Figure 3-28. (a) Paging to a static swap area. (b) Backing up pages dynamically.
In Fig. 3-28(a), a page table with eight pages is shown. Pages 0, 3, 4, and 6 are
in main memory. Pages 1, 2, 5, and 7 are on disk. The swap area on disk is as large
as the process virtual address space (eight pages), with each page having a fixed
location to which it is written when it is evicted from main memory. Calculating
this address requires knowing only where the process’ paging area begins, since
238
MEMORY MANAGEMENT
CHAP. 3
pages are stored in it contiguously in order of their virtual page number. A page
that is in memory always has a shadow copy on disk, but this copy may be out of
date if the page has been modified since being loaded. The shaded pages in memory indicate pages not present in memory. The shaded pages on the disk are (in
principle) superseded by the copies in memory, although if a memory page has to
be swapped back to disk and it has not been modified since it was loaded, the
(shaded) disk copy will be used.
In Fig. 3-28(b), pages do not have fixed addresses on disk. When a page is
swapped out, an empty disk page is chosen on the fly and the disk map (which has
room for one disk address per virtual page) is updated accordingly. A page in
memory has no copy on disk. The pages’ entries in the disk map contain an invalid
disk address or a bit marking them as not in use.
Having a fixed swap partition is not always possible. For example, no disk or
SSD partitions may be available. In this case, one or more large, preallocated files
within the normal file system can be used. Windows uses this approach. However,
an optimization can be used here to reduce the amount of nonvolatile storage space
needed. Since the program text of every process came from some (executable) file
in the file system, the executable file can be used as the swap area. Better yet, since
the program text is generally read only, when memory is tight and program pages
have to be evicted from memory, they are just discarded and read in again from the
executable file when needed. Shared libraries can also work this way.
3.6.6 Separation of Policy and Mechanism
An important principle for managing the complexity of any system is to split
policy from mechanism. We will illustrate how this principle can be applied to
memory management by having most of the memory manager run as a user-level
process—a separation that was first done in Mach (Young et al., 1987) on which
we base the discussion below.
A simple example of how policy and mechanism can be separated is shown in
Fig. 3-29. Here the memory management system is divided into three parts:
1. A low-level MMU handler.
2. A page fault handler that is part of the kernel.
3. An external pager running in user space.
All the details of how the MMU works are encapsulated in the MMU handler,
which is machine-dependent code and has to be rewritten for each new platform
the operating system is ported to. The page-fault handler is machine-independent
code and contains most of the mechanism for paging. The policy is largely determined by the external pager, which runs as a user process.
When a process starts up, the external pager is notified in order to set up the
process’ page map and allocate the necessary backing store on nonvolatile storage
SEC. 3.6
3. Request page
Main memory
User
space
User
process
Disk
External
pager
2. Needed
page
4. Page
arrives
5. Here
is page
1. Page
fault
Kernel
space
239
IMPLEMENTATION ISSUES
Fault
handler
6. Map
page in
MMU
handler
Figure 3-29. Page fault handling with an external pager.
if need be. As the process runs, it may map new objects into its address space, so
the external pager is once again notified.
Once the process starts running, it may get a page fault. The fault handler figures out which virtual page is needed and sends a message to the external pager,
telling it the problem. The external pager then reads the needed page in from the
nonvolatile storage and copies it to a portion of its own address space. Then it tells
the fault handler where the page is. The fault handler then unmaps the page from
the external pager’s address space and asks the MMU handler to put it into the
user’s address space at the right place. Then the user process can be restarted.
This implementation leaves open where the page replacement algorithm is put.
It would be cleanest to have it in the external pager, but there are some problems
with this approach. Principal among these is that the external pager does not have
access to the R and M bits of all the pages. These bits play a role in many of the
paging algorithms. Thus, either some mechanism is needed to pass this information up to the external pager, or the page replacement algorithm must go in the kernel. In the latter case, the fault handler tells the external pager which page it has
selected for eviction and provides the data, either by mapping it into the external
pager’s address space or including it in a message. Either way, the external pager
writes the data to nonvolatile storage.
The main advantage of this implementation is more modular code and greater
flexibility. The main disadvantage is the extra overhead of crossing the user-kernel
boundary several times and the overhead of the various messages being sent
between the pieces of the system. The subject is controversial, but as computers
get faster and faster, and the software gets more and more complex, in the long run
sacrificing some performance for more reliable software may well be acceptable to
most implementers and users. Some operating systems that implement paging in
240
MEMORY MANAGEMENT
CHAP. 3
the operating system kernel, such as Linux, nowadays also offer support for ondemand paging in user processes (see for instance userfaultfd).
3.7 SEGMENTATION
Despite paging, the virtual memory discussed so far is one dimensional
because the virtual addresses go from 0 to some maximum address, one address
after another. For many problems, having two or more separate virtual address spaces may be much better than having only one. For example, a compiler has many
tables that are built up as compilation proceeds, possibly including
1. The source text being saved for the printed listing (on batch systems).
2. The symbol table, containing the names and attributes of variables.
3. The table containing all the integer and floating-point constants used.
4. The parse tree, containing the syntactic analysis of the program.
5. The stack used for procedure calls within the compiler.
Each of the first four tables grows continuously as compilation proceeds. The last
one grows and shrinks in unpredictable ways during compilation. In a one-dimensional memory, these five tables would have to be allocated contiguous chunks of
virtual address space, as in Fig. 3-30.
Consider what happens if a program has a much larger than usual number of
variables but a normal amount of everything else. The chunk of address space allocated for the symbol table may fill up, but there may be lots of room in the other
tables. What is needed is a way of freeing the programmer from having to manage
the expanding and contracting tables, in the same way that virtual memory eliminates the worry of organizing the program into overlays.
A straightforward and quite general solution is to provide the machine with
many completely independent address spaces, which are called segments. Each
segment consists of a linear sequence of addresses, starting at 0 and going up to
some maximum value. The length of each segment may be anything from 0 to the
maximum address allowed. Different segments may, and usually do, have different
lengths. Moreover, segment lengths may change during execution. The length of a
stack segment may be increased whenever something is pushed onto the stack and
decreased whenever something is popped off the stack.
Because each segment constitutes a separate address space, different segments
can grow or shrink independently without affecting each other. If a stack in a certain segment needs more address space to grow, it can have it, because there is
nothing else in its address space to bump into. Of course, a segment can fill up, but
segments are usually very large, so this occurrence is rare. To specify an address
in this segmented or two-dimensional memory, the program must supply a two-part
SEC. 3.7
241
SEGMENTATION
Virtual address space
Call stack
Free
Address space
allocated to the
parse tree
Parse tree
Space currently being
used by the parse tree
Constant table
Source text
Symbol table
Symbol table has
bumped into the
source text table
Figure 3-30. In a one-dimensional address space with growing tables, one table
may bump into another.
address, a segment number, and an address within the segment. Figure 3-31 illustrates a segmented memory being used for the compiler tables discussed earlier.
Five independent segments are shown here.
20K
16K
16K
12K
12K
12K
8K
8K
8K
4K
4K
0K
0K
12K
Symbol
table
Segment
0
Source
text
4K
0K
Segment
1
Parse
tree
Constants
Segment
2
8K
Call
stack
4K
0K
0K
Segment
3
Segment
4
Figure 3-31. A segmented memory allows each table to grow or shrink independently of the other tables.
242
MEMORY MANAGEMENT
CHAP. 3
We emphasize here that a segment is a logical entity, which the programmer is
aware of and uses as a logical entity. A segment might contain a procedure, or an
array, or a stack, or a collection of scalar variables, but usually it does not contain a
mixture of different types.
A segmented memory has other advantages besides simplifying the handling of
data structures that are growing or shrinking. If each procedure occupies a separate segment, with address 0 as its starting address, the linking of procedures compiled separately is greatly simplified. After all the procedures that constitute a program have been compiled and linked up, a procedure call to the procedure in segment n will use the two-part address (n, 0) to address word 0 (the entry point).
If the procedure in segment n is subsequently modified and recompiled, no
other procedures need be changed (because no starting addresses have been modified), even if the new version is larger than the old one. With a one-dimensional
memory, the procedures are packed tightly right up next to each other, with no address space between them. Consequently, changing one procedure’s size can affect
the starting address of all the other (unrelated) procedures in the segment. This, in
turn, requires modifying all procedures that call any of the moved procedures, in
order to incorporate their new starting addresses. If a program contains hundreds
of procedures, this process can be costly.
Segmentation also facilitates sharing procedures or data between several processes. A common example is the shared library. Modern workstations that run
advanced window systems often have extremely large graphical libraries compiled
into nearly every program. In a segmented system, the graphical library can be put
in a segment and shared by multiple processes, eliminating the need for having it in
every process’ address space. While it is also possible to have shared libraries in
pure paging systems, it is more complicated. In effect, these systems do it by simulating segmentation.
Since each segment forms a logical entity that programmers know about, such
as a procedure, or an array, different segments can have different kinds of protection. A procedure segment can be specified as execute only, prohibiting attempts
to read from or store into it. A floating-point array can be specified as read/write
but not execute, and attempts to jump to it will be caught. Such protection is helpful in catching bugs. Paging and segmentation are compared in Fig. 3-32.
3.7.1 Implementation of Pure Segmentation
The implementation of segmentation differs from paging in an essential way:
pages are of fixed size and segments are not. Figure 3-33(a) shows an example of
physical memory initially containing five segments. Now consider what happens if
segment 1 is evicted and segment 7, which is smaller, is put in its place. We arrive
at the memory configuration of Fig. 3-33(b). Between segment 7 and segment 2 is
an unused area—that is, a hole. Then segment 4 is replaced by segment 5, as in
Fig. 3-33(c), and segment 3 is replaced by segment 6, as in Fig. 3-33(d). After the
SEC. 3.7
243
SEGMENTATION
Paging
Consideration
Segmentation
Need the programmer be aware
that this technique is being used?
No
Yes
How many linear address
spaces are there?
1
Many
Can the total address space
exceed the size of physical
memory?
Yes
Yes
Can procedures and data be
distinguished and separately
protected?
No
Yes
Can tables whose size fluctuates
be accommodated easily?
No
Yes
Is sharing of procedures
between users facilitated?
No
Yes
Why was this technique
invented?
To get a large
linear address
space without
having to buy
more physical
memory
To allow programs
and data to be broken
up into logically
independent address
spaces and to aid
sharing and
protection
Figure 3-32. Comparison of paging and segmentation.
system has been running for a while, memory will be divided up into a number of
chunks, some containing segments and some containing holes. This phenomenon,
called checkerboarding or external fragmentation, wastes memory in the holes.
It can be dealt with by compaction, as shown in Fig. 3-33(e).
3.7.2 Segmentation with Paging: MULTICS
If the segments are large, it may be inconvenient, or even impossible, to keep
them in main memory in their entirety. This leads to the idea of paging them, so
that only those pages of a segment that are actually needed have to be around. Several significant systems have supported paged segments. In this section, we will
describe the first one: MULTICS. Its design strongly influenced the Intel x86
which similarly offered segmentation and paging up until the x86-64.
The MULTICS operating system was one of the most influential operating systems ever, having had a major influence on topics as disparate as UNIX, the x86
memory architecture, TLBs, and cloud computing. It was started as a research
project at M.I.T. and went live in 1969. The last MULTICS system was shut down
in 2000, a run of 31 years. Few other operating systems have lasted more-or-less
244
MEMORY MANAGEMENT
CHAP. 3
(3K)
(3K)
Segment 5
(4K)
Segment 5
(4K)
Segment 4
(7K)
Segment 4
(7K)
Segment 3
(8K)
Segment 3
(8K)
Segment 3
(8K)
Segment 2
(5K)
Segment 2
(5K)
Segment 2
(5K)
Segment 2
(5K)
(3K)
(3K)
(3K)
Segment 2
(5K)
Segment 7
(5K)
Segment 7
(5K)
Segment 7
(5K)
Segment 7
(5K)
Segment 0
(4K)
Segment 0
(4K)
Segment 0
(4K)
Segment 0
(4K)
Segment 0
(4K)
(a)
(b)
(c)
(d)
(e)
Segment 1
(8K)
(10K)
(4K)
Segment 6
(4K)
Segment 5
(4K)
Segment 6
(4K)
Figure 3-33. (a)–(d) Development of checkerboarding. (e) Removal of the
checkerboarding by compaction.
unmodified anywhere near that long. While operating systems called Windows
have also been around that long, Windows 11 has absolutely nothing in common
with Windows 1.0 except the name and the fact that it was written by Microsoft.
Even more to the point, the ideas developed in MULTICS are as valid and useful
now as they were in 1965, when the first paper was published (Corbató and Vyssotsky, 1965). For this reason, we will now spend a little bit of time looking at the
most innovative aspect of MULTICS, the virtual memory architecture. More information about MULTICS can be found at www.multicians.org.
MULTICS ran on the Honeywell 6000 machines and their descendants and
provided each program with a virtual memory of up to 218 segments, each of which
was up to 65,536 (36-bit) words long. To implement this, the MULTICS designers
chose to treat each segment as a virtual memory and to page it, combining the
advantages of paging (uniform page size and not having to keep the whole segment
in memory if only part of it was being used) with the advantages of segmentation
(ease of programming, modularity, protection, sharing).
Each MULTICS program had a segment table, with one descriptor per segment. Since there were potentially more than a quarter of a million entries in the
table, the segment table was itself a segment and was paged. A segment descriptor
contained an indication of whether the segment was in main memory or not. If any
part of the segment was in memory, the segment was considered to be in memory,
and its page table was in memory. If the segment was in memory, its descriptor
contained an 18-bit pointer to its page table, as in Fig. 3-34(a). Because physical
addresses were 24 bits and pages were aligned on 64-byte boundaries (implying
that the low-order 6 bits of page addresses were 000000), only 18 bits were needed
in the descriptor to store a page table address. The descriptor also contained the
SEC. 3.7
245
SEGMENTATION
segment size, the protection bits, and other items. Figure 3-34(b) illustrates a segment descriptor. The address of the segment in secondary memory was not in the
segment descriptor but in another table used by the segment fault handler.
36 bits
Page 2 entry
Page 1 entry
Segment 6 descriptor
Page 0 entry
Segment 5 descriptor
Page table for segment 3
Segment 4 descriptor
Segment 3 descriptor
Segment 2 descriptor
Segment 1 descriptor
Page 2 entry
Segment 0 descriptor
Page 1 entry
Descriptor segment
Page 0 entry
Page table for segment 1
(a)
18
9
Main memory address
of the page table
Segment length
(in pages)
1 1 1
3
3
Page size:
0 = 1024 words
1 = 64 words
0 = segment is paged
1 = segment is not paged
Miscellaneous bits
Protection bits
(b)
Figure 3-34. The MULTICS virtual memory. (a) The descriptor segment pointed to the page tables. (b) A segment descriptor. The numbers are the field
lengths.
Each segment was an ordinary virtual address space and was paged in the same
way as the nonsegmented paged memory described earlier in this chapter. The normal page size was 1024 words (although a few small segments used by MULTICS
itself were not paged or were paged in units of 64 words to save physical memory).
246
MEMORY MANAGEMENT
CHAP. 3
An address in MULTICS consisted of two parts: the segment and the address
within the segment. The address within the segment was further divided into a page
number and a word within the page, as shown in Fig. 3-35. When a memory reference occurred, the following algorithm was carried out.
1. The segment number was used to find the segment descriptor.
2. A check was made to see if the segment’s page table was in memory.
If it was, it was located. If it was not, a segment fault occurred. If
there was a protection violation, a fault (trap) occurred.
3. The page table entry for the requested virtual page was examined. If
the page itself was not in memory, a page fault was triggered. If it
was in memory, the main-memory address of the start of the page was
extracted from the page table entry.
4. The offset was added to the page origin to give the main memory
address where the word was located.
5. The read or store finally took place.
Address within
the segment
Segment number
Page
number
Offset within
the page
18
6
10
Figure 3-35. A 34-bit MULTICS virtual address.
This process is illustrated in Fig. 3-36. For simplicity, the fact that the descriptor segment was itself paged has been omitted. What really happened was that a
register (the descriptor base register) was used to locate the descriptor segment’s
page table, which, in turn, pointed to the pages of the descriptor segment. Once the
descriptor for the needed segment was been found, the addressing proceeded as
shown in Fig. 3-36.
As you have no doubt guessed by now, if the preceding algorithm were actually carried out by the operating system on every instruction, programs would not
run very fast and users would not be happy campers. In reality, the MULTICS
hardware contained a 16-word high-speed TLB that could search all its entries in
parallel for a given key. This was the first system to have a TLB, something used
in all modern architectures. It is illustrated in Fig. 3-37. When an address was
presented to the computer, the addressing hardware first checked to see if the virtual address was in the TLB. If so, it got the page frame number directly from the
TLB and formed the actual address of the referenced word without having to look
in the descriptor segment or page table.
SEC. 3.7
247
SEGMENTATION
MULTICS virtual address
Page
number
Segment number
Offset
Word
Descriptor
Segment
number
Descriptor
segment
Page frame
Offset
Page
number
Page
table
Page
Figure 3-36. Conversion of a two-part MULTICS address into a main memory address.
Is this
entry
used?
Comparison
field
Segment
number
Virtual
page
Page
frame
Protection
Age
4
1
7
Read/write
13
1
6
0
2
Read only
10
1
12
3
1
Read/write
2
1
2
1
0
Execute only
7
1
2
2
12
Execute only
9
1
0
Figure 3-37. A simplified version of the MULTICS TLB. The existence of two
page sizes made the actual TLB more complicated.
The addresses of the 16 most recently referenced pages were kept in the TLB.
Programs whose working set was smaller than the TLB size came to equilibrium
with the addresses of the entire working set in the TLB and therefore ran
efficiently; otherwise, there were TLB faults.
248
MEMORY MANAGEMENT
CHAP. 3
3.7.3 Segmentation with Paging: The Intel x86
Up until the x86-64, the virtual memory system of the x86 resembled that of
MULTICS in many ways, including the presence of both segmentation and paging.
Whereas MULTICS had 256K independent segments, each up to 64K 36-bit
words, the x86 has 16K independent segments, each holding up to 1 billion 32-bit
words. Although there are fewer segments, the larger segment size is far more
important, as few programs need more than 1000 segments, but many programs
need large segments. As of x86-64, segmentation is considered obsolete and is no
longer supported, except in legacy mode. Although some vestiges of the old segmentation mechanisms are still available in x86-64’s native mode, mostly for compatibility, they no longer serve the same role and no longer offer true segmentation.
The x86-32, however, still comes equipped with the whole shebang.
So why did Intel kill what was a variant of the perfectly good MULTICS memory model that it supported for close to three decades? Probably the main reason is
that neither UNIX nor Windows ever used it, even though it was quite efficient
because it eliminated system calls, turning them into lightning-fast procedure calls
to the relevant address within a protected operating system segment. None of the
developers of any UNIX or Windows system wanted to change their memory model to something that was x86 specific because it would surely break portability to
other platforms. Since the software was not using the feature, Intel got tired of
wasting chip area to support it and removed it from the 64-bit CPUs.
All in all, one has to give credit to the x86 designers. Given the conflicting
goals of implementing pure paging, pure segmentation, and paged segments, while
at the same time being compatible with the 286, and doing all of this efficiently,
the resulting design is surprisingly simple and clean.
3.8 RESEARCH ON MEMORY MANAGEMENT
Memory management is an active research area and every year brings a new
harvest of publications to improve a system’s security, performance, or both. Also,
while traditional memory management topics, especially paging algorithms for
uniprocessor CPUs, has largely died off, researchers now look at new types of storage, or at incorporating memory on remote machines (Ruan et al., 2020). Also,
some people never say die and even the good old fork system call has been given a
do-over. Observing that the performance of fork has become a bottleneck for memory intensive applications, as all the page tables must be copied first, even if the
data and code pages themselves are shared copy-on-write, as a logical next step,
researchers decided to also share the page tables copy-on-write (Zhao, 2021).
Memory management in datacenters and clouds is complicated. For instance, a
big problem arises when a virtual machine is happily humming along when all of a
SEC. 3.8
RESEARCH ON MEMORY MANAGEMENT
249
sudden the hypervisor needs an update. While you may migrate the virtual
machines to another node, this turns out to be inefficient and in-place updates are
often possible, even without a reboot, by preserving the memory pages of the virtual machine (Russinovich, 2021). Another issue that makes memory management in
these environments different from traditional settings is that in data centers many
applications run on a complex software stack where each layer does some memory
management and the applications are able to adapt their performance to the available memory, rendering the working set model ineffective. Performance can be
improved by inserting policies and mechanisms in each layer to coordinate memory management (Lion et al., 2021).
Integrating new forms of memory and storage into the regular memory hierarchy is not easy and the adoption of persistent memory in existing systems has
been a bumpy ride (Neal et al., 2020). Much research tries to make the integration
more seamless. For instance, researchers have designed techniques for conversion
of regular DRAM addresses to persistent memory addresses (Lee et al., 2019).
Others have used persistent memory to convert existing distributed in-memory
storage systems into persistent, crash-consistent versions with low overhead and
minimal code changes (Zhang et al., 2020).
Many aspects of memory management have become a battleground for security researchers. For instance, reducing the memory footprint of a system by means
of memory deduplication turns out to be a highly security sensitive operation. Who
knew? For instance, attackers may detect that a page has been deduplicated and
thus learn what another process has in its address space (Bosman and Bos, 2016).
To eliminate the threat, deduplication must be designed such that one can no longer
distinguish between deduplicated and non-deduplicated pages (Oliverio, et al.,
2017).
Many attacks on the operating system depend on getting memory lined up in
the right way. For instance, attackers may be able to corrupt some important value
(e.g., the return address from a procedure call), but only if the corresponding object
is at a specific location. Such heap lay-out feng shui is complicated to perform
from a user process and researchers have looked for ways of automating this process (Chen and Xing, 2019).
Finally, there has been substantial work on operating systems due to Meltdown
and Spectre vulnerabilities in popular CPUs (see also Chapter 9). In particular, it
has led to radical and expensive changes in Linux on many processors. Where the
Linux kernel was originally mapped into the address space of every process as a
measure to speed up system calls (by obviating the need to change page tables for
a system call), Meltdown required strict page table isolation. As it made the context switch much more expensive, the Linux developers were furious with Intel.
The names initially proposed for the expensive fix were ‘‘User Address Space Separation’’ and ‘‘Forcefully Unmap Complete Kernel With Interrupt Trampolines,’’
but eventually they settled on kernel page table isolation (kpti), as the acronym is
considerably less offensive.
250
MEMORY MANAGEMENT
CHAP. 3
3.9 SUMMARY
In this chapter, we have examined memory management. We saw that the simplest systems do not swap or page at all. Once a program is loaded into memory, it
remains there in place until it finishes. Some operating systems allow only one
process at a time in memory, while others support multiprogramming. This model
is still common in small, embedded real-time systems.
The next step up is swapping. When swapping is used, the system can handle
more processes than it has room for in memory. Processes for which there is no
room are swapped out to the disk or SSD. Free space in memory and on nonvolatile storage can be kept track of with a bitmap or a hole list.
Modern computers often have some form of virtual memory. In the simplest
form, each process’ address space is divided up into uniform-sized blocks called
pages, which can be placed into any available page frame in memory. There are
many page replacement algorithms; two of the better algorithms are aging and
WSClock.
To make paging systems work well, choosing an algorithm is not enough;
attention to such issues as determining the working set, memory allocation policy,
and page size is required.
Segmentation helps in handling data structures that can change size during
execution and simplifies linking and sharing. It also facilitates providing different
protection for different segments. Sometimes segmentation and paging are combined to provide a two-dimensional virtual memory. The MULTICS system and the
32-bit Intel x86 support segmentation and paging. Still, it is clear that few operating system developers care deeply about segmentation (because they are married to
a different memory model).
PROBLEMS
1. In Fig. 3-3 the base and limit registers contain the same value, 16,384. Is this just an
accident, or are they always the same? If it is not an accident, why are they the same in
this example?
2. In this problem, you are to compare the storage needed to keep track of free memory
using a bitmap versus using a linked list. The 8-GB memory is allocated in units of n
bytes. For the linked list, assume that memory consists of an alternating sequence of
segments and holes, each 1 MB. Also assume that each node in the linked list needs a
32-bit memory address, a 16-bit length, and a 16-bit next-node field. How many bytes
of storage is required for each method? Which one is better?
CHAP. 3
PROBLEMS
251
3. Consider a swapping system in which memory consists of the following hole sizes in
memory order: 10 MB, 4 MB, 20 MB, 18 MB, 7 MB, 9 MB, 12 MB, and 15 MB.
Which hole is taken for successive segment requests of
(a) 12 MB
(b) 10 MB
(c) 9 MB
for first fit? Now repeat the question for best fit, worst fit, and next fit.
4. The first overlay managers and overlay sections were written by hand by programmers.
In principle, could this be done automatically by the compiler for a system with limited
memory? If so, how, and what difficulties would arise?
5. In what situations in modern computing might an overlay-style memory system be
effective, and why?
6. What is the difference between a physical address and a virtual address?
7. For each of the following decimal virtual addresses, compute the virtual page number
and offset for a 4-KB page and for an 8-KB page: 20000, 32768, 60000.
8. Using the page table of Fig. 3-9, give the physical address corresponding to each of the
following virtual addresses:
(a) 2000
(b) 8200
(c) 16536
9. What kind of hardware support is needed for a paged virtual memory to work?
10. Consider the following C program:
int X[N];
int step = M;
/* M is some predefined constant */
for (int i = 0; i < N; i += step) X[i] = X[i] + 1;
(a) If this program is run on a machine with a 4-KB page size and 64-entry TLB, what
values of M and N will cause a TLB miss for every execution of the inner loop?
(b) Would your answer in part (a) be different if the loop were repeated many times?
Explain.
11. The amount of disk space that must be available for page storage is related to the maximum number of processes, n, the number of bytes in the virtual address space, v, and
the number of bytes of RAM, r. Give an expression for the worst-case disk-space
requirements. How realistic is this amount?
12. If an instruction takes 2 nsec and a page fault takes an additional n nsec, give a formula
for the effective instruction time if page faults occur every k instructions.
13. Suppose that a machine has 48-bit virtual addresses and 32-bit physical addresses.
(a) If pages are 4 KB, how many entries are in the page table if it has only a single
level? Explain.
252
MEMORY MANAGEMENT
CHAP. 3
(b) Suppose this same system has a TLB (Translation Lookaside Buffer) with 32
entries. Furthermore, suppose that a program contains instructions that fit into one
page and it sequentially reads long integer elements from an array that spans thousands of pages. How effective will the TLB be for this case?
14. You are given the following data about a virtual memory system:
(a) The TLB can hold 1024 entries and can be accessed in 1 clock cycle (1 nsec).
(b) A page table entry can be found in 100 clock cycles or 100 nsec.
(c) The average page replacement time is 6 msec.
If page references are handled by the TLB 99% of the time, and only 0.01% lead to a
page fault, what is the effective address-translation time?
15. Some operating systems, Linux in particular, have a single virtual address space, with
some set of addresses designated for the kernel, and another set of addresses designated for user-space processes. The 64-bit Linux kernel supports a maximum of
4,194,304 processes in the process table, and the kernel is allocated half the virtual
address space. If memory address space is divided evenly across all processes, how
much virtual address space would be allocated to each process at a minimum, with the
maximum number of processes running?
16. The 32-bit Linux kernel supports a maximum of 32768 processes in the process table,
and the kernel is allocated 1,073,741,824 (1 GiB) of the virtual address space. If memory address space is divided evenly across all processes, how much virtual address
space would be allocated to each process at a minimum, with the maximum number of
processes running?
17. Section 3.3.4 states that the Pentium Pro extended each entry in the page table hierarchy to 64 bits but still could only address only 4 GB of memory. Explain how this
statement can be true when page table entries have 64 bits.
18. A computer with a 32-bit address uses a two-level page table. Virtual addresses are
split into a 9-bit top-level page table field, an 11-bit second-level page table field, and
an offset. How large are the pages and how many are there in the address space?
19. Suppose that a 32-bit virtual address is broken up into four fields, a, b, c, and d. The
first three are used for a three-level page table system. The fourth field, d, is the offset.
Does the number of pages depend on the sizes of all four fields? If not, which ones
matter and which do not?
20. A computer has 32-bit virtual addresses and 4-KB pages. The program and data together fit in the lowest page (0–4095) The stack fits in the highest page. How many
entries are needed in the page table if traditional (one-level) paging is used? How
many page table entries are needed for two-level paging, with 10 bits in each part?
21. Below is an execution trace of a program fragment for a computer with 512-byte
pages. The program is located at address 1020, and its stack pointer is at 8192 (the
stack grows toward 0). Give the page reference string generated by this program. Each
instruction occupies 4 bytes (1 word) including immediate constants. Both instruction
and data references count in the reference string.
CHAP. 3
PROBLEMS
253
Load word 6144 into register 0
Push register 0 onto the stack
Call a procedure at 5120, stacking the return address
Subtract the immediate constant 16 from the stack pointer
Compare the actual parameter to the immediate constant 4
Jump if equal to 5152
22. A computer whose processes have 1024 pages in their address spaces keeps its page
tables in memory. The overhead required for reading a word from the page table is 5
nsec. To reduce this overhead, the computer has a TLB, which holds 32 (virtual page,
physical page frame) pairs, and can do a lookup in 1 nsec. What hit rate is needed to
reduce the mean overhead to 2 nsec?
23. The VAX was the dominant computer at university computer science departments during most of the 1980s. The TLB on the VAX did not contain an R bit. Nevertheless,
these supposedly intelligent people kept buying VAXes. Was this just due to their loyalty to the VAX’ predecessor, the PDP-11, or was there some other reason they put up
with this for years?
24. A machine has 48-bit virtual addresses and 32-bit physical addresses. Pages are 8 KB.
How many entries are needed for a single-level linear page table?
25. A computer with an 8-KB page, a 256-KB main memory, and a 64-GB virtual address
space uses an inverted page table to implement its virtual memory. How big should the
hash table be to ensure a mean hash chain length of less than 1? Assume that the hashtable size is a power of two.
26. A student in a compiler design course proposes to the professor a project of writing a
compiler that will produce a list of page references that can be used to implement the
optimal page replacement algorithm. Is this possible? Why or why not? Is there anything that could be done to improve paging efficiency at run time?
27. Suppose that the virtual page reference stream contains repetitions of long sequences
of page references followed occasionally by a random page reference. For example, the
sequence: 0, 1, ... , 511, 431, 0, 1, ... , 511, 332, 0, 1, ... consists of repetitions of the
sequence 0, 1, ... , 511 followed by a random reference to pages 431 and 332.
(a) Why will the standard replacement algorithms (LRU, FIFO, clock) not be effective
in handling this workload for a page allocation that is less than the sequence
length?
(b) If this program were allocated 500 page frames, describe a page replacement
approach that would perform much better than the LRU, FIFO, or clock algorithms.
28. If FIFO page replacement is used with four page frames and eight pages, how many
page faults will occur with the reference string 0172327103 if the four frames are initially empty? Now repeat this problem for LRU.
29. Consider the page sequence of Fig. 3-15(b). Suppose that the R bits for the pages B
through A are 11011011, respectively. Which page will second chance remove?
30. A small computer on a smart card has four page frames. At the first clock tick, the R
bits are 0111 (page 0 is 0, the rest are 1). At subsequent clock ticks, the values are
254
MEMORY MANAGEMENT
CHAP. 3
1011, 1010, 1101, 0010, 1010, 1100, and 0001. If the aging algorithm is used with an
8-bit counter, give the values of the four counters after the last tick.
31. Give a simple example of a page reference sequence where the first page selected for
replacement will be different for the clock and LRU page replacement algorithms.
Assume that a process is allocated 3=three frames, and the reference string contains
page numbers from the set 0, 1, 2, 3.
32. A student has claimed that ‘‘in the abstract, the basic page replacement algorithms
(FIFO, LRU, optimal) are identical except for the attribute used for selecting the page
to be replaced.’’
(a) What is that attribute for the FIFO algorithm? LRU algorithm? Optimal algorithm?
(b) Give the generic algorithm for these page replacement algorithms.
33. How long does it take to load a 64-KB program from a disk whose average seek time is
5 msec, whose rotation time is 5 msec, and whose tracks hold 1 MB
(a) for a 2-KB page size?
(b) for a 4-KB page size?
The pages are spread randomly around the disk and the number of cylinders is so large
that the chance of two pages being on the same cylinder is negligible.
34. Consider the FIFO page replacement algorithm and the following reference string:
123412512345
When the number of page frames increases from three to four, does the number of page
faults go down, stay the same, or go up? Explain your answer.
35. A computer has four page frames. The time of loading, time of last access, and the R
and M bits for each page are as shown below (the times are in clock ticks):
Page
Loaded
Last ref.
R
M
0
126
280
1
0
1
230
265
0
1
2
140
270
0
0
3
110
285
1
1
(a) Which page will NRU replace?
(b) Which page will FIFO replace?
(c) Which page will LRU replace?
(d) Which page will second chance replace?
36. Suppose that two processes A and B share a page that is not in memory. If process A
faults on the shared page, the page table entry for process A must be updated once the
page is read into memory.
(a) Under what conditions should the page table update for process B be delayed even
though the handling of process A’s page fault will bring the shared page into memory? Explain.
(b) What is the potential cost of delaying the page table update?
CHAP. 3
PROBLEMS
255
37. Consider the following two-dimensional array:
int
X[64][64];
Suppose that a system has four page frames and each frame is 128 words (an integer
occupies one word). Programs that manipulate the X array fit into exactly one page
and always occupy page 0. The data are swapped in and out of the other three frames.
The X array is stored in row-major order (i.e., X[0][1] follows X[0][0] in memory).
Which of the two code fragments shown below will generate the lowest number of
page faults? Explain and compute the total number of page faults.
Fragment A
for (int j = 0; j < 64; j++)
for (int i = 0; i < 64; i++) X[i][j] = 0;
Fragment B
for (int i = 0; i < 64; i++)
for (int j = 0; j < 64; j++) X[i][j] = 0;
38. One of the first timesharing machines, the DEC PDP-1, had a (core) memory of 4K
18-bit words. It held one process at a time in its memory. When the scheduler decided
to run another process, the process in memory was written to a paging drum, with 4K
18-bit words around the circumference of the drum. The drum could start writing (or
reading) at any word, rather than only at word 0. Why do you suppose this drum was
chosen?
39. A computer provides each process with 65,536 bytes of address space divided into
pages of 4096 bytes each. A particular program has a text size of 32,768 bytes, a data
size of 16,386 bytes, and a stack size of 15,870 bytes. Will this program fit in the
machine’s address space? Suppose that instead of 4096 bytes, the page size were 512
bytes, would it then fit? Each page must contain either text, data, or stack, not a mixture of two or three of them.
40. Can a page be in two working sets at the same time? Explain.
41. If a page is shared between two processes, is it possible that the page is read-only for
one process and read-write for the other? Why or why not?
42. It has been observed that the number of instructions executed between page faults is
directly proportional to the number of page frames allocated to a program. If the available memory is doubled, the mean interval between page faults is also doubled. Suppose that a normal instruction takes 1 microsec, but if a page fault occurs, it takes 2001
µ sec (i.e., 2 msec) to handle the fault. If a program takes 60 sec to run, during which
time it gets 15,000 page faults, how long would it take to run if twice as much memory
were available?
43. A group of operating system designers for the Frugal Computer Company are thinking
about ways to reduce the amount of backing store needed in their new operating system. The project manager has just suggested not bothering to save the program text in
the swap area at all, but just page it in directly from the binary file whenever it is needed. Under what conditions, if any, does this idea work for the program text? Under
what conditions, if any, does it work for the data?
256
MEMORY MANAGEMENT
CHAP. 3
44. A machine-language instruction to load a 32-bit word into a register contains the 32-bit
address of the word to be loaded. What is the maximum number of page faults this
instruction can cause?
45. Explain the difference between internal fragmentation and external fragmentation.
Which one occurs in paging systems? Which one occurs in systems using pure segmentation?
46. When segmentation and paging are both being used, as in MULTICS, first the segment
descriptor must be looked up, then the page descriptor. Does the TLB also work this
way, with two levels of lookup?
47. We consider a program which has the two segments shown below consisting of instructions in segment 0, and read/write data in segment 1. Segment 0 has read/execute protection, and segment 1 has just read/write protection. The memory system is a demandpaged virtual memory system with virtual addresses that have a 4-bit page number, and
a 10-bit offset. The page tables and protection are as follows (all numbers in the table
are in decimal):
Segment 0
Segment 1
Read/Execute
Read/Write
Virtual Page #
Page frame #
Virtual Page #
Page frame #
0
2
0
On Disk
1
On Disk
1
14
2
11
2
9
3
5
3
6
4
On Disk
4
On Disk
5
On Disk
5
13
6
4
6
8
7
3
7
12
For each of the following cases, either give the real (actual) memory address which
results from dynamic address translation or identify the type of fault which occurs
(either page or protection fault).
(a) Fetch from segment 1, page 1, offset 3
(b) Store into segment 0, page 0, offset 16
(c) Fetch from segment 1, page 4, offset 28
(d) Jump to location in segment 1, page 3, offset 32
48. Can you think of any situations where supporting virtual memory would be a bad idea,
and what would be gained by not having to support virtual memory? Explain.
49. Virtual memory provides a mechanism for isolating one process from another. What
memory management difficulties would be involved in allowing two operating systems
to run concurrently? How might these difficulties be addressed?
50. Plot a histogram and calculate the mean and median of the sizes of executable binary
files on a computer to which you have access. On a Windows system, look at all .exe
and .dll files; on a UNIX system look at all executable files in /bin, /usr/bin, and
CHAP. 3
PROBLEMS
257
/local/bin that are not scripts (or use the file utility to find all executables). Determine
the optimal page size for this computer just considering the code (not data). Consider
internal fragmentation and page table size, making some reasonable assumption about
the size of a page table entry. Assume that all programs are equally likely to be run and
thus should be weighted equally.
51. Write a program that simulates a paging system using the aging algorithm. The number
of page frames is a parameter. The sequence of page references should be read from a
file. For a given input file, plot the number of page faults per 1000 memory references
as a function of the number of page frames available.
52. Write a program that simulates a toy paging system that uses the WSClock algorithm.
The system is a toy in that we will assume there are no write references (not very realistic), and process termination and creation are ignored (eternal life). The inputs are:
• The reclamation age threshold
• The clock interrupt interval expressed as number of memory references
• A file containing the sequence of page references
(a) Describe the basic data structures and algorithms in your implementation.
(b) Show that your simulation behaves as expected for a simple (but nontrivial) input
example.
(c) Plot the number of page faults and working set size per 1000 memory references.
(d) Explain what is needed to extend the program to handle a page reference stream
that also includes writes.
53. Write a program that demonstrates the effect of TLB misses on the effective memory
access time by measuring the per-access time it takes to stride through a large array.
(a) Explain the main concepts behind the program, and describe what you expect the
output to show for some practical virtual memory architecture.
(b) Run the program on some computer and explain how well the data fit your expectations.
(c) Repeat part (b) but for an older computer with a different architecture and explain
any major differences in the output.
54. Write a program that will demonstrate the difference between using a local page
replacement policy and a global one for the simple case of two processes. You will
need a routine that can generate a page reference string based on a statistical model.
This model has N states numbered from 0 to N < 1 representing each of the possible
page references and a probability pi associated with each state i representing the
chance that the next reference is to the same page. Otherwise, the next page reference
will be one of the other pages with equal probability.
(a) Demonstrate that the page reference string-generation routine behaves properly for
some small N .
(b) Compute the page fault rate for a small example in which there is one process and a
fixed number of page frames. Explain why the behavior is correct.
(c) Repeat part (b) with two processes with independent page reference sequences and
twice as many page frames as in part (b).
(d) Repeat part (c) but using a global policy instead of a local one. Also, contrast the
per-process page fault rate with that of the local policy approach.
258
MEMORY MANAGEMENT
CHAP. 3
55. Write a program that can be used to compare the effectiveness of adding a tag field to
TLB entries when control is toggled between two programs. The tag field is used to
effectively label each entry with the process id. Note that a nontagged TLB can be simulated by requiring that all TLB entries have the same tag at any one time. The inputs
will be:
• The number of TLB entries available
• The clock interrupt interval expressed as number of memory references
• A file containing a sequence of (process, page references) entries
• The cost to update one TLB entry
(a) Describe the basic data structures and algorithms in your implementation.
b) Show that your simulation behaves as expected for a simple (but nontrivial) input
example.
(c) Plot the number of TLB updates per 1000 references.
4
FILE SYSTEMS
All computer applications need to store and retrieve information. While a process is running, it can store a limited amount of information in physical RAM. For
many applications, the amount of memory is far too small and some even need
many terabytes of storage.
A second problem with keeping information in RAM is that when the process
terminates, the information is lost. For many applications (e.g., for databases), the
information must be retained for weeks, months, or even forever. Having it vanish
when the process using it terminates is unacceptable. Furthermore, it must not go
away when a computer crash kills the process or power goes off during an electrical storm.
A third problem is that it is frequently necessary for multiple processes to
access (parts of) the information at the same time. If we have an online telephone
directory stored inside the address space of a single process, only that process can
access it, unless it is shared explicitly. The way to solve this problem is to make the
information itself independent of any one process.
Thus, we have three essential requirements for long-term information storage:
1. It must be possible to store a very large amount of information.
2. The information must survive the termination of the process using it.
3. Multiple processes must be able to access the information at once.
Magnetic disks have been used for years for this long-term storage. While such
disks are still used extensively, solid-state drives (SSDs) have also become hugely
259
260
FILE SYSTEMS
CHAP. 4
popular, complementing or replacing their magnetic counterparts. Compared to
hard disks, they do not have any moving parts that may break, and offer fast random access. Tapes and optical disks are no longer as popular as they used to be and
have much lower performance. Nowadays, if they are used at all, it is typically for
backups. We will study magnetic hard disks and SSDs more in Chap. 5. For the
moment, you can think of both as disk-like, even though strictly speaking an SSD
is not a disk at all. Here, ‘‘disk-like’’ means that it supports an interface that appears to be a linear sequence of fixed-size blocks and supporting two operations:
1. Read block k
2. Write block k
In reality there are more, but with these two operations one could, in principle,
solve the long-term storage problem.
However, these are very inconvenient operations, especially on large systems
used by many applications and possibly multiple users (e.g., on a server). Just a
few of the questions that quickly arise are:
1. How do you find information?
2. How do you keep one user from reading another user’s data?
3. How do you know which blocks are free?
and there are many more.
Just as we saw how the operating system abstracted away the concept of the
processor to create the abstraction of a process and how it abstracted away the concept of physical memory to offer processes (virtual) address spaces, we can solve
this problem with a new abstraction: the file. Together, the abstractions of processes (and threads), address spaces, and files are the most important concepts relating to operating systems. If you really understand these three concepts from beginning to end, you are well on your way to becoming an operating systems expert.
Files are logical units of information created by processes. A disk will usually
contain thousands or even millions of them, each one independent of the others. In
fact, if you think of each file as a kind of address space, you are not that far off,
except that they are used to model the disk instead of modeling the RAM.
Processes can read existing files and create new ones if need be. Information
stored in files must be persistent, that is, not be affected by process creation and
termination. A file should disappear only when its owner explicitly removes it.
Although operations for reading and writing files are the most common ones, there
exist many others, some of which we will examine below.
Files are managed by the operating system. How they are structured, named,
accessed, used, protected, implemented, and managed are major topics in operating
system design. As a whole, that part of the operating system dealing with files is
known as the file system and is the subject of this chapter.
261
From the user’s standpoint, the most important aspect of a file system is how it
appears, in other words, what constitutes a file, how files are named and protected,
what operations are allowed on files, and so on. The details of whether linked lists
or bitmaps are used to keep track of free storage and how many sectors there are in
a logical disk block are of no interest, although they are of great importance to the
designers of the file system. For this reason, we have structured the chapter as several sections. The first two are concerned with the user interface to files and directories, respectively. Then comes a detailed discussion of how the file system is
implemented and managed. Finally, we give some examples of real file systems.
4.1 FILES
In the following pages, we will look at files from the user’s point of view, that
is, how they are used and what properties they have.
4.1.1 File Naming
A file is an abstraction mechanism. It provides a way to store information on
the disk and read it back later. This must be done in a way that shields the user
from the details of how and where the information is stored, and how the disks
actually work.
Probably the most important characteristic of any abstraction mechanism is the
way the objects being managed are named, so we will start our examination of file
systems with the subject of file naming. When a process creates a file, it gives the
file a name. When the process terminates, the file continues to exist and can be
accessed by other processes using its name.
The exact rules for file naming vary somewhat from system to system, but all
current operating systems allow strings of letters as legal file names. Thus andrea,
bruce, and cathy are possible file names. Frequently digits and special characters
are also permitted, so names like 2, urgent!, and Fig.2-14 are often valid as well.
Some older file systems, such as the one that was used in MS-DOS in a century
long ago, limit file names to eight letters maximum, but most modern systems support file names of up to 255 characters or more.
Some file systems distinguish between upper- and lowercase letters, whereas
others do not. UNIX falls in the first category; the old MS-DOS falls in the second. Thus, a UNIX system can have all of the following as three distinct files:
maria, Maria, and MARIA. In MS-DOS, all these names refer to the same file.
An aside on file systems is probably in order here. Older versions of Windows
(such as Windows 95 and Windows 98) used the MS-DOS file system, called
FAT-16, and thus inherited many of its properties, such as how file names are constructed. Admittedly, Windows 98 introduced some extensions to FAT-16, leading
to FAT-32, but these two are quite similar. Modern versions of Windows all still
support the FAT file systems, even though they also have a much more advanced
262
FILE SYSTEMS
CHAP. 4
native file system (NTFS) that has different properties (such as file names in Unicode). We will discuss NTFS in Chap. 11. There is also a second file system for
Windows, known as ReFS (Resilient File System), but that one is targeted at the
server version of Windows. In this chapter, when we refer to the MS-DOS or FAT
file systems, we mean FAT-16 and FAT-32 as used on Windows unless specified
otherwise. We will discuss the FAT file systems later in this chapter and NTFS in
Chap. 12, where we will examine Windows 10 in detail. Incidentally, there is also
an even newer FAT-like file system, known as exFAT file system, a Microsoft
extension to FAT-32 that is optimized for flash drives and large file systems.
Many operating systems support two-part file names, with the two parts separated by a period, as in prog.c. The part following the period is called the file
extension and usually indicates something about the file. In MS-DOS, for example, file names were 1–8 characters, plus an optional extension of 1–3 characters.
In UNIX, the size of the extension, if any, is up to the user, and a file may even
have two or more extensions, as in homepage.html.zip, where .html indicates a Web
page in HTML and .zip indicates that the file (homepage.html) has been compressed using the zip program. Some of the more common file extensions and their
meanings are shown in Fig. 4-1.
Extension
Meaning
.bak
Backup file
.c
C source program
.gif
Compuserve Graphical Interchange Format image
.html
World Wide Web HyperText Markup Language document
.jpg
Still picture encoded with the JPEG standard
.mp3
Music encoded in MPEG layer 3 audio format
.mpg
Movie encoded with the MPEG standard
.o
Object file (compiler output, not yet linked)
.pdf
Por table Document Format file
.ps
PostScript file
.tex
Input for the TEX formatting program
.txt
General text file
.zip
Compressed archive
Figure 4-1. Some typical file extensions.
In some systems (e.g., all flavors of UNIX), file extensions are just conventions and are not enforced by the operating system. A file named file.txt might be
some kind of text file, but that name is more to remind the owner than to convey
any actual information to the computer. On the other hand, a C compiler may actually insist that files it is to compile end in .c, and it may refuse to compile them if
they do not. However, the operating system does not care.
SEC. 4.1
263
FILES
Conventions like this are especially useful when the same program can handle
several different kinds of files. The C compiler, for example, can be given a list of
several files to compile and link together, some of them C files and some of them
assembly-language files. The extension then becomes essential for the compiler to
tell which are C files, which are assembly files, and which are other files.
In contrast, Windows is aware of the extensions and assigns meaning to them.
Users (or processes) can register extensions with the operating system and specify
for each one which program ‘‘owns’’ that extension. When a user double clicks on
a file name, the program assigned to its file extension is launched with the file as
parameter. For example, double clicking on file.docx starts Microsoft Word with
file.docx as the initial file to edit. In contrast, Photoshop will not open file ending
in .docx, no matter how often or hard you click on the file name, because it knows
that .docx files are not image files.
4.1.2 File Structure
Files can be structured in any of several ways. Three common possibilities are
depicted in Fig. 4-2. The file in Fig. 4-2(a) is an unstructured sequence of bytes.
In effect, the operating system does not know or care what is in the file. All it sees
are bytes. Any meaning must be imposed by user-level programs. Both UNIX and
Windows use this approach.
1 Byte
1 Record
Cat
Cow
Dog
Hen
(a)
(b)
Ibis
Ant
Fox
Pig
Goat
Lion
Owl
Pony
Rat
Worm
Lamb
(c)
Figure 4-2. Three kinds of files. (a) Byte sequence. (b) Record sequence.
(c) Tree.
Having the operating system regard files as nothing more than byte sequences
provides the maximum amount of flexibility. User programs can put anything they
want in their files and name them any way that they find convenient. The operating
system does not help, but it also does not get in the way. For users who want to do
264
FILE SYSTEMS
CHAP. 4
unusual things, the latter can be very important. All versions of UNIX (including
Linux and MacOS) as well as Windows use this file model. It is worth noting that
in this chapter when we talk about UNIX the text usually applies MacOS (which
was based on Berkeley UNIX) and Linux (which was carefully designed to be
compatible with UNIX).
The first step up in structure is illustrated in Fig. 4-2(b). In this model, a file is
a sequence of fixed-length records, each with some internal structure. Central to
the idea of a file being a sequence of records is the idea that the read operation
returns one record and the write operation overwrites or appends one record. As a
historical note, in decades gone by, when the 80-column punched card was pretty
much the only input medium available, many (mainframe) operating systems based
their file systems on files consisting of 80-character records, in effect, card images.
These systems also supported files of 132-character records, which were intended
for the line printer (which in those days were big chain printers having 132 columns). Programs read input in units of 80 characters and wrote it in units of 132
characters, although the final 52 could be spaces, of course. No current generalpurpose system uses this model as its primary file system any more, but back in the
days of 80-column punched cards and 132-character line printer paper, this was a
common model on mainframe computers.
The third kind of file structure is shown in Fig. 4-2(c). In this organization, a
file consists of a tree of records, not necessarily all the same length, each containing a key field in a fixed position in the record. The tree is sorted on the key
field, to allow rapid searching for a particular key.
The basic operation here is not to get the ‘‘next’’ record, although that is also
possible, but to get the record with a specific key. For the zoo file of Fig. 4-2(c),
one could ask the system to get the record whose key is pony, for example, without
worrying about its exact position in the file. Furthermore, new records can be
added to the file, with the operating system, and not the user, deciding where to
place them. This type of file is clearly quite different from the unstructured byte
streams used in UNIX and Windows and is used on some large mainframe computers for commercial data processing.
4.1.3 File Types
Many operating systems support several types of files. UNIX (again, including
MacOS and Linux) and Windows, for example, have regular files and directories.
UNIX also has character and block special files. Regular files are the ones that
contain user information. All the files of Fig. 4-2 are regular files since these are
the files most users deal with. Directories are system files for maintaining the
structure of the file system. We will study directories below. Character special
files are related to input/output and used to model serial I/O devices, such as terminals, printers, and networks. Block special files are used to model disks. In this
chapter, we will be primarily interested in regular files.
SEC. 4.1
FILES
265
Regular files are generally either ASCII files or binary files. ASCII files consist of lines of text. In some systems, each line is terminated by a carriage return
character. In others, the line feed character is used. Some systems (e.g., Windows)
use both. Lines need not all be of the same length.
The great advantage of ASCII files is that they can be displayed and printed as
is, and they can be edited with any text editor. Furthermore, if large numbers of
programs use ASCII files for input and output, it is easy to connect the output of
one program to the input of another, as in shell pipelines. (The interprocess
plumbing is not any easier, but interpreting the information certainly is if a standard convention, such as ASCII, is used for expressing it.)
Other files are binary, which just means that they are not ASCII files. Listing
them on the printer gives an incomprehensible listing full of random junk. Usually,
they have some internal structure known to programs that use them.
For example, in Fig. 4-3(a) we see a simple executable binary file taken from
an early version of UNIX. Although technically the file is just a sequence of bytes,
the operating system will execute a file only if it has the proper format. It has five
sections: header, text, data, relocation bits, and symbol table. The header starts
with a magic number, identifying the file as an executable file (to prevent the accidental execution of a file not in this format). Then come the sizes of the various
pieces of the file, the address at which execution starts, and some flag bits. After
the header are the text and data of the program itself. These are loaded into memory and relocated using the relocation bits. The symbol table is for debugging.
Our second example of a binary file is an archive, also from UNIX. It consists
of a collection of library procedures (modules) compiled but not linked. Each one
is prefaced by a header telling its name, creation date, owner, protection code, and
size. Just as with the executable file, the module headers are full of binary numbers. Copying them to the printer would produce complete gibberish.
Every operating system must recognize at least one file type: its own executable file; some recognize more. The old TOPS-20 system (for the DECsystem
20) went so far as to examine the creation time of any file to be executed. Then it
located the source file and saw whether the source had been modified since the
binary was made. If it had been, it automatically recompiled the source. In UNIX
terms, the make program had been built into the shell. The file extensions were
mandatory, so it could tell which binary program was derived from which source.
Having strongly typed files like this causes problems whenever the user does
anything that the system designers did not expect. Consider, as an example, a system in which program output files have extension .dat (data files). If a user writes
a program formatter that reads a .c file (C program), transforms it (e.g., by converting it to a standard indentation layout), and then writes the transformed file as output, the output file will be of type .dat. If the user tries to offer this to the C compiler to compile it, the system will refuse because it has the wrong extension.
Attempts to copy file.dat to file.c will be rejected by the system as invalid (to protect the user against mistakes).
266
FILE SYSTEMS
Magic number
CHAP. 4
Module
name
Header
Text size
Header
Data size
Date
BSS size
Symbol table size
Object
module
Owner
Protection
Entry point
Size
Flags
Header
Text
Object
module
Data
Header
Relocation
bits
Symbol
table
(a)
Object
module
(b)
Figure 4-3. (a) An executable file. (b) An archive.
While this kind of ‘‘user friendliness’’ may help novices, it drives experienced
users up the wall since they have to devote considerable effort to circumventing the
operating system’s idea of what is reasonable and what is not.
Most operating systems offer a slew of tools to examine files. For instance, on
UNIX you can use the file utility to examine the type of files. It uses heuristics to
determine that something is a text file, a directory, an executable, etc. Examples of
its use can be found in Fig. 4-4.
4.1.4 File Access
Early operating systems provided only one kind of file access: sequential
access. In these systems, a process could read all the bytes or records in a file in
order, starting at the beginning, but could not skip around and read them out of
SEC. 4.1
267
FILES
Command
Result
file README.txt
UTF-8 Unicode text
file hjb.sh
POSIX shell script, ASCII text executable
file Makefile
makefile script, ASCII text
file /usr/bin/less
symbolic link to /bin/less
file /bin/
director y
file /bin/less
ELF 64-bit LSB shared object, x86-64 [...more information...]
Figure 4-4. Finding out file types.
order. Sequential files could be rewound, however, so they could be read as often
as needed. Sequential files were convenient when the storage medium was magnetic tape rather than disk.
When disks came into use for storing files, it became possible to read the bytes
or records of a file out of order, or to access records by key rather than by position.
Files whose bytes or records can be read in any order are called random-access
files. They are required by many applications.
Random access files are essential for many applications, for example, database
systems. If an airline customer calls up and wants to reserve a seat on a particular
flight, the reservation program must be able to access the record for that flight
without having to read the records for thousands of other flights first.
Two methods can be used for specifying where to start reading. In the first
one, every read operation gives the position in the file to start reading at. In the
second one, a special operation, seek, is provided to set the current position. After
a seek, the file can be read sequentially from the now-current position. The latter
method is used in UNIX and Windows.
4.1.5 File Attributes
Every file has a name and its data. In addition, all operating systems associate
other information with each file, for example, the date and time the file was last
modified and the file’s size. We will call these extra items the file’s attributes.
Some people call them metadata. The list of attributes varies considerably from
system to system. The table of Fig. 4-5 shows some of the possibilities, but other
ones also exist. No existing system has all of these, but each one is present in
some system.
The first four attributes relate to the file’s protection and tell who may access it
and who may not. All kinds of schemes are possible, some of which we will study
later. In some systems the user must present a password to access a file, in which
case the password must be one of the attributes.
The flags are bits or short fields that control or enable some specific property.
Hidden files, for example, do not appear in listings of all the files. The archive flag
268
FILE SYSTEMS
Attribute
CHAP. 4
Meaning
Protection
Who can access the file and in what way
Password
Password needed to access the file
Creator
ID of the person who created the file
Owner
Current owner
Read-only flag
0 for read/write; 1 for read only
Hidden flag
0 for normal; 1 for do not display in listings
System flag
0 for normal files; 1 for system file
Archive flag
0 for has been backed up; 1 for needs to be backed up
ASCII/binar y flag
0 for ASCII file; 1 for binary file
Random access flag
0 for sequential access only; 1 for random access
Temporar y flag
0 for normal; 1 for delete file on process exit
Lock flags
0 for unlocked; nonzero for locked
Record length
Number of bytes in a record
Key position
Offset of the key within each record
Key length
Number of bytes in the key field
Creation time
Date and time the file was created
Time of last access
Date and time the file was last accessed
Time of last change
Date and time the file was last changed
Current size
Number of bytes in the file
Maximum size
Number of bytes the file may grow to
Figure 4-5. Some possible file attributes.
is a bit that keeps track of whether the file has been backed up recently. The backup program clears it, and the operating system sets it whenever a file is changed.
In this way, the backup program can tell which files need backing up. The temporary flag allows a file to be marked for automatic deletion when the process that
created it terminates.
The record-length, key-position, and key-length fields are only present in files
whose records can be looked up using a key. They provide the information required
to find the keys.
The times keep track of when the file was created, most recently accessed, and
most recently modified. These are useful for a variety of purposes. For example, a
source file that has been modified after the creation of the corresponding object file
needs to be recompiled. These fields provide the necessary information.
The current size tells how big the file is at present. Some old mainframe operating systems required the maximum size to be specified when the file was created,
in order to let the operating system reserve the maximum amount of storage in
advance. Personal-computer operating systems are thankfully clever enough to do
without this feature nowadays.
SEC. 4.1
FILES
269
4.1.6 File Operations
Files exist to store information and allow it to be retrieved later. Different systems provide different operations to allow storage and retrieval. Below is a discussion of the most common system calls relating to files.
1. Create. The file is created with no data. The purpose of the call is to
announce that the file is coming and to set some of the attributes.
2. Delete. When the file is no longer needed, it has to be deleted to free
up disk space. There is always a system call for this purpose.
3. Open. Before using a file, a process must open it. The purpose of the
open call is to allow the system to fetch the attributes and list of disk
addresses into main memory for rapid access on later calls.
4. Close. When all the accesses are finished, the attributes and disk
addresses are no longer needed, so the file should be closed to free up
internal table space. Many systems encourage this by imposing a
maximum number of open files on processes. A disk is written in
blocks, and closing a file forces writing of the file’s last block, even
though that block may not be entirely full yet.
5. Read. Data are read from file. Usually, the bytes come from the current position. The caller must specify how many data are needed and
must also provide a buffer to put them in.
6. Write. Data are written to the file again, usually at the current position. If the current position is the end of the file, the file’s size
increases. If the current position is in the middle of the file, existing
data are overwritten and lost forever.
7. Append. This call is a restricted form of write. It can add data only to
the end of the file. Systems that provide a minimal set of system calls
rarely have append, but some systems have this call.
8. Seek. For random-access files, a method is needed to specify from
where to take the data. One common approach is a system call, seek,
that repositions the file pointer to a specific place in the file. After this
call has completed, data can be read from, or written to, that position.
9. Get attributes. Processes often need to read file attributes to do their
work. For example, the UNIX make program is commonly used to
manage software development projects consisting of many source
files. When make is called, it examines the modification times of all
the source and object files and arranges for the minimum number of
compilations required to bring everything up to date. To do its job, it
must look at the attributes, namely, the modification times.
270
FILE SYSTEMS
CHAP. 4
10. Set attributes. Some of the attributes are user settable and can be
changed after the file has been created. This system call makes that
possible. The protection-mode information is an obvious example.
Most of the flags also fall in this category.
11. Rename. This call is not essential because a file that needs to be
renamed can be copied and then the original file deleted. However,
renaming a 50-GB movie by copying it and then deleting the original
will take a long time.
4.1.7 An Example Program Using File-System Calls
In this section, we will examine a simple UNIX program that copies one file
from its source file to a destination file. It is listed in Fig. 4-6. The program has
minimal functionality and even worse error reporting, but it gives a reasonable idea
of how some of the system calls related to files work.
The program, copyfile, can be called, for example, by the command line
copyfile abc xyz
to copy the file abc to xyz. If xyz already exists, it will be overwritten. Otherwise,
it will be created. The program must be called with exactly two arguments, both
legal file names. The first is the source; the second is the output file.
The four #include statements near the top of the program cause a large number
of definitions and function prototypes to be included in the program. These are
needed to make the program conformant to the relevant international standards, but
will not concern us further. The next line is a function prototype for main, something required by ANSI C, but also not important for our purposes.
The first #define statement is a macro definition that defines the character
string BUF SIZE as a macro that expands into the number 4096. The program
will read and write in chunks of 4096 bytes. It is considered good programming
practice to give names to constants like this. The second #define statement determines who can access the output file.
The main program is called main, and it has two arguments, argc and argv.
These are supplied by the operating system when the program is called. The first
one tells how many strings were present on the command line that invoked the program, including the program name. It should be 3. The second one is an array of
pointers to the arguments. In the example call given above, the elements of this
array would contain pointers to the following values:
argv[0] = "copyfile"
argv[1] = "abc"
argv[2] = "xyz"
It is via this array that the program accesses its arguments.
SEC. 4.1
FILES
271
/* File copy program. Error checking and reporting is minimal. */
#include <sys/types.h>
#include <fcntl.h>
#include <stdlib.h>
#include <unistd.h>
/* include necessary header files */
int main(int argc, char *argv[]);
/* ANSI prototype */
#define BUF SIZE 4096
#define OUTPUT MODE 0700
/* use a buffer size of 4096 bytes */
/* protection bits for output file */
int main(int argc, char *argv[])
{
int in fd, out fd, rd count, wt count;
char buffer[BUF SIZE];
if (argc != 3) exit(1);
/* syntax error if argc is not 3 */
/* Open the input file and create the output file */
/* open the source file */
in fd = open(argv[1], O RDONLY);
/* if it cannot be opened, exit */
if (in fd < 0) exit(2);
out fd = creat(argv[2], OUTPUT MODE); /* create the destination file */
/* if it cannot be created, exit */
if (out fd < 0) exit(3);
/* Copy loop */
while (TRUE) {
rd count = read(in fd, buffer, BUF SIZE); /* read a block of data */
/* if end of file or error, exit loop */
if (rd count <= 0) break;
wt count = write(out fd, buffer, rd count); /* write data */
/* wt count <= 0 is an error */
if (wt count <= 0) exit(4);
}
/* Close the files */
close(in fd);
close(out fd);
if (rd count == 0)
exit(0);
else
exit(5);
}
/* no error on last read */
/* error on last read */
Figure 4-6. A simple program to copy a file.
SP 1v
Five variables are declared. The first two, in fd and out fd, will hold the file
descriptors, small integers returned when a file is opened. The next two, rd count
and wt count, are the byte counts returned by the read and write system calls,
respectively. The last one, buffer, is the buffer used to hold the data read and supply
the data to be written.
272
FILE SYSTEMS
CHAP. 4
The first actual statement checks argc to see if it is 3. If not, it exits with status
code 1. Any status code other than 0 means that an error has occurred. The status
code is the only error reporting present in this program. A production version
would normally print error messages as well.
Then we try to open the source file and create the destination file. If the source
file is successfully opened, the system assigns a small integer to in fd, to identify
the file. Subsequent calls must include this integer so that the system knows which
file it wants. Similarly, if the destination is successfully created, out fd is given a
value to identify it. The second argument to creat sets the protection mode. If
either the open or the create fails, the corresponding file descriptor is set to <1, and
the program exits with an error code.
Now comes the copy loop. It starts by trying to read in 4 KB of data to buffer.
It does this by calling the library procedure read, which actually invokes the read
system call. The first parameter identifies the file, the second gives the buffer, and
the third tells how many bytes to read. The value assigned to rd count gives the
number of bytes actually read. Normally, this will be 4096, except if fewer bytes
are remaining in the file. When the end of the file has been reached, it will be 0. If
rd count is ever zero or negative, the copying cannot continue, so the break statement is executed to terminate the (otherwise endless) loop.
The call to write outputs the buffer to the destination file. The first parameter
identifies the file, the second gives the buffer, and the third tells how many bytes to
write, analogous to read. Note that the byte count is the number of bytes actually
read, not BUF SIZE. This point is important because the last read will not return
4096 unless the file just happens to be a multiple of 4 KB.
When the entire file has been processed, the first call beyond the end of file
will return 0 to rd count, which will make it exit the loop. At this point, the two
files are closed and the program exits with a status indicating normal termination.
Although the Windows system calls are different from those of UNIX, the general structure of a command-line Windows program to copy a file is moderately
similar to that of Fig. 4-6. We will examine the Windows calls in Chap. 11.
4.2 DIRECTORIES
To keep track of files, file systems normally have directories or folders, which
are themselves files. In this section, we will discuss directories, their organization,
their properties, and the operations that can be performed on them.
4.2.1 Single-Level Directory Systems
The simplest form of directory system is having one directory containing all
the files. Sometimes it is called the root directory, but since it is the only one, the
name does not matter much. On the first personal computers, this system was
SEC. 4.2
DIRECTORIES
273
common, in part because there was only one user. Interestingly enough, the world’s
first supercomputer, the CDC 6600, also had only a single directory for all files,
even though it was used by many users at once. This decision was no doubt made
to keep the software design simple.
An example of a system with one directory is given in Fig. 4-7. Here the
directory contains four files. The advantages of this scheme are its simplicity and
the ability to locate files quickly—there is only one place to look, after all. It is
sometimes still used on simple embedded devices such as digital cameras and
some portable music players.
Root directory
A
B
C
D
Figure 4-7. A single-level directory system containing four files.
Biologist Ernst Haeckel once said ‘‘ontogeny recapitulates phylogeny.’’ It’s not
entirely accurate, but there is a grain of truth in it. Something analogous happens in
the computer world. Some concept was in vogue on, say, mainframe computers,
then discarded as they got more powerful, but picked up later on minicomputers.
Then it was discarded there and later picked up on personal computers. Then it was
discarded there and later picked up further down the food chain.
So we frequently see concepts (like having one directory for all files) no longer
used on powerful computers, but now being used on simple embedded devices like
digital cameras and portable music players. For this reason through this chapter
(and, indeed, the entire book), we will often discuss ideas that were once popular
on mainframes, minicomputers, or personal computers, but have since been discarded. Not only is this a good historical lesson, but often these ideas make perfect
sense on yet lower-end devices. The chip on your credit card really does not need
the full-blown hierarchical directory system we are about to explore. The simple
file system used on the CDC 6600 supercomputer in the 1960s will do just fine,
thank you. So when you read about some old concept here, do not think ‘‘how oldfashioned.’’ Think: Would that work on an RFID (Radio Frequency IDentification)
chip? that costs 5 cents and is used on a public-transit payment card? It just might.
4.2.2 Hierarchical Directory Systems
The single level is adequate for very simple dedicated applications (and was
even used on the first personal computers), but for modern users with thousands of
files, it would be impossible to find anything if all files were in a single directory.
Consequently, a way is needed to group related files together. A professor, for
example, might have a collection of files that together form a book that she is writing, a second collection containing student programs submitted for another course,
274
FILE SYSTEMS
CHAP. 4
a third group containing the code of an advanced compiler-writing system she is
building, a fourth group containing grant proposals, as well as other files for electronic mail, minutes of meetings, papers she is writing, games, and so on.
What is needed is a hierarchy (i.e., a tree of directories). With this approach,
there can be as many directories as are needed to group the files in natural ways.
Furthermore, if multiple users share a common file server, as is the case on many
company networks, each user can have a private root directory for his or her own
hierarchy. This approach is shown in Fig. 4-8. Here, the directories A, B, and C
contained in the root directory each belong to a different user, two of whom have
created subdirectories for projects they are working on.
Root directory
User
directory
A
A
B
B
B
C
B
B
C
C
C
C
User subdirectories
C
C
C
C
User file
Figure 4-8. A hierarchical directory system.
The ability for users to create an arbitrary number of subdirectories provides a
powerful structuring tool for users to organize their work. For this reason, all modern file systems are organized in this manner. It is worth noting that an hierarchical
file system is one of many things that was pioneered by Multics in the 1960s.
4.2.3 Path Names
When the file system is organized as a directory tree, some way is needed for
specifying file names. Two different methods are commonly used. In the first
method, each file is given an absolute path name consisting of the path from the
root directory to the file. As an example, the path /usr/ast/mailbox means that the
root directory contains a subdirectory usr, which in turn contains a subdirectory
ast, which contains the file mailbox. Absolute path names always start at the root
directory and are unique. In UNIX the components of the path are separated by /.
In Windows the separator is \ . In MULTICS it was >. Thus, the same path name
would be written as follows in these three systems:
Windows
UNIX
MULTICS
\usr\ast\mailbox
/usr/ast/mailbox
>usr>ast>mailbox
SEC. 4.2
DIRECTORIES
275
No matter which character is used, if the first character of the path name is the separator, then the path is absolute.
The other kind of name is the relative path name. This is used in conjunction
with the concept of the working directory (also called the current directory). A
user can designate one directory as the current working directory, in which case all
path names not beginning at the root directory are taken relative to the working
directory. For example, if the current working directory is /usr/hjb, then the file
whose absolute path is /usr/hjb/mailbox can be referenced simply as mailbox. In
other words, the UNIX command
cp /usr/hjb/mailbox /usr/hjb/mailbox.bak
and the command
cp mailbox mailbox.bak
do exactly the same thing if the working directory is /usr/hjb. The relative form is
often more convenient, but it does the same thing as the absolute form.
Some programs need to access a specific file without regard to what the working directory is. In that case, they should always use absolute path names. For
example, a spelling checker might need to read /usr/lib/dictionary to do its work.
It should use the full, absolute path name in this case because it does not know
what the working directory will be when it is called. The absolute path name will
always work, no matter what the working directory is.
Of course, if the spelling checker needs a large number of files from /usr/lib,
an alternative approach is for it to issue a system call to change its working directory to /usr/lib, and then use just dictionary as the first parameter to open. By
explicitly changing the working directory, it knows for sure where it is in the directory tree, so it can then use relative paths.
Each process has its own working directory. When it changes its working directory and later exits, no other processes are affected and no traces of the change
are left behind. In this way, a process can change its working directory whenever it
is convenient. On the other hand, if a library procedure changes the working directory and does not change back to where it was when it is finished, the rest of the
program may not work since its assumption about where it is may now suddenly be
invalid. For this reason, library procedures rarely change the working directory,
and when they must, they always change it back again before returning.
Most operating systems that support a hierarchical directory system have two
special entries in every directory, ‘‘.’’ and ‘‘..’’, generally pronounced ‘‘dot’’ and
‘‘dotdot.’’ Dot refers to the current directory; dotdot refers to its parent (except in
the root directory, where it refers to itself). To see how these are used, consider the
UNIX file tree of Fig. 4-9. A certain process has /usr/ast as its working directory.
It can use .. to go higher up the tree. For example, it can copy the file /usr/lib/dictionary to its own directory using the command
cp ../lib/dictionary .
276
FILE SYSTEMS
CHAP. 4
The first path instructs the system to go upward (to the usr directory), then to go
down to the directory lib to find the file dictionary.
/
bin
Root directory
etc
lib
usr
tmp
bin
etc
lib
tmp
usr
ast
hjb
lib
ast
lib
dict.
hjb
/usr/hjb
Figure 4-9. A UNIX directory tree.
The second argument (dot) names the current directory. When the cp command
gets a directory name (including dot) as its last argument, it copies all the files to
that directory. Of course, a more normal way to do the copy would be to use the
full absolute path name of the source file:
cp /usr/lib/dictionary .
Here the use of dot saves the user the trouble of typing dictionary a second time.
Nevertheless, typing
cp /usr/lib/dictionary dictionar y
also works fine, as does
cp /usr/lib/dictionary /usr/ast/dictionar y
All of these do exactly the same thing.
SEC. 4.2
DIRECTORIES
277
4.2.4 Directory Operations
The allowed system calls for managing directories exhibit more variation from
system to system than system calls for files. To give an impression of what they
are and how they work, we will give a sample (taken from UNIX).
1. Create. A directory is created. It is empty except for dot and dotdot,
which are put there automatically by the mkdir program.
2. Delete. A directory is deleted. Only an empty directory can be
deleted. A directory containing only dot and dotdot is considered
empty as these cannot be deleted.
3. Opendir. Directories can be read. For example, to list all the files in a
directory, a listing program opens the directory to read out the names
of all the files it contains. Before a directory can be read, it must be
opened, analogous to opening and reading a file.
4. Closedir. When a directory has been read, it should be closed to free
up internal table space.
5. Readdir. This call returns the next entry in an open directory. Formerly, it was possible to read directories using the usual read system
call, but that approach has the disadvantage of forcing the programmer to know and deal with the internal structure of directories.
In contrast, readdir always returns one entry in a standard format, no
matter which of the possible directory structures is being used.
6. Rename. In many respects, directories are just like files and can be
renamed the same way files can be.
7. Link. Linking is a technique that allows a file to appear in more than
one directory. This system call specifies an existing file and a path
name, and creates a link from the existing file to the name specified
by the path. In this way, the same file may appear in multiple directories. A link of this kind, which increments the counter in the file’s
i-node (to keep track of the number of directory entries containing the
file), is sometimes called a hard link.
8. Unlink. A directory entry is removed. If the file being unlinked is
only present in one directory (the normal case), it is removed from the
file system. If it is present in multiple directories, only the path name
specified is removed. The others remain. In UNIX, the system call
for deleting files (discussed earlier) is, in fact, unlink.
The above list gives the most important calls, but there are a few others as well, for
example, for managing the protection information associated with a directory.
278
FILE SYSTEMS
CHAP. 4
A variant on the idea of linking files is the symbolic link (sometimes called a
shortcut or alias). Instead, of having two names point to the same internal data
structure representing a file, a name can be created that points to a tiny file naming
another file. When the first file is used, for example, opened, the file system follows the path and finds the name at the end. Then it starts the lookup process all
over using the new name. Symbolic links have the advantage that they can cross
disk boundaries and even name files on remote computers. Their implementation is
somewhat less efficient than hard links though.
4.3 FILE-SYSTEM IMPLEMENTATION
Now it is time to turn from the user’s view of the file system to the implementor’s view. Users are concerned with how files are named, what operations are
allowed on them, what the directory tree looks like, and similar interface issues.
Implementers are interested in how files and directories are stored, how disk space
is managed, and how to make everything work efficiently and reliably. In the following sections, we will examine a number of these areas to see what the issues
and trade-offs are.
4.3.1 File-System Layout
File systems are stored on disks. Most disks can be divided up into one or
more partitions, with independent file systems on each partition. The layout
depends on whether you have an old computer with a BIOS and a master boot
record, or a modern UEFI-based system.
Old School: The Master Boot Record
On older systems, sector 0 of the disk is called the MBR (Master Boot
Record) and is used to boot the computer. The end of the MBR contains the partition table. This table gives the starting and ending addresses of each partition. One
of the partitions in the table is marked as active. When the computer is booted, the
BIOS reads in and executes the MBR. The first thing the MBR program does is
locate the active partition, read in its first block, which is called the boot block,
and execute it. The program in the boot block loads the operating system contained
in that partition. For uniformity, every partition starts with a boot block, even if it
does not contain a bootable operating system. Besides, it might contain one in the
future.
Other than starting with a boot block, the layout of a disk partition varies a lot
from file system to file system. Often the file system will contain some of the items
shown in Fig. 4-10. The first one is the superblock. It contains all the key parameters about the file system and is read into memory when the computer is booted or
SEC. 4.3
FILE-SYSTEM IMPLEMENTATION
279
the file system is first touched. Typical information in the superblock includes a
magic number to identify the file-system type, the number of blocks in the file system, and other key administrative information.
Entire disk
Disk partition
Partition table
MBR
Boot block
Superblock
Free space mgmt
I-nodes
Root dir
Files and directories
Figure 4-10. A possible file-system layout.
Next might come information about free blocks in the file system, for example
in the form of a bitmap or a list of pointers. This might be followed by the i-nodes,
an array of data structures, one per file, telling all about the file. After that might
come the root directory, which contains the top of the file-system tree. Finally, the
remainder of the disk contains all the other directories and files.
New School: Unified Extensible Firmware Interface
Unfortunately, booting in the way described above is slow, architecture-dependent, and limited to smaller disks (up to 2 TB) and Intel therefore proposed the UEFI (Unified Extensible Firmware Interface) as a replacement. is
now the most popular way to boot personal computer systems. It fixes many of the
problems of the old-style BIOS and MBR: fast booting, different architectures, and
disk sizes up to 8 ZiB. It is also quite complex.
Rather than relying on a Master Boot Record residing in sector 0 of the boot
device, UEFI looks for the location of the partition table in the second block of
the device. It reserves the first block as a special marker for software that expects
an MBR here. The marker essentially says: No MBR here!
The GPT (GUID Partition Table), meanwhile, contains information about the
location of the various partitions on the disk. GUID stands for globally unique
identifiers. As shown in Fig. 4-11, UEFI keeps a backup of the GPT in the last
block. A GPT contains the start and end of each partition. Once the GPT is found,
the firmware has enough functionality to read file systems of specific types.
According to the UEFI standard the firmware should support at least FAT file system types. One such file system is placed in a special disk partition, known as the
280
FILE SYSTEMS
CHAP. 4
EFI system partition (ESP). Rather than a single magic boot sector, the boot process can now use a proper file system containing programs, configuration files, and
anything else that may be useful during boot. Moreover, UEFI expects the
firmware to be able to execute programs in a specific format, called PE (Portable
Executable). In other words, the firmware under UEFI looks like a small operating
system itself with an understanding of disk partitions, file systems, executables,
etc.
...
Partition n
Partition 2
Start
GPT
Partition 0
Partition 2 ...
Partition 1
...
Block N
End
Start
Partition n
Partition
Table Header
Partition 2
Space for legacy
systems that
expect MBR here
Block 2
Partition 1
Block 1
Partition 0
Block 0
Partition
Table Header
End
Partition of many blocks
Backup GPT
Figure 4-11. Layout for UEFI with partition table.
4.3.2 Implementing Files
Probably the most important issue in implementing file storage is keeping
track of which disk blocks go with which file. Various methods are used in different operating systems. In this section, we will examine a few of them.
Contiguous Allocation
The simplest allocation scheme is to store each file as a contiguous run of disk
blocks. Thus on a disk with 1-KB blocks, a 50-KB file would be allocated 50 consecutive blocks. With 2-KB blocks, it would be allocated 25 consecutive blocks.
We see an example of contiguous storage allocation in Fig. 4-12(a). Here the
first 40 disk blocks are shown, starting with block 0 on the left. Initially, the disk
was empty. Then a file A, of length four blocks, was written to disk starting at the
beginning (block 0). After that a six-block file, B, was written starting right after
the end of file A.
Note that each file begins at the start of a new block, so that if file A was really
3½ blocks, some space is wasted at the end of the last block. In the figure, a total
of seven files are shown, each one starting at the block following the end of the
previous one. Shading is used just to make it easier to tell the files apart. It has no
actual significance in terms of storage.
Contiguous disk-space allocation has two significant advantages. First, it is
simple to implement because keeping track of where a file’s blocks are is reduced
to remembering two numbers: the disk address of the first block and the number of
blocks in the file. Given the number of the first block, the number of any other
block can be found by a simple addition.
SEC. 4.3
281
FILE-SYSTEM IMPLEMENTATION
File A
(4 blocks)
File E
(12 blocks)
File C
(6 blocks)
File G
(3 blocks)
…
File B
(3 blocks)
File D
(5 blocks)
File F
(6 blocks)
(a)
(File A)
(File C)
(File E)
(File G)
…
File B
5 Free blocks
6 Free blocks
(b)
Figure 4-12. (a) Contiguous allocation of disk space for seven files. (b) The
state of the disk after files D and F have been removed.
Second, the read performance is excellent even on a magnetic disk because the
entire file can be read from the disk in a single operation. Only one seek is needed
(to the first block). After that, no more seeks or rotational delays are needed, so
data come in at the full bandwidth of the disk. Thus contiguous allocation is simple
to implement and has high performance. We will talk about sequential versus random accesses on SSDs later.
Unfortunately, contiguous allocation also has a very serious drawback: over the
course of time, the disk becomes fragmented. To see how this comes about, examine Fig. 4-12(b). Here two files, D and F, have been removed. When a file is
removed, its blocks are naturally freed, leaving a run of free blocks on the disk.
The disk is not compacted on the spot to squeeze out the hole, since that would
involve copying all the blocks following the hole, potentially millions of blocks,
which would take hours or even days with large disks. As a result, the disk ultimately consists of files and holes, as illustrated in the figure.
Initially, this fragmentation is not a problem, since each new file can be written
at the end of disk, following the previous one. However, eventually the disk will fill
up and it will become necessary to either compact the disk, which is prohibitively
expensive, or to reuse the free space in the holes. Reusing the space requires maintaining a list of holes, which is doable. However, when a new file is to be created,
it is necessary to know its final size in order to choose a hole that is big enough.
Imagine the consequences of such a design. The user starts a recording application in order to create a video. The first thing the program asks is how many
bytes the final video will be. The question must be answered or the program will
not continue. If the number given ultimately proves too small, the program has to
282
FILE SYSTEMS
CHAP. 4
terminate prematurely because the disk hole is full and there is no place to put the
rest of the file. If the user tries to avoid this problem by giving an unrealistically
large number as the final size, say, 100 GB, the editor may be unable to find such a
large hole and announce that the file cannot be created. Of course, the user would
be free to start the program again, say 50 GB this time, and so on until a suitable
hole was located. Still, this scheme is not likely to lead to happy users.
Linked-List Allocation
The second method for storing files is to keep each one as a linked list of disk
blocks, as shown in Fig. 4-13. The first part of each block is used as a pointer to
the next one. The rest of the block is for data.
File A
0
Physical
block
File
block
0
File
block
1
File
block
2
File
block
3
File
block
4
4
7
2
10
12
File B
0
Physical
block
File
block
0
File
block
1
File
block
2
File
block
3
6
3
11
14
Figure 4-13. Storing a file as a linked list of disk blocks.
Unlike contiguous allocation, every disk block can be used in this method. No
space is lost to disk fragmentation (except for internal fragmentation in the last
block). Also, it is sufficient for the directory entry to merely store the disk address
of the first block. The rest can be found starting there.
On the other hand, although reading a file sequentially is straightforward, random access is extremely slow. To get to block n, the operating system has to start
at the beginning and read the n < 1 blocks prior to it, one at a time. Clearly, doing
so many reads will be painfully slow.
Also, the amount of data storage in a block is no longer a power of two because the pointer takes up a few bytes. While not fatal, having a peculiar size is
less efficient because many programs read and write in blocks whose size is a power of two. With the first few bytes of each block occupied by a pointer to the next
block, reads of the full block size require acquiring and concatenating information
from two disk blocks, which generates extra overhead due to the copying.
SEC. 4.3
FILE-SYSTEM IMPLEMENTATION
283
Linked-List Allocation Using a Table in Memory
Both disadvantages of the linked-list allocation can be eliminated by taking the
pointer word from each disk block and putting it in a table in memory. Figure 4-14
shows what the table looks like for the example of Fig. 4-13. In both figures, we
have two files. File A uses disk blocks 4, 7, 2, 10, and 12, in that order, and file B
uses disk blocks 6, 3, 11, and 14, in that order. Using the table of Fig. 4-14, we can
start with block 4 and follow the chain all the way to the end. The same can be
done starting with block 6. Both chains are terminated with a special marker (e.g.,
< 1) that is not a valid block number. Such a table in main memory is called a FAT
(File Allocation Table).
Physical
block
0
1
2
10
3
11
4
7
File A starts here
6
3
File B starts here
7
2
5
8
9
10
12
11
14
12
-1
13
14
15
-1
Unused block
Figure 4-14. Linked-list allocation using a file-allocation table in main memory.
Using this organization, the entire block is available for data. Furthermore, random access is much easier. Although the chain must still be followed to find a
given offset within the file, the chain is entirely in memory, so it can be followed
without making any disk references. Like the previous method, it is sufficient for
the directory entry to keep a single integer (the starting block number) and still be
able to locate all the blocks, no matter how large the file is.
The primary disadvantage of this method is that the entire table must be in
memory all the time to make it work. With a 1-TB disk and a 1-KB block size, the
table needs 1 billion entries, one for each of the 1 billion disk blocks. Each entry
has to be a minimum of 3 bytes. For speed in lookup, they should be 4 bytes. Thus
284
FILE SYSTEMS
CHAP. 4
the table will take up 3 GB or 2.4 GB of main memory all the time, depending on
whether the system is optimized for space or time. Not wildly practical. Clearly the
FAT idea does not scale well to large disks. Nevertheless, it was the original MSDOS file system and is still fully supported by all versions of Windows though
(and UEFI). Versions of the FAT file system are still commonly used on the SD
cards used in digital cameras, electronic picture frames, music players, and other
portable electronic devices, as well as in other embedded applications.
I-nodes
Our last method for keeping track of which blocks belong to which file is to
associate with each file a data structure called an i-node (index-node), which lists
the attributes and disk addresses of the file’s blocks. A simple example is depicted
in Fig. 4-15. Given the i-node, it is then possible to find all the blocks of the file.
The big advantage of this scheme over linked files using an in-memory table is that
the i-node needs to be in memory only when the corresponding file is open. If each
i-node occupies n bytes and a maximum of k files may be open at once, the total
memory occupied by the array holding the i-nodes for the open files is only kn
bytes. Only this much space need be reserved in advance.
File Attributes
Address of disk block 0
Address of disk block 1
Address of disk block 2
Address of disk block 3
Address of disk block 4
Address of disk block 5
Address of disk block 6
Address of disk block 7
Address of block of pointers
Disk block
containing
additional
disk addresses
Figure 4-15. An example i-node.
This array is usually far smaller than the space occupied by the file table
described in the previous section. The reason is simple. The table for holding the
SEC. 4.3
FILE-SYSTEM IMPLEMENTATION
285
linked list of all disk blocks is proportional in size to the disk itself. If the disk has
n blocks, the table needs n entries. As disks grow larger, this table grows linearly
with them. In contrast, the i-node scheme requires an array in memory whose size
is proportional to the maximum number of files that may be open at once. It does
not matter if the disk is 500 GB, 500 TB, or 500 PB.
One problem with i-nodes is that if each one has room for a fixed number of
disk addresses, what happens when a file grows beyond this limit? One solution is
to reserve the last disk address not for a data block, but instead for the address of a
block containing more disk-block addresses, as shown in Fig. 4-15. Even more
advanced would be two or more such blocks containing disk addresses or even disk
blocks pointing to other disk blocks full of addresses. We will come back to inodes when studying UNIX in Chap. 10. Similarly, the Windows NTFS file system uses a similar idea, only with bigger i-nodes that can also contain small files.
4.3.3 Implementing Directories
Before a file can be read, it must be opened. When a file is opened, the operating system uses the path name supplied by the user to locate the directory entry on
the disk. The directory entry provides the information needed to find the disk
blocks. Depending on the system, this information may be the disk address of the
entire file (with contiguous allocation), the number of the first block (both linked-list schemes), or the number of the i-node. In all cases, the main function of the
directory system is to map the ASCII name of the file onto the information needed
to locate the data.
A closely related issue is where the attributes should be stored. Every file system maintains various file attributes, such as each file’s owner and creation time,
and they must be stored somewhere. One obvious possibility is to store them
directly in the directory entry. Some systems do precisely that. This option is
shown in Fig. 4-16(a). In this simple design, a directory consists of a list of fixedsize entries, one per file, containing a (fixed-length) file name, a structure of the
file attributes, and one or more disk addresses (up to some maximum) telling
where the disk blocks are.
For systems that use i-nodes, another possibility for storing the attributes is in
the i-nodes, rather than in the directory entries. In that case, the directory entry can
be shorter: just a file name and an i-node number. This approach is illustrated in
Fig. 4-16(b). As we shall see later, this method has some advantages over putting
them in the directory entry.
So far we have made the implicit assumption that files have short, fixed-length
names. In MS-DOS files have a 1–8 character base name and an optional extension
of 1–3 characters. In UNIX Version 7, file names were 1–14 characters, including
any extensions. However, nearly all modern operating systems support longer, variable-length file names. How can these be implemented?
286
FILE SYSTEMS
games
attributes
games
mail
attributes
mail
news
attributes
news
work
attributes
work
(a)
(b)
CHAP. 4
Data structure
containing the
attributes
Figure 4-16. (a) A simple directory containing fixed-size entries with the disk addresses
and attributes in the directory entry. (b) A directory in which each entry just
refers to an i-node.
The simplest approach is to set a limit on file-name length, typically 255 characters, and then use one of the designs of Fig. 4-16 with 255 characters reserved
for each file name. This approach is simple, but wastes a great deal of directory
space, since few files have such long names. For efficiency reasons, a different
structure is desirable.
One alternative is to give up the idea that all directory entries are the same size.
With this method, each directory entry contains a fixed portion, typically starting
with the length of the entry, and then followed by data with a fixed format, usually
including the owner, creation time, protection information, and other attributes.
This fixed-length header is followed by the actual file name, however long it may
be, as shown in Fig. 4-17(a) in big-endian format (as used by some CPUs). In this
example we have three files, project-budget, personnel, and foo. Each file name is
terminated by a special character (usually 0), which is represented in the figure by
a box with a cross in it. To allow each directory entry to begin on a word boundary, each file name is filled out to an integral number of words, shown by shaded
boxes in the figure.
A disadvantage of this method is that when a file is removed, a variable-sized
gap is introduced into the directory into which the next file to be entered may not
fit. This problem is essentially the same one we saw with contiguous disk files,
only now compacting the directory is feasible because it is entirely in memory.
Another problem is that a single directory entry may span multiple pages, so a
page fault may occur while reading a file name.
Another way to handle variable-length names is to make the directory entries
themselves all fixed length and keep the file names together in a heap at the end of
the directory, as shown in Fig. 4-17(b). This method has the advantage that when
an entry is removed, the next file entered will always fit there. Of course, the heap
must be managed and page faults can still occur while processing file names. One
very minor win here is that there is no longer any real need for file names to begin
SEC. 4.3
Entry
for one
file
287
FILE-SYSTEM IMPLEMENTATION
p
e
b
e
File 1 entry length
Pointer to file 1's name
File 1 attributes
File 1 attributes
r
o
j
c
t
u
d
g
t
File 2 entry length
Entry
for one
file
Pointer to file 2's name
File 2 attributes
Pointer to file 3's name
File 3 attributes
File 2 attributes
p
o
l
e
n
r
n
File 3 entry length
File 3 attributes
f
o
o
(a)
s
e
p
e
b
e
e
n
r
c
u
t
r
n
f
o
t
d
s
e
o
j
g
p
o
l
o
Heap
(b)
Figure 4-17. Two ways of handling long file names in a directory. (a) In-line.
(b) In a heap.
at word boundaries, so no filler characters are needed after file names in
Fig. 4-17(b) as they are in Fig. 4-17(a).
In all of the designs so far, directories are searched linearly from beginning to
end when a file name has to be looked up. For extremely long directories, linear
searching can be slow. One way to speed up the search is to use a hash table in
each directory. Call the size of the table n. To enter a file name, the name is hashed
onto a value between 0 and n < 1, for example, by dividing it by n and taking the
remainder. Alternatively, the words comprising the file name can be added up and
this quantity divided by n, or something similar.
Either way, the table entry corresponding to the hash code is inspected. If it is
unused, a pointer is placed there to the file entry. File entries follow the hash table.
If that slot is already in use, a linked list is constructed, headed at the table entry
and threading through all entries with the same hash value.
Looking up a file follows the same procedure. The file name is hashed to select
a hash-table entry. All the entries on the chain headed at that slot are checked to
see if the file name is present. If the name is not on the chain, the file is not present in the directory.
Using a hash table has the advantage of much faster lookup, but the disadvantage of a much more complex administration. It is only really a serious candidate
288
FILE SYSTEMS
CHAP. 4
in systems where it is expected that directories will routinely contain hundreds or
thousands of files.
A different way to speed up searching large directories is to cache the results
of searches. Before starting a search, a check is first made to see if the file name is
in the cache. If so, it can be located immediately. Of course, caching only works
if a relatively small number of files comprise the majority of the lookups.
4.3.4 Shared Files
When several users are working together on a project, they often need to share
files. As a result, it is often convenient for a shared file to appear simultaneously
in different directories belonging to different users. Figure 4-18 shows the file system of Fig. 4-8 again, only with one of C’s files now present in one of B’s directories as well. The connection between B’s directory and the shared file is called a
link. The file system itself is now a DAG (Directed Acyclic Graph), rather than a
tree. Having the file system be a DAG complicates maintenance, but such is life.
Root directory
A
A
B
B
C
B
B
C
C
C
B
?
C
C
C
C
Shared file
Figure 4-18. File system containing a shared file.
Sharing files is convenient, but it also introduces some problems. To start
with, if directories really do contain disk addresses, then a copy of the disk addresses will have to be made in B’s directory when the file is linked. If either B or C
subsequently appends to the file, the new blocks will be listed only in the directory
of the user doing the append. The changes will not be visible to the other user, thus
defeating the purpose of sharing.
This problem can be solved in two ways. In the first solution, disk blocks are
not listed in directories, but in a little data structure associated with the file itself.
The directories would then just have pointers to the little data structure. This is the
approach used in UNIX (where the little data structure is the i-node).
SEC. 4.3
289
FILE-SYSTEM IMPLEMENTATION
In the second solution, B links to one of C’s files by having the system create a
new file, of type LINK, and entering that file in B’s directory. The new file contains just the path name of the file to which it is linked. When B reads from the
linked file, the operating system sees that the file being read from is of type LINK,
looks up the name of the file, and reads that file. This approach is called symbolic
linking, to contrast it with traditional (hard) linking, as discussed earlier.
Each of these methods has its drawbacks. In the first method, at the moment
that B links to the shared file, the i-node records the file’s owner as C. Creating a
link does not change the ownership (see Fig. 4-19), but it does increase the link
count in the i-node, so the system knows how many directory entries currently
point to the file.
C's directory
B's directory C's directory
B's directory
Owner = C
Count = 1
Owner = C
Count = 2
Owner = C
Count = 1
(a)
(b)
(c)
Figure 4-19. (a) Situation prior to linking. (b) After the link is created. (c) After
the original owner removes the file.
If C subsequently tries to remove the file, the system is faced with a problem.
If it removes the file and clears the i-node, B will have a directory entry pointing to
an invalid i-node. If the i-node is later reassigned to another file, B’s link will
point to the wrong file. The system can see from the count in the i-node that the
file is still in use, but there is no easy way for it to find all the directory entries for
the file, in order to erase them. Pointers to the directories cannot be stored in the inode because there can be an unlimited number of directories.
The only thing to do is remove C’s directory entry, but leave the i-node intact,
with count set to 1, as shown in Fig. 4-19(c). We now have a situation in which B
is the only user having a directory entry for a file owned by C. If the system does
accounting or has quotas, C will continue to be billed for the file until B decides to
remove it, if ever, at which time the count goes to 0 and the file is deleted.
With symbolic links this problem does not arise because only the true owner
has a pointer to the i-node. Users who have linked to the file just have path names,
not i-node pointers. When the owner removes the file, it is destroyed. Subsequent
attempts to use the file via a symbolic link will fail when the system is unable to
locate the file. Removing a symbolic link does not affect the file at all.
290
FILE SYSTEMS
CHAP. 4
The problem with symbolic links is the extra overhead required. The file containing the path must be read, then the path must be parsed and followed, component by component, until the i-node is reached. All of this activity may require a
considerable number of extra disk accesses. Furthermore, an extra i-node is needed
for each symbolic link, as is an extra disk block to store the path, although if the
path name is short, the system could store it in the i-node itself, as a kind of optimization. Symbolic links have the advantage that they can be used to link to files
on machines anywhere in the world, by simply providing the network address of
the machine where the file resides in addition to its path on that machine.
There is also another problem introduced by links, symbolic or otherwise.
When links are allowed, files can have two or more paths. Programs that start at a
given directory and find all the files in that directory and its subdirectories will
locate a linked file multiple times. For example, a program that dumps all the files
in a directory and its subdirectories onto a backup drive may make multiple copies
of a linked file. Furthermore, if the backup drive is then read into another machine,
unless the dump program is clever, the linked file may be copied twice onto the
disk, instead of being linked.
4.3.5 Log-Structured File Systems
Changes in technology are putting pressure on current file systems. Let us consider computers with (magnetic) hard disks. In the next section, we will look at
SSDs. In systems with hard disks, the CPUs keep getting faster, the disks are
becoming much bigger and cheaper (but not much faster), and memories are growing exponentially in size. The one parameter that is not improving by leaps and
bounds is disk seek time.
The combination of these factors led to a performance bottleneck in file systems. Research done at Berkeley attempted to alleviate this problem by designing a
completely new kind of file system, LFS (the Log-structured File System). In this
section, we will briefly describe how LFS works. For a more complete treatment,
see the original paper on LFS (Rosenblum and Ousterhout, 1991).
The idea that drove the LFS design is that as CPUs get faster and RAM memories get larger, disk caches are also increasing rapidly. Consequently, it is now possible to satisfy a very substantial fraction of all read requests directly from the
file-system cache, with no disk access needed. It follows from this observation
that in the future, most disk accesses will be writes, so the read-ahead mechanism
used in some file systems to fetch blocks before they are needed no longer gains
much performance.
To make matters worse, in most file systems, writes are done in very small
chunks. Small writes are highly inefficient, since a 50- µ sec disk write is often preceded by a 10-msec seek and a 4-msec rotational delay. With these parameters,
disk efficiency drops to a fraction of 1%.
SEC. 4.3
FILE-SYSTEM IMPLEMENTATION
291
To see where all the small writes come from, consider creating a new file on a
UNIX system. To write this file, the i-node for the directory, the directory block,
the i-node for the file, and the file itself must all be written. While these writes can
be delayed, doing so exposes the file system to serious consistency problems if a
crash occurs before the writes are done. For this reason, the i-node writes are generally done immediately.
From this reasoning, the LFS designers decided to reimplement the UNIX file
system in such a way as to achieve the full bandwidth of the disk, even in the face
of a workload consisting in large part of small random writes. The basic idea is to
structure the entire disk as a great big log.
Periodically, and when there is a special need for it, all the pending writes
being buffered in memory are collected into a single segment and written to the
disk as a single contiguous segment at the end of the log. A single segment may
thus contain i-nodes, directory blocks, and data blocks, all mixed together. At the
start of each segment is a segment summary, telling what can be found in the segment. If the average segment can be made to be about 1 MB, almost the full bandwidth of the disk can be utilized.
In this design, i-nodes still exist and even have the same structure as in UNIX,
but they are now scattered all over the log, instead of being at a fixed position on
the disk. Nevertheless, when an i-node is located, locating the blocks is done in the
usual way. Of course, finding an i-node is now much harder, since its address cannot simply be calculated from its i-number, as in UNIX. To make it possible to
find i-nodes, an i-node map, indexed by i-number, is maintained. Entry i in this
map points to i-node i on the disk. The map is kept on disk, but it is also cached,
so the most heavily used parts will be in memory most of the time.
To summarize what we have said so far, all writes are initially buffered in
memory, and periodically all the buffered writes are written to the disk in a single
segment, at the end of the log. Opening a file now consists of using the map to
locate the i-node for the file. Once the i-node has been located, the addresses of
the blocks can be found from it. All of the blocks will themselves be in segments,
somewhere in the log.
If disks were infinitely large, the above description would be the entire story.
However, real disks are finite, so eventually the log will occupy the entire disk, at
which time no new segments can be written to the log. Fortunately, many existing
segments may have blocks that are no longer needed. For example, if a file is overwritten, its i-node will now point to the new blocks, but the old ones will still be
occupying space in previously written segments.
To deal with this problem, LFS has a cleaner thread that spends its time scanning the log circularly to compact it. It starts out by reading the summary of the
first segment in the log to see which i-nodes and files are there. It then checks the
current i-node map to see if the i-nodes are still current and file blocks are still in
use. If not, that information is discarded. The i-nodes and blocks that are still in
use go into memory to be written out in the next segment. The original segment is
292
FILE SYSTEMS
CHAP. 4
then marked as free, so that the log can use it for new data. In this manner, the
cleaner moves along the log, removing old segments from the back and putting any
live data into memory for rewriting in the next segment. Consequently, the disk is a
big circular buffer, with the writer thread adding new segments to the front and the
cleaner thread removing old ones from the back.
The bookkeeping here is nontrivial, since when a file block is written back to a
new segment, the i-node of the file (somewhere in the log) must be located,
updated, and put into memory to be written out in the next segment. The i-node
map must then be updated to point to the new copy. Nevertheless, it is possible to
do the administration, and the performance results show that all this complexity is
worthwhile. Measurements given in the papers cited above show that LFS outperforms UNIX by an order of magnitude on small writes, while having a performance that is as good as or better than UNIX for reads and large writes.
4.3.6 Journaling File Systems
Log-structured file systems are an interesting idea in general and one of the
ideas inherent in them, robustness in the face of failure, can also be applied to more
conventional file systems. The basic idea here is to keep a log of what the file system is going to do before it does it, so that if the system crashes before it can do its
planned work, upon rebooting the system can look in the log to see what was going
on at the time of the crash and finish the job. Such file systems, called journaling
file systems, are very popular. Microsoft’s NTFS file system and the Linux ext4
and ReiserFS file systems all use journaling. MacOS offers journaling file systems
as an option. Journaling is the default and it is widely used. Below we will give a
brief introduction to this topic.
To see the nature of the problem, consider a simple garden-variety operation
that happens all the time: removing a file. This operation (in UNIX) requires three
steps:
1. Remove the file from its directory.
2. Release the i-node to the pool of free i-nodes.
3. Return all the disk blocks to the pool of free disk blocks.
In Windows, analogous steps are required. In the absence of system crashes, the
order in which these steps are taken does not matter; in the presence of crashes, it
does. Suppose that the first step is completed and then the system crashes. The inode and file blocks will not be accessible from any file, but will also not be available for reassignment; they are just off in limbo somewhere, decreasing the available resources. If the crash occurs after the second step, only the blocks are lost.
If the order of operations is changed and the i-node is released first, then after
rebooting, the i-node may be reassigned, but the old directory entry will continue
SEC. 4.3
FILE-SYSTEM IMPLEMENTATION
293
to point to it, hence to the wrong file. If the blocks are released first, then a crash
before the i-node is cleared will mean that a valid directory entry points to an inode listing blocks now in the free storage pool and which are likely to be reused
shortly, leading to two or more files randomly sharing the same blocks. None of
these outcomes are good.
What the journaling file system does is first write a log entry listing the three
actions to be completed. The log entry is then written to disk (and for good measure, possibly read back from the disk to verify that it was, in fact, written correctly). Only after the log entry has been written, do the various operations begin.
After the operations complete successfully, the log entry is erased. If the system
now crashes, upon recovery the file system can check the log to see if any operations were pending. If so, all of them can be rerun (multiple times in the event of
repeated crashes) until the file is correctly removed.
To make journaling work, the logged operations must be idempotent, which
means they can be repeated as often as needed without harm. Operations such as
‘‘Update the bitmap to mark i-node k or block n as free’’ can be repeated until the
cows come home with no danger. Similarly, searching a directory and removing
any entry called foobar is also idempotent. On the other hand, adding the newly
freed blocks from i-node K to the end of the free list is not idempotent since they
may already be there. The more-expensive operation ‘‘Search the list of free blocks
and add block n to it if it is not already present’’ is idempotent. Journaling file systems have to arrange their data structures and loggable operations so they all are
idempotent. Under these conditions, crash recovery can be made fast and secure.
For added reliability, a file system can introduce the database concept of an
atomic transaction. When this concept is used, a group of actions can be bracketed by the begin transaction and end transaction operations. The file system then
knows it must complete either all the bracketed operations or none of them, but not
any other combinations.
NTFS has an extensive journaling system and its structure is rarely corrupted
by system crashes. It has been in development since its first release with Windows
NT in 1993. The first Linux file system to do journaling was ReiserFS, but its popularity was impeded by the fact that it was incompatible with the then-standard
ext2 file system. In contrast, ext3 which was a less ambitious project than ReiserFS, also does journaling while maintaining compatibility with the previous ext2
system. Its successor, ext4 was similarly developed initially as a series of backward-compatible extensions to ext3.
4.3.7 Flash-based File Systems
SSDs uses flash memory and operates quite differently from hard disk drives.
Generally, NAND-based flash is used (rather than NOR-based flash) in SSDs.
Much of the difference is related to the physics that underpins the storage, which,
294
FILE SYSTEMS
CHAP. 4
however fascinating, is beyond the scope of this chapter. Irrespective of the flash
technology, here are important differences between hard disks and flash storage.
There are no moving parts in flash storage and the problems of seek times and
rotational delays that we mentioned in the previous section do not exist. This
means that the access time (latency) is much better—on the order of several tens of
microseconds instead of milliseconds. It also means that on SSDs there is not
much of a gap in performance between random and sequential reads. As we shall
see, random writes are still quite a bit more expensive—especially small ones.
Indeed, unlike magnetic disks, flash technology has asymmetric read and write
performance: reads are much faster than writes. For instance, where a read takes a
few tens of microseconds, a write can take hundreds. First, writes are slow because
of how the flash cells that implement the bits are programmed—this is physics and
not something we want to get into here. A second, and more impactful reason is
that you can only write a unit of data after you have erased a suitable area on the
device. In fact, flash memory distinguishes between a unit of I/O (often 4 KB), and
a unit of erase (often 64-256 units of I/O, so up to several MB). Sadly, the industry
takes great pleasure in confusing people and refers to a unit of I/O as a page and to
the unit of erase as a block, except for those publications that refer to the unit of
I/O as a block or even a sector and the unit of erase as a chunk. Of course, the
meaning of a page is also quite different from that of a memory page in the previous chapter and the meaning of a block does not match that of a disk block either.
To avoid confusion, we will use the terms flash page and flash block for the unit of
I/O and the unit of erase, respectively.
To write a flash page, the SSD must first erase a flash block—an expensive
operation taking hundreds of microseconds. Fortunately, after it has erased the
block, there are many free flash pages in that space and the SSD can now write the
flash pages in the flash block in order. In other words, it first writes flash page 0 in
the block, then 1, then 2, etc. It cannot write flash page 0, followed by 2, and then
1. Also, the SSD cannot really overwrite a flash page that was written earlier. It
first has to erase the entire flash block again (not just the page). Indeed, if you really wanted to overwrite some data in a file in-place, the SSD would need to save the
other flash pages in the block somewhere else, erase the block in its entirety, and
then rewrite the pages one by one—not a cheap operation at all! Instead, modifying data on an SSD simply makes the old flash page invalid and then rewrites the
new content in another block. If there are no blocks with free pages available, this
would require erasing a block first.
You do not want to keep writing the same flash pages all the time anyway, as
flash memory suffers from wear. Repeatedly writing and erasing takes its toll and
at some point the flash cells that hold the bits can no longer be used. A program/erase (P/E) cycle consists of erasing a cell and writing new content in it. Typical flash memory cells have a maximum endurance of a few thousand to a few
hundred thousand P/E cycles before they kick the bucket. In other words, it is
important to spread the wear across the flash memory cells as much as possible.
SEC. 4.3
295
FILE-SYSTEM IMPLEMENTATION
The device component that is responsible for handling such wear-leveling is
known as the FTL (Flash Translation Layer). It has many other responsibilities
also and it is sometimes referred to as the drive’s secret sauce. The secret sauce
typically runs on a simple processor with access to fast memory. It is shown on the
left in Fig. 4-20. The data are stored in the flash packages (FPs) on the right. Each
flash package consists of multiple dies and each die in turn contains a number of
so-called planes: collections of flash blocks containing flash pages.
die 0
Flash Translation
Layer (FTL)
Memory
FP
FP
FP
FP
FP
FP
die 1
Plane 0
Plane 1
Plane m
Plane 0
Plane 1
Plane m
FP
Block 0
Page 0
:
Page n
Block 0
Page 0
:
Page n
Block 0
Page 0
:
Page n
Block 0
Page 0
:
Page n
Block 0
Page 0
:
Page n
Block 0
Page 0
:
Page n
FP
Block 1
Page 0
:
Page n
Block 1
Page 0
:
Page n
Block 1
Page 0
:
Page n
Block 1
Page 0
:
Page n
Block 1
Page 0
:
Page n
"Secret sauce"
...
...
Block 1
Page 0
:
Page n
:
:
:
:
:
:
Block n
Page 0
:
Page n
Block n
Page 0
:
Page n
Block n
Page 0
:
Page n
Block n
Page 0
:
Page n
Block n
Page 0
:
Page n
Block n
Page 0
:
Page n
Figure 4-20. Components inside a typical flash SSD.
To access a specific flash page on the SSD, we need to address the corresponding die on the appropriate flash package, and on that die the right plane, block and
page—a rather complicated, hierarchical address! Unfortunately, this is not how
file systems work at all. The file system simply requests to read a disk block at a
linear, logical disk address. How does the SSD translate between these logical
addresses and the complex physical addresses on the device? Here is a hint: the
Flash Translation Layer was not given its name for nothing.
Much like the paging mechanism in virtual memory, the FTL uses translation
tables to indicate that logical block 54321 is really at die 0 of flash package 1, in
plane 2 and block 5. Such translation tables are handy also for wear leveling, since
the device is free to move a page to a different block (for instance, because it needs
to be updated), as long as it adjusts the mapping in the translation table.
The FTL also takes care of managing blocks and pages that are no longer
needed. Suppose that after deleting or moving data a few times, a flash block contains several invalid flash pages. Since only some of the pages are now valid, the
device can free up space by copying the remaining valid pages to a block which
still has free pages available and then erasing the original block. This is known as
garbage collection. In reality, things are much more complicated. For instance,
when do we do garbage collection? If we do it constantly and as early as possible,
it may interfere with the user’s I/O requests. If we do it too late, we may run out of
free blocks. A reasonable compromise is to do it during idle periods, when the
SSD is not busy otherwise.
296
FILE SYSTEMS
CHAP. 4
In addition, the garbage collector needs to select both a victim block (the flash
block to clean) and a target block (to which to write the live data still in the victim
block). Should it simply pick these blocks in a random or round-robin fashion, or
try to make a more informed decision? For instance, for the victim block, should it
select the flash block with the least amount of valid data, or perhaps avoid the flash
blocks that have a lot of wear already, or the blocks that contain a lot of ‘‘hot’’ data
(i.e., data that is likely to be written again in the near future anyway)? Likewise,
for the target block, should it pick based on the amount of available space, or the
amount of accumulated wear on the flash block? Moreover, should it try to group
hot and cold data to ensure that cold flash pages can mostly stay in the same block
with no need for moving them around, while hot pages will perhaps be updated
together close in time, so we can collect the updates in memory and then write
them out to a new flash block in one blast? The answer is: yes. And if you’re wondering which strategy is best, the answer is: it depends. Modern FTLs actually use
a combination of these techniques.
Clearly, garbage collection is complex and a lot of work. It also leads to an
interesting performance property. Suppose there are many flash blocks with invalid
pages, but all of them have only a small number of such pages. In that case, the
garbage collector will have to separate the valid from the invalid pages for many
blocks, each time coalescing the valid data in new blocks and erasing the old
blocks to free up space, at a significant cost in performance and wear. Can you now
see why small random writes may be costlier for garbage collection than sequential
ones?
In reality, small random writes are expensive regardless of garbage collection,
if they overwrite an existing flash page in a full block. The problem has to do with
the mappings in the translation tables. To save space, the FTL has two types of
mappings: per page and per block. If everything was mapped per-page, we would
need an enormous amount of memory to store the translation table. Where possible, therefore, the FTL tries to map a block of pages that belong together as a single entry. Unfortunately, that also means that modifying even a single byte in that
block will invalidate the entire block and lead to lots of additional writes. The
actual overheads of random writes depend on the garbage collection algorithm and
the overall FTL implementation, both of which are typically as secret (and wellguarded) as the formula for Coca Cola.
The decoupling of logical disk block addresses and physical flash addresses
creates an additional problem. With a hard disk drive, when the file system deletes
a file, it knows exactly which blocks on the disk are now free for reuse and can reuse them as it sees fit. This is not the case with SSDs. The file system may decide
to delete a file and mark the logical block addresses as free, but how is the SSD to
know which of its flash pages have been deleted and can therefore be safely
garbage collected? The answer is: it does not and needs to be told explicitly by the
file system. For this, the file system may use the TRIM command which tells the
SSD that certain flash pages are now free. Note that an SSD without the TRIM
SEC. 4.3
FILE-SYSTEM IMPLEMENTATION
297
command still works (and indeed some operating systems have worked without
TRIM for years), but less efficiently. In this case, the SSD would only discover that
the flash pages are invalid when the file system tries to overwrite them. We say that
the TRIM command helps bridge the semantic gap between the FTL and the file
system—the FTL does not have sufficient visibility to do its job efficiently without
some help. It is a major difference between file systems for hard disks and file systems for SSD.
Let us recap what we have learned about SSDs so far. We saw that flash devices have excellent sequential read, but also very good random read performance,
while random writes are slow (although still much faster than read or write
accesses to a disk). Also, we know that frequent writes to the same flash cells rapidly reduces their lifetimes. Finally, we saw that doing complex things at the FTL
is difficult due to the semantic gap.
The reason that we want a new file system for flash is not really the presence
or absence of a TRIM command, but rather that the unique properties of flash
make it a poor match for existing file systems such as NTFS or ext4. So what file
system would be a good match? Since most reads can be served from the cache, we
should look at the writes. We also know that we should avoid random writes and
spread the writes evenly for wear-leveling. By now you may be thinking: ‘‘Wait,
that sounds like a match for a log-structured file system,’’ and you would be right.
Log-structured file systems, with their immutable logs and sequential writes appear
to be a perfect fit for flash-based storage.
Of course, a log-structured file system on flash does not automatically solve all
problems. In particular, consider what happens when we update a large file. In
terms of Fig. 4-15, a large file will use the disk block containing additional disk
addresses that we see in the bottom right and that we will refer to as a (single) indirect block. Besides writing the updated flash page to a new block, the file system
also needs to update the indirect block, since the logical (disk) address of the file
data has changed. The update means that the flash page corresponding to the indirect block must be moved to another flash block. In addition, because the logical
address of the indirect block has now changed, the file system should also update
the i-node itself—leading to a new write to a new flash block. Finally, since the inode is now in a new logical disk block, the file system must also update the i-node
map, leading to another write on the SSD. In other words, a single file update leads
to a cascade of writes of the corresponding meta-data. In real log-structured file
systems, there may be more than one level of indirection (with double or even
triple indirect blocks), so there will be even more writes. The phenomenon is
known as the recursive update problem or wandering tree problem.
While recursive updates cannot be avoided altogether, it is possible to reduce
the impact. For instance, instead of storing the actual disk address of the i-node or
indirect block (in the i-node map and i-node, respectively), some file systems store
the i-node / indirect block number and then maintain, at a fixed logical disk location, a global mapping of these (constant) numbers to logical block addresses on
298
FILE SYSTEMS
CHAP. 4
disk. The advantage is that the file update in the example above only leads to an
update of the indirect block and the global mapping, but not of any of the intermediate mappings. This solution was adopted in the popular Flash-Friendly File System (F2FS) supported by the Linux kernel.
In summary, while people may think of flash as a drop-in replacement for
magnetic disks, it has led to many changes in the file system. This is nothing new.
When magnetic disks started replacing magnetic tape, they led to many changes
also. For instance, the seek operation was introduced and researchers started worrying about disk scheduling algorithms. In general, the introduction of new technology often leads to a flurry of activity and changes in the operating system to make
optimal use of the new capabilities.
4.3.8 Virtual File Systems
Many different file systems are in use—often on the same computer—even for
the same operating system. A Windows system may have a main NTFS file system, but also a legacy FAT-32 or FAT-16 drive or partition that contains old, but
still needed, data, and from time to time, a flash drive with its own unique file system may be required as well. Windows handles these disparate file systems by
identifying each one with a different drive letter, as in C:, D:, etc. When a process
opens a file, the drive letter is explicitly or implicitly present so Windows knows
which file system to pass the request to. There is no attempt to integrate heterogeneous file systems into a unified whole.
In contrast, all modern UNIX systems make a very serious attempt to integrate
multiple file systems into a single structure. A Linux system could have ext4 as
the root file system, with an ext3 partition mounted on /usr and a second hard disk
with a ReiserFS file system mounted on /home as well as an F2FS flash file system
temporarily mounted on /mnt. From the user’s point of view, there is a single filesystem hierarchy. That it happens to encompass multiple (incompatible) file systems is not visible to users or processes.
However, the presence of multiple file systems is very definitely visible to the
implementation, and since the pioneering work of Sun Microsystems (Kleiman,
1986), most UNIX systems have used the concept of a VFS (Virtual File System)
to try to integrate multiple file systems into an orderly structure. The key idea is to
abstract out that part of the file system that is common to all file systems and put
that code in a separate layer that calls the underlying concrete file systems to actually manage the data. The overall structure is illustrated in Fig. 4-21. The discussion below is not specific to Linux or FreeBSD or any other version of UNIX, but
gives the general flavor of how virtual file systems work in UNIX systems.
All system calls relating to files are directed to the virtual file system for initial
processing. These calls, coming from user processes, are the standard POSIX calls,
such as open, read, write, lseek, and so on. Thus the VFS has an ‘‘upper’’ interface
to user processes and it is the well-known POSIX interface.
SEC. 4.3
FILE-SYSTEM IMPLEMENTATION
299
User
process
POSIX
Virtual file system
VFS interface
File
system
FS 1
FS 2
FS 3
Buffer cache
Figure 4-21. Position of the virtual file system.
The VFS also has a ‘‘lower’’ interface to the concrete file systems, which is
labeled VFS interface in Fig. 4-21. This interface consists of several dozen function calls that the VFS can make to each file system to get work done. Thus to create a new file system that works with the VFS, the designers of the new file system
must make sure that it supplies the function calls the VFS requires. An obvious
example of such a function is one that reads a specific block from disk, puts it in
the file system’s buffer cache, and returns a pointer to it. Thus the VFS has two distinct interfaces: the upper one to the user processes and the lower one to the concrete file systems.
While most of the file systems under the VFS represent partitions on a local
disk, this is not always the case. In fact, the original motivation for Sun to build
the VFS was to support remote file systems using the NFS (Network File System)
protocol. The VFS design is such that as long as the concrete file system supplies
the functions the VFS requires, the VFS does not know or care where the data are
stored or what the underlying file system is like. It requires is the proper interface
to the underlying file systems.
Internally, most VFS implementations are essentially object oriented, even if
they are written in C rather than C++. There are several key object types that are
normally supported. These include the superblock (which describes a file system),
the v-node (which describes a file), and the directory (which describes a file system directory). Each of these has associated operations (methods) that the concrete
file systems must support. In addition, the VFS has some internal data structures
for its own use, including the mount table and an array of file descriptors to keep
track of all the open files in the user processes.
To understand how the VFS works, let us run through an example chronologically. When the system is booted, the root file system is registered with the VFS.
In addition, when other file systems are mounted, either at boot time or during
operation, they too must register with the VFS. When a file system registers, what
300
FILE SYSTEMS
CHAP. 4
it basically does is provide a list of the addresses of the functions the VFS requires,
either as one long call vector (table) or as several of them, one per VFS object, as
the VFS demands. Thus once a file system has registered with the VFS, the VFS
knows how to, say, read a block from it—it simply calls the fourth (or whatever)
function in the vector supplied by the file system. Similarly, the VFS then also
knows how to carry out every other function the concrete file system must supply:
it just calls the function whose address was supplied when the file system registered.
After a file system has been mounted, it can be used. For example, if a file system has been mounted on /usr and a process makes the call
open("/usr/include/unistd.h", O RDONLY)
while parsing the path, the VFS sees that a new file system has been mounted on
/usr and locates its superblock by searching the list of superblocks of mounted file
systems. Having done this, it can find the root directory of the mounted file system
and look up the path include/unistd.h there. The VFS then creates a v-node and
makes a call to the concrete file system to return all the information in the file’s inode. This information is copied into the v-node (in RAM), along with other information, most importantly the pointer to the table of functions to call for operations
on v-nodes, such as read, write, close, and so on.
After the v-node has been created, the VFS makes an entry in the file-descriptor table for the calling process and sets it to point to the new v-node. (For the
purists, the file descriptor actually points to another data structure that contains the
current file position and a pointer to the v-node, but this detail is not important for
our purposes here.) Finally, the VFS returns the file descriptor to the caller so it
can use it to read, write, and close the file.
Later when the process does a read using the file descriptor, the VFS locates
the v-node from the process and file descriptor tables and follows the pointer to the
table of functions, all of which are addresses within the concrete file system on
which the requested file resides. The function that handles read is now called and
code within the concrete file system goes and gets the requested block. The VFS
has no idea whether the data are coming from the local disk, a remote file system
over the network, a USB stick, or something different. The data structures involved
are shown in Fig. 4-22. Starting with the caller’s process number and the file descriptor, successively the v-node, read function pointer, and access function within
the concrete file system are located.
In this manner, it becomes relatively straightforward to add new file systems.
To make one, the designers first get a list of function calls the VFS expects and
then write their file system to provide all of them. Alternatively, if the file system
already exists and needs to be ported to the VFS, then they have to provide wrapper functions that do what the VFS needs, usually by making one or more native
calls to the underlying concrete file system.
SEC. 4.4
301
FILE-SYSTEM MANAGEMENT AND OPTIMIZATION
VFS
Process
table
File
descriptors
V-nodes
..
.
4
2
0
Function
pointers
..
.
..
.
write
read
open
Call from
VFS into
FS 1
Read
function
FS 1
Figure 4-22. A simplified view of the data structures and code used by the VFS
and concrete file system to do a read.
4.4 FILE-SYSTEM MANAGEMENT AND OPTIMIZATION
Making the file system work is one thing; making it work efficiently and
robustly in real life is something quite different. In the following sections, we will
look at some of the issues involved in managing disks.
4.4.1 Disk-Space Management
Files are normally stored on disk, so management of disk space is a major concern to file-system designers. Two general strategies are possible for storing an n
byte file: n consecutive bytes of disk space are allocated, or the file is split up into
a number of (not necessarily) contiguous blocks†. The same trade-off is present in
memory-management systems between pure segmentation and paging.
As we have seen, storing a file simply as a contiguous sequence of bytes has
†Disk blocks, not flash blocks. In general, ‘‘block’’ means disk block, unless explicitly stated otherwise.
302
FILE SYSTEMS
CHAP. 4
the obvious problem that if a file grows, it may have to be moved on the disk. The
same problem holds for segments in memory, except that moving a segment in
memory is a relatively fast operation compared to moving a file from one disk
position to another. For this reason, nearly all file systems chop files up into fixedsize blocks that need not be adjacent.
Block Size
Once it has been decided to store files in fixed-size blocks, the question arises
how big the block should be. Given the way hard disks are organized, the sector,
the track, and the cylinder are obvious candidates for the unit of allocation
(although these are all device dependent, which is a minus). In flash-based systems, the flash page size is another candidate, while in a paging system, the memory page size is also a major contender.
Since magnetic disks have served as the storage work horse for years and led
to many of the design choices, such as the common 4 KB block size still used
today, let us consider them first. On a hard disk, having a large block size means
that every file, even a 1-byte file, ties up an entire block. It also means that small
files waste a large amount of disk space. On the other hand, a small block size
means that most files will span multiple blocks and thus need multiple seeks and
rotational delays to read them, reducing performance. Thus if the allocation unit is
too large, we waste space; if it is too small, we waste time. The block size of 4 KB
is considered a reasonable compromise for average users.
As an example, consider a disk with 1 MB per track, a rotation time of 8.33
msec, and an average seek time of 5 msec. The time in milliseconds to read a block
of k bytes is then the sum of the seek, rotational delay, and transfer times:
5 + 4. 165 + (k/1000000) × 8. 33
The dashed curve of Fig. 4-23 shows the data rate for such a disk as a function of
block size. To compute the space efficiency, we need to make an assumption about
the mean file size. For simplicity, let us assume that all files are 4 KB. While this is
clearly not true in practice, it turns out that modern file systems are littered with
files of a few kilobytes in size (e.g., icons, emojis, and emails) so this is not a crazy
number either. The solid curve of Fig. 4-23 shows the space efficiency as a function of block size.
The two curves can be understood as follows. The access time for a block is
completely dominated by the seek time and rotational delay, so given that it is
going to cost 9 msec to access a block, the more data that are fetched, the better.
Hence, the data rate goes up almost linearly with block size (until the transfers take
so long that the transfer time begins to matter).
Now consider space efficiency. With 4-KB files and 1-KB, 2-KB, or 4-KB
blocks, files use 4, 2, and 1 block, respectively, with no wastage. With an 8-KB
block and 4-KB files, the space efficiency drops to 50%, and with a 16-KB block it
303
FILE-SYSTEM MANAGEMENT AND OPTIMIZATION
Data rate (MB/sec)
60
100%
50
80%
40
60%
30
40%
20
20%
10
0
1 KB
4 KB
16 KB
64 KB
256 KB
1MB
Disk space utilization
SEC. 4.4
0%
Figure 4-23. The dashed curve (left-hand scale) gives the data rate of a disk. The
solid curve (right-hand scale) gives the disk-space efficiency. All files are 4 KB.
is down to 25%. In reality, few files are an exact multiple of the disk block size, so
some space is always wasted in the last block of a file.
What the curves show, however, is that performance and space utilization are
inherently in conflict. Small blocks are bad for performance but good for diskspace utilization. For these data, no reasonable compromise is available. The size
closest to where the two curves cross is 64 KB, but the data rate is only 6.6 MB/sec
and the space efficiency is about 7%, neither of which is very good. Historically,
file systems have chosen sizes in the 1-KB to 4-KB range, but with disks now
exceeding multiple TB, it might be better to increase the block size and accept the
wasted disk space. Disk space is hardly in short supply any more.
So far we have looked at the optimal block size from the perspective of a hard
disk and observed that if the allocation unit is too large, we waste space, while if it
is too small, we waste time. With flash storage, we incur memory waste not just for
large disk blocks, but also for smaller ones that do not fill up a flash page.
Keeping Track of Free Blocks
Once a block size has been chosen, the next issue is how to keep track of free
blocks. Two methods are widely used, as shown in Fig. 4-24. The first one consists of using a linked list of disk blocks, with each block holding as many free
disk block numbers as will fit. With a 1-KB block and a 32-bit disk block number,
each block on the free list holds the numbers of 255 free blocks. (One slot is
required for the pointer to the next block.) Consider a 1-TB disk, which has about
1 billion disk blocks. To store all these addresses at 255 per block requires about 4
million blocks. Generally, free blocks are used to hold the free list, so the storage
is essentially free.
The other free-space management technique is the bitmap. A disk with n
blocks requires a bitmap with n bits. Free blocks are represented by 1s in the map,
304
FILE SYSTEMS
CHAP. 4
Free disk blocks: 16, 17, 18
42
230
86
1001101101101100
136
162
234
0110110111110111
210
612
897
1010110110110110
97
342
422
0110110110111011
41
214
140
1110111011101111
63
160
223
1101101010001111
21
664
223
0000111011010111
48
216
160
1011101101101111
262
320
126
1100100011101111
310
180
142
0111011101110111
516
482
141
1101111101110111
A 1-KB disk block can hold 256
32-bit disk block numbers
A bitmap
(a)
(b)
Figure 4-24. (a) Storing the free list on a linked list. (b) A bitmap.
allocated blocks by 0s (or vice versa). For our example 1-TB disk, we need 1 billion bits for the map, which requires around 130,000 1-KB blocks to store. It is
not surprising that the bitmap requires less space, since it uses 1 bit per block, vs.
32 bits in the linked-list model. Only if the disk is nearly full (i.e., has few free
blocks) will the linked-list scheme require fewer blocks than the bitmap.
If free blocks tend to come in long runs of consecutive blocks, the free-list system can be modified to keep track of runs of blocks rather than single blocks. An
8-, 16-, or 32-bit count could be associated with each block giving the number of
consecutive free blocks. In the best case, a basically empty disk could be represented by two numbers: the address of the first free block followed by the count of
free blocks. On the other hand, if the disk becomes severely fragmented, keeping
track of runs is less efficient than keeping track of individual blocks because not
only must the address be stored, but also the count.
This issue illustrates a problem operating system designers often have. There
are multiple data structures and algorithms that can be used to solve a problem, but
choosing the best one requires data that the designers do not have and will not have
until the system is deployed and heavily used. And even then, the data may not be
available. For instance, while we may measure the file size distribution and disk
usage in one or two environments, we have little idea if these numbers are representative of home computers, corporate computers, government computers, not to
mention tablets and smartphones, and others.
SEC. 4.4
FILE-SYSTEM MANAGEMENT AND OPTIMIZATION
305
Getting back to the free list method for a moment, only one block of pointers
need be kept in main memory. When a file is created, the needed blocks are taken
from the block of pointers. When it runs out, a new block of pointers is read in
from the disk. Similarly, when a file is deleted, its blocks are freed and added to
the block of pointers in main memory. When this block fills up, it is written to
disk.
Under certain circumstances, this method leads to unnecessary disk I/O. Consider the situation of Fig. 4-25(a), in which the block of pointers in memory has
room for only two more entries. If a three-block file is freed, the pointer block
overflows and has to be written to disk, leading to the situation of Fig. 4-25(b). If
a three-block file is now written, the full block of pointers has to be read in again,
taking us back to Fig. 4-25(a). If the three-block file just written was a temporary
file, when it is freed, another disk write is needed to write the full block of pointers
back to the disk. In short, when the block of pointers is almost empty, a series of
short-lived temporary files can cause a lot of disk I/O.
Disk
Main
memory
(a)
(b)
(c)
Figure 4-25. (a) An almost-full block of pointers to free disk blocks in memory
and three blocks of pointers on disk. (b) Result of freeing a three-block file.
(c) An alternative strategy for handling the three free blocks. The shaded entries
represent pointers to free disk blocks.
An alternative approach that avoids most of this disk I/O is to split the full
block of pointers. Thus instead of going from Fig. 4-25(a) to Fig. 4-25(b), we go
from Fig. 4-25(a) to Fig. 4-25(c) when three blocks are freed. Now the system can
handle a series of temporary files without doing any disk I/O. If the block in memory fills up, it is written to the disk, and the half-full block from the disk is read in.
The idea here is to keep most of the pointer blocks on disk full (to minimize disk
usage), but keep the one in memory about half full, so it can handle both file creation and file removal without disk I/O on the free list.
With a bitmap, it is also possible to keep just one block in memory, going to
disk for another only when it becomes completely full or empty. An additional
benefit of this approach is that by doing all the allocation from a single block of the
bitmap, the disk blocks will be close together, thus minimizing disk-arm motion.
306
FILE SYSTEMS
CHAP. 4
Since the bitmap is a fixed-size data structure, if the kernel is (partially) paged, the
bitmap can be put in virtual memory and have pages of it paged in as needed.
Disk Quotas
To prevent people from hogging too much disk space, multiuser operating systems often provide a mechanism for enforcing disk quotas. The idea is that the system administrator assigns each user a maximum allotment of files and blocks, and
the operating system makes sure that the users do not exceed their quotas. A typical mechanism is described below.
When a user opens a file, the attributes and disk addresses are located and put
into an open-file table in main memory. Among the attributes is an entry telling
who the owner is. Any increases in the file’s size will be charged to the owner’s
quota.
A second table contains the quota record for every user with a currently open
file, even if the file was opened by someone else. This table is shown in Fig. 4-26.
It is an extract from a quota file on disk for the users whose files are currently
open. When all the files are closed, the record is written back to the quota file.
Open file table
Attributes
disk addresses
User = 8
Quota pointer
Quota table
Soft block limit
Hard block limit
Current # of blocks
# Block warnings left
Soft file limit
Quota
record
for user 8
Hard file limit
Current # of files
# File warnings left
Figure 4-26. Quotas are kept track of on a per-user basis in a quota table.
When a new entry is made in the open-file table, a pointer to the owner’s quota
record is entered into it, to make it easy to find the various limits. Every time a
block is added to a file, the total number of blocks charged to the owner is incremented, and a check is made against both the hard and soft limits. The soft limit
may be exceeded, but the hard limit may not. An attempt to append to a file when
the hard block limit has been reached will result in an error. Analogous checks also
exist for the number of files to prevent a user from hogging all the i-nodes.
SEC. 4.4
FILE-SYSTEM MANAGEMENT AND OPTIMIZATION
307
When a user attempts to log in, the system examines the quota file to see if the
user has exceeded the soft limit for either number of files or number of disk blocks.
If either limit has been violated, a warning is displayed, and the count of warnings
remaining is reduced by one. If the count ever gets to zero, the user has ignored
the warning one time too many, and is not permitted to log in. Getting permission
to log in again will require some discussion with the system administrator.
This method has the property that users may go above their soft limits during a
login session, provided they remove the excess before logging out. The hard limits
may never be exceeded.
4.4.2 File-System Backups
Destruction of a file system is often a far greater disaster than destruction of a
computer. If a computer is destroyed by fire, lightning surges, or a cup of coffee
poured onto the keyboard, it is annoying and will cost money, but generally a
replacement can be purchased with a minimum of fuss. Inexpensive personal computers can even be replaced within an hour by just going to a computer store
(except at universities, where issuing a purchase order takes three committees, five
signatures, and 90 days).
If a computer’s file system is irrevocably lost, whether due to hardware or software failures, restoring all the information will be difficult, time consuming, and in
many cases, impossible. For the people whose programs, documents, tax records,
customer files, databases, marketing plans, or other data are gone forever, the consequences can be catastrophic. While the file system cannot offer any protection
against physical destruction of the equipment and media, it can help protect the
information. It is pretty straightforward: make backups. But that is not quite as
simple as it sounds. Let us take a look.
Most people do not think making backups of their files is worth the time and
effort—until one fine day their disk abruptly dies, at which time most of them
undergo an instantaneous change of heart. Companies, however, (usually) well
understand the value of their data and generally do a backup at least once a day, to
a large disk or even good old-fashioned tape. Tape is still very cost efficient, costing less than $10/TB; no other medium comes close to that price. For companies
with petabytes or exabytes of data, cost of the backup medium matters. Nevertheless, making backups is not quite as trivial as it sounds, so we will examine some
of the related issues below.
Backups are generally made to handle one of two potential problems:
1. Recover from disaster
2. Recover from user mistakes
The first one covers getting the computer running again after a disk crash, fire,
flood, or some other natural catastrophe. In practice, these things do not happen
308
FILE SYSTEMS
CHAP. 4
very often, which is why many people do not bother with backups. These people
also tend not to have fire insurance on their houses for the same reason.
The second reason is that users often accidentally remove files that they later
need again. This problem occurs so often that when a file is ‘‘removed’’ in Windows, it is not deleted at all, but just moved to a special directory, the recycle bin,
so it can be fished out and restored easily later. Backups take this principle further
and allow files that were removed days, even weeks, ago to be restored from old
backup tapes.
Making a backup takes a long time and occupies a large amount of space, so
doing it efficiently and conveniently is important. These considerations raise the
following issues. First, should the entire file system be backed up or only part of
it? At many installations, the executable (binary) programs are kept in a limited
part of the file-system tree. It is not necessary to back up these files if they can all
be reinstalled from the manufacturer’s Website. Also, most systems have a directory for temporary files. There is usually no reason to back it up either. In UNIX,
all the special files (I/O devices) are kept in a directory /dev. Not only is backing
up this directory not necessary, it is downright dangerous because the backup program would hang forever if it tried to read each of these to completion. In short, it
is usually desirable to back up only specific directories and everything in them
rather than the entire file system.
Second, it is wasteful to back up files that have not changed since the previous
backup, which leads to the idea of incremental dumps. The simplest form of
incremental dumping is to make a complete dump (backup) periodically, say
weekly or monthly, and to make a daily dump of only those files that have been
modified since the last full dump. Even better is to dump only those files that have
changed since they were last dumped. While this scheme minimizes dumping time,
it makes recovery more complicated, because first the most recent full dump has to
be restored, followed by all the incremental dumps in reverse order. To ease recovery, more sophisticated incremental dumping schemes are often used.
Third, since immense amounts of data are typically dumped, it may be desirable to compress the data before writing them to backup storage. However, with
many compression algorithms, a single bad spot on the backup storage can foil the
decompression algorithm and make an entire file or even an entire backup storage
unreadable. Thus the decision to compress the backup stream must be carefully
considered.
Fourth, it is difficult to perform a backup on an active file system. If files and
directories are being added, deleted, and modified during the dumping process, the
resulting dump may be inconsistent. However, since making a dump may take
hours, it may be necessary to take the system offline for much of the night to make
the backup, something that is not always acceptable. For this reason, algorithms
have been devised for making rapid snapshots of the file-system state by copying
critical data structures, and then requiring future changes to files and directories to
copy the blocks instead of updating them in place (Hutchinson et al., 1999). In this
SEC. 4.4
FILE-SYSTEM MANAGEMENT AND OPTIMIZATION
309
way, the file system is effectively frozen at the moment of the snapshot, so it can
be backed up at leisure afterward.
Fifth and last, making backups introduces many nontechnical problems into an
organization. The best online security system in the world may be useless if the
system administrator keeps all the backup disks (or tapes) in his office and leaves it
open and unguarded whenever he walks down the hall to get coffee. All a spy has
to do is pop in for a second, put one tiny disk or tape in his pocket, and saunter off
jauntily. Goodbye security. Also, making a daily backup has little use if the fire
that burns down the computers also burns up all the backup media. For this reason,
the backups should be kept off-site, but that introduces more security risks because
now two sites must be secured. While these practical administration issues should
be taken into account in any organization, below we will discuss only the technical
issues involved in making file-system backups.
Two strategies can be used for dumping a disk to a backup medium: a physical
dump or a logical dump. A physical dump starts at block 0 of the disk, writes all
the disk blocks onto the output disk in order, and stops when it has copied the last
one. Such a program is so simple that it can probably be made 100% bug free,
something that can probably not be said about any other useful program.
Nevertheless, it is worth making several comments about physical dumping.
For one thing, there is no value in backing up unused disk blocks. If the dumping
program can obtain access to the free-block data structure, it can avoid dumping
unused blocks. However, skipping unused blocks requires writing the number of
each block in front of the block (or the equivalent), since it is no longer true that
block k on the backup was block k on the disk.
A second concern is dumping bad blocks. It is nearly impossible to manufacture large disks without any defects. Some bad blocks are always present. Sometimes when a low-level format is done, the bad blocks are detected, marked as bad,
and replaced by spare blocks reserved at the end of each track for just such emergencies. In many cases, the disk controller handles bad-block replacement transparently without the operating system even knowing about it.
However, sometimes blocks go bad after formatting, in which case the operating system will eventually detect them. Usually, it solves the problem by creating a
‘‘file’’ consisting of all the bad blocks—just to make sure they never appear in the
free-block pool and are never assigned. Needless to say, this file is completely
unreadable.
If all bad blocks are remapped by the disk controller and hidden from the operating system as just described, physical dumping works fine. On the other hand, if
they are visible to the operating system and maintained in one or more bad-block
files or bitmaps, it is absolutely essential that the physical dumping program get
access to this information and avoid dumping them to prevent endless disk read
errors while trying to back up the bad-block file.
Windows systems have paging and hibernation files that are not needed in the
event of a restore and should not be backed up in the first place. Specific systems
310
FILE SYSTEMS
CHAP. 4
may also have other internal files that should not be backed up, so the dumping
program needs to be aware of them.
The main advantages of physical dumping are simplicity and great speed (basically, it can run at the speed of the disk). The main disadvantages are the inability
to skip selected directories, make incremental dumps, and restore individual files
upon request. For these reasons, most installations make logical dumps.
A logical dump starts at one or more specified directories and recursively
dumps all files and directories found there that have changed since some given
base date (e.g., the last backup for an incremental dump or system installation for a
full dump). Thus, in a logical dump, the dump disk gets a series of carefully identified directories and files, which makes it easy to restore a specific file or directory
upon request.
Since logical dumping is the most common form, let us examine a common
algorithm in detail using the example of Fig. 4-27 to guide us. Most UNIX systems
use this algorithm. In the figure, we see a file tree with directories (squares) and
files (circles). The shaded items have been modified since the base date and thus
need to be dumped. The unshaded ones do not need to be dumped.
1
2
3
4
Directory
that has not
changed
5
16
18
6
17
19
7
8
Root directory
10
9
11
12
13
27
20
22
14
21
23
15
File that
has changed
24
29
28
25
30
31
26
32
File that has
not changed
Figure 4-27. A file system to be dumped. The squares are directories and the circles are files. The shaded items have been modified since the last dump. Each
directory and file is labeled by its i-node number.
This algorithm also dumps all directories (even unmodified ones) that lie on
the path to a modified file or directory for two reasons. The first reason is to make
it possible to restore the dumped files and directories to a fresh file system on a different computer. In this way, the dump and restore programs can be used to transport entire file systems between computers.
SEC. 4.4
FILE-SYSTEM MANAGEMENT AND OPTIMIZATION
311
The second reason for dumping unmodified directories above modified files is
to make it possible to incrementally restore a single file (possibly to handle recovery from user mistakes rather than system failure). Suppose that a full file-system
dump is done Sunday evening and an incremental dump is done on Monday
evening. On Tuesday, the directory /usr/jhs/proj/nr3 is removed, along with all the
directories and files under it. On Wednesday morning bright and early, suppose the
user wants to restore the file /usr/jhs/proj/nr3/plans/summary. However, it is not
possible to just restore the file summary because there is no place to put it. The
directories nr3 and plans must be restored first. To get their owners, modes, times,
and whatever, correct, these directories must be present on the dump disk even
though they themselves were not modified since the previous full dump.
The dump algorithm maintains a bitmap indexed by i-node number with several bits per i-node. Bits will be set and cleared in this map as the algorithm proceeds. The algorithm operates in four phases. Phase 1 begins at the starting directory (the root in this example) and examines all the entries in it. For each modified
file, its i-node is marked in the bitmap. Each directory is also marked (whether or
not it has been modified) and then recursively inspected.
At the end of phase 1, all modified files and all directories have been marked in
the bitmap, as shown (by shading) in Fig. 4-28(a). Phase 2 conceptually recursively walks the tree again, unmarking any directories that have no modified files
or directories in them or under them. This phase leaves the bitmap as shown in
Fig. 4-28(b). Note that directories 10, 11, 14, 27, 29, and 30 are now unmarked
because they contain nothing under them that has been modified. They will not be
dumped. By way of contrast, directories 5 and 6 will be dumped even though they
themselves have not been modified because they will be needed to restore today’s
changes to a fresh machine. For efficiency, phases 1 and 2 can be combined in one
tree walk.
(a)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
(b)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
(c)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
(d)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
Figure 4-28. Bitmaps used by the logical dumping algorithm.
At this point, it is known which directories and files must be dumped. These
are the ones that are marked in Fig. 4-28(b). Phase 3 then consists of scanning the
i-nodes in numerical order and dumping all the directories that are marked for
312
FILE SYSTEMS
CHAP. 4
dumping. These are shown in Fig. 4-28(c). Each directory is prefixed by the directory’s attributes (owner, times, etc.) so that they can be restored. Finally, in phase
4, the files marked in Fig. 4-28(d) are also dumped, again prefixed by their attributes. This completes the dump.
Restoring a file system from the dump disk is straightforward. To start with,
an empty file system is created on the disk. Then the most recent full dump is
restored. Since the directories appear first on the dump disk, they are all restored
first, giving a skeleton of the file system. Then the files themselves are restored.
This process is then repeated with the first incremental dump made after the full
dump, then the next one, and so on.
Although logical dumping is straightforward, there are a few tricky issues. For
one, since the free block list is not a file, it is not dumped and hence it must be reconstructed from scratch after all the dumps have been restored. Doing so is
always possible since the set of free blocks is just the complement of the set of
blocks contained in all the files combined.
Another issue is links. If a file is linked to two or more directories, it is important that the file is restored only one time and that all the directories that are supposed to point to it do so.
Still another issue is the fact that UNIX files may contain holes. It is permitted
to open a file, write a few bytes, then seek to a distant file offset and write a few
more bytes. The blocks in between are not part of the file and should not be
dumped and must not be restored. Core dump files often have a hole of hundreds
of megabytes between the data segment and the stack. If not handled properly,
each restored core file will fill this area with zeros and thus be the same size as the
virtual address space (e.g., 232 bytes, or worse yet, 264 bytes).
Finally, special files, named pipes, and the like (anything that is not a real file)
should never be dumped, no matter in which directory they may occur (they need
not be confined to /dev). For more information about file-system backups, see
Zwicky (1991) and Chervenak et al., (1998).
4.4.3 File-System Consistency
Another area where reliability is an issue is file-system consistency. Many file
systems read blocks, modify them, and write them out later. If the system crashes
before all the modified blocks have been written out, the file system can be left in
an inconsistent state. This problem is especially critical if some of the blocks that
have not been written out are i-node blocks, directory blocks, or blocks containing
the free list.
To deal with inconsistent file systems, most computers have a utility program
that checks file-system consistency. For example, UNIX has fsck; Windows has sfc
(and others). This utility can be run whenever the system is booted, especially
after a crash. The description below tells how fsck works. Sfc is somewhat different because it works on a different file system, but the general principle of using
SEC. 4.4
FILE-SYSTEM MANAGEMENT AND OPTIMIZATION
313
the file system’s inherent redundancy to repair it is still a valid one. All file-system
checkers verify each file system (disk partition) independently of the other ones. It
is also important to note that some file systems, such the journaling file systems
discussed earlier, are designed such that they do not require administrators to run a
separate file system consistency checker after a crash, because they can handle
most inconsistencies themselves.
Two kinds of consistency checks can be made: blocks and files. To check for
block consistency, the program builds two tables, each one containing a counter for
each block, initially set to 0. The counters in the first table keep track of how
many times each block is present in a file; the counters in the second table record
how often each block is present in the free list (or the bitmap of free blocks).
The program then reads all the i-nodes using a raw device, which ignores the
file structure and just returns all the disk blocks starting at 0. Starting from an inode, it is possible to build a list of all the block numbers used in the corresponding file. As each block number is read, its counter in the first table is incremented.
The program then examines the free list or bitmap to find all the blocks that are not
in use. Each occurrence of a block in the free list results in its counter in the second table being incremented.
If the file system is consistent, each block will have a 1 either in the first table
or in the second table, as illustrated in Fig. 4-29(a). However, as a result of a
crash, the tables might look like Fig. 4-29(b), in which block 2 does not occur in
either table. It will be reported as being a missing block. While missing blocks
do no real harm, they waste space and thus reduce the capacity of the disk. The solution to missing blocks is straightforward: the file system checker just adds them to
the free list.
Block number
Block number
0 1 2 3 4 5 6 7 8 9 101112131415
0 1 2 3 4 5 6 7 8 9 101112131415
1 1 0 1 0 1 1 1 1 0 0 1 1 1 0 0 Blocks in use
1 1 0 1 0 1 1 1 1 0 0 1 1 1 0 0 Blocks in use
0 0 1 0 1 0 0 0 0 1 1 0 0 0 1 1 Free blocks
0 0 0 0 1 0 0 0 0 1 1 0 0 0 1 1 Free blocks
(a)
(b)
Block number
0 1 2 3 4 5 6 7 8 9 101112131415
Block number
0 1 2 3 4 5 6 7 8 9 101112131415
1 1 0 1 0 1 1 1 1 0 0 1 1 1 0 0 Blocks in use
1 1 0 1 0 2 1 1 1 0 0 1 1 1 0 0 Blocks in use
0 0 1 0 2 0 0 0 0 1 1 0 0 0 1 1 Free blocks
0 0 1 0 1 0 0 0 0 1 1 0 0 0 1 1 Free blocks
(c)
(d)
Figure 4-29. File-system states. (a) Consistent. (b) Missing block. (c) Duplicate block in free list. (d) Duplicate data block.
Another situation that might occur is that of Fig. 4-29(c). Here we see a block,
number 4, that occurs twice in the free list. (Duplicates can occur only if the free
314
FILE SYSTEMS
CHAP. 4
list is really a list; with a bitmap it is impossible.) The solution here is also simple:
rebuild the free list.
The worst thing that can happen is that the same data block is present in two or
more files, as shown in Fig. 4-29(d) with block 5. If either of these files is
removed, block 5 will be put on the free list, leading to a situation in which the
same block is both in use and free at the same time. If both files are removed, the
block will be put onto the free list twice.
The appropriate action for the file-system checker to take is to allocate a free
block, copy the contents of block 5 into it, and insert the copy into one of the files.
In this way, the information content of the files is unchanged (although almost
assuredly one is garbled), but the file-system structure is at least made consistent.
The error should be reported, to allow the user to inspect the damage.
In addition to checking to see that each block is properly accounted for, the
file-system checker also checks the directory system. It, too, uses a table of counters, but these are per file, rather than per block. It starts at the root directory and
recursively descends the tree, inspecting each directory in the file system. For
every i-node in every directory, it increments a counter for that file’s usage count.
Remember that due to hard links, a file may appear in two or more directories.
Symbolic links do not count and do not cause the counter for the target file to be
incremented.
When the checker is all done, it has a list, indexed by i-node number, telling
how many directories contain each file. It then compares these numbers with the
link counts stored in the i-nodes themselves. These counts start at 1 when a file is
created and are incremented each time a (hard) link is made to the file. In a consistent file system, both counts will agree. However, two kinds of errors can occur:
the link count in the i-node can be too high or it can be too low.
If the link count is higher than the number of directory entries, then even if all
the files are removed from the directories, the count will still be nonzero and the inode will not be removed. This error is not serious, but it wastes space on the disk
with files that are not in any directory. It should be fixed by setting the link count
in the i-node to the correct value.
The other error is potentially catastrophic. If two directory entries are linked
to a file, but the i-node says that there is only one, when either directory entry is
removed, the i-node count will go to zero. When an i-node count goes to zero, the
file system marks it as unused and releases all of its blocks. This action will result
in one of the directories now pointing to an unused i-node, whose blocks may soon
be assigned to other files. Again, the solution is just to force the link count in the inode to the actual number of directory entries.
These two operations, checking blocks and checking directories, are often integrated for efficiency reasons (i.e., only one pass over the i-nodes is required).
Other checks are also possible. For example, directories have a definite format,
with i-node numbers and ASCII names. If an i-node number is larger than the
number of i-nodes on the disk, the directory has been damaged.
SEC. 4.4
FILE-SYSTEM MANAGEMENT AND OPTIMIZATION
315
Furthermore, each i-node has a mode, some of which are legal but strange,
such as 007, which allows the owner and his group no access at all, but allows outsiders to read, write, and execute the file. It might be useful to at least report files
that give outsiders more rights than the owner. Directories with more than, say,
1000 entries are also suspicious. Files located in user directories, but which are
owned by the superuser and have the SETUID bit on, are potential security problems because such files acquire the powers of the superuser when executed by any
user. With a little effort, one can put together a fairly long list of technically legal
but still peculiar situations that might be worth reporting.
The previous paragraphs have discussed the problem of protecting the user
against crashes. Some file systems also worry about protecting the user against
himself. If the user intends to type
rm *.o
to remove all the files ending with .o (compiler-generated object files), but accidentally types
rm * .o
(note the space after the asterisk), rm will remove all the files in the current directory and then complain that it cannot find .o. This is a catastrophic error from
which recovery is virtually impossible without heroic efforts and special software.
In Windows, files that are removed are placed in the recycle bin (a special directory), from which they can later be retrieved if need be. Of course, no storage is
reclaimed until they are actually deleted from this directory.
4.4.4 File-System Performance
Access to hard disk is much slower than access to flash storage and much
slower still than access to memory. Reading a 32-bit memory word might take 10
nsec. Reading from a hard disk might proceed at 100 MB/sec, which is four times
slower per 32-bit word, but to this must be added 5–10 msec to seek to the track
and then wait for the desired sector to arrive under the read head. If only a single
word is needed, the memory access is on the order of a million times as fast as disk
access. As a result of this difference in access time, many file systems have been
designed with various optimizations to improve performance. In this section, we
will cover three of them.
Caching
The most common technique used to reduce disk accesses is the block cache
or buffer cache. (Cache is pronounced ‘‘cash’’ and is derived from the French
cacher, meaning to hide.) In this context, a cache is a collection of blocks that logically belong on the disk but are being kept in memory for performance reasons.
316
FILE SYSTEMS
CHAP. 4
Various algorithms can be used to manage the cache, but a common one is to
check all read requests to see if the needed block is in the cache. If it is, the read
request can be satisfied without a disk access. If the block is not in the cache, it is
first read into the cache and then copied to wherever it is needed. Subsequent
requests for the same block can be satisfied from the cache.
Operation of the cache is illustrated in Fig. 4-30. Since there are many (often
thousands of) blocks in the cache, some way is needed to determine quickly if a
given block is present. The usual way is to hash the device and disk address and
look up the result in a hash table. All the blocks with the same hash value are
chained together on a linked list so that the collision chain can be followed.
Hash table
Front (LRU)
Rear (MRU)
Figure 4-30. The buffer cache data structures.
When a block has to be loaded into a full cache, some block has to be removed
(and rewritten to the disk if it has been modified since being brought in). This
situation is very much like paging, and all the usual page-replacement algorithms
described in Chap. 3, such as FIFO, second chance, and LRU, are applicable. One
pleasant difference between paging and caching is that cache references are relatively infrequent, so that it is feasible to keep all the blocks in exact LRU order
with linked lists.
In Fig. 4-30, we see that in addition to the collision chains starting at the hash
table, there is also a bidirectional list running through all the blocks in the order of
usage, with the least recently used block on the front of this list and the most
recently used block at the end. When a block is referenced, it can be removed from
its position on the bidirectional list and put at the end. In this way, exact LRU
order can be maintained.
Unfortunately, there is a catch. Now that we have a situation in which exact
LRU is possible, it turns out that LRU is undesirable. The problem has to do with
the crashes and file-system consistency discussed in the previous section. If a critical block, such as an i-node block, is read into the cache and modified, but not
rewritten to the disk, a crash will leave the file system in an inconsistent state. If
the i-node block is put at the end of the LRU chain, it may be quite a while before
it reaches the front and is rewritten to the disk.
SEC. 4.4
FILE-SYSTEM MANAGEMENT AND OPTIMIZATION
317
Furthermore, some blocks, such as i-node blocks, are rarely referenced two
times within a short interval. These considerations lead to a modified LRU scheme,
taking two factors into account:
1. Is the block likely to be needed again soon?
2. Is the block essential to the consistency of the file system?
For both questions, blocks can be divided into categories such as i-node blocks,
indirect blocks, directory blocks, full data blocks, and partially full data blocks.
Blocks that will probably not be needed again soon go on the front, rather than the
rear of the LRU list, so their buffers will be reused quickly. Blocks that might be
needed again soon, such as a partly full block that is being written, go on the end
of the list, so they will stay around for a long time.
The second question is independent of the first one. If the block is essential to
the file-system consistency (basically, everything except data blocks), and it has
been modified, it should be written to disk immediately, regardless of which end of
the LRU list it is put on. By writing critical blocks quickly, we greatly reduce the
probability that a crash will wreck the file system. While a user may be unhappy if
one of his files is ruined in a crash, he is likely to be far more unhappy if the whole
file system is lost.
Even with this measure to keep the file-system integrity intact, it is undesirable
to keep data blocks in the cache too long before writing them out. Consider the
plight of someone who is using a personal computer to write a book. Even if our
writer periodically tells the editor to write the file being edited to the disk, there is
a good chance that everything will still be in the cache and nothing on the disk. If
the system crashes, the file-system structure will not be corrupted, but a whole
day’s work will be lost.
This situation need not happen often before we have a fairly unhappy user.
Systems take two approaches to dealing with it. The UNIX way is to have a system call, sync, which forces all the modified blocks out onto the disk immediately.
When the system is started up, a program, usually called update, is started up in
the background to sit in an endless loop issuing sync calls, sleeping for 30 sec
between calls. As a result, no more than 30 seconds of work is lost due to a crash.
Although Windows now has a system call equivalent to sync, called FlushFileBuffers, in the past it did not. Instead, it had a different strategy that was in some
ways better than the UNIX approach (and in some ways worse). What it did was
to write every modified block to disk as soon as it was written to the cache. Caches
in which all modified blocks are written back to the disk immediately are called
write-through caches. They require more disk I/O than nonwrite-through caches.
The difference between these two approaches can be seen when a program
writes a 1-KB block full, one character at a time. UNIX will collect all the characters in the cache and write the block out once every 30 seconds, or whenever the
block is removed from the cache. With a write-through cache, there is a disk access
318
FILE SYSTEMS
CHAP. 4
for every character written. Of course, most programs do internal buffering, so
they normally write not a character, but a line or a larger unit on each write system
call.
A consequence of this difference in caching strategy is that just removing a
disk from a UNIX system without doing a sync will almost always result in lost
data, and frequently in a corrupted file system as well. With write-through caching,
no problem arises. These differing strategies were chosen because UNIX was developed in an environment in which all disks were hard disks and not removable,
whereas the first Windows file system was inherited from MS-DOS, which started
out in the floppy-disk world. As hard disks became the norm, the UNIX approach,
with its better efficiency (but worse reliability), became the norm, and it is also
used now on Windows for hard disks. However, NTFS takes other measures (e.g.,
journaling) to improve reliability, as discussed earlier.
At this point, it is worth discussing the relationship between the buffer cache
and the page cache. Conceptually they are different in that a page cache caches
pages of files to optimize file I/O, while a buffer cache simply caches disk blocks.
The buffer cache, which predates the page cache, really behaves like disk, except
that the reads and writes access memory. The reason people added a page cache
was that it seemed a good idea to move the cache higher up in the stack, so file requests could be served without going through the file system code and all its complexities. Phrased differently: files are in the page cache and disk blocks in the
buffer cache. In addition, a cache at a higher level without need for the file system
made it easier to integrate it with the memory management subsystem—as befitting a component called page cache. However, it has probably not escaped your
notice that the files in the page cache are typically on disk also, so that their data
are now in both of the caches.
Some operating systems therefore integrate the buffer cache with the page
cache. This is especially attractive when memory-mapped files are supported. If a
file is mapped onto memory, then some of its pages may be in memory because
they were demand paged in. Such pages are hardly different from file blocks in the
buffer cache. In this case, they can be treated the same way, with a single cache for
both file blocks and pages. Even if the functions are still distinct, they point to the
same data. For instance, as most data has both a file and a block representation, the
buffer cache simply point into the page cache—leaving only one instance of the
data cached in memory.
Block Read Ahead
A second technique for improving perceived file-system performance is to try
to get blocks into the cache before they are needed to increase the hit rate. In particular, many files are read sequentially. When the file system is asked to produce
block k in a file, it does that, but when it is finished, it makes a sneaky check in the
cache to see if block k + 1 is already there. If it is not, it schedules a read for block
SEC. 4.4
FILE-SYSTEM MANAGEMENT AND OPTIMIZATION
319
k + 1 in the hope that when it is needed, it will have already arrived in the cache.
At the very least, it will be on the way.
Of course, this read-ahead strategy works only for files that are actually being
read sequentially. If a file is being randomly accessed, read ahead does not help.
In fact, it hurts by tying up disk bandwidth reading in useless blocks and removing
potentially useful blocks from the cache (and possibly tying up more disk bandwidth writing them back to disk if they are dirty). To see whether read ahead is
worth doing, the file system can keep track of the access patterns to each open file.
For example, a bit associated with each file can keep track of whether the file is in
‘‘sequential-access mode’’ or ‘‘random-access mode.’’ Initially, the file is given the
benefit of the doubt and put in sequential-access mode. However, whenever a seek
is done, the bit is cleared. If sequential reads start happening again, the bit is set
once again. In this way, the file system can make a reasonable guess about whether it should read ahead or not. If it gets it wrong once in a while, it is not a disaster, just a little bit of wasted disk bandwidth.
Reducing Disk-Arm Motion
Caching and read ahead are not the only ways to increase file-system performance. Another important technique for hard disks is to reduce the amount of
disk-arm motion by putting blocks that are likely to be accessed in sequence close
to each other, preferably in the same cylinder. When an output file is written, the
file system has to allocate the blocks one at a time, on demand. If the free blocks
are recorded in a bitmap, and the whole bitmap is in main memory, it is easy
enough to choose a free block as close as possible to the previous block. With a
free list, part of which is on disk, it is much harder to allocate blocks close together.
However, even with a free list, some block clustering can be done. The trick is
to keep track of disk storage not in blocks, but in groups of consecutive blocks. If
all sectors consist of 512 bytes, the system could use 1-KB blocks (2 sectors) but
allocate disk storage in units of 2 blocks (4 sectors). This is not the same as having
2-KB disk blocks, since the cache would still use 1-KB blocks and disk transfers
would still be 1 KB, but reading a file sequentially on an otherwise idle system
would reduce the number of seeks by a factor of two, considerably improving performance. A variation on the same theme is to take account of rotational positioning. When allocating blocks, the system attempts to place consecutive blocks
in a file in the same cylinder.
Another performance bottleneck in systems that use i-nodes or anything like
them is that reading even a short file requires two disk accesses: one for the i-node
and one for the block. In many file systems, the i-node placement is like the one
shown in Fig. 4-31(a). Here all the i-nodes are near the start of the disk, so the
average distance between an i-node and its blocks will be half the number of cylinders, requiring long seeks. This is clearly inefficient and needs to be improved.
320
FILE SYSTEMS
CHAP. 4
I-nodes are
located near
the start
of the disk
Disk is divided into
cylinder groups, each
with its own i-nodes
Cylinder group
(a)
(b)
Figure 4-31. (a) I-nodes placed at the start of the disk. (b) Disk divided into cylinder groups, each with its own blocks and i-nodes.
One easy performance improvement is to put the i-nodes in the middle of the
disk, rather than at the start, thus reducing the average seek between the i-node and
the first block by a factor of two. Another idea, shown in Fig. 4-31(b), is to divide
the disk into cylinder groups, each with its own i-nodes, blocks, and free list
(McKusick et al., 1984). When creating a new file, any i-node can be chosen, but
an attempt is made to find a block in the same cylinder group as the i-node. If
none is available, then a block in a nearby cylinder group is used.
Of course, disk-arm movement and rotation time are relevant only if the disk
has them and are not relevant for SSDs, which have no moving parts whatsoever.
For these drives, built on the same technology as flash cards, random (read)
accesses are just as fast as sequential ones and many of the problems of traditional
disks go away (only for new ones emerge).
4.4.5 Defragmenting Disks
When the operating system is initially installed, the programs and files it needs
are installed consecutively starting at the beginning of the disk, each one directly
following the previous one. All free disk space is in a single contiguous unit following the installed files. However, as time goes on, files are created and removed
and typically the disk becomes badly fragmented, with files and holes all over the
place. As a consequence, when a new file is created, the blocks used for it may be
spread all over the disk, giving poor performance.
The performance can be restored by moving files around to make them contiguous and to put all (or at least most) of the free space in one or more large contiguous regions on the disk. Windows has a program, defrag, that does precisely this.
Windows users should run it regularly, except on SSDs.
SEC. 4.4
FILE-SYSTEM MANAGEMENT AND OPTIMIZATION
321
Defragmentation works better on file systems that have a lot of free space in a
contiguous region at the end of the partition. This space allows the defragmentation
program to select fragmented files near the start of the partition and copy all their
blocks to the free space. Doing so frees up a contiguous block of space near the
start of the partition into which the original or other files can be placed contiguously. The process can then be repeated with the next chunk of disk space, etc.
Some files cannot be moved, including the paging file, the hibernation file, and
the journaling log, because the administration that would be required to do this is
more trouble than it is worth. In some systems, these are fixed-size contiguous
areas anyway, so they do not have to be defragmented. The one time when their
lack of mobility is a problem is when they happen to be near the end of the partition and the user wants to reduce the partition size. The only way to solve this
problem is to remove them altogether, resize the partition, and then recreate them
afterward.
Linux file systems (especially ext3 and ext4) generally suffer less from defragmentation than Windows systems due to the way disk blocks are selected, so manual defragmentation is rarely required. Also, SSDs do not suffer from fragmentation at all. In fact, defragmenting an SSD is counterproductive. Not only is
there no gain in performance, but SSDs wear out, so defragmenting them merely
shortens their lifetimes.
4.4.6 Compression and Deduplication
In the ‘‘Age of Data,’’ people tend to have, well, a lot of data. All these data
must find a home on a storage device and often that home fills up quickly with cat
pictures, cat videos, and other essential information. Of course, we can always buy
a new and bigger SSD, but it would be nice if we could prevent it from filling up
quite so quickly.
The simplest technique to use scarce storage space more efficiently is compression. Besides manually compressing files or folders, we can use a file system
that compresses specific folders or even all data automatically. File systems such as
NTFS (on Windows), Btrfs (Linux), and ZFS (on a variety of operating systems)
all offer compression as an option. The compression algorithms commonly look
for repeating sequences of data which they then encode efficiently. For instance,
when writing file data they may discover that the 133 bytes at offset 1737 in the
file are the same as the 133 bytes at offset 1500, so instead of writing the same
bytes again, they insert a marker (237,133)—indicating that these 133 bytes can be
found at a distance of 237 before the current offset.
Besides eliminating redundancy within a single file, several popular file systems also remove redundancy across files. On systems that store data from many
users, for instance in a cloud or server environment, it is common to find files that
contain the same data, as multiple users store the same documents, binaries, or
videos. Such data duplication is even more pronounced in backup storage. If users
322
FILE SYSTEMS
CHAP. 4
back up all their important files every week, each new backup probably contains
(mostly) the same data.
Rather than storing the same data multiple times, several file systems implement deduplication to eliminate duplicate copies—exactly like the deduplication
of pages in the memory subsystem that we discussed in the previous chapter. This
is a very common phenomenon in operating systems: a technique (in this case
deduplication) that is a good idea in one subsystem, is often a good idea in other
subsystems also. Here we discuss deduplication in file systems, but the technique
is also used in networking to prevent the same data from being sent over the network multiple times.
File system deduplication is possible at the granularity of files, portions of
files, or even individual disk blocks. Nowadays, many file systems perform deduplication on fixed-size chunks of, say, 128 KB. When the deduplication procedure
detects that two files contain chunks that are exactly the same, it will keep only a
single physical copy that is shared by both files. Of course, as soon as the chunk in
one of the files is overwritten, a unique copy must be made so that the changes do
not affect the other file.
Deduplication can be done inline or post-process. With inline deduplication,
the file system calculates a hash for every chunk that it is about to write and compares it to the hashes of existing chunks. If the chunk is already present, it will
refrain from actually writing out the data and instead add a reference to the existing
chunk. Of course, the additional calculations take time and slow down the write. In
contrast, post-process deduplication always writes out the data and performs the
hashing and comparisons in the background, without slowing down the process’
file operations. Which method is better is debated almost as hotly as which editor
is best, Emacs or Vi (even though the answer to that question is, of course, Emacs).
As the astute reader may have noticed, there is a problem with the use of
hashes to determine chunk equivalence: even if it happens rarely, the pigeonhole
principle says that chunks with different content may have the same hash. Some
implementations of deduplication gloss over this little inconvenience and accept
the (very low) probability of getting things wrong, but there also exist solutions
that verify whether the chunks are truly equivalent before deduplicating them.
4.4.7 Secure File Deletion and Disk Encryption
However sophisticated the access restrictions at the level of the operating system, the physical bits on the hard disk or SSDs can always be read back by taking
out the storage device and reading them back in another machine. This has many
implications. For instance, the operating system may ‘‘delete’’ a file by removing it
from the directories and freeing up the i-node for reuse, but that does not remove
the content of the file on disk. Thus, an attacker can simply read the raw disk
blocks to bypass all file system permissions, no matter how restrictive they are.
SEC. 4.4
FILE-SYSTEM MANAGEMENT AND OPTIMIZATION
323
In fact, securely deleting data on disk is not easy. If the disk is old and not
needed any more but the data must not fall into the wrong hands under any conditions, the best approach is to get a large flowerpot. Put in some thermite, put the
disk in, and cover it with more thermite. Then light it and watch it burn nicely at
2500°C. Recovery will be impossible, even for a pro. If you are unfamiliar with
the properties of thermite, it is strongly recommended that you do not try this at
home.
However, if you want to reuse the disk, this technique is clearly not appropriate. Even if you overwrite the original content with zeros, it may not be enough.
On some hard disks, data stored on the disk leave magnetic traces in areas close to
the actual tracks. So even if the normal content in the tracks is zeroed out, a highly
motivated and sophisticated attacker (such as a government intelligence agency)
could still recover the original content by carefully inspecting the adjacent areas. In
addition, there may be copies of the file in unexpected places on the disk (for instance, as a backup or in a cache), and these need to be wiped also. SSDs have
even worse problems, as the file system has no control over what flash blocks are
overwritten and when, since this is determined by the FTL. Usually by overwriting
a disk with three to seven passes, alternating zeros and random numbers, will
securely erase it though. There is software available to do this.
One way to make it impossible to recover data from disk, deleted or not, is by
encrypting everything that is on the disk. Full disk encryption is available on all
modern operating systems. As long as you do not write the password on a Post-It
note stuck somewhere on your computer, full disk encryption with a powerful
encryption algorithm will keep your data safe even if the disk falls in the hands of
the baddies.
Full disk encryption is sometimes also provided by the storage devices themselves in the form of Self-Encrypting Drives (SEDs) with onboard cryptographic
capabilities to do the encryption and decryption, leading to a performance boost as
the cryptographic calculations are offloaded from the CPU. Unfortunately,
researchers found that many SEDs have critical security weaknesses due to specification, design, and implementation issues (Meijer and Van Gastel, 2019).
As an example of full disk encryption, Windows makes use of the capabilities
of such SEDs if they are present. If not, it takes care of the encryption itself, using
a secret key, the volume master key, in a standard encryption algorithm called
Advanced Encryption Standard (AES). Full disk encryption on Windows was designed to be as unobtrusive as possible and many users are blissfully unaware that
their data are encrypted on disk. The volume master key used to encrypt or decrypt
the data on regular (i.e., non SED) storage devices can be obtained by decrypting
the (itself encrypted) key either with the user password or with the recovery key
(that was automatically generated the first time the file system was encrypted), or
by extracting the key from a special-purpose cryptoprocessor known as the Trusted
Platform Module, or TPM. Either way, once it has the key, Windows can encrypt or
decrypt the disk data as required.
324
FILE SYSTEMS
CHAP. 4
4.5 EXAMPLE FILE SYSTEMS
In the following sections, we will discuss several example file systems, ranging
from quite simple to more sophisticated. Since modern UNIX file systems and
Windows’s native file system are covered in the chapter on UNIX (Chap. 10) and
the chapter on Windows (Chap. 11), we will not cover those systems here. We
will, however, examine their predecessors below. As we have mentioned before,
variants of the MS-DOS file system are still in use in digital cameras, portable
music players, electronic picture frames, USB sticks, and other devices, so studying them is definitely still relevant.
4.5.1 The MS-DOS File System
The MS-DOS file system is the one the first IBM PCs came with. It was the
main file system up through Windows 98 and Windows ME. It is still supported
on Windows 10 and Windows 11. However, it and an extension of it (FAT-32)
have become widely used for many embedded systems. Most digital cameras use
it. Many MP3 players use it exclusively. Electronic picture frames use it. Some
memory cards use it. Many other simple devices that store music, images, and so
on still use it. It is still the preferred file system for disks and other devices that
need to be read by both Windows and MacOS. Thus, the number of electronic
devices using the MS-DOS file system is vastly larger now than at any time in the
past, and certainly much larger than the number using the more modern NTFS file
system. For that reason alone, it is worth looking at in some detail.
To read a file, an MS-DOS program must first make an open system call to get
a handle for it. The open system call specifies a path, which may be either absolute
or relative to the current working directory. The path is looked up component by
component until the final directory is located and read into memory. It is then
searched for the file to be opened.
Although MS-DOS directories are variable sized, they use a fixed-size 32-byte
directory entry. The format of an MS-DOS directory entry is shown in Fig. 4-32. It
contains the file name, attributes, creation date and time, starting block, and exact
file size. File names shorter than 8 + 3 characters are left justified and padded with
spaces on the right, in each field separately. The Attributes field is new and contains bits to indicate that a file is read-only, needs to be archived, is hidden, or is a
system file. Read-only files cannot be written. This is to protect them from accidental damage. The archived bit has no actual operating system function (i.e., MSDOS does not examine or set it). The intention is to allow user-level archive programs to clear it upon archiving a file and to have other programs set it when modifying a file. In this way, a backup program can just examine this attribute bit on
every file to see which files to back up. The hidden bit can be set to prevent a file
from appearing in directory listings. Its main use is to avoid confusing novice users
with files they might not understand. Finally, the system bit also hides files. In
SEC. 4.5
325
EXAMPLE FILE SYSTEMS
addition, system files cannot accidentally be deleted using the del command. The
main components of MS-DOS have this bit set.
Bytes
8
3
1
10
2
2
File name
Extension Attributes
2
4
Size
Reserved
Time Date
First
block
number
Figure 4-32. The MS-DOS directory entry.
The directory entry also contains the date and time the file was created or last
modified. The time is accurate only to ±2 sec because it is stored in a 2-byte field,
which can store only 65,536 unique values (a day contains 86,400 seconds). The
time field is subdivided into seconds (5 bits), minutes (6 bits), and hours (5 bits).
The date counts in days using three subfields: day (5 bits), month (4 bits), and year
< 1980 (7 bits). With a 7-bit number for the year and time beginning in 1980, the
highest expressible year is 2107. Thus. MS-DOS has a built-in Y2108 problem.
To avoid catastrophe, MS-DOS users should begin with Y2108 compliance as
early as possible. If MS-DOS had used the combined date and time fields as a
32-bit seconds counter, it could have represented every second exactly and delayed
the catastrophe until 2116.
MS-DOS stores the file size as a 32-bit number, so in theory files can be as
large as 4 GB. However, other limits (described below) restrict the maximum file
size to 2 GB or less. A surprisingly large part of the entry (10 bytes) is unused.
MS-DOS keeps track of file blocks via a file allocation table in main memory.
The directory entry contains the number of the first file block. This number is used
as an index into a 64K entry FAT in main memory. By following the chain, all the
blocks can be found. The operation of the FAT is illustrated in Fig. 4-14.
The FAT file system comes in three versions: FAT-12, FAT-16, and FAT-32,
depending on how many bits a disk address contains. Actually, FAT-32 is something of a misnomer, since only the low-order 28 bits of the disk addresses are
used. It should have been called FAT-28, but powers of two sound so much neater.
Another variant of the FAT file system is exFAT, which Microsoft introduced
for large removable devices. Apple licensed exFAT, so that there is one modern file
system that can be used to transfer files both ways between Windows and MacOS
computers. Since exFAT is proprietary and Microsoft has not released the specification, we will not discuss it further here.
For all FATs, the disk block can be set to some multiple of 512 bytes (possibly
different for each partition), with the set of allowed block sizes (called cluster
sizes by Microsoft) being different for each variant. The first version of MS-DOS
used FAT-12 with 512-byte blocks, giving a maximum partition size of 212 × 512
326
FILE SYSTEMS
CHAP. 4
bytes (actually only 4086 × 512 bytes because 10 of the disk addresses were used
as special markers, such as end of file, bad block, etc.). With these parameters, the
maximum disk partition size was about 2 MB and the size of the FAT table in
memory was 4096 entries of 2 bytes each. Using a 12-bit table entry would have
been too slow.
This system worked well for floppy disks, but when hard disks came out, it
became a problem. Microsoft solved the problem by allowing additional block
sizes of 1 KB, 2 KB, and 4 KB. This change preserved the structure and size of
the FAT-12 table, but allowed disk partitions of up to 16 MB.
Since MS-DOS supported four disk partitions per disk drive, the new FAT-12
file system worked up to 64-MB disks. Beyond that, something had to give. What
happened was the introduction of FAT-16, with 16-bit disk pointers. Additionally,
block sizes of 8 KB, 16 KB, and 32 KB were permitted. (32,768 is the largest
power of two that can be represented in 16 bits.) The FAT-16 table now occupied
128 KB of main memory all the time, but with the larger memories by then available, it was widely used and rapidly replaced the FAT-12 file system. The largest
disk partition that can be supported by FAT-16 is 2 GB (64K entries of 32 KB
each) and the largest disk, 8 GB, namely four partitions of 2 GB each. For quite a
while, that was good enough.
But not forever. For business letters, this limit is not a problem, but for storing
digital video using the DV standard, a 2-GB file holds just over 9 minutes of video.
As a consequence of the fact that a PC disk can support only four partitions, the
largest video that can be stored on a disk is about 38 minutes, no matter how large
the disk is. This limit also means that the largest video that can be edited on line is
less than 19 minutes, since both input and output files are needed.
Starting with the second release of Windows 95, the FAT-32 file system, with
its 28-bit disk addresses, was introduced and the version of MS-DOS underlying
Windows 95 was adapted to support FAT-32. In this system, partitions could theoretically be 228 × 215 bytes, but they are actually limited to 2 TB (2048 GB) because internally the system keeps track of partition sizes in 512-byte sectors using
a 32-bit number, and 29 × 232 is 2 TB. The maximum partition size for various
block sizes and all three FAT types is shown in Fig. 4-33.
In addition to supporting larger disks, the FAT-32 file system has two other
advantages over FAT-16. First, an 8-GB disk using FAT-32 can be a single partition. Using FAT-16 it has to be four partitions, which appears to the Windows user
as the C:, D:, E:, and F: logical disk drives. It is up to the user to decide which file
to place on which drive and keep track of what is where.
The other advantage of FAT-32 over FAT-16 is that for a given size disk partition, a smaller block size can be used. For example, for a 2-GB disk partition,
FAT-16 must use 32-KB blocks; otherwise with only 64K available disk addresses,
it cannot cover the whole partition. In contrast, FAT-32 can use, for example,
4-KB blocks for a 2-GB disk partition. The advantage of the smaller block size is
that most files are much shorter than 32 KB. If the block size is 32 KB, a file of 10
SEC. 4.5
327
EXAMPLE FILE SYSTEMS
Block size
FAT-12
0.5 KB
2 MB
1 KB
4 MB
FAT-16
FAT-32
2 KB
8 MB
128 MB
4 KB
16 MB
256 MB
1 TB
8 KB
512 MB
2 TB
16 KB
1024 MB
2 TB
32 KB
2048 MB
2 TB
Figure 4-33. Maximum partition size for different block sizes. The empty boxes
represent forbidden combinations.
bytes ties up 32 KB of disk space. If the average file is, say, 8 KB, then with a
32-KB block, three quarters of the disk will be wasted, not a terribly efficient way
to use the disk. With an 8-KB file and a 4-KB block, there is no disk wastage, but
the price paid is more RAM eaten up by the FAT. With a 4-KB block and a 2-GB
disk partition, there are 512K blocks, so the FAT must have 512K entries in memory (occupying 2 MB of RAM).
MS-DOS uses the FAT to keep track of free disk blocks. Any block that is not
currently allocated is marked with a special code. When MS-DOS needs a new
disk block, it searches the FAT for an entry containing this code. Thus no bitmap or
free list is required.
4.5.2 The UNIX V7 File System
Even early versions of UNIX had a fairly sophisticated multiuser file system
since it was derived from MULTICS. Below we will discuss the V7 file system,
the one for the PDP-11 that made UNIX famous. We will examine a modern
UNIX file system in the context of Linux in Chap. 10.
The file system is in the form of a tree starting at the root directory, with the
addition of links, forming a directed acyclic graph. File names can be up to 14
characters and can contain any ASCII characters except / (because that is the separator between components in a path) and NUL (because that is used to pad out
names shorter than 14 characters). NUL has the numerical value of 0.
A UNIX directory entry contains one entry for each file in that directory. Each
entry is extremely simple because UNIX uses the i-node scheme illustrated in
Fig. 4-15. A directory entry contains only two fields: the file name (14 bytes) and
the number of the i-node for that file (2 bytes), as shown in Fig. 4-34. These
parameters limit the number of files per file system to 64K.
Like the i-node of Fig. 4-15, the UNIX i-node contains some attributes. The
attributes contain the file size, three times (creation, last access, and last modification), owner, group, protection information, and a count of the number of directory
328
FILE SYSTEMS
Bytes
2
CHAP. 4
14
File name
I-node
number
Figure 4-34. A UNIX V7 directory entry.
entries that point to the i-node. The latter field is needed due to links. Whenever a
new link is made to an i-node, the count in the i-node is increased. When a link is
removed, the count is decremented. When it gets to 0, the i-node is reclaimed and
the disk blocks are put back in the free list.
Keeping track of disk blocks is done using a generalization of Fig. 4-15 in
order to handle very large files. The first 10 disk addresses are stored in the i-node
itself, so for small files, all the necessary information is right in the i-node, which
is fetched from disk to main memory when the file is opened. For somewhat larger
files, one of the addresses in the i-node is the address of a disk block called a single indirect block. This block contains additional disk addresses. If this still is
not enough, another address in the i-node, called a double indirect block, contains
the address of a block that contains a list of single indirect blocks. Each of these
single indirect blocks points to a few hundred data blocks. If even this is not
enough, a triple indirect block can also be used. The complete picture is given in
Fig. 4-35.
When a file is opened, the file system must take the file name supplied and
locate its disk blocks. Let us consider how the path name /usr/ast/mbox is looked
up. We will use UNIX as an example, but the algorithm is basically the same for
all hierarchical directory systems. First the file system locates the root directory.
In UNIX its i-node is located at a fixed place on the disk. From this i-node, it
locates the root directory, which can be anywhere on the disk, but say block 1.
After that it reads the root directory and looks up the first component of the
path, usr, in the root directory to find the i-node number of the file /usr. Locating
an i-node from its number is straightforward, since each one has a fixed location on
the disk. From this i-node, the system locates the directory for /usr and looks up
the next component, ast, in it. When it has found the entry for ast, it has the i-node
for the directory /usr/ast. From this i-node it can find the directory itself and look
up mbox. The i-node for this file is then read into memory and kept there until the
file is closed. The lookup process is illustrated in Fig. 4-36.
Relative path names are looked up the same way as absolute ones, only starting
from the working directory instead of from the root directory. Every directory has
entries for . and .. which are put there when the directory is created. The entry .
has the i-node number for the current directory, and the entry for .. has the i-node
SEC. 4.5
329
EXAMPLE FILE SYSTEMS
I-node
Disk addresses
Attributes
Single
indirect
block
Addresses of
data blocks
Double
indirect
block
Triple
indirect
block
Figure 4-35. A UNIX i-node.
Root directory
1
.
..
4
bin
1
I-node 6
is for /usr
Mode
size
times
Block 132
is /usr
directory
6
1
19
dick
I-node 26
is for
/usr/ast
Mode
size
times
Block 406
is /usr/ast
directory
26
6
64
grants
7
dev
30
erik
92
books
14
lib
51
jim
60
mbox
9
etc
26
ast
81
minix
6
usr
45
bal
17
src
8
tmp
Looking up
usr yields
i-node 6
132
I-node 6
says that
/usr is in
block 132
/usr/ast
is i-node
26
406
I-node 26
says that
/usr/ast is in
block 406
Figure 4-36. The steps in looking up /usr/ast/mbox.
/usr/ast/mbox
is i-node
60
330
FILE SYSTEMS
CHAP. 4
number for the parent directory. Thus, a procedure looking up ../chris/prog.c simply looks up .. in the working directory, finds the i-node number for the parent
directory, and searches that directory for chris. No special mechanism is needed to
handle these names. As far as the directory system is concerned, they are just
ordinary ASCII strings, just the same as any other names. The only bit of trickery
here is that .. in the root directory points to itself.
4.6 RESEARCH ON FILE SYSTEMS
File systems have always attracted more research than other parts of the operating system and that is still the case. Entire conferences such as FAST, MSST, and
NAS are devoted largely to file and storage systems.
A considerable amount of research addresses the reliability of storage and file
systems. A powerful way to guarantee reliability is to formally prove the safety of
your system even in the face of catastrophic events, such as crashes (Chen et al.,
2017). Also, with the rising popularity of SSDs as the primary storage medium, it
is interesting to look at how well they hold up in large enterprise storage systems
(Maneas et al., 2020).
As we have seen in this chapter, file systems are complex beasts and developing new file systems is not easy. Many operating systems allow file systems to be
developed in user space (e.g., the FUSE userspace filesystem framework on
Linux), but the performance is generally much lower. With new storage devices
such as low-level SSDs arriving on the market, the need for an agile storage stack
is important and research is needed to develop high-performance file systems
quickly (Miller et al., 2021). In fact, new advances in storage technology is driving
much of the research on file systems. For instance, how do we build efficient file
systems for new persistent memory (Chen et al., 2021; and Neal, 2021)? Or how
can we speed up file system checking (Domingo, 2021)? Even fragmentation creates different issues on hard disks and SSDs and requires different approaches
(Kesavan, 2019).
Storing increasing amounts of data on the same file system is challenging,
especially in mobile devices, leading to the development of new methods to compress the data without slowing down the system too much, for instance by taking
the access patterns of files into account (Ji et al., 2021). We have seen that as an
alternative to per-file or per-block compression, some file systems today support
deduplication across the entire system to prevent storing the same data twice.
Unfortunately, deduplication tends to lead to poor data locality and trying to obtain
good deduplication without performance loss due to lack of locality is difficult
(Zou, 2021). Of course, with data deduplicated all over the place, it becomes much
harder to estimate how much space if left or will be left when we delete a certain
file (Harnik, 2019).
SEC. 4.7
SUMMARY
331
4.7 SUMMARY
When seen from the outside, a file system is a collection of files and directories, plus operations on them. Files can be read and written, directories can be
created and destroyed, and files can be moved from directory to directory. Most
modern file systems support a hierarchical directory system in which directories
may have subdirectories and these may have subsubdirectories ad infinitum.
When seen from the inside, a file system looks quite different. The file system
designers have to be concerned with how storage is allocated and how the system
keeps track of which block goes with which file. Possibilities include contiguous
files, linked lists, file-allocation tables, and i-nodes. Different systems have different directory structures. Attributes can go in the directories or somewhere else
(e.g., an i-node). Disk space can be managed using free lists or bitmaps. File-system reliability is enhanced by making incremental dumps and by having a program
that can repair sick file systems. File-system performance is important and can be
enhanced in several ways, including caching, read ahead, and carefully placing the
blocks of a file close to each other. Log-structured file systems also improve performance by doing writes in large units.
Examples of file systems include ISO 9660, MS-DOS, and UNIX. These differ in many ways, including how they keep track of which blocks go with which
file, directory structure, and management of free disk space.
PROBLEMS
1. In Windows, when a user double clicks on a file listed by Windows Explorer, a program is run and given that file as a parameter. List two different ways the operating
system could know which program to run.
2. In early UNIX systems, executable files (a.out files) began with a very specific magic
number, not one chosen at random. These files began with a header, followed by the
text and data segments. Why do you think a very specific number was chosen for
executable files, whereas other file types had a more-or-less random magic number as
the first word?
3. In Fig. 4-5, one of the attributes is the record length. Why does the operating system
ever care about this?
4. Is the open system call in UNIX absolutely essential? What would the consequences
be of not having it?
5. Systems that support sequential files always have an operation to rewind files. Do systems that support random-access files need this, too?
6. Some operating systems provide a system call rename to give a file a new name. Is
there any difference at all between using this call to rename a file and just copying the
file to a new file with the new name, followed by deleting the old one?
332
FILE SYSTEMS
CHAP. 4
7. A simple operating system supports only a single directory but allows it to have arbitrarily many files with arbitrarily long file names. Can something approximating a hierarchical file system be simulated? How?
8. In UNIX and Windows, random access is done by having a special system call that
moves the ‘‘current position’’ pointer associated with a file to a given byte in the file.
Propose an alternative way to do random access without having this system call.
9. Consider the directory tree of Fig. 4-9. If /usr/jim is the working directory, what is the
absolute path name for the file whose relative path name is ../ast/x?
10. Contiguous allocation of files leads to disk fragmentation, as mentioned in the text,
because some space in the last disk block will be wasted in files whose length is not an
integral number of blocks. Is this internal fragmentation or external fragmentation?
Make an analogy with something discussed in the previous chapter.
11. Suppose a filesystem check reveals that a block has been allocated to two different
files, /home/hjb/dadjokes.txt and /etc/motd. Both are text files. The filesystem check
duplicates the block’s data and re-assigns /etc/motd to use the new block. Answer the
following questions. (i) In what realistic circumstance(s) could the data from both files
still remain correct and consistent with their original content? (ii) How might the user
investigate whether the files have been corrupted? (iii) If one or both of the files’ data
have been corrupted, what mechanisms might allow the user to recover the data?
12. One way to use contiguous allocation of the disk and not suffer from holes is to compact the disk every time a file is removed. Since all files are contiguous, copying a file
requires a seek and rotational delay to read the file, followed by the transfer at full
speed. Writing the file back requires the same work. Assuming a seek time of 5 msec,
a rotational delay of 4 msec, a transfer rate of 8 MB/sec, and an average file size of 8
KB, how long does it take to read a file into main memory and then write it back to the
disk at a new location? Using these numbers, how long would it take to compact half
of a 16-GB disk?
13. MacOS has symbolic links and also aliases. An alias is similar to a symbolic link;
however, unlike symbolic links, an alias stores additional metadata about the target file
(such as its inode number and file size) so that, if the target file is moved within the
same filesystem, accessing the alias will result in accessing the target file, as the
filesystem will search for and find the original target. How could this behavior be beneficial compared to symbolic links? How could it cause problems?
14. Following on the previous question, in earlier MacOS versions, if the target file is
moved and then another file is created with the original path of the target, the alias
would still find and use the moved target file (not the new file with the same
path/name). However, in versions of MacOS 10.2 or later, if the target file is moved
and another is created in the old location, the alias will connect to the new file. Does
this address the drawbacks from your answer to the previous question? Does it dampen
the benefits you noted?
15. Some digital consumer devices need to store data, for example as files. Name a modern
device that requires file storage and for which contiguous allocation would be a fine
idea.
CHAP. 4
PROBLEMS
333
16. Consider the i-node shown in Fig. 4-15. If it contains 10 direct addresses of 4 bytes
each and all disk blocks are 1024 KB, what is the largest possible file?
17. For a given class, the student records are stored in a file. The records are randomly
accessed and updated. Assume that each student’s record is of fixed size. Which of the
three allocation schemes (contiguous, linked and table/indexed) will be most appropriate?
18. Consider a file whose size varies between 4 KB and 4 MB during its lifetime. Which
of the three allocation schemes (contiguous, linked and table/indexed) will be most
appropriate?
19. It has been suggested that efficiency could be improved and disk space saved by storing the data of a short file within the i-node. For the i-node of Fig. 4-15, how many
bytes of data could be stored inside the i-node?
20. Two computer science students, Carolyn and Elinor, are having a discussion about inodes. Carolyn maintains that memories have gotten so large and so cheap that when a
file is opened, it is simpler and faster just to fetch a new copy of the i-node into the inode table, rather than search the entire table to see if it is already there. Elinor disagrees. Who is right?
21. Name one advantage of hard links over symbolic links and one advantage of symbolic
links over hard links.
22. Explain how hard links and soft links differ with respective to i-node allocations.
23. Consider a 4-TB disk that uses 8-KB blocks and the free-list method. How many block
addresses can be stored in one block?
24. Free disk space can be kept track of using a free list or a bitmap. Disk addresses
require D bits. For a disk with B blocks, F of which are free, state the condition under
which the free list uses less space than the bitmap. For D having the value 16 bits,
express your answer as a percentage of the disk space that must be free.
25. The beginning of a free-space bitmap looks like this after the disk partition is first formatted: 1000 0000 0000 0000 (the first block is used by the root directory). The system always searches for free blocks starting at the lowest-numbered block, so after
writing file A, which uses six blocks, the bitmap looks like this: 1111 1110 0000 0000.
Show the bitmap after each of the following additional actions:
(a) File B is written, using five blocks.
(b) File A is deleted.
(c) File C is written, using eight blocks.
(d) File B is deleted.
26. What would happen if the bitmap or free list containing the information about free disk
blocks was completely lost due to a crash? Is there any way to recover from this disaster, or is it bye-bye disk? Discuss your answers for UNIX and the FAT-16 file system
separately.
27. Oliver Owl’s night job at the university computing center is to change the tapes used
for overnight data backups. While waiting for each tape to complete, he works on writing his thesis that proves Shakespeare’s plays were written by extraterrestrial visitors.
334
FILE SYSTEMS
CHAP. 4
His text processor runs on the system being backed up since that is the only one they
have. Is there a problem with this arrangement?
28. We discussed making incremental dumps in some detail in the text. In Windows it is
easy to tell when to dump a file because every file has an archive bit. This bit is missing in UNIX. How do UNIX backup programs know which files to dump?
29. Suppose that file 21 in Fig. 4-27 was not modified since the last dump. In what way
would the four bitmaps of Fig. 4-28 be different?
30. It has been suggested that the first part of each UNIX file be kept in the same disk
block as its i-node. What good would this do?
31. Consider Fig. 4-29. Is it possible that for some particular block number the counters in
both lists have the value 2? How should this problem be corrected?
32. The performance of a file system depends upon the cache hit rate (fraction of blocks
found in the cache). If it takes 1 msec to satisfy a request from the cache, but 40 msec
to satisfy a request if a disk read is needed, give a formula for the mean time required
to satisfy a request if the hit rate is h. Plot this function for values of h varying from 0
to 1.0.
33. For an external USB hard drive attached to a computer, which is more suitable: a writethrough cache or a block cache?
34. Consider an application where students’ records are stored in a file. The application
takes a student ID as input and subsequently reads, updates, and writes the corresponding student record; this is repeated till the application quits. Would the ‘‘block readahead’’ technique be useful here?
35. Discuss the design issues involved in selecting the appropriate block size for a file system.
36. Consider a disk that has 10 data blocks starting from block 14 through 23. Let there be
2 files on the disk: f1 and f2. The directory structure lists that the first data blocks of f1
and f2 are, respectively, 22 and 16. Given the FAT table entries as below, what are the
data blocks allotted to f1 and f2?
(14,18); (15,17); (16,23); (17,21); (18,20); (19,15); (20, <1); (21, <1); (22,19); (23,14).
In the above notation, (x, y) indicates that the value stored in table entry x points to data
block y.
37. In the text, we discussed two major ways to identify file type: file extensions and investigation of file content (e.g., by using headers and magic numbers). Many modern
UNIX filesystems support extended attributes which can store additional metadata for
a file, including file type. This data is stored as part of the file’s attribute data (in the
same way that file size and permissions are stored). How is the extended attribute
approach for storing files better or worse than the file extension approach or identifying file type by content?
38. Consider the idea behind Fig. 4-23, but now for a disk with a mean seek time of 8
msec, a rotational rate of 15,000 rpm, and 262,144 bytes per track. What are the data
rates for block sizes of 1 KB, 2 KB, and 4 KB, respectively?
CHAP. 4
PROBLEMS
335
39. In this chapter, we have seen that SSDs do their best to avoid writing the same memory
cells frequently (because of the wear). However, many SSDs offer much more functionality than what we presented so far. For instance, many controllers implement compression. Explain why compression may help with reducing the wear.
40. Given a disk-block size of 4 KB and block-pointer address value of 4 bytes, what is the
largest file size (in bytes) that can be accessed using 11 direct addresses and one indirect block?
41. The MS-DOS FAT-16 table contains 64K entries. Suppose that one of the bits had been
needed for some other purpose and that the table contained exactly 32,768 entries
instead. With no other changes, what would the largest MS-DOS file have been under
this condition?
42. Files in MS-DOS have to compete for space in the FAT-16 table in memory. If one file
uses k entries, that is k entries that are not available to any other file, what constraint
does this place on the total length of all files combined?
43. How many disk operations are needed to fetch the i-node for a file with the path name
/usr/ast/courses/os/handout.t? Assume that the i-node for the root directory is in memory, but nothing else along the path is in memory. Also assume that all directories fit in
one disk block.
44. In many UNIX systems, the i-nodes are kept at the start of the disk. An alternative
design is to allocate an i-node when a file is created and put the i-node at the start of
the first block of the file. Discuss the pros and cons of this alternative.
45. Write a program that reverses the bytes of a file, so that the last byte is now first and
the first byte is now last. It must work with an arbitrarily long file, but try to make it
reasonably efficient.
46. Write a program that starts at a given directory and descends the file tree from that
point recording the sizes of all the files it finds. When it is all done, it should print a
histogram of the file sizes using a bin width specified as a parameter (e.g., with 1024,
file sizes of 0 to 1023 go in one bin, 1024 to 2047 go in the next bin, etc.).
47. Write a program that scans all directories in a UNIX file system and finds and locates
all i-nodes with a hard link count of two or more. For each such file, it lists together all
file names that point to the file.
48. Write a new version of the UNIX ls program. This version takes as an argument one or
more directory names and for each directory lists all the files in that directory, one line
per file. Each field should be formatted in a reasonable way given its type. List only
the first disk address, if any.
49. Implement a program to measure the impact of application-level buffer sizes on read
time. This involves writing to and reading from a large file (say, 2 GB). Vary the application buffer size (say, from 64 bytes to 4 KB). Use timing measurement routines (such
as gettimeofday and getitimer on UNIX) to measure the time taken for different buffer
sizes. Analyze the results and report your findings: does buffer size make a difference
to the overall write time and per-write time?
336
FILE SYSTEMS
CHAP. 4
50. Implement a simulated file system that will be fully contained in a single regular file
stored on the disk. This disk file will contain directories, i-nodes, free-block information, file data blocks, etc. Choose appropriate algorithms for maintaining free-block
information and for allocating data blocks (contiguous, indexed, linked). Your program
will accept system commands from the user to perform file system operations, including at least one to create/delete directories, create/delete/open files, read/write from/to
a selected file, and to list directory contents.
5
INPUT/OUTPUT
In addition to providing abstractions such as processes, address spaces, and
files, an operating system also controls all the computer’s I/O (Input/Output) devices. It must issue commands to the devices, catch interrupts, and handle errors.
It should also provide an interface between the devices and the rest of the system
that is simple and easy to use. To the extent possible, the interface should be the
same for all devices (device independence). The I/O code represents a significant
fraction of the total operating system. How the operating system manages I/O is
the subject of this chapter.
This chapter is organized as follows. We will look first at some of the principles of I/O hardware and then at I/O software in general. I/O software can be
structured in layers, with each having a well-defined task. We will look at these
layers to see what they do and how they fit together.
Next, we will look at several I/O devices in detail: disks, clocks, keyboards,
and displays. For each device, we will look at its hardware and software. Finally,
we will consider power management.
5.1 PRINCIPLES OF I/O HARDWARE
Different people look at I/O hardware in different ways. Electrical engineers
look at it in terms of chips, wires, power supplies, motors, and all the other physical components that comprise the hardware. Programmers look at the interface
337
338
INPUT/OUTPUT
CHAP. 5
presented to the software—the commands the hardware accepts, the functions it
carries out, and the errors that can be reported back. In this book, we are concerned with programming I/O devices, not designing, building, or maintaining them, so
our interest is in how the hardware is programmed, not how it works inside. Nevertheless, the programming of many I/O devices is often intimately connected with
their internal operation. In the next three sections, we will provide a little general
background on I/O hardware as it relates to programming. It may be regarded as a
review and expansion of the introductory material in Sec. 1.3.
5.1.1 I/O Devices
I/O devices can be roughly divided into two categories: block devices and
character devices. A block device is one that stores information in fixed-size
blocks, each one with its own address. Common block sizes range from 512 to
65,536 bytes. All transfers are in units of one or more entire (consecutive) blocks.
The essential property of a block device is that it is possible to read or write each
block independently of all the other ones. Hard disks and SSDs (Solid State
Drives) are common block devices, and so are magnetic tape drives that are now
commonly found in computer museums but are also still in use in data centers and
have been the go-to solution for really big mass storage for over half a century. An
LTO-8 Ultrium tape, for example, can store 12 TB, be read at 750 MB/s, and is
expected to last 30 years. It costs under $100.
The other type of I/O device is the character device. A character device delivers or accepts a stream of characters, without regard to any block structure. It is
not addressable and does not have a seek operation. Printers, network interfaces,
mice (for pointing), rats (for psychology lab experiments), and most other devices
that are not disk-like can be seen as character devices.
This classification scheme is not perfect. Some devices do not fit in. Clocks,
for example, are not block addressable. Nor do they generate or accept character
streams. All they do is cause interrupts at well-defined intervals. Memory-mapped
screens do not fit the model well either. Nor do touch screens, for that matter. Still,
the model of block and character devices is general enough that it can be used as a
basis for making some of the operating system software dealing with I/O device
independent. The file system, for example, deals just with abstract block devices
and leaves the device-dependent part to lower-level software.
I/O devices cover a huge range in speeds, which puts considerable pressure on
the software to perform well over many orders of magnitude in data rates. Figure
5-1 shows the data rates of some common devices.
5.1.2 Device Controllers
I/O units often consist of a mechanical component and an electronic component. It is possible to separate the two parts to provide a more modular and general
design. The electronic component is called the device controller or adapter. On
SEC. 5.1
339
PRINCIPLES OF I/O HARDWARE
Device
Data rate
Keyboard
10 bytes/sec
Mouse
100 bytes/sec
56K modem
Bluetooth 5 BLE
7 KB/sec
256 KB/sec
Scanner at 300 dpi
1 MB/sec
Digital video recorder
3.5 MB/sec
802.11n Wireless
37.5 MB/sec
USB 2.0
60 MB/sec
16x Blu-ray disc
72 MB/sec
Gigabit Ethernet
125 MB/sec
SATA 3 disk drive
600 MB/sec
USB 3.0
625 MB/sec
Single-lane PCIe 3.0 bus
985 MB/sec
802.11ax Wireless
1.25 GB/sec
PCIe Gen 3.0 NVMe M.2 SSD (reading)
3.5 GB/sec
USB 4.0
PCI Express 6.0
5 GB/sec
126 GB/sec
Figure 5-1. Some typical device, network, and bus data rates.
personal computers, it often takes the form of a chip on the motherboard or a
printed circuit card that can be inserted into a (PCIe) expansion slot. The mechanical component is the device itself. This arrangement is shown in Fig. 1-6.
The controller card usually has a connector on it, into which a cable leading to
the device itself can be plugged. Many controllers can handle two, four, eight, or
even more identical devices. If the interface between the controller and device is a
standard interface, either an official ANSI, IEEE, or ISO standard or a de facto
one, then companies can make controllers or devices that fit that interface. Many
companies, for example, make disk drives that match the SATA, SCSI, USB, or
Thunderbolt, interfaces.
The interface between the controller and the device is often a very low-level
one. A disk, for example, might have 3,000,000 tracks, each formatted with between 200 and 500 sectors of 4,096 bytes each. What actually comes off the drive,
however, is a serial bit stream, starting with a preamble, then followed by the
8 × 4, 096 = 32, 768 bits in a sector, and finally a checksum, or ECC (Error-Correcting Code). The preamble is written when the disk is formatted and contains
the cylinder and sector number, the sector size, and similar data, as well as synchronization information.
The controller’s job is to convert the serial bit stream into a block of bytes and
perform any error correction that is necessary. The block of bytes is typically first
340
INPUT/OUTPUT
CHAP. 5
assembled, bit by bit, in a buffer inside the controller. After its checksum has been
verified and the block has been declared to be error free, it can then be copied to
main memory.
The controller for an LCD display monitor also works as a bit serial device at
an equally low level. It reads bytes containing the characters to be displayed from
memory and generates the signals to modify the polarization of the backlight for
the corresponding pixels in order to write them on screen. If it were not for the
display controller, the operating system programmer would have to explicitly program the electric fields of all pixels. With the controller, the operating system initializes the controller with a few parameters, such as the number of characters or
pixels per line and number of lines per screen, and lets the controller take care of
actually driving the electric fields.
In a very short time, LCD screens have completely replaced the old CRT
(Cathode Ray Tube) monitors. CRT monitors fire a beam of electrons onto a fluorescent screen. Using magnetic fields, the system is able to bend the beam and
draw pixels on the screen. Compared to LCD screens, CRT monitors were bulky,
power hungry, and fragile. Moreover, the resolution on today’s (Retina) LCD
screens is so good that the human eye is unable to distinguish individual pixels. It
is hard to imagine today that laptops in the past came with a small CRT screen that
made them more than 20 cm deep with a nice work-out weight of around 12 kg.
5.1.3 Memory-Mapped I/O
Each controller has a few registers that are used for communicating with the
CPU. By writing into these registers, the operating system can command the device to deliver data, accept data, switch itself on or off, or otherwise perform some
action. By reading from these registers, the operating system can learn what the
device’s state is, whether it is prepared to accept a new command, and so on.
In addition to the control registers, many devices have a data buffer that the
operating system can read and write. For example, a common way for computers to
display pixels on the screen is to have a video RAM, which is basically just a data
buffer, available for programs or the operating system to write into.
The issue thus arises of how the CPU communicates with the control registers
and also with the device data buffers. Two alternatives exist. In the first approach,
each control register is assigned an I/O port number, an 8- or 16-bit integer. The
set of all the I/O ports form the I/O port space, which is protected so that ordinary
user programs cannot access it (only the operating system can). Using a special
I/O instruction such as
IN REG,PORT,
the CPU can read in control register PORT and store the result in CPU register
REG. Similarly, using
OUT PORT,REG
SEC. 5.1
341
PRINCIPLES OF I/O HARDWARE
the CPU can write the contents of REG to a control register. Most early computers,
including nearly all mainframes, such as the IBM 360 and all of its successors,
worked this way.
In this scheme, the address spaces for memory and I/O are different, as shown
in Fig. 5-2(a). The instructions
IN R0,4
and
MOV R0,4
are completely different in this design. The former reads the contents of I/O port 4
and puts it in R0, whereas the latter reads the contents of memory word 4 and puts
it in R0. The 4s in these examples refer to different and unrelated address spaces.
Two address
0xFFFF…
One address space
Two address spaces
Memory
I/O ports
0
(a)
(b)
(c)
Figure 5-2. (a) Separate I/O and memory space. (b) Memory-mapped I/O.
(c) Hybrid.
The second approach, introduced with the PDP-11, is to map all the control
registers into the memory space, as shown in Fig. 5-2(b). Each control register is
assigned a unique memory address to which no memory is assigned. This system is
called memory-mapped I/O. In most systems, the assigned addresses are at or
near the top of the address space. A hybrid scheme, with memory-mapped I/O
data buffers and separate I/O ports for the control registers, is shown in Fig. 5-2(c).
The x86 uses this architecture, with addresses 640K to 1M < 1 being reserved for
device data buffers in IBM PC compatibles, in addition to I/O ports 0 to 64K < 1.
As an aside, assigning 360K addresses for I/O devices on the original PC was
an absurdly large number and limited the amount of memory that could be put on a
PC. Having 4K I/O addresses would have been plenty. But back when memory
cost $1 per byte, no one thought that anyone would want to have 640 KB on a PC,
let alone 900 KB or more. What the designers did not realize was how fast memory prices would tumble. Nowadays, you would be hard pressed to find a notebook
computer with less than 4,000,000 KB of RAM.
342
INPUT/OUTPUT
CHAP. 5
How do these schemes actually work in practice? In all cases, when the CPU
wants to read a word, either from memory or from an I/O port, it puts the address it
needs on the bus’ address lines and then asserts a READ signal on a bus’ control
line. A second signal line is used to tell whether I/O space or memory space is
needed. If it is memory space, the memory responds to the request. If it is I/O
space, the I/O device responds to the request. If there is only memory space [as in
Fig. 5-2(b)], every memory module and every I/O device compares the address
lines to the range of addresses that it services. If the address falls in its range, it
responds to the request. Since no address is ever assigned to both memory and an
I/O device, there is no ambiguity and no conflict.
These two schemes for addressing the controllers have different strengths and
weaknesses. Let us start with the advantages of memory-mapped I/O. First of all,
if special I/O instructions are needed to read and write the device control registers,
access to them requires the use of assembly code since there is no way to execute
an IN or OUT instruction in C or C++. Calling such a procedure adds overhead to
controlling I/O. In contrast, with memory-mapped I/O, device control registers are
just variables in memory and can be addressed in C the same way as any other
variables. Thus, with memory-mapped I/O, an I/O device driver can be written
entirely in C. Without memory-mapped I/O, some assembly code is needed.
Second, with memory-mapped I/O, no special protection mechanism is needed
to keep user processes from performing I/O. All the operating system has to do is
refrain from putting that portion of the address space containing the control registers in any user’s virtual address space. Even better yet, if each device has its control registers on a different page of the address space, the operating system can
give a user control over specific devices but not others by simply including the desired pages in its page table. Such a scheme can allow different device drivers to be
run in different user-mode address spaces, not only reducing kernel size but also
keeping one driver from interfering with others. This also prevents a driver crash
from taking down the entire system. Some microkernels (e.g., MINIX 3) work like
this.
Third, with memory-mapped I/O, every instruction that can reference memory
can also reference control registers. For example, if there is an instruction, TEST,
that tests a memory word for 0, it can also be used to test a control register for 0,
which might be the signal that the device is idle and can accept a new command.
The assembly language code might look like this:
LOOP: TEST PORT 4
BEQ READY
BRANCH LOOP
READY:
// check if por t 4 is 0
// if it is 0, go to ready
// otherwise, continue testing
If memory-mapped I/O is not present, the control register must first be read into
the CPU, then tested, requiring two instructions instead of just one. In the case of
SEC. 5.1
343
PRINCIPLES OF I/O HARDWARE
the loop given above, a fourth instruction has to be added, slightly slowing down
the responsiveness of detecting an idle device.
In computer design, practically everything involves trade-offs, and that is the
case here, too. Memory-mapped I/O also has its disadvantages. First, most computers nowadays have some form of caching of memory words. Caching a device
control register would be disastrous. Consider the assembly-code loop given above
in the presence of caching. The first reference to PORT 4 would cause it to be
cached. Subsequent references would just take the value from the cache and not
even ask the device. Then when the device finally became ready, the software
would have no way of finding out. Instead, the loop would go on forever.
To prevent this situation with memory-mapped I/O, the hardware has to be able
to selectively disable caching, for example, on a per-page basis. This feature adds
extra complexity to both the hardware and the operating system, which has to manage the selective caching.
Second, if there is only one address space, then all memory modules and all
I/O devices must examine all memory references to see which ones to respond to.
If the computer has a single bus, as in Fig. 5-3(a), having everyone look at every
address is straightforward.
CPU reads and writes of memory
go over this high-bandwidth bus
CPU
Memory
All addresses (memory
and I/O) go here
(a)
I/O
CPU
Memory
I/O
This memory port is
to allow I/O devices
access to memory
Bus
(b)
Figure 5-3. (a) A single-bus architecture. (b) A dual-bus memory architecture.
However, the trend in modern personal computers is to have a dedicated highspeed memory bus, as shown in Fig. 5-3(b). The bus is tailored to optimize memory performance, with no compromises for the sake of slow I/O devices. x86 systems can have multiple buses (memory, PCIe, SCSI, and USB), as shown in
Fig. 1-12.
The trouble with having a separate memory bus on memory-mapped machines
is that the I/O devices have no way of seeing memory addresses as they go by on
the memory bus, so they have no way of responding to them. Again, special measures have to be taken to make memory-mapped I/O work on a system with multiple
344
INPUT/OUTPUT
CHAP. 5
buses. One possibility is to first send all memory references to the memory. If the
memory fails to respond, then the CPU tries the other buses. This design can be
made to work but requires additional hardware complexity.
A second possible design is to put a snooping device on the memory bus to
pass all addresses presented to potentially interested I/O devices. The problem here
is that I/O devices may not be able to process requests at the speed the memory
can.
A third possible design, and one that would well match the design sketched in
Fig. 1-12, is to filter addresses in the memory controller. In that case, the memory
controller chip contains range registers that are preloaded at boot time. For example, 640K to 1M < 1 could be marked as a nonmemory range. Addresses that fall
within one of the ranges marked as nonmemory are forwarded to devices instead of
to memory. The main disadvantage of this scheme is the need for figuring out at
boot time which memory addresses are not really memory addresses. Thus each
scheme has arguments for and against it, so compromises and trade-offs are inevitable, especially when backward compatibility with legacy systems is important.
5.1.4 Direct Memory Access
No matter whether a CPU does or does not have memory-mapped I/O, it needs
to address the device controllers to exchange data with them. The CPU can request
data from an I/O controller one byte at a time, but doing so wastes the CPU’s time,
so a different scheme, called DMA (Direct Memory Access) is often used. To
simplify the explanation, we assume that the CPU accesses all devices and memory
via a single system bus that connects the CPU, the memory, and the I/O devices, as
shown in Fig. 5-4. We already know that the real organization in modern systems
is more complicated, but all the principles are the same. The operating system can
only use DMA if the hardware has a DMA controller, which most systems do.
Sometimes this controller is integrated into disk controllers and other controllers,
but such a design requires a separate DMA controller for each device. More commonly, a single DMA controller is available (e.g., on the motherboard) for regulating transfers to multiple devices, often concurrently.
No matter where it is physically located, the DMA controller has access to the
system bus independent of the CPU, as shown in Fig. 5-4. It contains several registers that can be written and read by the CPU. These include a memory address
register, a byte count register, and one or more control registers. The control registers specify the I/O port to use, the direction of the transfer (reading from the I/O
device or writing to the I/O device), the transfer unit (byte at a time or word at a
time), and the number of bytes to transfer in one burst.
To explain how DMA works, consider how data are read from, say, a disk. Let
us first look at how disk reads occur when DMA is not used. First, the disk controller reads the block (one or more sectors) from the drive serially, bit by bit, until
the entire block is stored in the controller’s internal buffer. Next, it computes the
SEC. 5.1
345
PRINCIPLES OF I/O HARDWARE
Drive
CPU
1. CPU
programs
the DMA
controller
DMA
controller
Main
memory
Disk
controller
Buffer
Address
Count
Control
Interrupt when
done
4. Ack
2. DMA requests
transfer to memory
3. Data transferred
Bus
Figure 5-4. Operation of a DMA transfer.
checksum to verify that no read errors have occurred. Then the controller causes
an interrupt. When the operating system starts running, it can read the disk block
from the controller’s buffer a byte or a word at a time by executing a loop, with
each iteration reading one byte or word from a controller device register and storing it in main memory.
When DMA is used, the procedure is different. First the CPU programs the
DMA controller by setting its registers so it knows what to transfer where (step 1
in Fig. 5-4). It also issues a command to the disk controller telling it to read data
from the disk into its internal buffer and verify the checksum. When valid data are
in the disk controller’s buffer, DMA can begin.
The DMA controller initiates the transfer by issuing a read request over the bus
to the disk controller (step 2). This read request looks like any other read request,
and the disk controller does not know (or care) whether it came from the CPU or
from a DMA controller. Typically, the memory address to write to is on the bus’
address lines, so when the disk controller fetches the next word from its internal
buffer, it knows where to write it. The write to memory is another standard bus
cycle (step 3). When the write is complete, the disk controller sends an acknowledgement signal to the DMA controller, also over the bus (step 4). The DMA controller then increments the memory address to use and decrements the byte count.
If the byte count is still greater than 0, steps 2 through 4 are then repeated until the
count reaches 0. At that time, the DMA controller interrupts the CPU to let it
know that the transfer is now complete. When the operating system starts up, it
does not have to copy the disk block to memory; it is already there.
DMA controllers vary considerably in their sophistication. The simplest ones
handle one transfer at a time, as described above. More complex ones can be programmed to handle multiple transfers at the same time. Such controllers have multiple sets of registers internally, one for each channel. The CPU starts by loading
346
INPUT/OUTPUT
CHAP. 5
each set of registers with the relevant parameters for its transfer. Each transfer must
use a different device controller. After each word is transferred (steps 2 through 4)
in Fig. 5-4, the DMA controller decides which device to service next. It may be set
up to use a round-robin algorithm, or it may have a priority scheme design to favor
some devices over others. Multiple requests to different device controllers may be
pending at the same time, provided that there is an unambiguous way to tell the
acknowledgements apart. Often a different acknowledgement line on the bus is
used for each DMA channel for this reason.
Many buses can operate in two modes: word-at-a-time mode and block mode.
Often, DMA controllers can also operate in either mode. In the former mode, the
operation is as described above: the DMA controller requests the transfer of one
word and gets it. If the CPU also wants the bus, it has to wait. The mechanism is
called cycle stealing because the device controller sneaks in and steals an occasional bus cycle from the CPU once in a while, delaying it slightly. In block mode,
the DMA controller tells the device to acquire the bus, issue a series of transfers,
then release the bus. This form of operation is called burst mode. It is more efficient than cycle stealing because acquiring the bus takes time and multiple words
can be transferred for the price of one bus acquisition. The downside to burst mode
is that it can block the CPU and other devices for a substantial period if a long
burst is being transferred.
In the model we have been discussing, sometimes called fly-by mode, the
DMA controller tells the device controller to transfer the data directly to main
memory. An alternative mode that some DMA controllers use is to have the device
controller send the word to the DMA controller, which then issues a second bus
request to write the word to wherever it is supposed to go. This scheme requires an
extra bus cycle per word transferred, but is more flexible in that it can also perform
device-to-device copies and even memory-to-memory copies (by first issuing a
read to memory and then issuing a write to memory at a different address).
Most DMA controllers use physical memory addresses for their transfers.
Using physical addresses requires the operating system to convert the virtual address of the intended memory buffer into a physical address and write this physical
address into the DMA controller’s address register. An alternative scheme used in a
few DMA controllers is to write virtual addresses into the DMA controller instead.
Then the DMA controller must use the MMU to have the virtual-to-physical translation done. Only when the MMU is part of the memory (possible, but rare), rather
than part of the CPU, can virtual addresses be put on the bus. In Chap. 7, we will
see that an IOMMU (an MMU for I/O) offers similar functionality: it translates the
virtual addresses used by devices to physical addresses. In other words, the virtual
address of a buffer used by a device may be different from the virtual address used
for the same buffer by the CPU, while both are different from the corresponding
physical address.
We mentioned earlier that before DMA can start, the disk first reads data into
its internal buffer. You may be wondering why the controller does not just store the
SEC. 5.1
PRINCIPLES OF I/O HARDWARE
347
bytes in main memory as soon as it gets them from the disk. In other words, why
does it need an internal buffer? There are two reasons. First, by doing internal
buffering, the disk controller can verify the checksum before starting a transfer. If
the checksum is incorrect, an error is signaled and no transfer is done.
The second reason is that once a disk transfer has started, the bits keep arriving
from the disk at a constant rate, whether the controller is ready for them or not. If
the controller tried to write data directly to memory, it would have to go over the
system bus for each word transferred. If the bus were busy due to some other device using it (e.g., in burst mode), the controller would have to wait. If the next
disk word arrived before the previous one had been stored, the controller would
have to store it somewhere. If the bus were very busy, the controller might end up
storing quite a few words and having a lot of administration to do as well. When
the block is buffered internally, the bus is not needed until the DMA begins, so the
design of the controller is much simpler because the DMA transfer to memory is
not time critical. (Some older controllers did, in fact, go directly to memory with
only a small amount of internal buffering, but when the bus was very busy, a transfer might have had to be terminated with a buffer overrun error.)
Some computers do not use DMA. The argument against it might be that the
main CPU is often far faster than the DMA controller and can do the job much
faster (when the limiting factor is not the speed of the I/O device). If there is no
other work for it to do, having the (fast) CPU wait for the (slow) DMA controller
to finish is pointless. Also, getting rid of the DMA controller and having the CPU
do all the work in software saves money, important on low-end (embedded) computers.
5.1.5 Interrupts Revisited
We briefly introduced interrupts in Sec. 1.3.4, but there is more to be said.
Before we start, you should know that the literature is confusing when it comes to
interrupts. Textbooks and Web pages may use the term to refer to hardware interrupts, traps, exceptions, faults, and a few other things. What do these terms mean?
We generally use trap to refer to a deliberate action by the program code, for
instance, a trap into the kernel for a system call. A fault or exception is similar,
except that it is generally not deliberate. For instance, the program may trigger a
segmentation fault when it tries to access memory that it is not allowed to access or
wants to learn what 100 divided by zero is. In contrast, we will now talk mostly
about hardware interrupts, where a device such as printer or a network sends a signal to the CPU. The reason all these terms are frequently clubbed together is that
they are handled in similar ways, even if they are triggered differently. In this section, we look at the hardware side. In Section 5.3, we will turn to the further handling of interrupts by the software.
Figure 5-5 shows the interrupt structure in a typical personal computer system.
In this respect, a smartphone or tablet works the same way. At the hardware level,
348
INPUT/OUTPUT
CHAP. 5
interrupts work as follows. When an I/O device has finished the work given to it, it
causes an interrupt (assuming that interrupts have been enabled by the operating
system), by asserting a signal on a bus line that it has been assigned. This signal is
detected by the interrupt controller chip on the motherboard, which then decides
what to do.
CPU
3. CPU acks
interrupt
2. Controller
issues
interrupt
Interrupt
controller
1. Device is finished
Disk
11 12 1
10
2
9
3
8
4
7 6 5
Keyboard
Clock
Printer
Bus
Figure 5-5. How an interrupt happens. The connections between the devices and
the controller actually use interrupt lines on the bus rather than dedicated wires.
If no other interrupts are pending, the interrupt controller handles the interrupt
immediately. However, if another interrupt is in progress, or another device has
made a simultaneous request on a higher-priority interrupt request line on the bus,
the device is just ignored for the moment. In this case, it continues to assert an
interrupt signal on the bus until it is serviced by the CPU.
To handle the interrupt, the controller puts a number on the address lines specifying which device wants attention and asserts a signal to interrupt the CPU.
The interrupt signal causes the CPU to stop what it is doing and start doing
something else. The number on the address lines is used as an index into a table
called the interrupt vector to fetch a new program counter. This program counter
points to the start of the corresponding interrupt-service procedure. Typically traps,
exceptions, and interrupts use the same mechanism from this point on, often sharing the same interrupt vector. The location of the interrupt vector can be hardwired
into the machine or it can be anywhere in memory, with a CPU register (loaded by
the operating system) pointing to its origin.
Shortly after it starts running, the interrupt-service procedure acknowledges
the interrupt by writing a certain value to one of the interrupt controller’s I/O ports.
This acknowledgement tells the controller that it is free to issue another interrupt.
By having the CPU delay this acknowledgement until it is ready to handle the next
interrupt, race conditions involving multiple (almost simultaneous) interrupts can
be avoided. As an aside, some (older) computers do not have a centralized interrupt controller, so each device controller requests its own interrupts.
The hardware always saves certain information before starting the service procedure. Which information is saved and where it is saved vary greatly from CPU to
SEC. 5.1
PRINCIPLES OF I/O HARDWARE
349
CPU. As a bare minimum, the program counter must be saved, so the interrupted
process can be restarted. At the other extreme, all the visible registers and a large
number of internal registers may be saved as well.
Another issue is where to save this information. One option is to put it in internal registers that the operating system can read out as needed. However, a problem
with this approach is that the interrupt controller cannot be acknowledged until all
potentially relevant information has been read out, lest a second interrupt overwrite
the internal registers saving the state. This strategy leads to long dead times when
interrupts are disabled and possibly to lost interrupts and lost data.
Consequently, most CPUs save the information on the stack. However, this
approach, too, has problems. To start with: whose stack? If the current stack is
used, it may well be a user process stack. The stack pointer may not even be legal,
which would cause a fatal error when the hardware tried to write some words at the
address pointed to. Also, it might point near the end of a page. After several memory writes, the page boundary might be exceeded and a page fault generated. Having a page fault occur during the hardware interrupt processing creates a bigger
problem: where to save the state to handle the page fault?
If the kernel stack is used, there is a much better chance of the stack pointer
being legal and pointing to a pinned page. However, switching into kernel mode
may require changing MMU contexts and will probably invalidate most or all of
the cache and TLB. Reloading all of these, statically or dynamically, will increase
the time to process an interrupt and thus waste CPU time at a critical moment.
So far, we have discussed interrupt handling mostly from a hardware perspective. However, there is a lot of software involved in I/O also. We will look at the
I/O software stack in detail in Sec. 5.3.
Precise and Imprecise Interrupts
Another problem is caused by the fact that most modern CPUs are heavily
pipelined and often superscalar (internally parallel). In older systems, after each
instruction was finished executing, the microprogram or hardware checked to see if
there was an interrupt pending. If so, the program counter and PSW were pushed
onto the stack and the interrupt sequence begun. After the interrupt handler ran, the
reverse process took place, with the old PSW and program counter popped from
the stack and the previous process continued.
This model makes the implicit assumption that if an interrupt occurs just after
some instruction, all the instructions up to and including that instruction have been
executed completely, and no instructions after it have executed at all. On older
machines, this assumption was always valid. On modern ones it may not be.
For starters, consider the pipeline model of Fig. 1-7(a). What happens if an
interrupt occurs while the pipeline is full (the usual case)? Many instructions are
in various stages of execution. When the interrupt occurs, the value of the program
counter may not reflect the correct boundary between executed instructions and
350
INPUT/OUTPUT
CHAP. 5
nonexecuted instructions. In fact, many instructions may have been partially
executed, with different instructions being more or less complete. In this situation,
the program counter most likely reflects the address of the next instruction to be
fetched and pushed into the pipeline rather than the address of the instruction that
just was processed by the execution unit.
On a superscalar machine, such as that of Fig. 1-7(b), things are even worse.
CPU instructions may be internally decomposed into so-called micro-operations
and these micro-operations may execute out of order, depending on the availability
of internal resources such as functional units and registers (see also Sec. 2.5.9). At
the time of an interrupt, some instructions issued long ago may not be anywhere
near completion and others started more recently may be (almost) done. This is not
a problem as the CPU will simply buffer the results of each instruction until all
previous instructions have also completed and then commit all of them in order.
However, it means that at the point when an interrupt is signaled, there may be
many instructions in various states of completeness, and there is not much of relation between them and the program counter at all.
An interrupt that leaves the machine in a well-defined state is called a precise
interrupt (Walker and Cragon, 1995). Such an interrupt has four properties:
1. The PC (Program Counter) is saved in a known place.
2. All instructions before the one pointed to by the PC have completed.
3. No instruction beyond the one pointed to by the PC has finished.
4. The execution state of the instruction pointed to by the PC is known.
Note that even with precise interrupts there is no prohibition on instructions
beyond the one pointed to by the PC from starting. It is just that any changes they
make to registers or memory must be completely undone when the interrupt happens. This is what many processor architectures, including the x86, try to do. Since
the CPU erases all visible effects as if these instructions never executed, we call the
instructions transient. Such transient execution occurs for many reasons (Ragab
et al., 2021). We already saw that a fault or interrupt that happens during the execution of an instruction while some later instructions have already completed requires
the processor to throw away the results of these later instructions. However, modern CPUs employ many more tricks to improve performance. For instance, the
CPU may speculate on the outcome of a conditional branch. If the outcome of an if
condition was TRUE the last 50 times, the CPU will assume that it will be true the
51st time also and speculatively start fetching and executing the instructions for the
TRUE branch. Of course, if the 51st time was different and the outcome is really
FALSE, these instructions must now be made transient. Transient execution has
been the source of all sorts of security trouble, but that is not what we need to discuss now and we save it for Chapter 9.
SEC. 5.1
351
PRINCIPLES OF I/O HARDWARE
Meanwhile, when an interrupt occurs, what should happen to the instruction to
which the PC is currently pointing? It is permitted that this instruction has been
executed. It is also permitted that it has not been executed. However, it must be
clear which case applies. Often, if the interrupt is an I/O interrupt, the instruction
will not yet have started. However, if the interrupt is really a trap or page fault, then
the PC generally points to the instruction that caused the fault so it can be restarted
later. The situation of Fig. 5-6(a) illustrates a precise interrupt. All instructions up
to the program counter (316) have completed and none of those beyond it have
started (or have been rolled back to undo their effects).
PC
Not executed
Not executed
Not executed
Not executed
Fully executed
Fully executed
Fully executed
Fully executed
(a)
332
328
324
320
316
312
308
304
300
PC
Not executed
10% executed
40% executed
35% executed
20% executed
60% executed
80% executed
Fully executed
332
328
324
320
316
312
308
304
300
(b)
Figure 5-6. (a) A precise interrupt. (b) An imprecise interrupt.
An interrupt that does not meet these requirements is called an imprecise
interrupt and makes life most unpleasant for the operating system writer, who
now has to figure out what has happened and what still has to happen. Fig. 5-6(b)
illustrates an imprecise interrupt, where different instructions near the program
counter are in different stages of completion, with older ones not necessarily more
complete than younger ones. Machines with imprecise interrupts usually vomit a
large amount of internal state onto the stack to give the operating system the possibility of figuring out what was going on. The code necessary to restart the machine
is typically exceedingly complicated. Also, saving a large amount of information to
memory on every interrupt makes interrupts slow and recovery even worse. This
leads to the ironic situation of having very fast superscalar CPUs sometimes being
unsuitable for real-time work due to slow interrupts.
Some computers are designed so that some kinds of interrupts and traps are
precise and others are not. For example, having I/O interrupts be precise but traps
due to fatal programming errors be imprecise is not so bad since no attempt need
be made to restart a running process after it has divided by zero. At that point,
having done something that is infinitely bad, it is toast anyway. Some machines
have a bit that can be set to force all interrupts to be precise. The downside of setting this bit is that it forces the CPU to carefully log everything it is doing and
maintain shadow copies of registers so it can generate a precise interrupt at any
instant. All this overhead has a major impact on performance.
352
INPUT/OUTPUT
CHAP. 5
Some superscalar machines, such as the x86 family, have precise interrupts to
allow old software to work correctly. The price paid for backward compatibility
with precise interrupts is extremely complex interrupt logic within the CPU to
make sure that when the interrupt controller signals that it wants to cause an interrupt, all instructions up to some point are allowed to finish and none beyond that
point are allowed to have any noticeable effect on the machine state. Here the price
is paid not in time, but in chip area and in complexity of the design. If precise
interrupts were not required for backward compatibility purposes, this chip area
would be available for larger on-chip caches, making the CPU faster. On the other
hand, imprecise interrupts make the operating system far more complicated, less
secure due to the complexity, and slower, so it is hard to tell which approach is
really better.
Also, as mentioned earlier, we will see in Chap. 9 that all the instructions of
which the effects on the machine state have been undone (and that are therefore
transient) may be problematic still from a security perspective. The reason is that
not all effects are undone. In particular, they leave traces deep in the micro-architecture (where we find the cache and the TLB and other components) which an
attacker may use to leak sensitive information.
5.2 PRINCIPLES OF I/O SOFTWARE
Let us now turn away from the I/O hardware and look at the I/O software. First
we will look at its goals and then at the different ways I/O can be done from the
point of view of the operating system.
5.2.1 Goals of the I/O Software
A key concept in the design of I/O software is known as device independence.
What it means is that we should be able to write programs that can access any I/O
device without having to specify the device in advance. For example, a program
that reads a file as input should be able to read a file on a hard disk, an SSD, or a
USB stick without having to be modified for each different device. Similarly, one
should be able to type a command such as
sor t <input >output
and have it work with input coming from any kind of storage device or the
keyboard and the output going to any kind of storage device or the screen. It is up
to the operating system to take care of the problems caused by the fact that these
devices really are different and require very different command sequences to read
or write.
Closely related to device independence is the goal of uniform naming. The
name of a file or a device should simply be a string or an integer and not depend on
SEC. 5.2
PRINCIPLES OF I/O SOFTWARE
353
the device in any way. In UNIX, all disks can be integrated in the file-system hierarchy in arbitrary ways so the user need not be aware of which name corresponds
to which device. For example, a USB stick can be mounted on top of the directory
/usr/ast/backup so that copying a file to /usr/ast/backup/monday copies the file to
the USB stick. In this way, all files and devices are addressed the same way: by a
path name.
Another important issue for I/O software is error handling. In general, errors
should be handled as close to the hardware as possible. If the controller discovers
a read error, it should try to correct the error itself if it can. If it cannot, then the
device driver should handle it, perhaps by just trying to read the block again. Many
errors are transient, such as read errors caused by specks of dust on the read head
of a disk, and will frequently go away if the operation is repeated. Only if the
lower layers are not able to deal with the problem should the upper layers be told
about it. In many cases, error recovery can be done transparently at a low level
without the upper levels even knowing about the error.
Still another important issue is that of synchronous (blocking) versus asynchronous (interrupt-driven) transfers. Most physical I/O is asynchronous, that is,
the CPU starts the transfer and goes off to do something else until the interrupt
arrives. User programs are much easier to write if the I/O operations are blocking—after a read system call the program is automatically suspended until the data
are available in the buffer. It is up to the operating system to make operations that
are actually interrupt-driven look blocking to the user programs. However, some
very high-performance applications need to control all the details of the I/O, so
operating systems also make asynchronous I/O available to them.
Another issue for the I/O software is buffering. Often data that come off a device cannot be stored directly in their final destination. For example, when a packet
comes in off the network, the operating system does not know where to put it until
it has stored the packet somewhere and examined it to see which port it is addressed to. Also, some devices have severe real-time constraints (for example, digital
audio devices), so the data must be put into an output buffer in advance to decouple
the rate at which the buffer is filled from the rate at which it is emptied, in order to
avoid buffer underruns. Buffering involves considerable copying and often has a
major impact on I/O performance.
The final concept that we will mention here is sharable versus dedicated devices. Some I/O devices, such as disks and SSDs, can be used by many users at the
same time. No problems are caused by multiple users having open files on the
same storage devices at the same time. Other devices, such as printers, have to be
dedicated to a single user until that user is finished. Then another user can have the
printer. Having two or more users writing characters intermixed at random to the
same page will definitely not work. Scanners are like that as well. Introducing
dedicated (unshared) devices also introduces a variety of problems, such as deadlocks. Again, the operating system must be able to handle both shared and dedicated devices in a way that avoids problems.
354
INPUT/OUTPUT
CHAP. 5
5.2.2 Programmed I/O
There are three fundamentally different ways that I/O can be performed. In
this section we will look at the first one (programmed I/O). In the next two sections we will examine the others (interrupt-driven I/O and I/O using DMA). The
simplest form of I/O is to have the CPU do all the work. This method is called programmed I/O.
It is simplest to illustrate how programmed I/O works by means of an example.
Consider a user process that wants to print the eight-character string ‘‘ABCDEFGH’’ on the printer via a serial interface. Displays on small embedded systems
sometimes work this way. The software first assembles the string in a buffer in
user space, as shown in Fig. 5-7(a).
User
space
String to
be printed
Printed
page
Printed
page
ABCD
EFGH
A
Next
Kernel
space
(a)
AB
Next
ABCD
EFGH
ABCD
EFGH
(b)
(c)
Figure 5-7. Steps in printing a string.
The user process then acquires the printer for writing by making a system call
to open it. If the printer is currently in use by another process, this call will fail
and return an error code or will block until the printer is available, depending on
the operating system and the parameters of the call. Once it has the printer, the user
process makes a system call telling the operating system to print the string.
The operating system then (usually) copies the buffer with the string to an
array, in kernel space, where it is more easily accessed (because the kernel may
have to change the memory map to get at user space) and also safe from modification by the user process. It then checks to see if the printer is currently available.
If not, it waits until it is. As soon as the printer is available, the operating system
copies the first character to the printer’s data register, in this example using memory-mapped I/O. This action activates the printer. The character may not appear yet
because some printers buffer a line or a page before printing anything. In
Fig. 5-7(b), however, we see that the first character has been printed and that the
system has marked the ‘‘B’’ as the next character to be printed.
SEC. 5.2
PRINCIPLES OF I/O SOFTWARE
355
As soon as it has copied the first character to the printer, the operating system
checks to see if the printer is ready to accept another one. Generally, the printer has
a second register, which gives its status. The act of writing to the data register
causes the status to become not ready. When the printer controller has processed
the current character, it indicates its availability by setting some bit in its status register or putting some value in it.
At this point, the operating system waits for the printer to become ready again.
When that happens, it prints the next character, as shown in Fig. 5-7(c). This loop
continues until the entire string has been printed. Then control returns to the user
process.
The actions followed by the operating system are briefly summarized in
Fig. 5-8. First, the data are copied to the kernel. Then the operating system enters
a tight loop, outputting the characters one at a time. The essential aspect of programmed I/O, clearly illustrated in this figure, is that after outputting a character,
the CPU continuously polls the device to see if it is ready to accept another one.
This behavior is often called polling or busy waiting.
copy from user(buffer, p, count);
for (i = 0; i < count; i++) {
while (*printer status reg != READY) ;
*printer data register = p[i];
}
return to user( );
/* p is the kernel buffer */
/* loop on every character */
/* loop until ready */
/* output one character */
Figure 5-8. Writing a string to the printer using programmed I/O.
Programmed I/O is simple but has the disadvantage of tying up the CPU full time
until all the I/O is done. If the time to ‘‘print’’ a character is very short (because all
the printer is doing is copying the new character to an internal buffer), then busy
waiting is fine. Also, in an embedded system, where the CPU has nothing else to
do, busy waiting is fine. However, in more complex systems, where the CPU has
other work to do, busy waiting is inefficient. A better I/O method is needed.
5.2.3 Interrupt-Driven I/O
Now let us consider the case of printing on a printer that does not buffer characters but prints each one as soon as it arrives. If the printer can print, say 100
characters/sec, each character takes 10 msec to print. This means that after every
character is written to the printer’s data register, the CPU will sit in an idle loop for
10 msec waiting to be allowed to output the next character. This is more than
enough time to do a context switch and run some other process for nearly all of the
10 msec that would otherwise be wasted.
The way to allow the CPU to do something else while waiting for the printer to
become ready is to use interrupts. When the system call to print the string is made,
356
INPUT/OUTPUT
CHAP. 5
the buffer is copied to kernel space, as we showed earlier, and the first character is
copied to the printer as soon as it is willing to accept a character. At that point, the
CPU calls the scheduler and some other process is run. The process that asked for
the string to be printed is blocked until the entire string has printed. The work done
on the system call is shown in Fig. 5-9(a).
copy from user(buffer, p, count);
enable interrupts( );
while (*printer status reg != READY) ;
*printer data register = p[0];
scheduler( );
(a)
if (count == 0) {
unblock user( );
} else {
*printer data register = p[i];
count = count < 1;
i = i + 1;
}
acknowledge interrupt( );
return from interrupt( );
(b)
Figure 5-9. Writing a string to the printer using interrupt-driven I/O. (a) Code
executed at the time the print system call is made. (b) Interrupt service procedure
for the printer.
When the printer has printed the character and is prepared to accept the next
one, it generates an interrupt. This interrupt stops the current process and saves its
state. Then the printer interrupt-service procedure is run. A crude version of this
code is shown in Fig. 5-9(b). If there are no more characters to print, the interrupt
handler takes some action to unblock the user. Otherwise, it outputs the next character, acknowledges the interrupt, and returns to the process that was running just
before the interrupt, which continues from where it left off.
5.2.4 I/O Using DMA
An obvious disadvantage of interrupt-driven I/O is that an interrupt occurs on
every character. Interrupts take time, so this scheme wastes a certain amount of
CPU time. A solution is to use DMA. Here the idea is to let the DMA controller
feed the characters to the printer one at time, without the CPU being bothered. In
essence, DMA is programmed I/O, only with the DMA controller doing all the
work, instead of the main CPU. This strategy requires special hardware (the DMA
controller) but frees up the CPU during the I/O to do other work. An outline of the
code is given in Fig. 5-10.
The big win with DMA is reducing the number of interrupts from one per
character to one per buffer printed. If there are many characters and interrupts are
slow, this can be a major improvement. On the other hand, the DMA controller is
usually much slower than the main CPU. If the DMA controller is not capable of
driving the device at full speed, or the CPU usually has nothing to do anyway
while waiting for the DMA interrupt, then interrupt-driven I/O or even programmed I/O may be better. Most of the time, though, DMA is worth it.
SEC. 5.3
I/O SOFTWARE LAYERS
copy from user(buffer, p, count);
set up DMA controller( );
scheduler( );
357
acknowledge interrupt( );
unblock user( );
return from interrupt( );
(a)
(b)
Figure 5-10. Printing a string using DMA. (a) Code executed when the print
system call is made. (b) Interrupt-service procedure.
5.3 I/O SOFTWARE LAYERS
I/O software is typically organized in four layers, as shown in Fig. 5-11. Each
layer has a well-defined function to perform and a well-defined interface to the
adjacent layers. The functionality and interfaces differ from system to system, so
the discussion that follows, which examines all the layers starting at the bottom, is
not specific to one machine.
User-level I/O software
Device-independent operating system software
Device drivers
Interrupt handlers
Hardware
Figure 5-11. Layers of the I/O software system.
5.3.1 Interrupt Handlers
While programmed I/O is occasionally useful, for most I/O, interrupts are an
unpleasant fact of life and cannot be avoided. They should be hidden away, deep in
the bowels of the operating system, so that as little of the operating system as possible knows about them. The best way to hide them is to have the driver starting an
I/O operation block until the I/O has completed and the interrupt occurs. The driver
can block itself, for example, by doing a down on a semaphore, a wait on a condition variable, a receive on a message, or something similar.
When the interrupt happens, the interrupt procedure does whatever it has to in
order to handle the interrupt. Then it can unblock the driver that was waiting for it.
In some cases it will just complete up on a semaphore. In others, it will do a signal
on a condition variable in a monitor. In still others, it will send a message to the
blocked driver. In all cases, the net effect of the interrupt will be that a driver that
358
INPUT/OUTPUT
CHAP. 5
was previously blocked will now be able to run. This model works best if drivers
are structured as processes (either in kernel mode or user mode), with their own
states, stacks, and program counters.
Of course, reality is not quite so simple. Processing an interrupt is not just a
matter of taking the interrupt, doing an up on some semaphore, and then executing
an IRET instruction to return from the interrupt to the previous process. There is a
great deal more work involved for the operating system. We will now give an outline of this work as a series of steps that must be performed in software after the
hardware interrupt discussed earlier has completed. It should be noted that the
details are highly system dependent, so some of the steps listed below may not be
needed on a particular machine, and steps not listed may be required. Also, the
steps that do occur may be in a different order on some machines.
1. Save any registers (including the PSW) that have not already been
saved by the interrupt hardware.
2. Set up a context for the interrupt-service procedure. Doing this may
involve setting up the TLB, MMU, and a page table.
3. Set up a stack for the interrupt service-procedure.
4. Acknowledge the interrupt controller. If there is no centralized interrupt controller, reenable interrupts.
5. Copy the registers from where they were saved (possibly some stack)
to the process table.
6. Run the interrupt-service procedure. Typically, it will extract information from the interrupting device controller’s registers.
7. Choose which process to run next. If the interrupt has caused some
high-priority process that was blocked to become ready, it may be
chosen to run now.
8. Set up the MMU context for the process to run next. Some TLB setup may also be needed.
9. Load the new process’ registers, including its PSW.
10. Start running the new process.
As can be seen, interrupt processing is far from trivial. It also takes a considerable
number of CPU instructions, especially on machines in which virtual memory is
present and page tables have to be set up or the state of the MMU stored (e.g., the
R and M bits). On some machines, the TLB and CPU cache may also have to be
managed when switching between user and kernel modes, which takes additional
machine cycles if many entries need to be purged.
SEC. 5.3
I/O SOFTWARE LAYERS
359
5.3.2 Device Drivers
Earlier in this chapter we looked at what device controllers do. We saw that
each controller has some device registers used to give it commands or some device
registers used to read out its status or both. The number of device registers and the
nature of the commands vary radically from device to device. For example, a
mouse driver has to accept information from the mouse telling it how far it has
moved and which buttons are currently depressed. In contrast, a disk driver may
have to know all about sectors, tracks, cylinders, heads, arm motion, motor drives,
head settling times, and all the other mechanics of making the disk work properly.
Obviously, these drivers will be very different.
Consequently, each I/O device attached to a computer needs some device-specific code for controlling it. This code, called the device driver, is generally written by the device’s manufacturer and delivered along with the device. Since each
operating system needs its own drivers, device manufacturers commonly supply
drivers for several popular operating systems.
Each device driver normally handles one device type, or at most, one class of
closely related devices. For example, a SATA disk driver can usually handle multiple SATA SSDs and SATA disks of different sizes and different speeds. On the
other hand, a mouse and joystick are so different that different drivers are usually
required. However, there is no technical restriction on having one device driver
control multiple unrelated devices. It is just not a good idea in most cases.
Sometimes though, wildly different devices are based on the same underlying
technology. The best-known example is probably USB (Universal Serial Bus). It
is not called ‘‘universal’’ for nothing. USB devices include disks, mice, memory
sticks, cameras, keyboards, mini-fans, robots, credit card readers, bar code scanners, rechargeable shavers, paper shredders, disco balls, and thermometers. They
all use USB and yet they all do very different things. The trick is that USB drivers
are typically stacked, like a TCP/IP stack in networks. At the bottom, typically in
hardware, we find the USB link layer (serial I/O) that handles hardware stuff like
signaling and decoding a stream of signals to USB packets. It is used by higher
layers that deal with the data packets and the common functionality for USB that is
shared by most devices. On top of that, finally, we find the higher-layer APIs such
as the interfaces for mass storage, cameras, etc. Thus, we still have separate device
drivers, even though they share part of the protocol stack.
In order to access the device’s hardware, meaning the controller’s registers, the
device driver normally has to be part of the operating system kernel, at least with
current architectures. Actually, it is possible to construct drivers that run in user
space, with system calls for reading and writing the device registers. This design
isolates the kernel from the drivers and the drivers from each other, eliminating a
major source of system crashes—buggy drivers that interfere with the kernel in one
way or another. For building highly reliable systems, this is definitely a good way
to go. An example of a system in which the device drivers run as user processes is
360
INPUT/OUTPUT
CHAP. 5
MINIX 3 (www.minix3.org). However, since most other desktop operating systems
run their drivers in the kernel, that is the model we will consider here.
Since the designers of every operating system know that pieces of code (drivers) written by outsiders will be installed in it, it needs to have an architecture that
allows such installation. This means having a well-defined model of what a driver
does and how it interacts with the rest of the operating system. Device drivers are
normally positioned below the rest of the operating system, as is illustrated in
Fig. 5-12.
User process
User
program
User
space
Rest of the operating system
Kernel
space
Hardware
Printer
driver
Camera
driver
Printer controller
Camera controller
SSD
driver
SSD controller
Devices
Figure 5-12. Logical positioning of device drivers. In reality, all communication
between drivers and device controllers goes over the bus.
Operating systems usually classify drivers into one of a small number of categories. The most common categories are the block devices, such as disks, which
contain multiple data blocks that can be addressed independently, and the character
devices, such as keyboards and printers, which generate or accept a stream of characters.
Most operating systems define a standard interface that all block drivers must
support and a second standard interface that all character drivers must support.
These interfaces consist of a number of procedures that the rest of the operating
SEC. 5.3
I/O SOFTWARE LAYERS
361
system can call to get the driver to do work for it. Typical procedures are those to
read a block (block device) or write a character string (character device).
In some systems, the operating system is a single binary program that contains
all of the drivers it will need compiled into it. This scheme was the norm for years
with UNIX systems because they were run by computer centers and I/O devices
rarely changed. If a new device was added, the system administrator simply recompiled the kernel with the new driver to build a new binary.
With the advent of personal computers, with their myriad I/O devices, this
model no longer worked. Few users are capable of recompiling or relinking the
kernel, even if they have the source code or object modules, which is not always
the case. Instead, operating systems, starting with MS-DOS, went over to a model
in which drivers were dynamically loaded into the system during execution. Different systems handle loading drivers in different ways.
A device driver has several functions. The most obvious one is to accept
abstract read and write requests from the device-independent software above it and
see that they are carried out. But there are also a few other functions it must perform. For example, the driver must initialize the device, if needed. It may also
need to manage its power requirements and log events.
Many device drivers have a similar general structure. A typical driver starts
out by checking the input parameters to see if they are valid. If not, an error is
returned. If they are valid, a translation from abstract to concrete terms may be
needed. For a disk driver, this may mean converting a linear block number into the
head, track, sector, and cylinder numbers for the disk’s geometry, while for SSDs
the block number should be mapped on the appropriate flash block and page.
Next the driver may check if the device is currently in use. If it is, the request
will be queued for later processing. If the device is idle, the hardware status will
be examined to see if the request can be handled now. It may be necessary to
switch the device on or start a motor before transfers can be begun. On inkjet
printers, the print head has to do a little dance before it can start printing. Once the
device is on and ready to go, the actual control can begin.
Controlling the device means issuing a sequence of commands to it. The driver
is the place where the command sequence is determined, depending on what has to
be done. After the driver knows which commands it is going to issue, it starts writing them into the controller’s device registers. After each command is written to
the controller, it may be necessary to check to see if the controller accepted the
command and is prepared to accept the next one. This sequence continues until all
the commands have been issued. Some controllers can be given a linked list of
commands (in memory) and told to read and process them all by itself without further help from the operating system.
After the commands have been issued, one of two situations will apply. In
many cases, the device driver must wait until the controller does some work for it,
so it blocks itself until the interrupt comes in to unblock it. In other cases, however, the operation finishes without delay, so the driver does not need to block. As an
362
INPUT/OUTPUT
CHAP. 5
example of the latter situation, scrolling the screen requires just writing a few bytes
into the controller’s registers. No mechanical motion is needed, so the entire operation can be completed in nanoseconds.
In the former case, the blocked driver will be awakened by the interrupt. In the
latter case, it will never go to sleep. Either way, after the operation has been completed, the driver must check for errors. If everything is all right, the driver may
have some data to pass to the device-independent software (e.g., a block just read).
Finally, it returns some status information for error reporting back to its caller. If
any other requests are queued, one of them can now be selected and started. If
nothing is queued, the driver blocks waiting for the next request.
This simple model is only a rough approximation to reality. Many factors make
the code much more complicated. For one thing, an I/O device may complete
while a driver is running, interrupting the driver. The interrupt may cause a device
driver to run. In fact, it may cause the current driver to run. For example, while the
network driver is processing an incoming packet, another packet may arrive. Consequently, drivers have to be reentrant, meaning that a running driver has to
expect that it will be called a second time before the first call has completed.
In a hot-pluggable system, devices can be added or removed while the computer is running. As a result, while a driver is busy reading from some device, the
system may inform it that the user has suddenly removed that device from the system. Not only must the current I/O transfer be terminated without damaging any
kernel data structures, but any pending requests for the now-vanished device must
also be gracefully removed from the system and their callers given the bad news.
Furthermore, the unexpected addition of new devices may cause the kernel to juggle resources (e.g., interrupt request lines), taking old ones away from the driver
and giving it new ones in their place.
Drivers are not allowed to make system calls, but they often need to interact
with the rest of the kernel. Usually, calls to certain kernel procedures are permitted.
For example, there are usually calls to allocate and deallocate hardwired pages of
memory for use as buffers. Other useful calls are needed to manage the MMU,
timers, the DMA controller, the interrupt controller, and so on.
5.3.3 Device-Independent I/O Software
Although some of the I/O software is device specific, other parts of it are device independent. The exact boundary between the drivers and the device-independent software is system (and device) dependent, because some functions that could
be done in a device-independent way may actually be done in the drivers, for
efficiency or other reasons. The functions shown in Fig. 5-13 are typically done in
the device-independent software.
The basic function of the device-independent software is to perform the I/O
functions that are common to all devices and to provide a uniform interface to the
user-level software. We will now look at the above issues in more detail.
SEC. 5.3
363
I/O SOFTWARE LAYERS
Uniform interfacing for device drivers
Buffering
Error reporting
Allocating and releasing dedicated devices
Providing a device-independent block size
Figure 5-13. Functions of the device-independent I/O software.
Uniform Interfacing for Device Drivers
A major issue in an operating system is how to make all I/O devices and drivers look more or less the same. If disks, printers, keyboards, and so on, are all
interfaced in different ways, every time a new device comes along, the operating
system must be modified for the new device. Having to hack on the operating system for each new device is not a good idea.
One aspect of this issue is the interface between the device drivers and the rest
of the operating system. In Fig. 5-14(a), we illustrate a situation in which each device driver has a different interface to the operating system. What this means is that
the driver functions available for the system to call differ from driver to driver. It
might also mean that the kernel functions that the driver needs also differ from
driver to driver. Taken together, it means that interfacing each new driver requires a
lot of new programming effort.
Operating system
Operating system
SATA disk driver USB disk driver SCSI disk driver
SATA disk driver USB disk driver SCSI disk driver
(a)
(b)
Figure 5-14. (a) Without a standard driver interface. (b) With a standard driver
interface.
In contrast, in Fig. 5-14(b), we show a different design in which all drivers
have the same interface. Now it becomes much easier to plug in a new driver, providing it conforms to the driver interface. It also means that driver writers know
what is expected of them. In practice, not all devices are absolutely identical, but
364
INPUT/OUTPUT
CHAP. 5
usually there are only a small number of device types and even these are generally
almost the same, or the differ in only minor ways.
The way this works is as follows. For each class of devices, such as disks or
printers, the operating system defines a set of functions that the driver must supply.
For a disk these would naturally include read and write, but also turning the power
on and off, formatting, and other disky things. Often the driver holds a table with
pointers into itself for these functions. When the driver is loaded, the operating
system records the address of this table of function pointers, so when it needs to
call one of the functions, it can make an indirect call via this table. This table of
function pointers defines the interface between the driver and the rest of the operating system. All devices of a given class (disks, printers, etc.) must obey it.
Another aspect of having a uniform interface is how I/O devices are named.
The device-independent software takes care of mapping symbolic device names
onto the proper driver. For example, in UNIX a device name, such as /dev/disk0,
uniquely specifies the i-node for a special file, and this i-node contains the major
device number, which is used to locate the appropriate driver. The i-node also
contains the minor device number, which is passed as a parameter to the driver in
order to specify the unit to be read or written. All devices have major and minor
numbers, and all drivers are accessed by using the major device number to select
the driver.
Closely related to naming is protection. How does the system prevent users
from accessing devices that they are not entitled to access? In both UNIX and
Windows, devices appear in the file system as named objects, which means that the
usual protection rules for files also apply to I/O devices. The system administrator
can then set the proper permissions for each device.
Buffering
Buffering is also an issue, both for block and character devices, for a variety of
reasons. To see one of them, consider a process that wants to read data from an
(VDSL—Very High Bitrate Digital Subscriber Line) modem, something many
people use at home to connect to the Internet. One possible strategy for dealing
with the incoming characters is to have the user process do a read system call and
block waiting for one character. Each arriving character causes an interrupt. The
interrupt-service procedure hands the character to the user process and unblocks it.
After putting the character somewhere, the process reads another character and
blocks again. This model is indicated in Fig. 5-15(a).
The trouble with this way of doing business is that the user process has to be
started up for every incoming character. Allowing a process to run many times for
short runs is inefficient, so this design is not a good one.
An improvement is shown in Fig. 5-15(b). Here the user process provides an
n-character buffer in user space and does a read of n characters. The interrupt-
SEC. 5.3
365
I/O SOFTWARE LAYERS
User process
User
space
Kernel
space
2
2
1
1
3
Modem
Modem
Modem
Modem
(a)
(b)
(c)
(d)
Figure 5-15. (a) Unbuffered input. (b) Buffering in user space. (c) Buffering in
the kernel followed by copying to user space. (d) Double buffering in the kernel.
service procedure puts incoming characters in this buffer until it is completely full.
Only then does it wakes up the user process. This scheme is far more efficient than
the previous one, but it has a drawback: what happens if the buffer is paged out
when a character arrives? The buffer could be locked in memory, but if many
processes start locking pages in memory willy nilly, the pool of available pages
will shrink and performance will degrade.
Yet another approach is to create a buffer inside the kernel and have the interrupt handler put the characters there, as shown in Fig. 5-15(c). When this buffer is
full, the page with the user buffer is brought in, if needed, and the buffer copied
there in one operation. This scheme is far more efficient.
However, even this improved scheme suffers from a problem: What happens to
characters that arrive while the page with the user buffer is being brought in from
the disk? Since the buffer is full, there is no place to put them. A way out is to
have a second kernel buffer. After the first buffer fills up, but before it has been
emptied, the second one is used, as shown in Fig. 5-15(d). When the second buffer
fills up, it is available to be copied to the user (assuming the user has asked for it).
While the second buffer is being copied to user space, the first one can be used for
new characters. In this way, the two buffers take turns: while one is being copied
to user space, the other is accumulating new input. A buffering scheme like this is
called double buffering.
Another common form of buffering is the circular buffer. It consists of a
region of memory and two pointers. One pointer points to the next free word,
where new data can be placed. The other pointer points to the first word of data in
the buffer that has not been removed yet. In many situations, the hardware
advances the first pointer as it adds new data (e.g., just arriving from the network)
and the operating system advances the second pointer as it removes and processes
data. Both pointers wrap around, going back to the bottom when they hit the top.
366
INPUT/OUTPUT
CHAP. 5
Buffering is also important on output. Consider, for example, how output is
done to the modem without buffering using the model of Fig. 5-15(b). The user
process executes a write system call to output n characters. The system has two
choices at this point. It can block the user until all the characters have been written, but this could take a very long time over a slow telephone line. It could also
release the user immediately and do the I/O while the user computes some more,
but this leads to an even worse problem: how does the user process know that the
output has been completed and it can reuse the buffer? The system could generate
a signal or software interrupt, but that style of programming is difficult and prone
to race conditions. A much better solution is for the kernel to copy the data to a
kernel buffer, analogous to Fig. 5-15(c) (but the other way), and unblock the caller
immediately. Now it does not matter when the actual I/O has been completed. The
user is free to reuse the buffer the instant it is unblocked.
Buffering is a widely used technique, but it has a downside as well. If data get
buffered too many times, performance suffers. Consider, for example, the network
of Fig. 5-16. When a user performs a system call to write to the network, the kernel copies the packet to a kernel buffer to allow the user to proceed immediately
(step 1). At this point, the user program can reuse the buffer.
User process
User
space
Kernel
space
5
1
2
4
Network
controller
3
Network
Figure 5-16. Networking may involve many copies of a packet.
When the driver is called, it copies the data to the controller for output (step 2).
The reason it does not output to the wire directly from kernel memory is that once
a packet transmission has been started, it must continue at a uniform speed. The
driver cannot guarantee that it can get to memory at a uniform speed because DMA
channels and other I/O devices may be stealing many cycles. Failing to get a word
transmitted on time would ruin the packet. By buffering the packet inside the controller, this problem is avoided.
After the packet has been copied to the controller’s internal buffer, it is copied
out onto the network (step 3). Bits arrive at the receiver shortly after being sent, so
SEC. 5.3
I/O SOFTWARE LAYERS
367
just after the last bit has been sent, that bit arrives at the receiver, where the packet
has been buffered in the controller. Next the packet is copied to the receiver’s kernel buffer (step 4). Finally, it is copied to the receiving process’ buffer (step 5).
Usually, the receiver then sends back an acknowledgement. When the sender gets
the acknowledgement, it is free to send the next packet. However, it should be
clear that all this copying is going to slow down the transmission rate considerably
because all the steps must happen sequentially.
Error Reporting
Errors are far more common in the context of I/O than in other contexts. When
they occur, the operating system must handle them as best it can. Many errors are
device specific and must be handled by the appropriate driver, but the framework
for error handling is device independent.
One class of I/O errors is programming errors. These occur when a process
asks for something impossible, such as writing to an input device (keyboard, scanner, mouse, etc.) or reading from an output device (printer, plotter, etc.). Other
errors are providing an invalid buffer address or other parameter, and specifying an
invalid device (e.g., drive 3 when the system has only two drives), and so on. The
action to take on these errors is straightforward: just report back an error code to
the caller.
Another class of errors is the class of actual I/O errors, for example, trying to
write a block that has been damaged or trying to read from a camera that has been
switched off. In these circumstances, it is up to the driver to determine what to do.
If the driver does not know what to do, it may pass the problem back up to device-independent software.
What this software does depends on the environment and the nature of the
error. If it is a simple read error and there is an interactive user available, it may
display a dialog box asking the user what to do. The options may include retrying a
certain number of times, ignoring the error, or killing the calling process. If there
is no user available, probably the only real option is to have the system call fail
with an error code.
However, some errors cannot be handled this way. For example, a critical data
structure, such as the root directory or free block list, may have been destroyed. In
this case, the system may have to display an error message and terminate. There is
not much else it can do.
Allocating and Releasing Dedicated Devices
Some devices, such as printers, can be used only by a single process at any
given moment. It is up to the operating system to examine requests for device
usage and accept or reject them, depending on whether the requested device is
available or not. A simple way to handle these requests is to require processes to
368
INPUT/OUTPUT
CHAP. 5
perform opens on the special files for devices directly. If the device is unavailable,
the open fails. Closing such a dedicated device then releases it.
An alternative approach is to have special mechanisms for requesting and
releasing dedicated devices. An attempt to acquire a device that is not available
blocks the caller instead of failing. Blocked processes are put on a queue. Sooner
or later, the requested device becomes available and the first process on the queue
is allowed to acquire it and continue execution.
Device-Independent Block Size
Different SSDs have different flash page sizes, while different disks may have
different sector sizes. It is up to the device-independent software to hide this fact
and provide a uniform block size to higher layers, for example, by treating several
sectors or flash pages as a single logical block. In this way, the higher layers deal
only with abstract devices that all use the same logical block size, independent of
the physical sector size. Similarly, some character devices deliver their data one
byte at a time (e.g., mice), while others deliver theirs in larger units (e.g., Ethernet
interfaces). These differences may also be hidden.
5.3.4 User-Space I/O Software
Although most of the I/O software is within the operating system, a small portion of it consists of libraries linked together with user programs, and even whole
programs running outside the kernel. System calls, including the I/O system calls,
are normally made by library procedures. When a C program contains the call
count = write(fd, buffer, nbytes);
the library procedure write might be linked with the program and contained in the
binary program present in memory at run time. In other systems, libraries can be
loaded during program execution. Either way, the collection of all these library
procedures is clearly part of the I/O system.
While most of these procedures do little more than put their parameters in the
appropriate place for the system call, other I/O procedures actually do real work.
In particular, formatting of input and output is done by library procedures. One
example from C is printf, which takes a format string and possibly some variables
as input, builds an ASCII string, and then calls write to output the string. As an
example of printf, consider the statement
printf("The square of %3d is %6d\n", i, i*i);
It formats a string consisting of the 14-character string ‘‘The square of ’’ followed
by the value i as a 3-character string, then the 4-character string ‘‘ is ’’, then i 2 as 6
characters, and finally a line feed.
SEC. 5.3
369
I/O SOFTWARE LAYERS
An example of a similar procedure for input is scanf, which reads input and
stores it into variables described in a format string using the same syntax as printf.
The standard I/O library contains a number of procedures that involve I/O and all
run as part of user programs.
Not all user-level I/O software consists of library procedures. Another important category is the spooling system. Spooling is a way of dealing with dedicated
I/O devices in a multiprogramming system. Consider a typical spooled device: a
printer. Although it would be technically easy to let any user process open the
character special file for the printer, suppose a process opened it and then did nothing for hours. No other process could print anything.
Instead what is done is to create a special process, called a daemon, and a special directory, called a spooling directory. To print a file, a process first generates
the entire file to be printed and puts it in the spooling directory. It is up to the daemon, which is the only process having permission to use the printer’s special file,
to print the files in the directory. By protecting the special file against direct use by
users, the problem of having someone keeping it open unnecessarily long is eliminated.
Figure 5-17 summarizes the I/O system, showing all the layers and the principal functions of each layer. Starting at the bottom, the layers are the hardware,
interrupt handlers, device drivers, device-independent software, and finally the user
processes.
Layer
I/O
request
User processes
Device-independent
software
I/O
reply
I/O functions
Make I/O call; format I/O; spooling
Naming, protection, blocking, buffering, allocation
Device drivers
Set up device registers; check status
Interrupt handlers
Wake up driver when I/O completed
Hardware
Perform I/O operation
Figure 5-17. Layers of the I/O system and the main functions of each layer.
The arrows in Fig. 5-17 show the flow of control. When a user program tries to
read a block from a file, for example, the operating system is invoked to carry out
the call. The device-independent software looks for it, say, in the buffer cache. If
the needed block is not there, it calls the device driver to issue the request to the
hardware to go get it from the SSD or disk. The process is then blocked until this
operation has been completed and the data are safely available in the caller’s buffer. The operation may take milliseconds, which is too long for the CPU to be idle.
370
INPUT/OUTPUT
CHAP. 5
When the SSD or disk is finished, the hardware generates an interrupt. The
interrupt handler is run to discover what has happened, that is, which device wants
attention right now. It then extracts the status from the device and wakes up the
sleeping process to finish off the I/O request and let the user process continue.
5.4 MASS STORAGE: DISK AND SSD
Now we will begin studying some real I/O devices. We will begin with storage
devices. In later sections, we examine clocks, keyboards, and displays. Modern
storage devices come in a variety of types. The most common ones are magnetic
hard disks and SSDs. For distribution of programs, data, and movies, old fogeys
may still use optical disks (DVDs and Blu-ray), but these are rapidly going out of
fashion and will not be discussed in this (undeniably fashionable) book. Instead,
we briefly discuss magnetic disks and SSDs. We will start with the former, as it is a
nice case study.
5.4.1 Magnetic Disks
Magnetic disks are characterized by the fact that reads and writes are equally
fast, which makes them suitable as secondary memory (paging, file systems, etc.).
Arrays of these disks are sometimes used to provide highly reliable storage. They
are organized into cylinders, each one containing as many tracks as there are heads
stacked vertically. The tracks are divided into sectors, with typically up to several
hundreds of sectors around the circumference. The number of heads varies from 1
to about 16.
Older disks had little electronics and just delivered a simple serial bit stream.
On these disks, the controller did most of the work. On other disks, in particular,
SATA (Serial ATA) disks, the drive itself contains a microcontroller that does considerable work and allows the real disk controller to issue a set of higher-level
commands. The controller does track caching, bad-block remapping, and more.
A device feature that has important implications for the disk driver is the possibility of a controller doing seeks on two or more drives at the same time. These are
known as overlapped seeks. While the controller and software are waiting for a
seek to complete on one drive, the controller can initiate a seek on another drive.
Many controllers can also read or write on one drive while seeking on one or more
other drives. Moreover, a system with several hard disks with integrated controllers
can operate them simultaneously, at least to the extent of transferring between the
disk and the controller’s buffer memory. However, only one transfer between the
controller and the main memory is possible at once. The ability to perform two or
more operations at the same time can reduce the average access time considerably.
If we compare the standard storage medium of the original IBM PC (a floppy
disk) with a modern hard disk, such as the Seagate IronWolf Pro, we see that many
SEC. 5.4
371
MASS STORAGE: DISK AND SSD
things have changed. First, the disk capacity of the old floppy disk was 360 KB, or
about one third of the capacity needed to store the PDF of just this chapter. In contrast, the IronWolf packs as much as 18 TB—an increase of give or take 8 orders of
magnitude. The authors solemnly promise never to make the chapter that big. The
transfer rate also went up from around 23 KB/sec to 250 MB/sec, a jump of 4
orders of magnitude. The latency, however, improved more marginally, from
around 100 msec to 4 msec. Better, but you may find it a bit underwhelming.
One thing to be aware of in looking at the specifications of modern hard disks
is that the geometry specified, and used by the driver software, is almost always
different from the physical format. On old disks, the number of sectors per track
was the same for all cylinders. For instance, the IBM PC floppy disk had 9 sectors
of 512 bytes on every track. Modern disks, on the other hand, are divided into
zones with more sectors on the outer zones than the inner ones. Figure 5-18(a) illustrates a tiny disk with two zones. The outer zone has 32 sectors per track; the
inner one has 16 sectors per track. A real disk easily has tens of zones, with the
number of sectors increasing per zone as one goes out from the innermost to the
outermost zone.
30
31
0
1
23
2
26
21
20
4
19
5
7
7 8 9
16
89
10
15
11
12
13
14
11
20
10
21
18
6
9
17
8
6
11 12 13
10
2
7
3 4 5
23
1
6
24 2
5
5
0 1
2
22
0
3
4
15
14
24
22
3
27
28
29
12
13
14
15
16
17
18
19
Figure 5-18. (a) Physical geometry of a disk with two zones. (b) A possible virtual geometry for this disk.
To hide the details of how many sectors each track has, most modern disks
have a virtual geometry that is presented to the operating system. The software is
instructed to act as though there are x cylinders, y heads, and z sectors per track.
The controller then remaps a request for (x, y, z) onto the real cylinder, head, and
sector. A possible virtual geometry for the physical disk of Fig. 5-18(a) is shown
in Fig. 5-18(b). In both cases, the disk has 192 sectors, only the published arrangement is different than the real one. Simplifying the addressing even more, modern
372
INPUT/OUTPUT
CHAP. 5
disks now support a system called logical block addressing, in which disk sectors
are just numbered consecutively starting at 0, without regard to the disk geometry.
Disk Formatting
A hard disk consists of a stack of aluminum, alloy, or glass platters typically
3.5 inch in diameter (or 2.5 inch on notebook computers). On each platter is
deposited a thin magnetizable metal oxide. After manufacturing, there is no information whatsoever on the disk.
Before the disk can be used, each platter must receive a low-level format done
by software. The format consists of a series of concentric tracks, each containing
some number of sectors, with short gaps between the sectors. The format of a sector is shown in Fig. 5-19.
Preamble
Data
ECC
Figure 5-19. A disk sector.
The preamble starts with a certain bit pattern that allows the hardware to recognize the start of the sector. It also contains the cylinder and sector numbers and
some other information. The size of the data portion is determined by the lowlevel formatting program. Most disks use 512-byte sectors. The ECC field contains redundant information that can be used to recover from read errors. The size
and content of this field varies from manufacturer to manufacturer, depending on
how much disk space the designer is willing to give up for higher reliability and
how complex an ECC code the controller can handle. A 16-byte ECC field is not
unusual. Furthermore, all hard disks have some number of spare sectors allocated
to be used to replace sectors with a manufacturing defect.
The position of sector 0 on each track is offset from the previous track when
the low-level format is laid down. This offset, called cylinder skew, is done to
improve performance. The idea is to allow the disk to read multiple tracks in one
continuous operation without losing data. The nature of the problem can be seen
by looking at Fig. 5-18(a). Suppose that a request needs 18 sectors starting at sector 0 on the innermost track. Reading the first 16 sectors takes one disk rotation,
but a seek is needed to move outward one track to get the 17th sector. By the time
the head has moved one track, sector 0 has rotated past the head so an entire rotation is needed until it comes by again. That problem is eliminated by offsetting the
sectors as shown in Fig. 5-20.
The amount of cylinder skew depends on the drive geometry. For example, a
10,000-RPM (Revolutions Per Minute) drive rotates in 6 msec. If a track contains
300 sectors, a new sector passes under the head every 20 µ sec. If the track-to-track
seek time is 800 µ sec, 40 sectors will pass by during the seek, so the cylinder skew
373
MASS STORAGE: DISK AND SSD
27
6
3
31
28
30
8 9
6
5
3
2
0
31
4
1
32
4
13
10
9
8
5
2
13
23 24 25 2
62
7
22
21
20
7
18
17
10 11
12
7
1
29
19
11 12 13 14 15
10
16
4
0
15 16 17 1
8
12
14
1
5
4
1
1
1
6
13
5
17
8
12
18
7 18 19
1
6
1
1
20
1
15
21
21 22
0
2
14
9
1
2
26
30
2
9
25
29
1
5
5
24
28
0
4
8
10
27
12
23
26
30 31
29
11
2 3
3 4 5 6 7
1
2
7 8 9 10
11
25
28
0
3
22
11
24
1
0
31
26
25
27
10
31 0 1 2
20
22
22 23 24
23
21
25
20
5 26 27
26
24 2
19
28
3
2
29
2
2
27 28 29 30 3
17
30
9
30
29
28
18
Direction of disk
rotation
19
5 6
8
21
4
7
20
1
6
18
19
21
16
17
14
13
12
13
16
15
14
9
SEC. 5.4
7
6
4
3
Figure 5-20. An illustration of cylinder skew.
should be at least 40 sectors, rather than the three sectors shown in Fig. 5-20. It is
worth mentioning that switching between heads also takes a finite time, so there is
head skew as well as cylinder skew, but head skew is not very large, usually much
less than one sector time.
As a result of the low-level formatting, disk capacity is reduced, depending on
the sizes of the preamble, intersector gap, and ECC, as well as the number of spare
sectors reserved. Often the formatted capacity is 20% lower than the unformatted
capacity. The spare sectors do not count toward the formatted capacity, so all disks
of a given type have exactly the same capacity when shipped, independent of how
many bad sectors they actually have (if the number of bad sectors exceeds the
number of spares, the drive will be rejected and not shipped).
There is considerable confusion about disk capacity because some manufacturers advertised the unformatted capacity to make their drives look larger than they
in reality are. For example, let us consider a drive whose unformatted capacity is
20 × 1012 bytes. This might be sold as a 20-TB disk. However, after formatting,
possibly only 17 × 1012 bytes are available for data. To add to the confusion, the
operating system may report this capacity as 15 TB, not 17 TB, because software
considers a memory of 1 TB to be 240 (1.099.511.627.776) bytes, not 1012
(1,000,000,000,000) bytes. It would be better if this were reported as 15 TiB.
374
INPUT/OUTPUT
CHAP. 5
To make things even worse, in the world of data communications, 1 Tbps
means 1,000,000,000,000 bits/sec because the prefix Tera really does mean 1012 (a
kilometer is 1000 meters, not 1024 meters, after all). Only with memory and disk
sizes do kilo, mega, giga, tera, peta, exa, and zetta mean 210 , 220 , 230 , 240 , 250 , 260 ,
and 270 , respectively.
To avoid confusion, some authors use the prefixes kilo, mega, giga, tera, peta,
exa, and zetta to mean 103 , 106 , 109 , 1012 , 1015 , 1018 , and 1021 respectively, while
using kibi, mebi, gibi, tebi, pebi, exbi, and zebi to mean 210 , 220 , 230 , etc. However, the use of the ‘‘b’’ prefixes is relatively rare. Just in case you like really big
numbers, one yottabyte is 1024 and a yobibyte 280 bytes.
Formatting also affects performance. If a track on a 10,000-RPM disk has 300
sectors of 512 bytes each, it takes 6 msec to read the 153,600 bytes on the track for
a data rate of 25,600,000 bytes/sec or 24.4 MB/sec. It is not possible to go faster
than this, no matter what kind of interface is present, even if it is a SATA interface
at 6 GB/sec.
Actually reading continuously at this rate requires a large buffer in the controller. Consider, for example, a controller with a one-sector buffer that has been
given a command to read two consecutive sectors. After reading the first sector
from the disk and doing the ECC calculation, the data must be transferred to main
memory. While this transfer is taking place, the next sector will fly by the head.
When the copy to memory is complete, the controller will have to wait almost an
entire rotation time for the second sector to come around again.
This problem can be eliminated by numbering the sectors in an interleaved
fashion when formatting the disk. In Fig. 5-21(a), we see the usual numbering pattern (ignoring cylinder skew here). In Fig. 5-21(b), we see single interleaving,
which gives the controller some breathing space between consecutive sectors in
order to copy the buffer to main memory.
7
0
6
5
4
3
(a)
7
1
3
2
6
0
2
5
(b)
5
4
2
1
7
0
3
6
4
1
(c)
Figure 5-21. (a) No interleaving. (b) Single interleaving. (c) Double interleaving.
If the copying process is very slow, the double interleaving of Fig. 5-22(c)
may be needed. If the controller has a buffer of only one sector, it does not matter
whether the copying from the buffer to main memory is done by the controller, the
main CPU, or even a DMA chip; it still takes some time. To avoid the need for
SEC. 5.4
MASS STORAGE: DISK AND SSD
375
interleaving, the controller should be able to buffer an entire track. With hundreds
of MB of memory, most modern controllers can buffer many entire tracks.
After low-level formatting is completed, the disk is partitioned. Logically, each
partition is like a separate disk. Partitions are needed to allow multiple operating
systems to coexist. Also, in some cases, a partition can be used for swapping. In
older other computers, sector 0 contains the MBR (Master Boot Record), which
contains some boot code plus the partition table at the end. The MBR, and thus
support for partition tables, first appeared in IBM PCs in 1983 to support the thenmassive 10-MB hard drive in the PC XT. Disks have grown a bit since then. As
MBR partition entries in most systems are limited to 32 bits, the maximum disk
size that can be supported with 512 B sectors is 2 TB. For this reason, most operating since now also support the new GPT (GUID Partition Table), which supports
disk sizes up to 9.4 ZB (9,444,732,965,739,290,426,880 bytes or some 8 ZiB). At
the time this book went to press, this was considered a lot of bytes.
The partition table gives the starting sector and size of each partition. You can
see more about the GPT in UEFI in Sec. 4.3. If there are four partitions and all of
them are for Windows, they will be called C:, D:, E:, and F: and treated as separate
drives. If three of them are for Windows and one is for UNIX, then Windows will
call its partitions C:, D:, and E:. If a USB drive is added, it will be F:. To be able
to boot from the hard disk, one partition must be marked as active in the partition
table.
The final step in preparing a disk for use is to perform a high-level format of
each partition (separately). This operation lays down a boot block, the free storage
administration (free list or bitmap), root directory, and an empty file system. It
also puts a code in the partition table entry telling which file system is used in the
partition because many operating systems support multiple incompatible file systems (for historical reasons). At this point the system can be booted.
We already saw in Chap. 1 that when the power is turned on, the BIOS runs
initially and reads the GPT. It then finds the appropriate bootloader and executes i
to boot the operating system.
Disk Arm Scheduling Algorithms
In this section, we will look at some issues related to disk drivers in general.
First, consider how long it takes to read or write a disk block. The time required is
determined by three factors:
1. Seek time (the time to move the arm to the proper cylinder).
2. Rotational delay (how long for the proper sector to appear under the
reading head).
3. Actual data transfer time.
For most disks, the seek time dominates the other two times, so reducing the mean
seek time can improve system performance substantially.
376
INPUT/OUTPUT
CHAP. 5
If the disk driver accepts requests one at a time and carries them out in that
order, that is, FCFS (First-Come, First-Served), little can be done to optimize
seek time. However, another strategy is possible when the disk is heavily loaded. It
is likely that while the arm is seeking on behalf of one request, other disk requests
may be generated by other processes. Many disk drivers maintain a table, indexed
by cylinder number, with all the pending requests for each cylinder chained together in a linked list headed by the table entries.
Given this kind of data structure, we can improve upon the first-come, firstserved scheduling algorithm. To see how, consider an imaginary disk with 40 cylinders. A request comes in to read a block on cylinder 11. While the seek to cylinder 11 is in progress, new requests come in for cylinders 1, 36, 16, 34, 9, and 12, in
that order. They are entered into the table of pending requests, with a separate linked list for each cylinder. The requests are shown in Fig. 5-22.
Initial
position
X
Time
0
X
5
Pending
requests
X X
10
X
15
X
20
25
30
X
35 Cylinder
Sequence of seeks
Figure 5-22. Shortest Seek First (SSF) disk scheduling algorithm.
When the current request (for cylinder 11) is finished, the disk driver has a
choice of which request to handle next. Using FCFS, it would go next to cylinder
1, then to 36, and so on. This algorithm would require arm motions of 10, 35, 20,
18, 25, and 3, respectively, for a total of 111 cylinders.
Alternatively, it could always handle the closest request next, to minimize seek
time. Given the requests of Fig. 5-22, the sequence is 12, 9, 16, 1, 34, and 36,
shown as the jagged line at the bottom of Fig. 5-22. With this sequence, the arm
motions are 1, 3, 7, 15, 33, and 2, for a total of 61 cylinders. This algorithm, called
SSF (Shortest Seek First), cuts the total arm motion almost in half versus FCFS.
Unfortunately, SSF has a problem. Suppose more requests keep coming in
while the requests of Fig. 5-22 are being processed. For example, if, after going to
cylinder 16, a new request for cylinder 8 is present, that request will have priority
over cylinder 1. If a request for cylinder 13 then comes in, the arm will next go to
13, instead of 1. With a heavily loaded disk, the arm will tend to stay in the middle
of the disk most of the time, so requests at either extreme will have to wait until a
statistical fluctuation in the load causes there to be no requests near the middle.
Requests far from the middle may get poor service. The goals of minimal response
time and fairness are in conflict here.
SEC. 5.4
377
MASS STORAGE: DISK AND SSD
Tall buildings also have to deal with this trade-off. The problem of scheduling
an elevator in a tall building is similar to that of scheduling a disk arm. Requests
come in continuously calling the elevator to floors (cylinders) at random. The computer running the elevator could easily keep track of the sequence in which customers pushed the call button and service them using FCFS or SSF.
However, most elevators use a different algorithm in order to reconcile the
mutually conflicting goals of efficiency and fairness. They keep moving in the
same direction until there are no more outstanding requests in that direction, then
they switch directions. This algorithm, known both in the disk world and the elevator world as the elevator algorithm, requires the software to maintain 1 bit: the
current direction bit, UP or DOWN. When a request finishes, the disk or elevator
driver checks the bit. If it is UP, the arm or cabin is moved to the next highest
pending request. If no requests are pending at higher positions, the direction bit is
reversed. When the bit is set to DOWN, the move is to the next lowest requested
position, if any. If no request is pending, it just stops and waits. In big office towers, when there are no requests pending, the software might send the cabin to the
ground floor, since it is more likely to be need there shortly than on, say, the 19th
floor. Disk software does not usually try to speculatively preposition the head anywhere.
Figure 5-23 shows the elevator algorithm using the same seven requests as
Fig. 5-22, assuming the direction bit was initially UP. The order in which the cylinders are serviced is 12, 16, 34, 36, 9, and 1, which yields arm motions of 1, 4, 18,
2, 27, and 8, for a total of 60 cylinders. In this case, the elevator algorithm is slightly better than SSF, although it is usually worse. One nice property the elevator
algorithm has is that given any collection of requests, the upper bound on the total
motion is fixed: it is just twice the number of cylinders.
Initial
position
X
Time
0
X
5
X X
10
X
15
X
20
25
30
X
35 Cylinder
Sequence of seeks
Figure 5-23. The elevator algorithm for scheduling disk requests.
A slight modification of this algorithm that has a smaller variance in response
times (Teory, 1972) is to always scan in the same direction. When the highest-numbered cylinder with a pending request has been serviced, the arm goes to the
378
INPUT/OUTPUT
CHAP. 5
lowest-numbered cylinder with a pending request and then continues moving in an
upward direction. In effect, the lowest-numbered cylinder is thought of as being
just above the highest-numbered cylinder.
Some disk controllers provide a way for the software to inspect the current sector number under the head. With such a controller, another optimization is possible. If two or more requests for the same cylinder are pending, the driver can
issue a request for the sector that will pass under the head next. Note that when
multiple tracks are present in a cylinder, consecutive requests can be for different
tracks with no penalty. The controller can select any of its heads almost instantaneously (head selection involves neither arm motion nor rotational delay).
If the disk has the property that seek time is much faster than the rotational
delay, then a different optimization should be used. Pending requests should be
sorted by sector number, and as soon as the next sector is about to pass under the
head, the arm should be zipped over to the right track to read or write it.
With a modern hard disk, the seek and rotational delays so dominate performance that reading one or two sectors at a time is very inefficient. For this reason, many disk controllers always read and cache multiple sectors, even when only
one is requested. Typically any request to read a sector will cause that sector and
much or all the rest of the current track to be read, depending upon how much
space is available in the controller’s cache memory. The Seagate IronWolf hard
disk described earlier has a 256-MB cache, for example. The use of the cache is
determined dynamically by the controller. In the simplest case, the cache is divided
into two sections, one for reads and one for writes. If a subsequent read can be satisfied out of the controller’s cache, it can return the requested data immediately.
It is worth noting that the disk controller’s cache is completely independent of
the operating system’s cache. The controller’s cache usually holds blocks that have
not actually been requested, but which were convenient to read because they just
happened to pass under the head as a side effect of some other read. In contrast,
any cache maintained by the operating system will consist of blocks that were
explicitly read and which the operating system thinks might be needed again in the
near future (e.g., a disk block holding a directory block).
When several drives are present on the same controller, the operating system
should maintain a pending request table for each drive separately. Whenever any
drive is idle, a seek should be issued to move its arm to the cylinder where it will
be needed next (assuming the controller allows overlapped seeks). When the current transfer finishes, a check can be made to see if any drives are positioned on the
correct cylinder. If one or more are, the next transfer can be started on a drive that
is already on the right cylinder. If none of the arms is in the right place, the driver
should issue a new seek on the drive that just completed a transfer and wait until
the next interrupt to see which arm gets to its destination first.
It is important to realize that all of the above disk-scheduling algorithms tacitly
assume that the real disk geometry is the same as the virtual geometry. If it is not,
then scheduling disk requests makes no sense because the operating system cannot
SEC. 5.4
MASS STORAGE: DISK AND SSD
379
really tell whether cylinder 40 or cylinder 200 is closer to cylinder 39. On the
other hand, if the disk controller can accept multiple outstanding requests, it can
use these scheduling algorithms internally. In that case, the algorithms are still
valid, but one level down, inside the controller.
Error Handling
Disk manufacturers are constantly pushing the limits of the technology by
increasing linear bit densities. The IronWolf hard disk of our examples packs as
many a 2470 Kbits per inch on average. Recording that many bits per inch
requires an extremely uniform substrate and a very fine oxide coating. Unfortunately, it is not possible to manufacture a disk to such specifications without defects.
As soon as manufacturing technology has improved to the point where it is possible to operate flawlessly at such densities, disk designers will go to higher densities to increase the capacity. Doing so will probably reintroduce defects.
Manufacturing defects introduce bad sectors, that is, sectors that do not correctly read back the value just written to them. If the defect is very small, say, only
a few bits, it is possible to use the bad sector and just let the ECC correct the errors
every time. If the defect is bigger, the error cannot be masked.
There are two general approaches to bad blocks: deal with them in the controller or deal with them in the operating system. In the former approach, before
the disk is shipped from the factory, it is tested and a list of bad sectors is written
onto the disk. For each bad sector, one of the spares is substituted for it.
There are two ways to do this substitution. In Fig. 5-24(a), we see a single
disk track with 30 data sectors and two spares. Sector 7 is defective. What the controller can do is remap one of the spares as sector 7 as shown in Fig. 5-24(b). The
other way is to shift all the sectors up one, as shown in Fig. 5-24(c). In both cases
the controller has to know which sector is which. It can keep track of this information through internal tables (one per track) or by rewriting the preambles to give
the remapped sector numbers. If the preambles are rewritten, the method of
Fig. 5-24(c) is more work (because 23 preambles must be rewritten) but ultimately
gives better performance because an entire track can still be read in one rotation.
Errors can also develop during normal operation after the drive has been
installed. The first line of defense upon getting an error that the ECC cannot handle
is to just try the read again. Some read errors are transient, that is, are caused by
specks of dust under the head and will go away on a second attempt. If the controller notices that it is getting repeated errors on a certain sector, it can switch to a
spare before the sector has died completely. In this way, no data are lost and the
operating system and user do not even notice the problem. Usually, the method of
Fig. 5-24(b) has to be used since the other sectors might now contain data. Using
the method of Fig. 5-24(c) would require not only rewriting the preambles, but
copying all the data as well.
380
INPUT/OUTPUT
0 1
29
2
28
3
27
4
26
5
25
Spare
6
Bad
sectors
24
sector
23
8
9
22
10
21
11
20
12
19
13
18 17
16 15 14
0 1
29 7
2
28
3
27
4
26
5
25 Replacement
6
sector
24
23
8
9
22
10
21
11
20
12
19
13
18 17
16 15 14
(a)
(b)
CHAP. 5
28 29
27
26
25
24
23
22
21
20
19
18
17 16
0 1
2
15 14 13
3
4
5
6
7
8
9
10
11
12
(c)
Figure 5-24. (a) A disk track with a bad sector. (b) Substituting a spare for the
bad sector. (c) Shifting all the sectors to bypass the bad one.
Earlier we said there were two general approaches to handling errors: handle
them in the controller or in the operating system. If the controller does not have
the capability to transparently remap sectors as we have discussed, the operating
system must do the same thing in software. This means that it must first acquire a
list of bad sectors, either by reading them from the disk, or simply testing the entire
disk itself. Once it knows which sectors are bad, it can build remapping tables. If
the operating system wants to use the approach of Fig. 5-24(c), it must shift the
data in sectors 7 through 29 up one sector.
If the operating system is handling the remapping, it must make sure that bad
sectors do not occur in any files and also do not occur in the free list or bitmap.
One way to do this is to create a secret file consisting of all the bad sectors. If this
file is not entered into the file system, users will not accidentally read it (or worse
yet, free it).
However, there is still another problem: backups. If the disk is backed up file
by file, it is important that the backup utility not try to copy the bad block file. To
prevent this, the operating system has to hide the bad block file so well that even a
backup utility cannot find it. If the disk is backed up sector by sector rather than
file by file, it will be difficult, if not impossible, to prevent read errors during backup. The only hope is that the backup program has enough smarts to give up after 10
failed reads and continue with the next sector.
Bad sectors are not the only source of errors. Seek errors caused by mechanical
problems in the arm also occur. The controller keeps track of the arm position
internally. To perform a seek, it issues a command to the arm motor to move the
arm to the new cylinder. When the arm gets to its destination, the controller reads
the actual cylinder number from the preamble of the next sector. If the arm is in the
wrong place, a seek error has occurred.
Most hard disk controllers correct seek errors automatically, but most of the
old floppy controllers used in the 1980s and 1990s just set an error bit and left the
rest to the driver. The driver handled this error by issuing a recalibrate command,
SEC. 5.4
MASS STORAGE: DISK AND SSD
381
to move the arm as far out as it would go and reset the controller’s internal idea of
the current cylinder to 0. Usually this solved the problem. If it did not, the drive
had to be repaired.
As we have just seen, the controller is really a specialized little computer, complete with software, variables, buffers, and occasionally, bugs. Sometimes an
unusual sequence of events, such as an interrupt on one drive occurring simultaneously with a recalibrate command for another drive will trigger a bug and cause
the controller to go into a loop or lose track of what it was doing. Controller
designers usually plan for the worst and provide a pin on the chip which, when
asserted, forces the controller to forget whatever it was doing and reset itself. If all
else fails, the disk driver can set a bit to invoke this signal and reset the controller.
If that does not help, all the driver can do is print a message and give up.
Recalibrating a disk makes a funny noise but otherwise normally is not disturbing. However, there is one situation where recalibration is a problem: systems with
real-time constraints. When a video is being played off (or served from) a hard
disk, or files from a hard disk are being burned onto a Blu-ray disc, it is essential
that the bits arrive from the hard disk at a uniform rate. Under these circumstances,
recalibrations insert gaps into the bit stream and are unacceptable. Special drives,
called AV disks (Audio Visual disks), which never recalibrate are available for
such applications.
Anecdotally, a highly convincing demonstration of how advanced disk controllers have become was given by the Dutch hacker Jeroen Domburg, who hacked
a modern disk controller to make it run custom code. It turns out the disk controller
is equipped with a fairly powerful multicore ARM processor and has easily enough
resources to run Linux. If the bad guys hack your hard drive in this way, they will
be able to see and modify all data you transfer to and from the disk. Even reinstalling the operating from scratch will not remove the infection, as the disk controller itself is malicious and serves as a permanent backdoor. Alternatively, you
can collect a stack of broken hard drives from your local recycling center and build
your own cluster computer for free.
5.4.2 Solid State Drives (SSDs)
As we saw in Sec. 4.3.7, SSDs are fast, have asymmetric read and write performance, and contain no moving parts. They come in different guises. For
instance, there are some that conform to the SATA standard for storage devices that
is also used for magnetic disks. However, since SATA was designed for mechanical
disks that are slow compared to flash technology, more and more SSDs now interface to the rest of the system using NVMe (Non-Volatile Memory Express).
NVMe is a standard to exploit better the speed of the fast PCI Express connection
between the SSD and the rest of the system, as well as the parallelism available in
the SSD itself.
382
INPUT/OUTPUT
CHAP. 5
For instance, since modern computers have multiple cores and the SSD consists of many (flash) pages, blocks and, ultimately, chips, it pays off to process requests in parallel. To make this possible, NVMe supports multiple queues. At the
very least, NVMe offers one command request queue (known as a submission
queue in NVMe terminology) and one reply queue (known as a completion queue)
per core. To perform storage requests, a core will first write I/O commands in its
request queue and then write to the doorbell register when the commands are ready
to execute. The doorbell triggers the controller on the SSD to process the entries in
some order (e.g., in the order in which they were received, or in order of priority).
When the request completes, it will write the result as a status code in the reply
queue.
NVMe queues have multiple advantages. First, where SATA offers only a single queue with a small number of entries, NVMe allows many (and longer)
queues—up to 64K queues with up to 64K I/O commands entries each. Each
queue is processed in parallel, thus allowing the controller to push more commands
to the flash chips and speeding up the storage I/O significantly. Second, because of
this, since the overall computer system now needs fewer devices to support the
same number of I/O operations, this also reduces the power and cooling requirements. As a bonus, NVMe allows file systems more direct access to the PCIe bus†
and SSD, meaning that fewer layers of software are involved in NVMe than in
SATA operations.
If our SSD uses NVMe, the operating system needs a driver for NVMe also.
Often, this driver in turn consists of multiple components, such as a module that is
more or less hardware independent, a module specifically for PCIe, a module for
TCP, etc. This is not uncommon and drivers often consist of a number of logical
components. The good news here is that the SSD interface is standardized by
NVMe, and so the operating system needs only a single driver to handle all conforming SSDs. Nowadays, all major operating systems provide support for NVMe,
and thus for NVMe SSDs.
Stable Storage
As we have seen, disks sometimes make errors. Good sectors can suddenly
become bad sectors. Whole drives can die unexpectedly. For some applications, it
is essential that data never be lost or corrupted, even in the face of disk and CPU
errors. Ideally, a disk should simply work all the time with no errors. Unfortunately, that is not achievable. What is achievable is a disk subsystem that has the following property: when a write is issued to it, the disk either correctly writes the
data or it does nothing, leaving the existing data intact. Such a system is called stable storage and is implemented in software (Lampson and Sturgis, 1979). The
goal is to keep the disk consistent at all costs. Below we will describe a slight variant of the original idea.
† Actually, NVMe can even handle devices attached through other means than PCIe (including TCP
connections over the network!), but for our purposes the PCIe is the only one of interest.
SEC. 5.4
MASS STORAGE: DISK AND SSD
383
Before describing the algorithm, it is important to have a clear model of the
possible errors. The model assumes that when a disk writes a block (one or more
sectors), either the write is correct or it is incorrect and this error can be detected
on a subsequent read by examining the values of the ECC fields. In principle,
guaranteed error detection is never possible because with a, say, 16-byte ECC field
guarding a 512-byte sector, there are 24096 data values and only 2144 ECC values.
Thus if a block is garbled during writing but the ECC is not, there are billions upon
billions of incorrect combinations that yield the same ECC. If any of them occur,
the error will not be detected. On the whole, the probability of random data having
the proper 16-byte ECC is about 2<144 , which is small enough that we will call it
zero, even though it is really not.
The model also assumes that a correctly written sector can spontaneously go
bad and become unreadable. However, the assumption is that such events are so
rare that having the same sector go bad on a second (independent) drive during a
reasonable time interval (e.g., 1 day) is small enough to ignore.
The model also assumes the CPU can fail, in which case it just stops. Any disk
write in progress at the moment of failure also stops, leading to incorrect data in
one sector and an incorrect ECC that can later be detected. Under all these conditions, stable storage can be made 100% reliable in the sense of writes either working correctly or leaving the old data in place. Of course, it does not protect against
physical disasters, such as an earthquake happening and the computer falling 100
meters into a fissure and landing in a pool of boiling magma. It is tough to recover
from this condition in software.
Stable storage uses a pair of identical disks with the corresponding blocks
working together to form one error-free block. In the absence of errors, the corresponding blocks on both drives are the same. Either one can be read to get the same
result. To achieve this goal, the following three operations are defined:
1. Stable writes. A stable write consists of first writing the block on
drive 1, then reading it back to verify that it was written correctly. If
it was not, the write and reread are done again up to n times until they
work. After n consecutive failures, the block is remapped onto a spare
and the operation repeated until it succeeds, no matter how many
spares have to be tried. After the write to drive 1 has succeeded, the
corresponding block on drive 2 is written and reread, repeatedly if
need be, until it, too, finally succeeds. In the absence of CPU crashes,
when a stable write completes, the block has correctly been written
onto both drives and verified on both of them.
2. Stable reads. A stable read first reads the block from drive 1. If this
yields an incorrect ECC, the read is tried again, up to n times. If all
of these give bad ECCs, the corresponding block is read from drive 2.
Given the fact that a successful stable write leaves two good copies of
the block behind, and our assumption that the probability of the same
384
INPUT/OUTPUT
CHAP. 5
block spontaneously going bad on both drives in a reasonable time
interval is negligible, a stable read always succeeds.
3. Crash recovery. After a crash, a recovery program scans both disks
comparing corresponding blocks. If a pair of blocks are both good
and the same, nothing is done. If one of them has an ECC error, the
bad block is overwritten with the corresponding good block. If a pair
of blocks are both good but different, the block from drive 1 is written
onto drive 2.
In the absence of CPU crashes, this scheme always works because stable
writes always write two valid copies of every block and spontaneous errors are
assumed never to occur on both corresponding blocks at the same time. What
about in the presence of CPU crashes during stable writes? It depends on precisely
when the crash occurs. There are five possibilities, as depicted in Fig. 5-25.
ECC
error
Disk
1
2
Old
Old
Crash
(a)
Disk
1
Crash
(b)
Disk
Disk
2
1
2
1
Old
New
Old
New
Crash
(c)
Disk
2
Crash
(d)
1
2
New
New
Crash
(e)
Figure 5-25. Analysis of the influence of crashes on stable writes.
In Fig. 5-25(a), the CPU crash happens before either copy of the block is written. During recovery, neither will be changed and the old value will continue to
exist, which is allowed.
In Fig. 5-25(b), the CPU crashes during the write to drive 1, destroying the
contents of the block. However, the recovery program detects this error and
restores the block on drive 1 from drive 2. Thus, the effect of the crash is wiped
out and the old state is fully restored.
In Fig. 5-25(c), the CPU crash happens after drive 1 is written but before drive
2 is written. The point of no return has been passed here: the recovery program
copies the block from drive 1 to drive 2. The write succeeds.
Fig. 5-25(d) is like Fig. 5-25(b): during recovery, the good block overwrites the
bad block. Again, the final value of both blocks is the new one.
Finally, in Fig. 5-25(e), the recovery program sees that both blocks are the
same, so neither is changed and the write succeeds here, too.
Various optimizations and improvements are possible to this scheme. For
starters, comparing all the blocks pairwise after a crash is doable, but expensive. A
SEC. 5.4
MASS STORAGE: DISK AND SSD
385
huge improvement is to keep track of which block was being written during a stable write so that only one block has to be checked during recovery. Many computers have a small amount of nonvolatile RAM, which is a special CMOS memory powered by a lithium battery. Such batteries last for years, possibly even the
whole life of the computer. Unlike main memory, which is lost after a crash, nonvolatile RAM is not lost after a crash. The time of day is normally kept here (and
incremented by a special circuit), which is why computers still know what time it
is e