CS4023 – Operating Systems
Dr. Atif Azad
Atif.azad@ul.ie
1
3 Billion+ Internet Users
WWW
1969 1974
2.0 B 1/26/11
HTTP 0.9
RFC 675 TCP/IP
ARPANet
Internet
2.8 B
1990
Adapted from https://cs162.eecs.berkeley.edu/static/lectures/1.pdf (c) UC Berkeley 2015
A lot of this
growth is due
to Operating
Systems
2010
Example: What’s in a Search Query?
DNS
Servers
DNS
request
Datacenter
create
result
page
Search
Index
Page
store
Load
balancer
Ad Server
• Complex interaction of multiple components in multiple
administrative domains
– Systems, services, protocols, …
Adapted from https://cs162.eecs.berkeley.edu/static/lectures/1.pdf (c) UC Berkeley 2015
•
•
Why Study Operating Systems?
Possibly Design Operating Systems
Create systems that utilize the core concepts in operating systems.
– Repeating Patterns in Software Development - (Design Patterns)
•
Learn how to build complex systems:
– How can you manage complexity for future projects?
•
Almost everyone of you will write applications that utilise operating systems
– understanding operating systems concepts can help you write better applications
– some times, you may even have to replicate Operating Systems concepts (e.g. XBox)
•
Engineering issues:
– Why is the web so slow sometimes? Can you fix it?
– What features should be in the next mars Rover?
– How do large distributed systems work? (Skype, etc)
•
Buying and using a personal computer:
– Why different PCs with same CPU behave differently (Opteron, Itanium, Celeron, Pentium)?
– What is different among Windows XP, 2000, Linux, Mac OS …?
– Why does Microsoft have such a bad name?
•
Business issues:
– Should your organisation buy thin-clients vs PC?
•
Security, viruses, and worms
– What exposure do you have to worry about?
4
Goals for This Week
•
•
•
•
Why Operating Systems?
What is an Operating System?
What makes Operating Systems So Exciting?
How does this class operate?
Interactive is important!
Ask Questions!
Acknowledgement
• Significant material in this set of lectures has
been borrowed from:
– http://www.cs.berkeley.edu/~kubitron/courses/cs162
/. Copyright © 2010 UCB
– https://cs162.eecs.berkeley.edu/ Copyright © 2015
UCB
– http://www.os-book.com . © 2005 Silberschatz et al.
– Dr Patrick Healy at CSIS Department, University of
Limerick.
6
Class Website
• http://amnesia.csisdmz.ul.ie/4023/
• Primary source of information about the
module.
– Grading policy
– Tutorial exercises
– Lab exercises
– Course material
– Grades
– Extra Reading Material
7
Course Material
• Silberschatz, Galvin & Gagne Operating System
Concepts (Wiley) 9th Edition.
– Check with the University Book Shop
– On SL in library (ed.s 6 & 8) http://www.os-book.com/
– Chapters 1 – 12 (tentatively). Look up Review Questions
and Practice Exercises at the end of the chapter.
– Lectures don’t cover full chapters, but
• Exams do!
• Lab Exercises
• Tutorials
• Extra Reading Material (mandatory)!
8
Grading Policy and How to Help
Yourself
• Check Grading Policy at the website:
http://amnesia.csisdmz.ul.ie/4023/
• Attendance: Mandatory! First 5 minutes rule.
– Applies to labs, lectures and tutorials.
• The module is demanding ~ 10 hours/week
– But extra credit also available
– Check Study Advice on the class website.
9
Labs
• Will be supervised by a TA.
• lab exercise is due this week.
– Log into your linux accounts
– And read the instructions on the class website.
• Contact Liam O'Riordan about account problems.
• Programming lab exercises begin later - worth
20%.
– All in C language.
– Learn C mostly through self-help! start NOW!
– Get books or consult online tutorials
• http://www.cprogramming.com/begin.html
10
Tutorials
• Tutorials start this week
– Attempt exercises BEFORE coming to tutorials
• except in the first week - introductory session
– Hand in your attempt to TA to be marked present
– Inform the TA about most difficult questions at the
start of the session;
• not enough time to solve all questions in a tutorial
session
– Tutorial exercises deliberately long to cover most
material and prepare you for exam.
11
Material Availability
• Labs/tutorial exercises - uploaded by Friday
evening each week.
– check the class website each Friday evening.
• Lecture slides:
– uploaded at the start of each week
– will contain stuff for both lectures of the week
• but may be revised before the second lecture
12
Society as a System
• The world is a large parallel system
– Microprocessors in everything
– Vast infrastructure
Massive Cluster
Gigabit Ethernet
Clusters
Scalable, Reliable,
Secure Services
Databases
Information Collection
Remote Storage
Online Games
Commerce
…
13
Technology Trends: Moore’s Law
Moore’s Law
2X transistors/Chip Every 1.5 years
Gordon Moore (co-founder of
Intel) predicted in 1965 that the
transistor density of
semiconductor chips would
double roughly every 18 months.
Called “Moore’s Law”
Microprocessors have
become smaller, denser,
and more powerful.
14
People-to-Computer Ratio Over Time
From David Culler
• Today: Multiple CPUs/person!
– Approaching 100s?
15
New Challenge: Slowdown in Joy’s law
of Performance
Performance (vs. VAX-11/780)
10000
From Hennessy and Patterson, Computer Architecture: A
Quantitative Approach, 4th edition, Sept. 15, 2006
3X
??%/year
1000
52%/year
100
10
25%/year
 Sea change in chip
design: multiple “cores” or
processors per chip
1
1978 1980 1982 1984 1986 1988 1990 1992 1994 1996 1998 2000 2002 2004 2006
• Y-axis counts the clock speed (MHz). Notice the log scale
• We do not have a 10 GHz processor!
• Can not make the single processor much faster
Adapted from https://cs162.eecs.berkeley.edu/static/lectures/1.pdf (c) UC Berkeley 2015
16
The power density wall
• If Moore’s law continues to power a single
processor, incredible heat generates.
17
Adapted from https://cs162.eecs.berkeley.edu/static/lectures/1.pdf (c) UC Berkeley 2015
Way forward
• Can not expect the single processor to increase
performance.
– No free lunch (not anymore!)
• Redesign of hardware
– Multiprocessing systems
• Implications
– Redesign of software (both applications and Operating
Systems).
– Increasing complexity of software systems
• Recommended reading:
– The death of scaling
– The Free Lunch is Over: A Fundamental Turn towards
Concurrency in Software
18
Operator …
Switchboard Operator
Computer Operators
Adapted from https://cs162.eecs.berkeley.edu/static/lectures/1.pdf (c) UC Berkeley 2015
Computer System Organization
Book Section 1.2
• Computer-system operation
– One or more CPUs, device controllers
connect through common bus providing
access to shared memory
– Concurrent execution of CPUs and devices
competing for memory cycles
Bus
RAM
20
Computer-System Operation
• I/O devices and the CPU can
execute concurrently
• Each device controller is in charge
of a particular device type
• I/O is from the device to local
buffer of controller
– Why local buffer?
• Device controller informs CPU that
it has finished its operation by
causing an interrupt
• CPU moves data from/to main
memory to/from local buffers
21
I/O Structure and Interrupts
Book section 1.2.3
• Device controllers (DC) (hardware) handle devices.
– Contains memory: registers and local buffer.
• OS talks to DC through Device Drivers (DD)
(software)
• I/O begins:
– DD loads registers to indicate what is required.
– DC reads registers to decide what to do (read from KB)
– DC transfers data: device to/from local buffer.
– End of transfer: DC indicates to DD via an interrupt.
– OS transfers data from local buffer to memory, and
returns the address of that data.
22
Interrupt Timeline
Time
• Repeated, small data I/O can slow things down
• Direct Memory Access (such as disk copying).
23
Direct Memory Access (DMA)
• Instead of generating one interrupt per byte of
data:
– Copy the whole block of data from the buffer to main
memory and then interrupt the CPU.
• High speed I/O
• Note: for programmers:
– Avoid repetitive I/O in your programs.
– Can slow things down
– Java provides Buffered readers and writer classes.
24
Example: Some Mars Rover
(Pathfinder) Requirements
Program loaded
• Pathfinder hardware limitations/complexity:
into memory for
– 20Mhz processor, 128MB of DRAM
execution
– cameras, scientific instruments, batteries,
solar panels, and locomotion equipment
– Many independent processes work together
• Can’t hit reset button very easily!
– Must reboot itself if necessary
– Must always be able to receive commands from Earth
• Individual Programs must not interfere
– Suppose the MUT (Martian Universal Translator Module)
buggy
– Better not crash antenna positioning software!
• Further, all software may crash occasionally
– Automatic restart with diagnostics sent to Earth
– Periodic checkpoint of results saved?
• Certain functions time critical:
– Need to stop before hitting something
– Must track orbit of Earth for communication
Increasing Software Complexity
Windows 7
From MIT’s 6.033 course
26
How do we tame complexity?
• Every piece of computer hardware different
– Different CPU
• Pentium, PowerPC, ColdFire, ARM, MIPS
– Different amounts of memory, disk, …
– Different types of devices
• Mice, Keyboards, Sensors, Cameras, Fingerprint readers
– Different networking environment
• Cable, DSL, Wireless, Firewalls,…
• Questions:
– Does the programmer need to write a single program that
covers all operations of computer systems? Modularity?
– Does every program have to be altered for every piece of
hardware? Re-usability?
– Does a faulty program crash everything? Fault isolation?
– Does every program have access to all hardware? Security?
OS Tool: Virtual Machine
Abstraction
Application
Operating System
Hardware
Virtual Machine Interface
Physical Machine Interface
• Software Engineering Problem:
– Turn hardware/software quirks 
what programmers want/need
– Optimize for convenience, utilization, security, reliability, etc…
• For Any OS area (e.g. file systems, virtual memory,
networking, scheduling):
– What’s the hardware interface? (physical reality)
– What’s the application interface? (nicer abstraction)
Interfaces Define How to interact
software
instruction set
hardware
• Interfaces define:
– Boundaries
– Expectations (how will something behave and how to
invoke that behavior)
• Why do interfaces look the way that they do?
– History, Functionality, Stupidity, Bugs, Management
• Interfaces are a huge subject in Software Engineering!
– Important to define, understand and respect them!
Virtual Machines
• Software emulation of an abstract machine
– Make it look like hardware has features you want
– Programs from one hardware & OS on another one
• Programming simplicity
–
–
–
–
Each process thinks it has all memory/CPU time
Each process thinks it owns all devices
Different Devices appear to have same interface
Device Interfaces more powerful than raw hardware
• Bitmapped display  windowing system
• Ethernet card  reliable, ordered, networking (TCP/IP)
• Fault Isolation
– Processes unable to directly impact other processes
– Bugs cannot crash whole machine
• Protection and Portability
– Java interface safe and stable across many platforms.
– JVM is NOT an operating system though!
Book section
1.11.6
Virtual Machines: Layers of OSs
• Useful for OS development
– When OS crashes, restricted to one VM
– Can aid testing programs on other OSs
Interpretation vs Virtualisation
• Java virtual machine interprets
– .class files are not machine instructions
– .class files are translated into native instructions
• Can happen every time (e.g. in Lisp/Scheme)
• Or Just in Time Compilation (typical in Java)
• OS Virtualization
– A host OS natively compiled
– A virtual machine manager (VMM) natively compiled
– A guest OS natively compiled but running on top of
VMM
32
Virtual Machine Managers
• VMware :
http://www.vmware.com/download/player/)
– see section 1.12.2 on installing it
– Commercial Software
•
Book section
1.12.2
VirtualBox (https://www.virtualbox.org)
– See
http://people.westminstercollege.edu/faculty/ggagne
/osc/vm/index.html
– Instructions on how to install linux on top of virtual
box.
– Recommended for the book.
33
Study in Groups
• Your attention spans are not the same
• You have to study a LOT! (entire chapters)
• Can face your weak points (more likely to come
up in groups)
• Teaching is great for self learning
• Communication is key in job environment
• Advice on how to make effective groups:
– http://www.topuniversities.com/blog/joining-studygroup-benefits
– Google is your friend!
34
So, What is an Operating System?
• No Universal Definition, but a general
understanding.
Book section
1.1
– A program that acts as an intermediary
between a user of a computer and the
computer hardware
– What does it do? That depends upon your point
of view, which in turn depends on the
components of Computer System.
35
Four Components of a Computer System
Outside View
(User’s view)
Inside View
(computer’s
view)
User’s View: What an OS does.
• Personal computer users want convenience, ease of
use and good performance
– Don’t care about resource utilization
• But do care in shared computer such as mainframe or
minicomputer must keep all users happy
• Also in workstations, with most individual resources
but some shared (servers)
• Handheld: convenience of use.
• Embedded computers: fridges, automobiles, washing
machines etc may have little to no user view.
37
System’s View: What an OS Does
• Fair and Efficient Resource allocator:
– CPU time, memory, I/O devices etc to be used by
conflicting requests.
• Control Program
– Supervise proper execution of programs. Prevents
errors (such as one buggy program crashing the
system)
– Manages the operation of I/O devices (see section
1.2.3 for details)
• Like Government, facilitates the operation
– But does it do anything useful itself?
38
Another View on What’s an OS
• “everything that the vendor ships when we order
an operating systems”.
• Contentious because it varies.
– Graphical User Interface? Is it a part of an OS?
• Not always.
– What about email programs? Or web browsers?
• Safari on iphone. Internet Explorer on Windows?
• A more accepted definition:
– A program that is running on the computer all the
time: Kernel.
39
What is OS: summary
– A program that acts as an intermediary between a
user of a computer and the computer hardware
– A program that allocates computer resources such
as CPU time, memory and I/O access to all the
programs running, efficiently and fairly.
– A control program that supervises the execution
of everything on the computer: they should
execute correctly (as allowed by OS) and they
should not affect other programs (fault isolation)
40
Fetch-Execute Cycle
• Process: Program loaded into the main memory (RAM)
• CPU fetches one instruction into instruction register at
a time and executes it.
• Execution:
– Decodes the instruction: may cause fetching operands
from memory.
– May store the result of execution back in the memory.
• Fetches next instruction and executes
– Interrupts cause instructions to be fetched from a different
sequence stored outside the executing program.
41
Common Functions of Interrupts
Book Section 1.2
• Interrupt transfers control to the interrupt service
routine generally, through the interrupt vector, which
contains the addresses of all the service routines
• Interrupt architecture must save the address of the
interrupted instruction
• An operating system is interrupt driven
• Interrupts should be handled quickly.
• Types of interrupts:
– Hardware interrupt: a signal to CPU through system bus.
– Software interrupt:
• trap a special function call i.e. system call/monitor call
requesting an OS facility: print output, open a file etc.
• exception is a software-generated interrupt caused by an
error/abnormal execution: divide by zero, stepping in
debugging
Useful reading: http://flint.cs.yale.edu/cs422/doc/art-of-asm/pdf/CH17.PDF
Computer Startup
• bootstrap program is loaded at power-up or reboot
– Typically stored in ROM or EEPROM, generally known as
firmware
– Initializes all aspects of system
– Loads operating system kernel and starts execution
• ‘init’ is a system program or daemon; i.e. is it provides services
on behalf of OS but is outside kernel.
– Init loads other daemons.
– Example of a daemon is a line printer daemon looking after the print
jobs and connecting your computer to the print server.
– OS then waits for some “event” to occur.
• Bootstrap program is sometimes also called
bootloader.
– http://en.wikipedia.org/wiki/Booting#BOOT-LOADER
Computer Startup: Linux example
Unified Extensible
Firmware Interface
(UEFI)
Courtesy: http://www.ibm.com/developerworks/library/l-linuxboot/
44
Storage Structure
• Main memory – only large storage media that the CPU can
access directly
Section 1.2.2
– Random access
– Typically volatile
• Secondary storage – extension of main memory that provides
large nonvolatile storage capacity
• Hard disks – rigid metal or glass platters covered with
magnetic recording material
– Disk surface is logically divided into tracks, which are subdivided
into sectors
– The disk controller determines the logical interaction between
the device and the computer
• Solid-state disks – faster than hard disks, nonvolatile
– Various technologies e.g. a mixture of RAM and magnetic disk
– Becoming more popular (in cameras, PDAs, USB etc)
http://uk.pcmag.com/storage-devicesreviews/8061/feature/ssd-vs-hdd-whats-the-
Storage Hierarchy
• Storage systems organized in hierarchy
– Speed
– Cost
– Volatility
• Caching – copying information into faster
storage system; main memory can be viewed as
a cache for secondary storage
– Faster storage checked first. if info there use it, else
copy data into cache.
– Cache smaller than storage being cached. Thus
cache management is an important design
problem.
Storage-Device Hierarchy
Speed and Cost
Volatile