TDDD63 Agenda Lecture I: Computer Systems Overview Introduction to Operating Systems Part II Building and Executing Programs Operating System Basics Lecture II: Interrupts and I/O Christoph Kessler System Calls IDA, Linköping University CPU Management Memory Management Copyright Notice: The slides partly use material from Silberschatz’s, Galvin’s and Gagne’s book (“Operating System Concepts”, course book in TDDB68). No part of the lecture notes may be reproduced in any form, due to the copyrights reserved by Wiley. These lecture notes should only be used for internal teaching purposes at the Linköping University. File System Protection and Security Christoph Kessler, IDA, Linköpings universitet. 2 C. Kessler, IDA, Linköpings universitet. Computer System Organization Computer-system operation Interrupts and I/O One or more CPUs, device controllers connected through common bus providing access to shared memory Concurrent execution of CPUs and devices, competing for memory cycles Copyright Notice: The slides partly use material from Silberschatz’s, Galvin’s and Gagne’s book (“Operating System Concepts”, course book in TDDB68). No part of the lecture notes may be reproduced in any form, due to the copyrights reserved by Wiley. These lecture notes should only be used for internal teaching purposes at the Linköping University. Memory bus Christoph Kessler, IDA, Linköpings universitet. 4 C. Kessler, IDA, Linköpings universitet. A Kernel I/O Structure Background: Interrupt Program execution (von-Neumann cycle) Device Controllers Physical Devices Instruction Fetch Instruction Decode Instruction Execute HARDWARE Device Drivers SOFTWARE by a processor C. Kessler, IDA, Linköpings universitet. Update Program Counter 5 C. Kessler, IDA, Linköpings universitet. No way to react to events not explicitly anticipated in the (user) program code 6 1 Background: Interrupt CPU – I/O Device Interaction (1) Program execution (von-Neumann cycle) I/O devices and the CPU can execute concurrently. by a processor with interrupt logic Each device controller has a local buffer. CPU moves data from/to main memory to/from local buffers Instruction Fetch Save processor state Instruction Decode I/O is from the device to local buffer of controller. Device controller informs CPU that it has finished its operation Execute interrupt service routine (ISR) by causing an interrupt. Instruction Execute Yes Check for Interrupt Restore processor state No Update Program Counter 7 C. Kessler, IDA, Linköpings universitet. I/O Interaction using Interrupt I/O Interaction using Interrupt Example: Read from the keyboard (KBD) System Bus CPU + Memory 0: initiate I/O operation KBD_RD Memory Device controller OS Kernel 3: ISR table ISR’s ... The device driver loads the appropriate registers within the device controller The device controller determines what action to take based on the registers Device 5: Device driver 1 ... Device driver KBD 4: ... A OS buffer User program code 8 C. Kessler, IDA, Linköpings universitet. A Local buffer 2: data + signal interrupt for KBD_RD 1: token 3: call ISR for KBD_RD 4: ISR uses device driver KBD, passes data to an OS buffer 5: return from interrupt 9 C. Kessler, IDA, Linköpings universitet. CPU – I/O Device Interaction (2) “read a character from the keyboard” The controller starts the transfer of data from the device to its local buffer Once the transfer is complete, the device controller informs the device driver via an interrupt that it has completed the transfer. 10 C. Kessler, IDA, Linköpings universitet. Synchronous vs. Asynchronous I/O DMA = Direct Memory Access allows for parallel activity of CPU and I/O data transfer More efficient for large-volume data transfer 1 2 4 Synchronous I/O 3 User process waits for I/O operation to complete Can only have 1 I/O request outstanding at any time C. Kessler, IDA, Linköpings universitet. Asynchronous I/O 11 C. Kessler, IDA, Linköpings universitet. User process continues in parallel to I/O operation, may eventually need to check for availability of the result 12 2 Interrupt (1) Interrupt (2) Interrupt transfers control to an interrupt service routine, A trap is a software-generated interrupt generally through the interrupt vector (IRV), a branch table that contains the start addresses of all the service routines. caused either by an error or a user request. Examples: Division by zero; Request for OS service An operating system is interrupt driven. Interrupt architecture must save the address of the interrupted instruction. How to determine which type of interrupt has occurred? polling vectored interrupt C. Kessler, IDA, Linköpings universitet. system: interrupt number indexes IRV 13 Interrupt Handling Interrupt Timeline Hardware + OS preserves the state of the CPU by 14 C. Kessler, IDA, Linköpings universitet. Store registers and program counter (address of interrupted instruction) for a device sending input Determines which type of interrupt has occurred: Polling (Continually checking a non-busy bit in device controllers’ status register) Vectored interrupt system: Interrupt signal, number indexes into IRV table Separate segments of code determine what action should be taken Interrupt for each type of interrupt C. Kessler, IDA, Linköpings universitet. Intel Pentium Processor Event Vector Table15 C. Kessler, IDA, Linköpings universitet. Interrupt 16 Operating System Operations Dual mode, system calls CPU management System Calls Batch processing Uniprogramming, Multiprogramming, Multitasking Process management Memory management System call mechanism System call API Passing parameters Types of system calls File system and mass storage management Protection and security Copyright Notice: The slides partly use material from Silberschatz’s, Galvin’s and Gagne’s book (“Operating System Concepts”, course book in TDDB68). No part of the lecture notes may be reproduced in any form, due to the copyrights reserved by Wiley. These lecture notes should only be used for internal teaching purposes at the Linköping University. Christoph Kessler, IDA, Linköpings universitet. C. Kessler, IDA, Linköpings universitet. 18 3 Example: I/O Protection Dual mode, System calls Dual-mode operation allows OS to protect itself and other system components All I/O instructions are privileged instructions. Must ensure that a user program could never gain Privileged control of the computer in kernel mode System (i.e., a user program that, as part of its execution, stores a new address in the interrupt vector) C. Kessler, IDA, Linköpings universitet. User mode and kernel mode (supervisor mode, privileged mode) 19 Dual-Mode Operation instructions only executable in kernel mode call changes mode to kernel, on return resets it to user Mode bit provided by hardware 20 C. Kessler, IDA, Linköpings universitet. Dual-Mode Operation (Cont.) Sharing system resources requires the operating system to When an interrupt or fault occurs, hardware switches to ensure that an incorrect program cannot cause other programs to execute incorrectly. kernel mode. System calls – call OS service Hardware support (mode bit in CPU status register) to differentiate between at least two modes of operations. User mode Interrupt/fault Execution done on behalf of a user Access only to memory addresses owned by the process kernel Kernel mode (also supervisor mode or system mode) Execution done on behalf of operating system. Privileged instructions instructions that may be harmful, e.g., system login, set priorities, system halt, I/O, etc. Unrestricted memory access C. Kessler, IDA, Linköpings universitet. user set user mode 21 Example of Using System Calls C. Kessler, IDA, Linköpings universitet. 22 System Call API – OS Relationship System call sequence to copy contents of one file to another C. Kessler, IDA, Linköpings universitet. 23 C. Kessler, IDA, Linköpings universitet. 24 4 System Call API Example of a System Call API Programming interface to the services provided by the OS ReadFile() function in Win32 API (function for reading from a file) Typically written in a high-level language (C or C++) Mostly accessed by programs via a high-level Application Program Interface (API) rather than direct system call use Most common system call APIs are Win32 API for Windows, POSIX API for POSIX-based systems (including virtually all versions of UNIX, Linux, Mac OS X) Parameters passed to ReadFile(): C. Kessler, IDA, Linköpings universitet. 25 System Call API Implementation file — the file to be read buffer — a buffer where the data will be read into and written from bytesToRead — the number of bytes to be read into the buffer bytesRead — the number of bytes read during the last read ovl — indicates if overlapped I/O is being used C. Kessler, IDA, Linköpings universitet. 26 Standard C Library Example System call implementation is hardware-specific, e.g. special trap instruction with a system call number passed in a register, indexing the interrupt vector (branch table) C program invoking printf(), which calls write() system call System call interface (usually, in C) invokes the intended system call in OS kernel and returns status of the system call and any return values Advantage: Caller does not need to know anything about how the system call is implemented Most details of OS interface hidden from programmer by API C. Kessler, IDA, Linköpings universitet. 27 Types of System Calls C. Kessler, IDA, Linköpings universitet. 28 Examples of System Calls Process control load, execute, end, abort, create, terminate, wait ... memory allocation and deallocation File management open, close, create, delete, read, write, get/set attributes... Device management request / release device, read, write, ... Information maintenance get / set time, date, system data, process / file attributes Communications create / delete connection, send, receive, ... C. Kessler, IDA, Linköpings universitet. 29 C. Kessler, IDA, Linköpings universitet. 30 5 Solaris 10 dtrace Following System Call System Programs dtrace: provide a convenient environment for program development and execution. dynamic tracing facility instrument the code to collect diagnostic information during runtime, e.g. entry/exit of functions File management Status information File modification Programming language support: Compilers, assemblers, debuggers... Program loading and execution Communications: Message passing, e-mail, web browser, ... Some of them are simply user interfaces to system calls; others are considerably more complex U = User mode Most users’ view of the operation system is defined by system programs, not the actual system calls K = Kernel mode 31 C. Kessler, IDA, Linköpings universitet. 32 C. Kessler, IDA, Linköpings universitet. Summary: Interrupts and I/O Operating System = OS Kernel + System Programs Mediates all accesses to system resources Interrupt-driven Error handling Controlled CPU Management access to system resources, e.g. – I/O devices, DMA – CPU time sharing Context Switching () CPU Scheduling … Dual-Mode (user mode, kernel mode) Processes Copyright Notice: The slides partly use material from Silberschatz’s, Galvin’s and Gagne’s book (“Operating System Concepts”, course book in TDDB68). No part of the lecture notes may be reproduced in any form, due to the copyrights reserved by Wiley. These lecture notes should only be used for internal teaching purposes at the Linköping University. System Call API for portability C. Kessler, IDA, Linköpings universitet. Evolution of Shared Computing Operating System Operations Batch Processing Dual mode, system calls CPU management Batch processing Uniprogramming, Multiprogramming, Multitasking Christoph Kessler, IDA, Linköpings universitet. 33 No interaction between the computer and user at all No overlap between I/O and CPU usage Multi-Programming Process management CPU is wasted if a job waits for I/O While one job waits for a resource, another job can run Interactive Processing Memory management File system and mass storage management Protection and security C. Kessler, IDA, Linköpings universitet. User interacts with computer requires real-time processing Helps explain the need for and the emergence of Operating Systems I/O and CPU usage overlap Time-Sharing / Multi-Tasking 35 Implemented through multi-programming multiple users executing single task each on same CPU Single user executing multiple tasks on same CPU Multi-Processor machines C. Kessler, IDA, Linköpings universitet. 36 6 Simple Batch Systems (1960’s) Early Systems - Bare Machine (1950’s) Reduce setup time by batching jobs with similar requirements Hardware - expensive; Human - inexpensive Add a card reader, Hire an operator Structure Large machines run from a console Single user system Programmer/User as operator Automatically transfers control from one job to another First rudimentary operating system Paper tape or punched cards Early Software Assemblers, compilers, linkers, loaders, device drivers, libraries of common subroutines Secure Execution One operator / no network Inefficient use of expensive resources User is not the operator (operator is an intermediary) Automatic job sequencing Low CPU utilization, high setup time 37 C. Kessler, IDA, Linköpings universitet. Resident monitor Holds initial control control transfers to a job when job completes control transfers back to the monitor Problem Need to distinguish jobs from each other Need to distinguish data from program Batch processing used today mainly for providing access to supercomputers – exclusive use of (groups of) servers for single HPC application programs 38 C. Kessler, IDA, Linköpings universitet. Batch Processing Memory Layout for a Simple Batch System Monitor One program at a time 39 C. Kessler, IDA, Linköpings universitet. Supervisor/Operator Control 40 C. Kessler, IDA, Linköpings universitet. Uniprogramming Secure monitor that controls job processing Special cards indicate what to do IBM 7094 %Fortran card, %load card Running User program prevented from performing I/O User separated from computer User(s) submit card deck(s) cards put on tape tape is processed by the operator output is written to a tape tape is printed on a printer 0 10 Running 110 Waiting 120 220 Process execution time: CPU: 10 + 10 time units I/O: 100 + 100 time units Problems Waiting I.e., I/O intensive (200/220 = 90.9%), CPU utilization 9.1% Long turnaround time - up to 2 days Low CPU utilization I/O and CPU could not overlap Slow mechanical devices C. Kessler, IDA, Linköpings universitet. Single user with single program cannot keep CPU and I/O devices busy at all times. 41 C. Kessler, IDA, Linköpings universitet. 42 7 Multiprogramming Multiprogramming with three programs needed for efficiency Multiprogramming organizes jobs (code and data) so CPU always has one to execute A subset of total jobs in system is kept in memory One job selected and run via job scheduling When it has to wait (e.g., for I/O), OS switches to another job (kernel memory) A Running B Waiting (printer) Running C Running Waiting (disk) Waiting Running Running Waiting (network) Waiting Running Waiting Running Running Running Waiting Running Running Running Waiting combined Memory layout for multiprogrammed system C. Kessler, IDA, Linköpings universitet. 43 C. Kessler, IDA, Linköpings universitet. Timesharing (Multitasking) CPU time sharing using timer interrupt extension of multiprogramming: Timer to prevent infinite loop / process hogging resources CPU switches jobs so frequently that users can interact with each job while it is running For interactive computer systems, the response time should be short (< 1 second) Each user has at least one program executing in memory processes If several jobs ready to run at the same time CPU scheduling If processes don’t fit in memory, swapping moves them in and out to run Virtual memory allows execution of processes not completely in memory C. Kessler, IDA, Linköpings universitet. 45 Set up to interrupt the computer after specific period System decrements counter at clock ticks When counter = zero, generate an interrupt So, OS regains control and can reschedule or terminate a program that exceeds allotted time C. Kessler, IDA, Linköpings universitet. 44 46 Interactive Processing CPU time sharing using timer interrupt On-line communication between the user and the system: Example: Switching between two processes when the OS finishes execution of one command, it seeks the next “control statement” from the user’s keyboard. On-line system must be available for users to access data and code. Relatively frequent context switching C. Kessler, IDA, Linköpings universitet. 47 ”time quantum” = usually a few milliseconds per process at a time C. Kessler, IDA, Linköpings universitet. 48 8 Process Management Process Concept A process is a program in execution. Process = a program in execution A unit of work within the system. Program is a passive entity, process is an active entity. Process needs resources to accomplish its task CPU, memory, I/O, files Initialization data Process termination reclaim of any reusable resources Single-threaded process: has one program counter specifying location of next instruction to execute Process executes instructions sequentially, one at a time, until completion Multi-threaded process: has one program counter per thread Typically, a system has many processes (some user, some system pr.) running concurrently on one or more CPUs Concurrency by multiplexing the CPUs among the processes / threads C. Kessler, IDA, Linköpings universitet. 49 A process includes: text section (loaded program) program counter stack (+ heap) data section (global variables) process in memory Remark: [SGG7] uses the terms job and process almost interchangeably. C. Kessler, IDA, Linköpings universitet. Process State Process Control Block (PCB) As a process executes, it changes state Information associated with each process new: The process is being created running: Instructions are being executed waiting: The process is waiting for some event to occur ready: The process is waiting to be assigned to a process CPU registers terminated: The process has finished execution CPU scheduling information 50 Process state Program counter Memory-management information Accounting information I/O status information C. Kessler, IDA, Linköpings universitet. 51 CPU Switch From Process to Process C. Kessler, IDA, Linköpings universitet. 52 Context Switch When CPU switches to another process, the system must save the state of the old process and load the saved state for the new process Context-switch time is overhead C. Kessler, IDA, Linköpings universitet. 53 the system does no useful work while switching time depends on hardware support C. Kessler, IDA, Linköpings universitet. 54 9 Process Creation in UNIX A typical tree of processes in Solaris int main() C program forking { fork system call a separate process Pid_t ret; creates new child process /* fork another process: */ exec system call ret = fork(); if (ret < 0) { /* error occurred */ used after a fork to replace fprintf ( stderr, "Fork Failed“ ); the process’ memory space exit(-1); with a new program } wait system call else if (ret == 0) { // I am child process: execlp ( "/bin/ls", "ls", NULL ); by parent, suspends parent execution until child process } else { // I am the parent process has terminated // of child process with PID==ret /* wait for child to complete: */ wait (NULL); printf ("Child complete"); exit(0); } C. Kessler, IDA, Linköpings universitet. } 55 56 C. Kessler, IDA, Linköpings universitet. Ready Queue And Various I/O Device Queues Process Scheduling Queues Job queue set of all processes in the system Ready queue set of all processes residing in main memory, ready and waiting to execute Device queues set of processes waiting for an I/O device Processes migrate among the various queues C. Kessler, IDA, Linköpings universitet. 57 58 C. Kessler, IDA, Linköpings universitet. Inter-Process Communication (IPC) Schedulers 2 Fundamental Models – Realization by OS Long-term scheduler (or job scheduler) for batch systems – new jobs for execution queued on disk selects which processes should be brought into the ready queue, and loads them into memory for execution controls the degree of multiprogramming invoked very infrequently (seconds, minutes) No long-term scheduler on UNIX and Windows; instead swapping, controlled by medium-term scheduler Short-term scheduler (or CPU scheduler) selects which ready process should be executed next invoked very frequently (milliseconds) must be fast C. Kessler, IDA, Linköpings universitet. IPC via Message Passing 59 Send and Recv system calls must be granted by the OS, may use kernel-space buffering for messages (M) C. Kessler, IDA, Linköpings universitet. IPC via Shared Memory Setup of a shared memory section (via system calls) must be granted by the OS, thereafter access by Load/Store by each process 60 10 Cooperating Processes Summary: Process Management Activities Independent process The OS is responsible for: cannot affect or be affected by execution of another process Creating and deleting both user and system processes Cooperating process Suspending and resuming processes can affect or be affected by execution of another process Providing mechanisms for process synchronization Advantages of process cooperation: Information sharing Computation speed-up Modularity Convenience Providing mechanisms for process communication Parallel Computing: TDDC78, TDDD56 In special cases: Providing mechanisms for deadlock handling Inter-Process Communication (IPC) shared memory message passing signals 61 C. Kessler, IDA, Linköpings universitet. 62 C. Kessler, IDA, Linköpings universitet. Operating System Operations Dual mode, system calls CPU management Memory Management Batch processing Uniprogramming, Multiprogramming, Multitasking Process management Memory management File system and mass storage management Protection and security Copyright Notice: The slides partly use material from Silberschatz’s, Galvin’s and Gagne’s book (“Operating System Concepts”, course book in TDDB68). No part of the lecture notes may be reproduced in any form, due to the copyrights reserved by Wiley. These lecture notes should only be used for internal teaching purposes at the Linköping University. Christoph Kessler, IDA, Linköpings universitet. 64 C. Kessler, IDA, Linköpings universitet. Simple Memory Protection Memory Management Use of a Base and Limit Register Memory: A large array of words or bytes, each with its own address Kernel memory Primary storage – directly accessible from CPU shared by CPU and I/O devices a volatile storage medium All data in memory before and after processing All instructions in memory in order to execute Memory management determines what is in memory when. Optimizing CPU utilization and memory utilization OS memory management activities Keeping track of which parts of memory are currently being used and by whom Deciding which processes (or parts thereof) and data to move into and out of memory Allocating and deallocating memory space as needed C. Kessler, IDA, Linköpings universitet. Hardware support for memory protection 65 C. Kessler, IDA, Linköpings universitet. 66 11 Virtual Memory Memory Protection Must provide memory protection at least for the interrupt vector and the interrupt service routines (in kernel memory) For memory protection, add two registers that determine the range of legal addresses that a process (running program) may access: Base register – holds the smallest legal physical memory address of this process. Only part of the program needs to be in memory for execution. When executing in kernel mode, the process has unrestricted access to both kernel’s and user’s memory The load instructions for the base and limit registers are privileged instructions, can only be set by system calls. This will later be extended to virtual memory ( TDDB68) C. Kessler, IDA, Linköpings universitet. 67 One can then allow programs larger than physical memory. Logical address space can therefore be much larger than physical address space. Allows address spaces to be shared by several processes. Allows for more efficient process creation. Limit register – contains the size of the range Memory outside the defined range is protected: Virtual memory provides separation of the user process’s logical memory (contiguous, starting at relative address 0) from physical memory. 68 C. Kessler, IDA, Linköpings universitet. Virtual Memory that is Larger than Physical Memory File Systems and Secondary Storage Management Copyright Notice: The slides partly use material from Silberschatz’s, Galvin’s and Gagne’s book (“Operating System Concepts”, course book in TDDB68). No part of the lecture notes may be reproduced in any form, due to the copyrights reserved by Wiley. These lecture notes should only be used for internal teaching purposes at the Linköping University. C. Kessler, IDA, Linköpings universitet. Christoph Kessler, IDA, Linköpings universitet. 69 Operating System Operations Mass-Storage Management (1) Dual mode, system calls Disks (including SSD) used to store data that do not fit in main memory or must be kept for a “long” period of time. CPU management Batch processing Uniprogramming, Multiprogramming, Multitasking Process management Secondary storage OS activities: Free-space management Storage allocation Disk scheduling Memory management File system and mass storage management Critical for system performance Protection and security Some storage need not be fast C. Kessler, IDA, Linköpings universitet. 71 Tertiary storage optical storage, magnetic tape... Still must be managed C. Kessler, IDA, Linköpings universitet. 72 12 Mass-Storage Management (2) Protection and Security OS provides uniform, logical view of information storage Protection – any mechanism for controlling access of processes or users Abstracts from physical to logical storage unit: file Each medium (disk, tape, flash, …) has different properties: access speed, capacity, data transfer rate, sequential/random access OS File-System management Files usually organized into directories OS activities include Creating and deleting files and directories Primitives to manipulate files and directories Access control Mapping files onto secondary storage Backup files to tertiary storage Security – defense of the system against internal and external attacks Huge range, including denial-of-service, worms, viruses, Trojan horses, buffer overflow / overread, identity theft, theft of service Policies for configuring / coordinating the OS’ protection mechanisms Systems generally first distinguish among users, to determine who can do what TDDB68 C. Kessler, IDA, Linköpings universitet. to resources defined by the OS User identities (user IDs, security IDs associated with all files, processes of that user Group IDs TDDB68 73 Summary: Operating System Services 74 C. Kessler, IDA, Linköpings universitet. Case study: LINUX Linux is a modern, free operating system based on UNIX standards. First developed as a small but self-contained kernel in 1991 by Linus Torvalds, with the major design goal of UNIX compatibility. Linux is a multiuser, multitasking system with a full set of UNIX-compatible tools.. Its file system adheres to traditional UNIX semantics, and it fully implements the standard UNIX networking model. Main design goals are speed, efficiency, and standardization. The core Linux operating system kernel is entirely original, but it can run much existing free UNIX software, resulting in an entire UNIX-compatible operating system free from proprietary code. C. Kessler, IDA, Linköpings universitet. 75 Where are We? Components of a Linux System Abstraction Like most UNIX implementations, Linux is composed of three main bodies of code; Software Eng the most important distinction being between the kernel and all other components. Kernel code executes in kernel mode with full access to all physical resources Methodology Software Engineering Compiler Design of the computer. 76 C. Kessler, IDA, Linköpings universitet. All kernel code and data structures are kept in the same single address space Operating Systems The system libraries define a standard set of functions through which applications interact with the kernel, and which implement much of the operating-system functionality that does not need the full privileges of kernel code. Systems Operating Systems, Compilers Programming Assembly Language Programming Machine Language The system utilities perform individual specialized management tasks. Computer Systems Discrete Mathematics Information Theory Digital Electronics Signals & Systems C. Kessler, IDA, Linköpings universitet. 77 Electronics Hardware Computer Architectures Computation Arithmetical & Logical Computations Information Bits, Bytes, Words, Representations Hardware Logical Gates & Networks Hardware Transistors, Circuits & Processors C. Kessler, IDA, Linköpings universitet. 78 13