• Overview
• Processes in UNIX
• Memory management in UNIX
• I/O in UNIX
• UNIX file system
2

• Why some programmers like UNIX better than
Windows?
– Although GUI may be easy for beginners, they provide little flexibility and no insight into how the system works!
– UNIX has GUI, too!
• History of UNIX
– UNICS in Bell Labs
– PDP-11 UNIX
– Portable UNIX, portable C compiler
– Berkeley UNIX (BSD)
– UNIX standards: POSIX from IEEE
– MINIX and Linux
3

• Designed by programmers, for programmers
• Simple, elegant and consistent
• Power and flexibility
– A small number of basic elements
– One program should do just one thing
– An infinite combination to suit the applications
• No useless redundancy
– Simple interface
4

User interface
Library interface
System call interface
Users
Standards utility programs
(shell, editors, compilers, etc)
Standard library
(e.g. open, close, read, write, fork, etc)
UNIX operating system
(process management, memory management, the file system, I/O, etc)
Hardware
(CPU, memory, disks, terminals, etc)
User mode
Kernel mode
5

• Wait for a command line
• Extract the first word from the command line, use it as program name, run the program
• Pass arguments
• Flags: argument controlling the operation or specify an optional value
• Wild cards and magic characters
• Standard input/output
• Filter and pipe symbol
– grep ter *.t | sort | head –20 | tail –5 > foo
• Background execution and shell scripts
6

• File and directory manipulation commands: cp, mv
• Filters: grep, head, sort, tail
• Program development tools: cc
• Text processing: vi, edit
• System administration: ps
• Miscellaneous: time
7

System calls Interrupts and traps
Terminal handling Sockets
File naming
Network protocols
File systems
Mapping
Page faults
Signal handling
Virtual memory
Raw tty
Cooked tyy
Line disciplines
Routing
Buffer cache
Page cache
Process creation and termination
Process scheduling
Character devices
Network device drivers
Disk device drivers Process dispatching
Hardware
8

• Overview
• Processes in UNIX
• Memory management in UNIX
• I/O in UNIX
• UNIX file system
9

• UNIX is a multiprogramming system
– User processes
– Daemons
• Create new processes by system call fork()
– Parent process and child process
– PID, parent process gets child PID from fork()
• Pipes between two processes
– Shell pipelines are implemented using pipes
• Signals: soft interrupt
10

pid=fork() Create a child process identical to the parent pid=waitpid(pid, &statloc, opts) Wait for a child to terminate s=execve(name, argv, envp) Replace a process’ core image exit(status) Terminate process execution and return status s=sigaction(sig, &act, &oldact) Define action to take on signals s=sigreturn(&context) Return from a signal s=sigprocmask(how, &set, &old) Examine or change the signal mask s=sigpending(set) s=sigsuspend(sigmask) s=kil(pid, sig)
Get the set of blocked signals
Replace the signal mask and suspend the process
Send a signal to a process residual=alarm(seconds) s=pause()
Sent the alarm clock
Suspend the caller until the next signal
11

• Not every UNIX has threads package
– Threads package becomes popular
• Threads can be implemented in either user space or kernel
– POSIX does not specify
12

Thread Call pthread_create pthread_exit pthread_join
Description
Create a new thread in the caller’s address space
Terminate the calling thread
Wait for a thread to terminate pthread_mutex_init Create a new mutex pthread_mutex_destroy Destroy a mutex pthread_mutex_lock Lock a mutex pthread_mutex_unlock Unlock a mutex pthread_cond_init Create a condition variable pthread_cond_destroy Destroy a condition variable pthread_cond_wait pthread_cond_signal
Wait on a condition variable
Release one thread waiting on a condition variable
13

• Two key data structures for processes
• Process table: always in main memory
– Scheduling parameters, memory image, signals, miscellaneous
– A process has one entry in the table
• User structure: in memory only when the process is in memory
– Machine registers, system call state, file descriptor table, accounting info, kernel stack
– A block per process, adjacent to the stack segment
14

PID=501 PID=748 PID=748 sh New process sh Same process sh
3. Exec call
1. Fork call
2. New sh
Fork code created
1.
Allocate child’s process table entry
2.
Fill child’s entry from parent
3.
Allocate child’s stack and user area
4.
Fill child’s user area from parent
5.
Allocate PID for child
6.
Set up child to share parent’s text
Exec code
4. Sh overlaid with ls
1.
Find the executable program
2.
Verify the execute permission
3.
Read and verify the header
4.
Copy arguments, environ to kernel
5.
Free the old address space
6.
Allocate new address space
7.
Copy page tables for data and stack
8.
Set up sharing of open files
9.
Copy parent’s registers to child
7.
Copy arguments, environ to stack
8.
Reset signals
9.
Initialize registers
15

• Threads in user space: a user space library
• Threads in kernel: may lead to many problems
– How to maintain the correct traditional UNIX semantics?
– File I/O
– Signal handling
– All solutions to these problems cause something to break somewhere
16

• System call clone
– pid=clone(func, stack_ptr, sharing_flgs, arg)
• Sharing_flgs determine whether the thread is in current process or in a new process
– In the same process: changes visible to other threads
• The new thread executes func
• The new thread has its own stack
17

• A two level algorithm
– Low-level algorithm: pick the process to run
– High-level algorithm: move processes between memory and disk
• Low-level algorithm uses multiple queues
– Priority=CPU_usage+nice+base
18

Highest
-4
…
Waiting for disk I/O
-3 Waiting for disk buffer
-2 Waiting for terminal input
2
3
Lowest
-1 Waiting for terminal output
0 Waiting for child to exist
0
1
User priority 0
User priority 1
User priority 2
User priority 3
…
Process waiting in kernel mode
Process waiting in user mode
19

• Scheduling is based on threads
• Three classes of threads
– Real-item FIFO, non-preemptive, highest priority
– Real-time round robin, preemptive, high priority
– Timesharing
• Scheduling based on priority and quantum
20

• Read & run first sector of the boot disk
– MBR has a small program (up to 512 bytes)
– Load program boot
• Boot reads the root directory of boot device
• Boot reads OS kernel, boot ends
• Kernel starts, set parameters & environment
• Initialization, build kernel data structure
• System auto-configuration, detect devices
• Load device drivers
• Start process 0
21

• Continue initialization
• Program the real-time clock
• Mount the root file system
• Create init (process 1) and page daemon
(process 2)
22

• Check flags of single-user/multi-user mode
• Single user mode
– Fork off a process running shell
– Wait for that process to exit
• Multi-user mode
– Fork off a process
• Run system initialization shell script /etc/rc
– Read /etc/ttys, fork off processes for terminals
• Run gtty
23

Process 0 gett y
Process 1 ini t
Terminal 0
Login: login
Terminal 1
Password:
Page daemo n
Process 2 sh
Terminal 2
% cp f1 f2 cp
24

• Dynamically loading
– One binary code can be used everywhere
– Drivers are loaded dynamically, even through a network
• Non-dynamically loading
– Only the system administrator can make kernel binary code
– No one can insert any component into the kernel
25

• Overview
• Processes in UNIX
• Memory management in UNIX
• I/O in UNIX
• UNIX file system
26

• Text segment: program’s executable code
– Read-only, size fixed after process is created
• Data segment: program’s variables, strings, and other data
– Initialized data and un-initialized data (BSS)
– Un-initialized global variables are in BSS, save space
– Size can change
• Stack segment
27

• Text segment & mapped file can be shared
28

• POSIX does not specify
• Common system calls s=brk(addr)
System call description
Change data segment size a=mmap(addr, len, prot, flags, fd, offset) Map a file in s=unmap(addr, len) Unmap a file
29

• High level scheduling: swapper
• What cause swapping?
– Fork() needs memory for a child process
– Brk() needs to expand a data segment
– Stack needs to be expanded
• Which process will be swapped out?
– The ones blocked/waiting for I/O
– The ones with high (priority + residence time)
• The ones consuming much CPU/staying long in mem
30

• Check processes on disk every few seconds
– Find processes are ready
– Select the one staying on disk longest
• If have enough free memory (easy swap), bring that process in
• Otherwise, swap out some processes in main memory
• No process is swapped out until it stays in memory for 2 seconds
31

• Do not use the working set model
• Done by kernel & page daemon (process 2)
• Never page out kernel and core map
– Core map records info about contents of the page frames
32

• Run every 250 msec by page daemon
• If insufficient free page frames, page daemon transfers pages to disk
• A modified version of the clock algorithm
33

• A three-level paging scheme
– Directory, middle, page + offset
• Buddy algorithm
32
32
64
32
16
16
32 32
16
8
8
16
8
8
8
8
8
8
32 32
32 32
8
4
4
8
8
8
4
4
8
8
8
4
4
16
34

• Overview
• Processes in UNIX
• Memory management in UNIX
• I/O in UNIX
• UNIX file system
35

• Integrate devices into file system as special files
– Each I/O device is assigned a path name
• Block special files and character special files
36

• Socket: interface to network
• Types of networking supported by sockets
– Reliable connection-oriented byte stream
– Reliable connection-oriented packet stream
– Unreliable packet transmission
Sending process
Receiving process
User space kernel space Socket
37

• POSIX specifies function calls for terminal
• Function call ioctl is used in many UNIX
Function call Description s=cfsetospeed(&termios, speed) Set the output speed s=cfsetispeed(&termios, speed) Set the input speed s=cfgetospeed(&termios, speed) Get the output speed s=cfgetispeed(&termios, speed) Get the input speed s=tcsetattr(fd, opt, &termios) s=tcgetattr(fd, &termios)
Set the attributes
Get the attributes
38

• Overview
• Processes in UNIX
• Memory management in UNIX
• I/O in UNIX
• UNIX file system
39

• A sequence of bytes
– No distinction between ASCII, binary, or any other kinds of files
• Directories are stored as files
• Important directories in UNIX systems
Directory bin dev etc lib usr
Contents
Binary (executable) programs
Special files for I/O devices
Miscellaneous system files
Libraries
User directories
40

• Only one file system (tree)
– One disk is mounted in another disk’s file tree
• UNIX allows user to lock files
– From single byte to an entire file
– Shared locks and exclusive locks
– A process specifies whether it wants to block if the lock cannot be placed
– Shared locked regions may overlap
41

System call fd=create(name, mode) fd=open(file, how, …) s=close(fd) n=read(fd, buffer, nbytes) n=write(fd, buffer, nbytes)
Description
One way to create a new file
Open a file for reading, writing or both
Close an open file
Read data from a file into a buffer
Write data from a buffer into a file position=lseek(fd, offset, whence) Move the file pointer s=stat(name, &buf) Get a file’s status information s=fstat(fd, &buf) Get a file’s status information s=pipe(&fd[0]) s=fcntl(fd, cmd, …)
Create a pipe
File locking and other operations
42

System call s=mkdir(path, mode)
Description
Create a new directory s=rmdir(path) Remove a directory s=link(oldpath, newpath) Create a link to an existing file s=unlink(path) s=chdir(path) dir=opendir(path) s=closedir(dir) dirent=readdir(dir)
Rewinddir(dir)
Unlink a file
Change the working directory
Open a directory for reading
Close a directory
Read one directory entry
Rewind a directory so it can be reread
43

• Boot block: contain code to boot computer
• Superblock: contain critical info about the layout of the file system
– # of i-nodes, # of disk blocks, the start of the list of free disk blocks, …
• I-nodes
• Data blocks
Boot block Super blk I-nodes Data blocks
44

• System call n=read(fd, buffer, nbytes)
• Start with file descriptor fd, find i-node
– Put pointer to the i-node in file descriptor?
• Only one current position for one file, recorded in inode
• Two processes open one file with different current position
– Put current position info in file descriptor?
• One file can have multiple current positions
• P1 and P2 open f at the same time, P2 want to write after P1 finishes
• P2 cannot get the correct current position
45

46

• Allow file names up to 255 characters
• Divide the disk up into cylinder groups
– Each with its own superblock, i-nodes and data blocks
– Whenever possible, blocks are allocated in the cylinder group containing the i-node
• Two block sizes
– Large files use large block size
47

• Three interfaces to UNIX:
– Shell, C library, and system calls
• Key concepts in UNIX
– Process, memory model, I/O, and file system
• Process management in UNIX
– Process table and user structure
• Memory model: text, data and stack
• I/O and file system
48