Chapter 15:
The Linux System II
Operating System Concepts Essentials – 2nd Edition
Silberschatz, Galvin and Gagne ©2013
Chapter 15: The Linux System
 Memory Management
 File Systems
 Input and Output
 IPC
 Network Structure
 Security
Operating System Concepts Essentials – 2nd Edition
15.2
Silberschatz, Galvin and Gagne ©2013
Memory Management
 Linux’s physical memory-management system
deals with:

Allocating / freeing pages, groups of pages, and
small blocks of memory
 Has additional mechanisms for handling virtual
memory

Memory mapped into address space of running
processes
 Splits memory into four zones due to hardware
characteristics

Architecture specific
Operating System Concepts Essentials – 2nd Edition
15.3
Silberschatz, Galvin and Gagne ©2013
Managing Physical Memory
 Page allocator allocates / frees all physical pages;\

Can allocate physically-contiguous pages on request
 Allocator uses buddy-heap algorithm to track
available physical pages

Each memory region paired with adjacent partner

Whenever two partner regions both freed, they
combine to form a larger region

If a small memory request cannot be satisfied by
allocating an existing small free region, then a larger
free region will be subdivided into two partners
Operating System Concepts Essentials – 2nd Edition
15.4
Silberschatz, Galvin and Gagne ©2013
Splitting of Memory in a Buddy Heap
Operating System Concepts Essentials – 2nd Edition
15.5
Silberschatz, Galvin and Gagne ©2013
Managing Physical Memory (Cont.)
 Memory allocation in Linux kernel occur either:

Statically (e.g., drivers reserve contiguous memory
during boot), or

Dynamically (via page allocator)
 Also uses slab allocator for kernel memory
 Page cache and virtual memory system also
manage physical memory

Page cache is kernel’s main cache for files and
main mechanism for block device I/O

Page cache stores entire pages of file contents for
local and network file I/O
Operating System Concepts Essentials – 2nd Edition
15.6
Silberschatz, Galvin and Gagne ©2013
Slab Allocator in Linux
Operating System Concepts Essentials – 2nd Edition
15.7
Silberschatz, Galvin and Gagne ©2013
Virtual Memory
 VM system maintains address space visible to
each process

It creates pages of virtual memory on demand

Manages loading pages from disk or swapping back
out to disk as required
 VM manager maintains two separate views of a
process’s address space

Logical view describing layout of address space
 Address
space: set of non-overlapping regions,
representing a continuous, page-aligned subset of
address space

Physical view of each address space (stored in
hardware page tables for the process)
Operating System Concepts Essentials – 2nd Edition
15.8
Silberschatz, Galvin and Gagne ©2013
Virtual Memory (Cont.)
 Virtual memory regions are characterized by:

Backing store- where pages for region come from
 Regions
usually backed by a file or by nothing
(demand-zero memory)

Region’s reaction to writes
 Page
sharing or copy-on-write
 Kernel creates new virtual address space
1. When a process runs a new program with
exec() system call
2. Upon creation of a new process by fork()
system call
Operating System Concepts Essentials – 2nd Edition
15.9
Silberschatz, Galvin and Gagne ©2013
Virtual Memory (Cont.)
 When executing a new program, process given new,
(completely empty) virtual-address space

Program-loading routines populate address space
with virtual-memory regions
Operating System Concepts Essentials – 2nd Edition
15.10
Silberschatz, Galvin and Gagne ©2013
Virtual Memory (Cont.)
 New process with fork()=> creates complete
copy of existing process’s virtual address space

Kernel copies parent process’s VMA descriptors;
creates new set of page tables for child

Parent’s page tables copied directly into child’s
 Reference

count of each page incremented
After fork, parent and child share same physical
pages of memory in their address spaces
Operating System Concepts Essentials – 2nd Edition
15.11
Silberschatz, Galvin and Gagne ©2013
Swapping and Paging
 VM paging system divided into two sections:

Pageout-policy algorithm decides which pages to
write out to disk, and when

Paging mechanism carries out transfer
 Pages
 Page

out to either swap device or normal files
Bitmap used to track used blocks in swap space
 Kept

data back into physical memory as needed
in physical memory
Allocator uses next-fit algorithm to try to write
contiguous runs
Operating System Concepts Essentials – 2nd Edition
15.12
Silberschatz, Galvin and Gagne ©2013
Kernel Virtual Memory
 Linux kernel reserves constant, architecture-
dependent region of every process’s virtual address
space for internal use
 Kernel virtual-memory area contains two regions:

Static area containing page table references to every
available physical page of memory in system

Remainder of reserved section not reserved for any
specific purpose
 Its
page-table entries can be modified to point to any
other areas of memory
Operating System Concepts Essentials – 2nd Edition
15.13
Silberschatz, Galvin and Gagne ©2013
Executing and Loading User Programs
 Linux maintains table of functions for loading
programs

Gives each function opportunity to try loading given file
when exec() system call made
 Multiple loader routines allows Linux to support both
ELF and a.out binary formats
 Initially, binary-file pages mapped into virtual memory

Page only loaded into physical memory on page fault
Operating System Concepts Essentials – 2nd Edition
15.14
Silberschatz, Galvin and Gagne ©2013
Executing and Loading User Programs
 ELF-format binary file consists of header followed by
several page-aligned sections

ELF loader works by reading header and mapping
sections of file into separate regions of virtual memory
Operating System Concepts Essentials – 2nd Edition
15.15
Silberschatz, Galvin and Gagne ©2013
Memory Layout for ELF Programs
Operating System Concepts Essentials – 2nd Edition
15.16
Silberschatz, Galvin and Gagne ©2013
Static and Dynamic Linking
 Static linking– library functions embedded directly
in program’s executable binary file

Main disadvantage: every program must contain
copies of exactly same system library functions
 Dynamic linking – loads system libraries into
memory only once

More efficient in terms of physical memory and diskspace usage
Operating System Concepts Essentials – 2nd Edition
15.17
Silberschatz, Galvin and Gagne ©2013
Static and Dynamic Linking (Cont.)
 Linux implements dynamic linking in user mode
through special linker library

Every dynamically linked program contains small
statically linked function called when process starts

Maps link library into memory

Link library determines dynamic libraries required by
process and names of variables / functions needed

Maps libraries into middle of virtual memory and
resolves references to symbols contained in libraries

Shared libraries compiled to be positionindependent code (PIC): can be loaded anywhere
Operating System Concepts Essentials – 2nd Edition
15.18
Silberschatz, Galvin and Gagne ©2013
Chapter 15: The Linux System
 Memory Management
 File Systems
 Input and Output
 IPC
 Network Structure
 Security
Operating System Concepts Essentials – 2nd Edition
15.19
Silberschatz, Galvin and Gagne ©2013
File Systems
 To user, Linux’s file system appears as hierarchical
directory tree obeying UNIX semantics
 Internally, kernel hides implementation details and
manages different file systems via virtual file system
(VFS) abstraction
Operating System Concepts Essentials – 2nd Edition
15.20
Silberschatz, Galvin and Gagne ©2013
File Systems (Cont.)
 Linux VFS designed around object-oriented
principles
 Composed of four components:

Inode object structure represent individual files

File object represents an open file

Superblock object represents an entire file system

Dentry object represents an individual directory
entry
Operating System Concepts Essentials – 2nd Edition
15.21
Silberschatz, Galvin and Gagne ©2013
The Linux ext3 File System
 ext3 is standard on-disk file system for Linux

Uses mechanism similar to BSD Fast File System
(FFS) for locating data

Supersedes older extfs, ext2 file systems

Work underway on ext4 adding features
 e.g.,extents

Many other file system choices with Linux distros
Operating System Concepts Essentials – 2nd Edition
15.22
Silberschatz, Galvin and Gagne ©2013
The Linux ext3 File System (Cont.)
 Main differences between ext2fs and FFS: disk
allocation policies

In FFS, disk allocated to files in blocks of 8Kb
 Blocks
subdivided into 1 KB fragments to store small
files or partially filled blocks at end of file

ext3 does not use fragments
 Performs
allocations in smaller units
 Default
block size varies as function of total size of file
system (e.g., 1, 2, 4 and 8 KB blocks)
Operating System Concepts Essentials – 2nd Edition
15.23
Silberschatz, Galvin and Gagne ©2013
The Linux ext3 File System (Cont.)

ext3 uses cluster allocation policies
 Designed
to place logically adjacent blocks into physically
adjacent blocks on disk
 Can
submit I/O request for several disk blocks as single
operation on block group

ext3 maintains bit map of free blocks in block group
 Searches
for free byte to allocate at least 8 blocks at once
Operating System Concepts Essentials – 2nd Edition
15.24
Silberschatz, Galvin and Gagne ©2013
Ext2fs Block-Allocation Policies
Operating System Concepts Essentials – 2nd Edition
15.25
Silberschatz, Galvin and Gagne ©2013
Journaling
 ext3 implements journaling

File system updates first written to log (transactions)
 Once
 Log
in log file, considered committed
file transactions eventually replayed over file system

On system crash, some transactions might be in journal
but not yet placed into file system

Must be completed once system recovers

No other consistency checking is needed after crash
 Much
faster than older methods
 Improves write performance on hard disks by turning
random I/O into sequential I/O
Operating System Concepts Essentials – 2nd Edition
15.26
Silberschatz, Galvin and Gagne ©2013
Chapter 15: The Linux System
 Memory Management
 File Systems
 Input and Output
 IPC
 Network Structure
 Security
Operating System Concepts Essentials – 2nd Edition
15.27
Silberschatz, Galvin and Gagne ©2013
Input and Output
 Linux device-oriented file system accesses disk
storage through two caches:

Data cached in page cache, which is unified with
virtual memory system

Metadata cached in buffer cache, a separate
cache indexed by physical disk block
Operating System Concepts Essentials – 2nd Edition
15.28
Silberschatz, Galvin and Gagne ©2013
Input and Output
 Linux splits all devices into three classes:

Block devices allow random access to completely
independent, fixed size blocks of data

Character devices include most other devices;
don’t need to support functionality of regular files

Network devices interfaced via kernel’s
networking subsystem
Operating System Concepts Essentials – 2nd Edition
15.29
Silberschatz, Galvin and Gagne ©2013
Block Devices
 Provide main interface to all system disk devices
 Block buffer cache serves two main purposes:

Acts as pool of buffers for active I/O

Serves as cache for completed I/O
 Request manager manages reading and writing
buffer contents to / from a block device driver
Operating System Concepts Essentials – 2nd Edition
15.30
Silberschatz, Galvin and Gagne ©2013
Block Devices
 Kernel 2.6 introduced Completely Fair Queuing
(CFQ)

Now default scheduler

Fundamentally different from elevator algorithms

Maintains set of lists, one for each process

Uses C-SCAN algorithm, with round robin between
all outstanding I/O from all processes

Four blocks from each process put on at once
Operating System Concepts Essentials – 2nd Edition
15.31
Silberschatz, Galvin and Gagne ©2013
Device-Driver Block Structure
Operating System Concepts Essentials – 2nd Edition
15.32
Silberschatz, Galvin and Gagne ©2013
Character Devices
 A device driver which does not offer random access
to fixed blocks of data
 Character device driver must register set of
functions that implement various file I/O operations
 Kernel performs almost no preprocessing of file read
or write request to character device

Simply passes on request to device

Main exception: special subset of character device
drivers which implement terminal devices,
 Kernel
maintains a standard interface
Operating System Concepts Essentials – 2nd Edition
15.33
Silberschatz, Galvin and Gagne ©2013
Chapter 15: The Linux System
 Memory Management
 File Systems
 Input and Output
 IPC
 Network Structure
 Security
Operating System Concepts Essentials – 2nd Edition
15.34
Silberschatz, Galvin and Gagne ©2013
Interprocess Communication
 Like UNIX, Linux uses signals to inform processes
that an event has occurred

Limited number of signals

Signals cannot carry information
 Only
that signal has occurred
 Linux kernel does not use signals to communicate
with processes running in kernel mode

Communication within kernel accomplished via
scheduling states and wait_queue structures
Operating System Concepts Essentials – 2nd Edition
15.35
Silberschatz, Galvin and Gagne ©2013
Interprocess Communication
 Also implements System V Unix semaphores

Process can wait for signal or semaphore

Semaphores scale better

Operations on multiple semaphores can be atomic
Operating System Concepts Essentials – 2nd Edition
15.36
Silberschatz, Galvin and Gagne ©2013
Passing Data Between Processes
 Pipe mechanism allows child process to inherit
communication channel to its parent

Data written to one end of pipe can be read at other
 Shared memory offers extremely fast way to
communicate

Any data written by one process can be immediately
read by any other process that has mapped that
region into its address space

Shared memory must be used in conjunction with
synchronization
Operating System Concepts Essentials – 2nd Edition
15.37
Silberschatz, Galvin and Gagne ©2013
Chapter 15: The Linux System
 Memory Management
 File Systems
 Input and Output
 IPC
 Network Structure
 Security
Operating System Concepts Essentials – 2nd Edition
15.38
Silberschatz, Galvin and Gagne ©2013
Network Structure
 Networking is key area of functionality for Linux

Supports standard Internet protocols for UNIX to
UNIX communications

Also implements protocols native to non-UNIX
operating systems
 Internally, networking in Linux kernel implemented
via three layers of software:

Socket interface

Protocol drivers

Network device drivers
Operating System Concepts Essentials – 2nd Edition
15.39
Silberschatz, Galvin and Gagne ©2013
Network Structure
 Internet protocol suite is most important set of
protocols in Linux networking system

Implements routing between different hosts anywhere
on the network

UDP, TCP and ICMP protocols built on top of routing
protocol
 Packets also pass to firewall management for
filtering based on firewall chains of rules
Operating System Concepts Essentials – 2nd Edition
15.40
Silberschatz, Galvin and Gagne ©2013
Chapter 15: The Linux System
 Memory Management
 File Systems
 Input and Output
 IPC
 Network Structure
 Security
Operating System Concepts Essentials – 2nd Edition
15.41
Silberschatz, Galvin and Gagne ©2013
Security
 Pluggable authentication modules (PAM) system
is available under Linux

Based on shared library used by any system
component that needs to authenticate users
 Access control performed through use of unique
numeric identifiers (uid and gid)
 Access control performed by assigning objects a
protections mask

Specifies which access modes (read, write, execute)
are to be granted to processes with owner, group, or
world access
Operating System Concepts Essentials – 2nd Edition
15.42
Silberschatz, Galvin and Gagne ©2013
End of Chapter 15
Operating System Concepts Essentials – 2nd Edition
Silberschatz, Galvin and Gagne ©2013