TDDI04
Concurrent Programming,
Operating Systems,
and Real-time Operating Systems
File System Implementation
§ File-System Structure
§ File-System Implementation
§ Directory Implementation
File System Implementation
§ Allocation Methods
§ Free-Space Management
§ Recovery
[SGG7] Chapter 11
Copyright Notice: The lecture notes are mainly based on Silberschatz’s, Galvin’s and Gagne’s book (“Operating System
Concepts”, 7th ed., Wiley, 2005). No part of the lecture notes may be reproduced in any form, due to the copyrights
reserved by Wiley. These lecture notes should only be used for internal teaching purposes at the Linköping University.
Acknowledgment: The lecture notes are originally compiled by C. Kessler, IDA.
Klas Arvidsson, IDA,
Linköpings universitet.
File-System Structure
TDDI04, K. Arvidsson, IDA, Linköpings universitet.
© Silberschatz, Galvin and Gagne 2005
11.2
In-Memory File System Structures
§ File system resides on
secondary storage (disks)
File API:
filenames, directories,
attributes, access...
§ File system organized into layers:
§ File control block (FCB) –
storage structure consisting of
information about a file
Logical block addresses
(1D array of blocks)
Physical block addresses
on disk (cylinder, sector,...)
(a) Create / open a file
read/write block
commands
Layered File System
TDDI04, K. Arvidsson, IDA, Linköpings universitet.
11.3
© Silberschatz, Galvin and Gagne 2005
Virtual File System (VFS)
(b) Reading an open file
TDDI04, K. Arvidsson, IDA, Linköpings universitet.
© Silberschatz, Galvin and Gagne 2005
11.4
Directory Implementation
Directory
VFS allows the same system call interface (the API) to be used for
different types
of file systems.
The API is to
the VFS interface,
rather than any specific
type of file system.
Files
F1 F2 F3 F4
Fn
§ Linear list of file names with pointer to the data blocks.
• simple to program
• time-consuming to execute
§ Hash Table
• decreases directory search time
•
•
TDDI04, K. Arvidsson, IDA, Linköpings universitet.
11.5
© Silberschatz, Galvin and Gagne 2005
collisions – situations where two file names hash to the
same location
fixed size
TDDI04, K. Arvidsson, IDA, Linköpings universitet.
11.6
© Silberschatz, Galvin and Gagne 2005
1
Allocation Methods
Contiguous Allocation
§ An allocation method refers to
how disk blocks are allocated for files
§ Each file occupies a set of
contiguous blocks on the disk
•
•
•
Contiguous allocation
§ Simple –
only starting location
(block index)
and length (# of blocks)
are required
Linked allocation
Indexed allocation
§ Random access
§ External fragmentation
§ Files cannot grow (easily)
TDDI04, K. Arvidsson, IDA, Linköpings universitet.
© Silberschatz, Galvin and Gagne 2005
11.7
TDDI04, K. Arvidsson, IDA, Linköpings universitet.
11.8
Extent-Based Systems
Linked Allocation
§ Many newer file systems (e.g., the Veritas File System)
use a modified contiguous allocation scheme
§ Each file is a linked list of disk blocks:
blocks may be scattered
anywhere on the disk.
© Silberschatz, Galvin and Gagne 2005
§ Extent-based file systems allocate disk blocks in extents
block
=
pointer
§ An extent is a contiguous set of blocks
•
•
Extents are allocated for file allocation
Simple
+ need only starting address
Free-space management
+ no external fragmentation
But very poor random access
– must scan list causing much
disk access
A file consists of one or more extents.
TDDI04, K. Arvidsson, IDA, Linköpings universitet.
© Silberschatz, Galvin and Gagne 2005
11.9
TDDI04, K. Arvidsson, IDA, Linköpings universitet.
11.10
© Silberschatz, Galvin and Gagne 2005
File-Allocation Table (FAT)
Indexed Allocation
File-allocation table (FAT) – disk-space allocation used by MS-DOS and OS/2.
§ Brings all pointers together into the index block
§ Logical view
in FAT
FAT resides in reserved section
at beginning of each disk volume
One entry for each disk block,
indexed by block number
index table
Entry for last block in a chain has table value -1
Unused blocks have table value 0
-> Variant of linked allocation
where references to blocks are implicit
TDDI04, K. Arvidsson, IDA, Linköpings universitet.
11.11
© Silberschatz, Galvin and Gagne 2005
TDDI04, K. Arvidsson, IDA, Linköpings universitet.
11.12
© Silberschatz, Galvin and Gagne 2005
2
Indexed Allocation (Cont.)
Multilevel-indexed allocation
§ No external fragmentation
• But overhead of index block (internal fragmentation)
§ How large should an index block be?
•
•
•
Small => very limited file size
Large => much wasted space for small files
Μ
Either way limited file size
§ Structure of index blocks
• List
outer-index
•
Tree
§ Random access
•
file
index table
Just disk access to index block(s) and searched block
TDDI04, K. Arvidsson, IDA, Linköpings universitet.
11.13
© Silberschatz, Galvin and Gagne 2005
TDDI04, K. Arvidsson, IDA, Linköpings universitet.
© Silberschatz, Galvin and Gagne 2005
11.14
Combined Scheme: UNIX inode
Free-Space Management (1)
Block size 4 KB
-> With 12 direct block pointers kept in the inode, 48 KB can be addressed directly.
§ Bit vector (32xn words for n blocks assuming 32 bits per word)
0 1
2
n-1
…
bit[i]
1 ⇒ block[i] free
0 ⇒ block[i] occupied
(block number of bit) == (bits per word) * (index of word) + (offset of bit within the word)
§ Easy to find free blocks or n consecutive free blocks
§ Requires extra space:
Block size of 2KiB (211 bytes)
Disk size of 1TiB (240 bytes)
240/211 = 229 blocks (bits) = 226 bytes (64MiB)
§ Must be in memory to be efficient
TDDI04, K. Arvidsson, IDA, Linköpings universitet.
11.15
© Silberschatz, Galvin and Gagne 2005
Free-Space Management (2)
TDDI04, K. Arvidsson, IDA, Linköpings universitet.
© Silberschatz, Galvin and Gagne 2005
11.16
Free-Space Management (3)
§ Grouping
a really free block
(n-1 references to free blocks)
§ Linked free space list on disk
•
•
Cannot get contiguous space easily
•
No waste of space
First ”free” block
Inefficient to allocate many blocks
(lot of disk access to traverse list)
TDDI04, K. Arvidsson, IDA, Linköpings universitet.
…
§ Counting
• Often, multiple subsequent blocks are allocated/freed
together
• For sequences of free blocks located subsequently on disk,
keep only reference to first one and length of sequence
11.17
© Silberschatz, Galvin and Gagne 2005
TDDI04, K. Arvidsson, IDA, Linköpings universitet.
11.18
© Silberschatz, Galvin and Gagne 2005
3
Page Cache
I/O Without a Unified Buffer Cache
§ A page cache caches pages rather than disk blocks
using virtual memory techniques
§ Memory-mapped I/O uses a page cache
§ Routine I/O through the file system
uses the buffer (disk) cache
(in main memory)
§ 2 scenarios: Unified or separate buffer cache
TDDI04, K. Arvidsson, IDA, Linköpings universitet.
11.19
© Silberschatz, Galvin and Gagne 2005
TDDI04, K. Arvidsson, IDA, Linköpings universitet.
11.20
© Silberschatz, Galvin and Gagne 2005
Unified Buffer Cache
Recovery
§ A unified buffer cache uses the same page cache to cache
both memory-mapped pages and ordinary file system I/O
§ Consistency checking – compares data in directory structure
with data blocks on disk, and tries to fix inconsistencies
§ Use system programs to back up data from disk to another
storage device (floppy disk, magnetic tape, other magnetic
disk, optical)
§ Recover lost file or disk by restoring data from backup
(in main memory)
TDDI04, K. Arvidsson, IDA, Linköpings universitet.
11.21
© Silberschatz, Galvin and Gagne 2005
TDDI04, K. Arvidsson, IDA, Linköpings universitet.
11.22
© Silberschatz, Galvin and Gagne 2005
4