Project 3: File System Design
COS318
Fall 2002
Last Time
• Web Server
server
client
u/
vivek/
public_html/
request
response
index.html cos318_02/
• Extensive use of a file system on server
machine without actually worrying about
how it was implemented.
Outline
•
•
•
•
•
•
Introduction
Design requirement
Helper functions
Implementation of file system calls
Hints
Reference
Introduction
• Why do we need a file system?
– Abstraction
– Protected access
– Security
• Simple Unix-like file system
– i-node
– directory tree
– links(symbolic/physical)
What does my file server
need to accomplish
• A New Client Program with
• New Methods:
– GET <dir name | file name>
• Fetch a file or dir listing and show them
– CREATE <dir name>
• Create a dir in server disk
– DELETE <file name | dir name>
• Delete a file or dir in server disk
• populated directories should not be deleted
– PUT <file name> <local file name> <startpos> <endpos>
• Create a new file in server disk, or append to an exsiting file
with contents from local file
[Notice: name uses absolute path]
More on file server
functionality
• Use our client program to create a
populated directory tree inside of server
disk, and then use your web server to
serve this content.
• More test programs:
– Randomly requests
– Maybe not just be PUT or GET: after some file
deletions, things will become more interesting.
– Coming soon…
So what do you have
• A Virtual Disk: a data structure
– Disk_name: dev_318
– Disk_pointer: used to access the disk(in fact a
file descriptor)
– Sector_size: 512 bytes
– Capacity: M, G …
• Rather than write to a real disk, you'll be
building a disk inside of a large file
• All development and debugging are in user
space
Helper Functions: not
tuxedo
•
•
•
•
•
•
GetExistingDisk(char *pathname)
CreateNewDisk(void)
CloseDisk(void)
ReadSector(int sectorNum, char *buf)
WriteSector(int sectorNum, char *buf)
Notice:
– You are allowed to access disk only with those
function calls;
Helper Functions(cont.)
• GetExistingDisk(char *pathname)
– returns a pointer to previously created disk in
directory pathname, or exits if no such disk
• CreateNewDisk(void)
– returns a pointer to a new disk
• CloseDisk(void)
– closes the virtual disk(dev_318)
Helper Functions(cont.)
• ReadSector(int sectorNum, char *buf)
– reads 512 bytes from disk
• WriteSector(int sectorNum, char *buf)
– writes 512 bytes from disk
• Notice:
– If any other user space function calls can facilitate your
work, feel free to use them.
• Using file system functions to build disk is
relatively easy compared to using really disk to
build file system
What do I need to do
• You should build on the code you wrote for
the previous assignments
– Good thing: we have a reference server
• The first step is to understand the
default file system provided by the base
code(helper functions)
• You'll have to understand at least one new
header with one of the new methods.
What do I need to
do(cont.)
• You'll have to write versions of various
system calls that you used in Project 1
– mkfs, open, close, read, and write
– Additionally, you'll also have to implement
versions of stat, mkdir, and a few others, also
function calls to getting file metadata and
directory information in order to properly build
directory listings
• No need to handle all of the various flags
and parameters, but should implement
enough to get the project working
correctly
One example-open()
• For example, the open() system call can be
used to try to test the existence of a file
in addition to creating it for writing
• O_CREATE: the file creation flag
– To properly handle “PUT" method
• Not always true: need other flags
– “GET” a non-existent file
mkfs
• File system creation
– Dos: format
– Linux: mkfs/mke2fs/mkfs.msdos
– Solaris: newfs
• Structure of a typical file system(unixlike)
– Superblock
• Describes layout of i-Nodes, bitmaps, blocks, root
directory, etc.
– i-Nodes
• Describes a file/dir – Link counter, list of blocks, size, etc.
(owner, permission, …)
– Block Allocation Bitmap
– Data blocks
mkfs(cont.)
• Creates the file system
–
–
–
–
Set up the Superblock
Flag all i-Nodes as type free (f)
Zero out the Alloc Bitmap
Create the root directory
•
•
•
•
Flag an i-Node as type directory (D)
Store it’s number in Superblock
Alloc one block in the bitmap
Initialize the ‘.’ and ‘..’ directory entries and set the
others as being INVALID
Block
Bootblock Super
Alloc
Kernel Block i-Nodes
Bitmap*
Procs
Data Blocks
SuperBlock
• Describes the layout of the file system
–
–
–
–
–
–
–
–
–
–
signature – A magic number
size – Total size of fs in blocks
root_inode – The root directory
inode_start – 1st block for i-Nodes
inode_blocks – Total i-Nodes blocks
bitmap_start – 1st block of bitmap
bitmap_blocks – Total bitmap blocks
alloc_start – 1st block for data
num_blocks – Total data blocks
free_blocks – Count of unallocated blocks
Bitmap
• Bitmap
– Indicates which data blocks are free,
and which are used
– Maps the data blocks only (not i-Nodes)
I-node
• Describes a file/directory
–
–
–
–
–
–
type – File, directory
opens – Count of open file handles
links – Count of links to this i-Node
size – Total byte count
direct[10] – List of data blocks
indirect[3] – List of indirect blocks
• Advantages
– Fast access to small files
– Supports very large files
– Support sparse files
Hints
• Break this project into two components expansion of the main server itself, and
implementation of the underlying disk
• Within the file system:
– Start with the creation of files and try to get
that working properly;
– Then build directories on top of that
mechanism;
– Finally work on directory listing support.
• For files: direct blocks first
More Hints
• Keep the high level interface as much as
possible
– By having wrappers that can either call the real
call or your replacement, you'll be able to do
testing quickly and efficiently
• Standard interface:
– Stat(const char *pathname, struct stat *buf)
stat
• Returns statistics for given a filename
–
–
–
–
–
inodeNo – The file’s i-Node number
Type – File or directory
Opens – The open count
Size – The byte count to EOF
numBlocks – Number of allocated blocks
Reference
• Check the manual page of the
function calls
– What they do
– What’s their interface
– What’s the most useful parameters
Wrap Up
•
•
•
•
Fun again? Probably….
Not too difficult.
But a little bit more coding.
Questions?