Laboratory work in TDDI04
Pintos Assignments 3, 4
Viacheslav Izosimov
2008-02-13
viaiz@ida.liu.se
Lab 3:: General Description
• Lab 3: “Execution, termination and synchronization
of user programs”
–
–
–
–
–
Handling program arguments
Execution of several user programs
Termination of a user program
Synchronization of shared data structures
Wait system call
• We will go through many issues one more
time!
Lab 3::Main Goals
•
•
•
•
Part A
Provide synchronization for multiple programs
Provide synchronization between programs
(the most important part of the lab)
•
•
•
Part B
Loading program arguments
(probably, was the most difficult part of the lab)
•
File synchronization: Not yet addressed. It is a part of
Lab 4
waiting until completion of start_process
Wait /
own
sema_d
pid1 = exec(“nasty_program”); // runs executable
4
ex
ec
uti
on
SP
ini : sta
tia ck
liz
ati
on
SP
:
st a
rt_
pro
ce
loa
din
g
Signal /
p
sema_u
pro
c
ge ess_
ne ex
rat ec
e p ute
ex
id :
e
ret c s
urn yst
ex
pid em
ec
uti
or call:
on
-1
Parent
ex
ec
s
ex
ec
uti
on
Child
(pid1)
ys
t em
pro
ca
ce
ll
ss_
ex
ec
ute
ss(
SP
)
Lab 3::Exec, Exit and Wait (1)
Lab 3::Exec, Exit and Wait (2)
Parent
Wait /
own
sema_d
ex
it
ex
ec
uti
on
ex
it_
c
wa
ge it:
t th
e
wa
sy i t
ste
m
ca
ll
Signal /
p
sema_u
pd
e
ex
pa it:
ss
ex
it_
c
od
e
) ; // the child exits with the exit code 0
ex
ec
uti
on
Child
(pid1)
0
ex
ec
uti
on
exit(
waiting
exit_code = wait(pid1) ; // the parent waits for a child process
Note that a parent can have several children!
5
Lab 3::Exec, Exit and Wait (3)
•
pid_t exec (const char *cmd_line)
•
Runs the executable whose name is given in
cmd_line, passing any given arguments, and
returns the new process’s program id (pid)
•
Must return pid -1, if the program cannot load or
run for any reason (!)
Lab 3::Exec, Exit and Wait (4)
•
void exit (int status)
•
Terminates the current user program, returning the
exit code status to the kernel.
status of 0 indicates success and nonzero values
indicate errors
Remember to free all the resources that will be not
needed anymore.
•
•
Lab 3::Exec, Exit and Wait (5)
•
int wait (pid t pid)
•
•
Provides synchronization between user programs.
"Parent" process waits until its pid-"child" process
dies (if the child is still running) and receives the
"child" exit code.
If the child has been finished, wait() should return
child's exit value without waiting.
•
Seems to be difficult…
pid = process ID
tid = thread ID
The Pintos kernel itself has to be
associated with pid!
Lab 3::Exec
Add your implementation of exec() functionalities
into process_execute() and process_start() in process.c
process_execute() {
start_process() {
…
loading – DONE!
Your Exec() system
tid = thread_create
initialization – DONE!
call in syscall.c {
putting program
…
…
generate pid from tid;
arguments into stack
f-eax = process_execute
wait until start_process();
is a PART B;
}
return pid or -1
signal to process_execute
}
}
pid = -1, if the program cannot load or run for any reason.
Use an array or a list to keep track of pid:s.
pid might equal tid, because we have only one thread per process.
Limit the number of user programs (t.ex. 64 or 128).
The Pintos kernel itself has to be
associated with pid!
userprog/process.c
Lab 3::Initialization (1)
Add process_init() function here with initialization of process
array, synchronization, assigning pid to the kernel, and so on…
Get file_name from the command line, use strtok_r
threads/init.c
Lab 3::Initialization (2)
Then call process_init() function somewhere here
Lab 3::Exit (1)
The most suitable place for Exit() functionalities
is in your implementation of systems calls in syscall.c
Your Exit() system
call in syscall.c {
get exit code from user;
save the exit code if needed;
thread_exit
}
thread_exit() {
…
process_exit
}
process_exit() {
…
clean up program’s
resources;
}
Exit() must return -1 to the “parent” program if something is wrong,
for example, if the child has caused a memory violation.
You should take care of it!
Clean up program’s resources before the exit!
printf("%s: exit(%d)\n", thread-name, thread-exit-value) before any exit.
(This is needed for testing purposes.)
Lab 3::Exit (2)
Lab 3::Wait
Once you get pid, just call process_wait()
(located in process.c) from Wait() system call:
Steps to accomplish wait():
1.Wait until the exit code of child pid is available
2.Get the exit code and remove it from the system
3.Return the exit code (or -1 if something is wrong)
Lab 3::Situations with Wait (1)
•
•
"Parent" exits without calling wait() while the
"child" is still running
"Child" exits before the "parent" and:
– "parent" calls wait() afterwards, or
– "parent" will exit without calling wait().
•
"Parent" calls wait() before the "child" exits.
+
All the situations above under the condition that
the child does not exit normally.
Lab 3::Situations with Wait (2)
•
"Parent" exits without calling wait() while the
"child" is still running
exit(0)
Child
Do not store the exit code!
Parent
exec(Child)
exit(0)
Lab 3::Situations with Wait (3)
•
"Child" exits before the "parent" and:
– "parent" calls wait() afterwards
exit(0)
Child
Destroy the exit value!
keep the exit value
Parent
exec(Child)
wait(Child)
exit(0)
wait() returns child’s exit value without waiting
Lab 3::Situations with Wait (4)
•
"Child" exits before the "parent" and:
– "parent" will exit without calling wait().
exit(0)
Child
Destroy the exit value!
keep the exit value
Parent
exec(Child)
exit(0)
You should keep child’s exit value until the parent exits
(since the child doesn’t know if the parent calls wait() later on)
Lab 3::Situations with Wait (5)
•
"Parent" calls wait() before the "child" exits.
Destroy the exit value!
exit(0)
Child
wait for the child
Parent
exec(Child)
wait(Child)
exit(0)
the parent waits for its child…
+
Lab 3::Situations with Wait (6)
All the situations above under the condition that
the child does not exit normally.
Lab 3::To Be Remembered…
Parts of the functions accessing shared resources
must be thread safe, e.g. employ synchronization
techniques such as locks and semaphores.
Particularly, access to global objects and data
must be synchronized.
Only one thread can have
access to the console at a time.
Other threads must wait until
completion of reading/writing.
21
Lab 3::Test (2)
• To debug:
/* grandfather.c
*/
#include <syscall.h>
#include <stdio.h>
int
main (void){
int pid1, pid2, pid3;
pid1 = exec("parent1");
wait(pid1);
pid2 = exec("parent2");
wait(pid2);
pid3 = exec("parent3");
wait(pid3);
exit(0);
}
/* parentX.c
*/
#include <syscall.h>
#include <stdio.h>
int
main (void){
int i;
int pid[10];
for(i = 0; i < 10; i++) {
pid[i] = exec("childX");
}
for(i = 0; i < 10; i++) {
wait(pid[i]);
}
exit(0);
}
22
Lab 3::Test (3)
/* childX.c
*/
#include <syscall.h>
#include <stdio.h>
int
main (void){
int i;
for(i = 0; i < 20000; i++) {
int a = (i * i) + (i * i);
int b = i;
i = a; a = b; i = b;
}
write(1,”PASS Lab X ON Time.\n”,20);
exit(0);
}
Create 1 grandfather,
3 “parents” and 3 “children”
in examples directory:
grandfather.c
parent1.c
parent2.c
parent3.c
child1.c
child2.c
child3.c
Replace “X” in all the places
in the code for “parents”
and “children” with 1, 2, and 3, respectively.
23
Lab 3::Test (4)
Modify Makefile in examples,
such that all the programs are compiled:
…
PROGS = … parent1 parent2 parent3 child1 \
child2 child3 grandfather
…
parent1_SRC = parent1.c
child1_SRC = child1.c
parent2_SRC = parent2.c
child2_SRC = child2.c
parent3_SRC = parent3.c
child3_SRC = child3.c
grandfather_SRC = grandfather.c
Compile the programs with gmake.
24
Lab 3::Test (5)
Copy them all on the Pintos disk in userprog
(don’t forget to create the disk and format it).
Start them as pintos --qemu -- run grandfather
As the output programs should print:
Firstly, all 10 “PASS Lab 1 ON Time.”
Secondly, all 10 “PASS Lab 2 ON Time.”
Thirdly, all 10 “PASS Lab 3 ON Time.”
Then, try to comment some wait() in grandfather and
parents to see how it behaves. The order should
be destroyed…
(Don’t forget to recompile the programs and copy on the disk again.
25
You may need to format the disk before.)
Lab 3::Part B (PB)
• Task: Load program arguments.
GOOD NEWS: You will receive the source code once
you are done with Part B
BAD NEWS: The code will contain a couple of nasty “bugs”
26
Lab 3::PB::Preparatory Steps (1)
CHANGE IT BACK!
• Lab 2::STEP 3
• into userprog/process.c, find setup_stack()
– *esp = PHYS_BASE;
– change to *esp = PHYS_BASE - 12;
• So that you have
– *esp = PHYS_BASE;
27
Lab 3::PB::Preparatory Steps (2)
• Before continuing, explain why you have
"KERNEL PANIC" after you have removed "-12".
What is wrong and why did the program work
before?
28
Lab 3::PB::Preparatory Steps (3)
• The user program with arguments should be called
with '...' from the Pintos command line:
• pintos --qemu -- run ‘nasty_program arg1 arg2
arg3’
• When the user program with arguments is called
from exec(), you have to call it like this:
• exec("nasty_program arg1 arg2 arg3")
• The implementation of all these things have to be
done in start_process()
29
pid = process ID
tid = thread ID
Lab 3::PB::Exec
Add your implementation of exec() functionalities
into process_execute() and process_start() in process.c
process_execute() {
start_process() {
…
loading – DONE!
Your Exec() system
tid = thread_create
initialization – DONE!
call in syscall.c {
putting program
…
…
generate pid from tid;
arguments into stack
f-eax = process_execute
wait until start_process();
is a PART B;
}
return pid or -1
signal to process_execute
}
}
pid = -1, if the program cannot load or run for any reason.
Use an array or a list to keep track of pid:s.
pid might equal tid, because we have only one thread per process.
Limit the number of user programs (t.ex. 64 or 128).
Lab 3::PB::Exec
Add command line parsing and
stack initialization code here!
31
Lab 3::PB::Implementation (1)
• STEP 1. Parse the string:
• Use strtok_r(), prototyped in lib/string.h
• Read comments in lib/string.c or man
page (run man strtok_r)
• Limit the number of arguments (for simplicity)
• STEP 2. Set up the stack:
• Necessary details about setting up the stack for
this task you can find in Program Startup
Details section of Pintos documentation. 32
Lab 3::PB::Implementation (2)
33
Lab 3::PB::Implementation (2)
• nasty_program arg1 arg2 arg3
• After parsing: nasty_program, arg1,
arg2, arg3
• Place the words at the top of the stack
• Align to 4-byte-words, add 0’s
• Reference the words through the pointers
(pointers should point to the addresses of the
words in the stack)
• Put the pointers to the stack (followed with
NULL pointer)
34
Lab 3::PB::Implementation (2)
• Put the address of the first pointer to the stack
• Put the number of words to the stack (the
number of arguments + 1)
• Put “faked” return address to the stack (e.g.
NULL). This is needed in order to meet x86
conventions for program arguments, even
though this return address will not be used.
• So, what we get if we assume that
PHYS_BASE is 0xc0000000 …
35
Address
Name
Type
word-align
Data
arg3\0
arg2\0
arg1\0
nasty_program\0
0
29 --> 32
0xbffffffb
argv[3][...]
0xbffffff6
argv[2][...]
0xbffffff1
argv[1][...]
0xbfffffe3
argv[0][...]
0xbfffffe0
0xbfffffdc
0xbfffffd8
0xbfffffd4
0xbfffffd0
0xbfffffcc
0xbfffffc8
argv[4]
argv[3]
argv[2]
argv[1]
argv[0]
argv
0
0xbffffffc
0xbffffff8
0xbffffff4
0xbfffffe6
0xbfffffd0
char *
char *
char *
char *
char *
char **
0xbfffffc4
argc
4
0
0xbfffffc0
return address
char[5]
char[5]
char[5]
char[14]
uint24_t
int
36
void (*) ()
Lab 3::Test (1)
• Now you are ready for a complete check!
• Run gmake check from userprog/build
• The following tests should pass:
• 1) Argument passing when executing:
args-none, args-single, args-multiple,
args-many, args-dbl-space
2) Different exec-tests:
exec-once, exec-arg, exec-multiple,
exec-missing, exec-bad-ptr
3) Wait-tests:
wait-simple, wait-twice, wait-killed,
37
wait-bad-pid
Lab 3::Test (2)
• Many of the other checks in gmake check belong to the second
lab. If any of them fails, then it means that something is wrong
with your implementation of Lab 2:
• sc-bad-sp, sc-bad-arg, sc-boundary,
sc-boundary-2, halt, exit, create-normal,
create-empty, create-null, create-bad-ptr,
create-long, create-exists, create-bound,
open-normal, open-missing, open-boundary,
open-empty, open-null, open-bad-ptr,
open-twice, close-normal, close-stdin,
close-stdout, close-bad-fd, read-bad-ptr,
read-boundary, read-zero, read-stdout,
read-bad-fd, write-normal, write-bad-ptr,
write-boundary, write-zero, write-stdin,
38
write-bad-fd
Lab 4:: General Description
• Lab 4: “File System”
– Synchronization of read-write operations
•One writer writes at a time
•Many readers can read
– Additional system calls to work with files
•seek()
•tell()
•filesize()
•remove()
– Creating and removing files without destroying
the file system
Lab 4:: General Description
• Lab 4: “File System”
– Synchronization of read-write operations
•One writer writes at a time
•Many readers can read
– Additional system calls to work with files
•seek()
•tell()
•filesize()
•remove()
– Creating and removing files without destroying
the file system
Lab 4::Files (1)
• filesys/file.[h|c] - operations on
files. A file object represents an open file.
• filesys/filesys.[h|c] - operations on
the file system.
• filesys/directory.[h|c] - operations
on directories.
• filesys/inode.[h|c] - the most
important part of the implementation related to
the file system. An inode object represents an
individual file (e.g. several open files fd1, fd2,
fd3 may belong to one inode “student.txt”).
41
Lab 4::Files (2)
• devices/disk.[h|c] - implementation of
the low-level access to the disk-drive.
• filesys/free-map.[h|c] implementation of the map of free disk sectors.
• These two last ones are not important for
you!
42
Lab 4::Reading/Writing (1)
• Several readers should be able to read from a
file at the same time
• Reading should be forbidden if the file content
is being changed by the writer
• Only one writer can write to a file at a time
• The writer must not write if at least one reader
is reading from the file
43
Lab 4::Reading/Writing (2)
• Readers should not starve
• Writers can starve
• However, think about solution to avoid the
problem of starvation, even though you are not
obliged to implement it
• Note that, in this lab assignment, you are not
asked to implement dynamic enlargement /
reducing of the file size (if you want to write
more than the file size currently is)
44
Lab 4::Reading/Writing (3)
• Ways to implement:
– Locks/Conditions
– Semaphores
• One file can be opened several times and
processes can attempt to read/write
simultaneously at any place in the file
• You need to apply synchronization not to
each file structure, but to the entire file!
• Synchronization can be done on one of three
levels: system calls, files, and inodes.
45
• Which level would you choose? Motivate!!!
Synchronization Pitfalls
• Inode counters in inode.c (for example,
open_cnt counter)
• Access to the disk drive during creating a file:
– Step 1: Creating an entity in the directory
– Step 2: Creating a tree of inodes
– Must not be interrupted!
• Global shared data such as FreeMap – a global
bitmap for the entire hard drive (critical section
problem!)
46
Lab 4:: Additional System Calls
• void seek (int fd, unsigned position)
– Sets the current position in the open file fd to position. If the
position exceeds the file size, it should be set to the end of file.
• unsigned tell (int fd)
– Returns the current position in the open file fd.
• int filesize (int fd)
– Returns the file size of the open file fd.
• bool remove (const char *file_name)
– Removes the file file_name.
– Note that the open files must not be deleted from the file system
before they are closed. If the file is to be removed, the operating
system should wait until the file is closed and only then it47can delete
it. This functionality has been already implemented!
Lab 4:: Create and Remove
• Creating and removing of files must not lead to
destructive consequences to the file system
• Create and remove are writing operations on the directories
(filesys/directory[.h|.c])
• Open is the reading operation on the directories
• In principle, synchronization of reading/writing operations
on directories should be handled as synchronization of files,
but…
• For the sake of making student life more convenient, in this
lab assignment, you are advised to handle synchronization of
48
directories as the critical section problem
Lab 4::Final Tests (1)
• The following tests should pass if your implementation is correct in
addition to the tests from Lab 2 and Lab 3:
• tests/filesys/base/lg-create
tests/filesys/base/lg-full
tests/filesys/base/lg-random
tests/filesys/base/lg-seq-block
tests/filesys/base/lg-seq-random
tests/filesys/base/sm-create
tests/filesys/base/sm-full
tests/filesys/base/sm-random
tests/filesys/base/sm-seq-block
tests/filesys/base/sm-seq-random
tests/filesys/base/syn-read
tests/filesys/base/syn-remove
tests/filesys/base/syn-write
tests/userprog/close-twice
tests/userprog/read-normal
tests/userprog/multi-recurse
tests/userprog/multi-child-fd
49
Lab 4::Final Tests (2)
•
In order to run the tests you need to do the following:
1.
Copy this Make.tests to "pintos/src/tests/userprog".
2.
Copy this Make.vars to "pintos/src/userprog".
3.
Go to "pintos/src/userprog".
4.
Clean up everything with "gmake clean".
5.
Run "gmake".
6.
Go to "pintos/src/userprog/build".
7.
Run "gmake check".
•
All 66 tests should pass if the implementation is correct.
50
Lab 4::Final Tests (3)
•
•
•
1.
2.
3.
4.
5.
6.
For testing your readers-writers algorithm, we provide the following user
programs: pfs.c, pfs_r.c, pfs_w1.c, and pfs_w2.c.
These programs try to emulate several readers and writers accessing the
same file.
In order to run these programs, you should do the following steps:
Copy these programs to "examples" directory.
Modify the following line in the "examples/Makefile":
PROGS = … pfs pfs_r pfs_w1 pfs_w2
Add the following lines to the "examples/Makefile":
pfs_SRC = pfs.c
pfs_r_SRC = pfs_r.c
pfs_w1_SRC = pfs_w1.c
pfs_w2_SRC = pfs_w2.c
Run gmake from "examples" directory.
Copy the executables pfs, pfs_r, pfs_w1 and pfs_w2 to the Pintos disk
(which is in "userprog/build").
51
Run pfs in Pintos.
Lab 4::Final Tests (4)
•
The result is copied into messages file, which should only contain the
word "cool" like this:
cool
cool
cool
cool
cool
cool
cool
cool
cool
cool
cool
cool
cool
cool
cool
52
...
Conclusion (1)
• In this course you do the first “real”
programming
• Learning of handing complex programming
tasks
• Self-management training
• Training of planning skills
• Working with a pile of extensive
documentation
• And, last but not least, understanding of the
basic concepts of operating systems
53
Conclusion (2)
Do not wait until the summer vacation!
Complete your assignments now!
54