Laboratory work in TDDI04
Pintos Assignments 2, 3
Viacheslav Izosimov
2008-01-28
viaiz@ida.liu.se
Instead of Motivation
•
The deadline for Lab 1 is extended
to 1st of February!
KERNEL PANIC!!!
YOUR PINTOS
IMPLEMENATION
IS NOT YET READY!
Lab 2
•
•
•
•
System calls
Single user program
Memory issues
Console
Lab 2 :: Systems Calls At High Level
• System calls:
– communication between the user program and the
kernel
– functions, called from the user program and
performed by the kernel
– computers often use interrupts to accomplish that
switch from user code to system code
Lab 2:: Systems Calls At High Level
user_program.c
bool flag;
flag = create(“file.txt”, 1000);
Calling
Return result back
corresponding
to the user program
function in
pintos/src/lib/user/syscall.c
System
call
wrapper
the wrapper
bool create (const char *file, unsigned initial_size) {
return syscall2 (SYS_CREATE, file, initial_size);
}
Calling
corresponding
exception in Pintos kernel
pintos/src/userprog/syscall.c
the kernel
Return result back
to the wrapper
User exception handler
// Your code here!
Lab 2::What to Implement?
• You will need to implement:
– create - creates a file.
– open - opens a file.
– close - closes a file.
– read - reads from a file or the console (the keyboard).
– write - writes to a file or the console (the monitor).
– halt - halts the processor.
– exit - Terminates a program and deallocates resources
occupied by the program, for example, closes all files
opened by the program.
Lab 2::Useful Files
• You have to have a look at:
– pintos/src/lib/user/syscall[.h|.c] – the wrapper…
– userprog/syscall[.h|.c] – Your implementation of system
calls!!!
– threads/interrupt.[h|c] – important structures!!!
– lib/syscall-nr.h – system call constants
–filesys/filesys.[h|c] Pintos file system implementation…
– examples/lab2test.c – the “official” test program for this lab
– filesys/file.[h|c] – useful functions for operations with files.
Things which you don’t find in “filesys”, you find here
–userprog/process.c Look up what is here and tell me…
Lab 2::How Do I Start? (1)
• STEP 1
• Understand what the user program is
• Look into “examples” directory and a couple of
the user programs, especially “halt.c”
• Look into Makefile in this directory to understand
how the user programs are compiled
• Compile the user programs by “gmake”
• STEP 2
• Prepare Pintos disk
• Copy one or two user programs to the disk
Lab 2::How Do I Start? (2)
• STEP 2 (Continuation) Dealing with Pintos disk
• In “userprog/build”: pintos-mkdisk fs.dsk 2
– This will create a file fs.dsk with a 2MB simulated disk in
the directory.
• Format: pintos --qemu -- -f -q.
• Copy user programs:
pintos --qemu -p programname -- -q
with new name
pintos --qemu -p programname -a newname -- -q
Lab 2::How Do I Start? (3)
• STEP 2 (Continuation) Dealing with Pintos disk
• Example:
• pintos --qemu -p ../../examples/halt
-a severe_student_program -- -q
• To copy a file from the disk:
pintos --qemu -g programname -- -q
• or
pintos --qemu -g programname -a newname -- -q
* -p = put, -g = get
• To run: pintos
• Example:
--qemu -- run programname
• pintos --qemu -- run severe_student_program
Lab 2::How Do I Start? (4)
• STEP 3
• into userprog/process.c, find setup_stack()
– *esp = PHYS_BASE;
– change to *esp = PHYS_BASE - 12;
• STEP 4
• Make sure that you understand the problems which may
arise if you access (in the kernel) data stored in the user
memory using the pointers provided by the user program
as system call arguments.
• (First, make sure that you understand what this sentence
means…)
Lab 2::Shutdown Example
•
•
•
•
•
•
#include "userprog/syscall.h“
halt.c
#include <stdio.h>
#include <syscall-nr.h>
#include <syscall.h>
#include "threads/interrupt.h“
int
#include "threads/thread.h“
static void syscall_handler (struct intr_frame *);
main (void){
voidsyscall_init (void) {
intr_register_int (0x30, 3, INTR_ON,
halt ();
syscall_handler, "syscall");
}
}
static void
syscall_handler (struct intr_frame *f UNUSED)
userprog/syscall.c printf ("system call!\n");
get a name of the system callthread_exit ();
! f->esp – stack pointer
SYS_HALT from the stack}
use power_off()
Lab 2::Shutdown Example
• static void syscall_handler (struct intr_frame *f) {
• // printf ("system call!\n");
•
int syscall_name;
•
syscall_name = *(int *)(f->esp);
•
switch(syscall_name) {
•
case SYS_HALT:
•
printf("halt system call\n");
•
power_off();
•
break;
•
default:
•
printf("unknown syscall %d\n",
syscall_name);
•
• }
}
13
Lab 2::Create Example
• bool create (const char *file, unsigned
initial_size)
• Example: create(“file.txt”, 1000);
How to get them?
Answer: f->esp
Hint:
… note that, in order to get
How to return a value?
a string, you will need get a void pointer
Answer: f->eax
from esp and then get a char pointer to which
points that void pointer. The char pointer will point to the first
element of the string …
Lab 2::Create Example
• void SysCall_Create(struct intr_frame *f) {
•
char *file_name;
•
int file_size;
•
printf("create system call\n");
•
file_name = (char *)*(int *)(f->esp + 4);
•
file_size = *(int *)(f->esp + 8);
•
if (filesys_create(file_name, file_size)) f->eax = true;
•
else f->eax = false;
• }
• …
• case SYS_CREATE:
•
SysCall_Create(f);
•
break;
• …
15
All the pointers on the variables,
which you get from the user program,
must be validated!
Problem 1: If the pointer above PHYS_BASE,
It points to Kernel memory! UNSAFE!
17
Memory Issues in Pintos
Kernel VM
Physical Memory
Kernel process
PHYS_BASE
User process
Page directory
If no entry?
a)Kill user
b) Handle page fault
Lab 2::Memory Issues in Pintos
Kernel VM
Physical Memory
Kernel process
PHYS_BASE
Check if the pointer is below
User process
Page directory
Check if the pointer is in the page directory
If no entry?
a)Kill user
b) Handle page fault
STRUCT THREAD
20
Lab 2::Shutdown Security (Safety?)
• static void syscall_handler (struct intr_frame *f) {
•
int syscall_name;
You have to implement
•
validate_user_ptr(f->esp);“validate_user_ptr(void *)”
•
syscall_name = *(int *)(f->esp); function to check if
•
switch(syscall_name) {
the pointer is
•
case SYS_HALT:
in the user space and
•
printf("halt system call\n");
in the page directory!
•
power_off();
•
break;
•
default:
•
printf("unknown syscall %d\n",
syscall_name);
•
• }
}
21
Lab 2::Create Security
• void SysCall_Create(struct intr_frame *f) {
•
char *file_name;
•
int file_size;
•
printf("create system call\n");
•
validate_user_ptr(f->esp + 4)
•
validate_user_ptr_str((void *)*(int *)(f->esp + 4))
•
file_name = (char *)*(int *)(f->esp + 4);
•
validate_user_ptr(f->esp + 8)
•
•
•
• }
A string can span across
several pages in memory!
All pointers of the string have
to be valid. You should provide
if (filesys_create(file_name, file_size))
f->eax solution
= true; to validate
an efficient
else f->eax = false;
as less string pointers as possible!
file_size = *(int *)(f->esp + 8);
22
Lab 2::Other System Calls
• To implement the system calls, which operate
with files, look at “filesys/filesys.[h|c]” and
“filesys/file.[h|c]” (except the console)
• Everything is already done there for you! Just
call the functions!
• But don’t forget validation of system call
parameters!
Lab 2::Open and Close
• int open (const char *file)
• void close (int fd)
• Sink when you manage file IDs but not too
deep!
• I suggest using an array of *file elements and
return indexes of this array as file IDs
• 0 and 1 IDs must be always reserved for the
console!
Lab 2::Read System Call
• int read (int fd, void *buffer, unsigned size)
• Reads size bytes from the file open as fd into
buffer. Returns the number of bytes actually
read (0 at end of file), or -1 if the file could not
be read (due to a condition other than end of
file).
• Fd 0 reads from the keyboard using
input_getc() (defined in devices/input.h).
Lab 2::Write System Call
• int write (int fd, const void *buffer, unsigned size)
• Writes size bytes from buffer to the open file fd
• Returns the number of bytes actually written or -1 if the
file could not be written
• The expected behavior is to write as many bytes as
possible up to end-of-file and return the actual number
written or -1 if no bytes could be written at all
• When fd=1 then the system call should write to the
console with just one call to putbuf() (check
lib/kernel/stdio.h and lib/kernel/console.c)
Lab 2::Console
• The reading from the console should be done in a
convenient way:
• You should implement echo
• The typing should be permitted until Enter is pressed
(ASCII code 13)
• Do not use the graphical qemu terminal for keyboard
input (it behaves strangely), instead use the terminal
from which you start pintos
• Only the requested number of characters passed in a read
system call should be stored (for example, 10), the rest
should be ignored
• Do not forget to add NULL character at the end of the
string that you have read.
Lab 2::Exit System Call
• void exit (int status)
• Terminates the current user program, returning
status to the kernel. Conventionally, a status of
0 indicates success and nonzero values indicate
errors. Remember to free all the resources will
be not needed anymore.
• This system call will be improved in the
following labs.
• Important: Free resource in process_exit()
function located in process.c
CLOSE ALL OPEN FILES
Lab 3:: General Description
• Lab 3: “Execution, termination and synchronization
of user programs”
–
–
–
–
–
Handling program arguments
Execution of several user programs
Termination of a user program
Synchronization of shared data structures
Wait system call
Lab 3::Main Goals
•
•
•
•
Part A
Provide synchronization for multiple programs
Provide synchronization between programs
(the most important part of the lab)
•
•
•
Part B
Loading program arguments
(probably, was the most difficult part of the lab)
•
File synchronization: Not yet addressed. It is a part of
Lab 4
Lab 3::Exec, Exit and Wait (1)
•
pid_t exec (const char *cmd_line)
•
Runs the executable whose name is given in
cmd_line, passing any given arguments, and
returns the new process’s program id (pid)
•
Must return pid -1, if the program cannot load or
run for any reason (!)
Lab 3::Exec, Exit and Wait (2)
•
void exit (int status)
•
Terminates the current user program, returning the
exit code status to the kernel.
status of 0 indicates success and nonzero values
indicate errors
Remember to free all the resources that will be not
needed anymore.
•
•
Lab 3::Exec, Exit and Wait (3)
•
int wait (pid t pid)
•
•
Provides synchronization between user programs.
"Parent" process waits until its pid-"child" process
dies (if the child is still running) and receives the
"child" exit code.
If the child has been finished, wait() should return
child's exit value without waiting.
•
Seems to be difficult? Not really…
pid = process ID
tid = thread ID
Lab 3::Exec
Add your implementation of exec() functionalities
into process_execute() and process_start() in process.c
process_execute() {
start_process() {
…
loading – DONE!
Your Exec() system
tid = thread_create
initialization – DONE!
call in syscall.c {
putting program
…
…
generate pid from tid;
arguments into stack
f-eax = process_execute
wait until start_process();
is a PART B;
}
return pid or -1
signal to process_execute
}
}
pid = -1, if the program cannot load or run for any reason.
Use an array or a list to keep track of pid:s.
pid might equal tid, because we have only one thread per process.
Limit the number of user programs (t.ex. 64 or 128).
Lab 3::Exit (1)
The most suitable place for Exit() functionalities
is in your implementation of systems calls in syscall.c
Your Exit() system
call in syscall.c {
get exit code from user;
save the exit code if needed;
thread_exit
}
thread_exit() {
…
process_exit
}
process_exit() {
…
clean up program’s
resources;
}
Exit() must return -1 to the “parent” program if something is wrong,
for example, if the child has caused a memory violation.
You should take care of it!
Clean up program’s resources before the exit!
printf("%s: exit(%d)\n", thread-name, thread-exit-value) before any exit.
(This is needed for testing purposes.)
Lab 3::Exit (2)
Lab 3::Wait
Once you get pid, just call process_wait()
(located in process.c) from Wait() system call:
Steps to accomplish wait():
1.Wait until the exit code of child pid is available
2.Get the exit code and remove it from the system
3.Return the exit code (or -1 if something is wrong)
Lab 3::Situations with Wait (1)
•
•
"Parent" exits without calling wait() while the
"child" is still running
"Child" exits before the "parent" and:
– "parent" calls wait() afterwards, or
– "parent" will exit without calling wait().
•
"Parent" calls wait() before the "child" exits.
+
All the situations above under the condition that
the child does not exit normally.
Lab 3::Situations with Wait (2)
•
"Parent" exits without calling wait() while the
"child" is still running
exit(0)
Child
Do not store the exit code!
Parent
exec(Child)
exit(0)
Lab 3::Situations with Wait (3)
•
"Child" exits before the "parent" and:
– "parent" calls wait() afterwards
exit(0)
Child
Destroy the exit value!
keep the exit value
Parent
exec(Child)
wait(Child)
exit(0)
wait() returns child’s exit value without waiting
Lab 3::Situations with Wait (4)
•
"Child" exits before the "parent" and:
– "parent" will exit without calling wait().
exit(0)
Child
Destroy the exit value!
keep the exit value
Parent
exec(Child)
exit(0)
You should keep child’s exit value until the parent exits
(since the child doesn’t know if the parent calls wait() later on)
Lab 3::Situations with Wait (5)
•
"Parent" calls wait() before the "child" exits.
Destroy the exit value!
exit(0)
Child
wait for the child
Parent
exec(Child)
wait(Child)
exit(0)
the parent waits for its child…
+
Lab 3::Situations with Wait (6)
All the situations above under the condition that
the child does not exit normally.
Do it as a homework!
Lab 3::To Hit The Target!
1. Parts of the functions accessing shared resources
must be thread safe, e.g. employ synchronization
techniques such as locks and semaphores.
2. Particularly, access to global objects and data
must be synchronized.
3. Only one thread can have
access to the console at a time.
Other threads must wait until
completion of reading/writing.
45
Lab 3::Test (2)
• To debug:
/* grandfather.c
*/
#include <syscall.h>
#include <stdio.h>
int
main (void){
int pid1, pid2, pid3;
pid1 = exec("parent1");
wait(pid1);
pid2 = exec("parent2");
wait(pid2);
pid3 = exec("parent3");
wait(pid3);
exit(0);
}
/* parentX.c
*/
#include <syscall.h>
#include <stdio.h>
int
main (void){
int i;
int pid[10];
for(i = 0; i < 10; i++) {
pid[i] = exec("childX");
}
for(i = 0; i < 10; i++) {
wait(pid[i]);
}
exit(0);
}
46
Lab 3::Test (3)
/* childX.c
*/
#include <syscall.h>
#include <stdio.h>
int
main (void){
int i;
for(i = 0; i < 20000; i++) {
int a = (i * i) + (i * i);
int b = i;
i = a; a = b; i = b;
}
write(1,”PASS Lab X ON Time.\n”,20);
exit(0);
}
Create 1 grandfather,
3 “parents” and 3 “children”
in examples directory:
grandfather.c
parent1.c
parent2.c
parent3.c
child1.c
child2.c
child3.c
Replace “X” in all the places
in the code for “parents”
and “children” with 1, 2, and 3, respectively.
47
Lab 3::Test (4)
Modify Makefile in examples,
such that all the programs are compiled:
…
PROGS = … parent1 parent2 parent3 child1 \
child2 child3 grandfather
…
parent1_SRC = parent1.c
child1_SRC = child1.c
parent2_SRC = parent2.c
child2_SRC = child2.c
parent3_SRC = parent3.c
child3_SRC = child3.c
grandfather_SRC = grandfather.c
Compile the programs with gmake.
48
Lab 3::Test (5)
Copy them all on the Pintos disk in userprog
(don’t forget to create the disk and format it).
Start them as pintos --qemu -- run grandfather
As the output programs should print:
Firstly, all 10 “PASS Lab 1 ON Time.”
Secondly, all 10 “PASS Lab 2 ON Time.”
Thirdly, all 10 “PASS Lab 3 ON Time.”
Then, try to comment some wait() in grandfather and
parents to see how it behaves. The order should
be destroyed…
(Don’t forget to recompile the programs and copy on the disk again.
49
You may need to format the disk before.)
Lab 3::Part B (PB)
• Task: Load program arguments.
GOOD NEWS: You will receive the source code once
you are done with Part B
BAD NEWS: The code will contain a couple of nasty “bugs”
50
Lab 3::PB::Preparatory Steps (1)
CHANGE IT BACK!
• Lab 2::STEP 3
• into userprog/process.c, find setup_stack()
– *esp = PHYS_BASE;
– change to *esp = PHYS_BASE - 12;
• So that you have
– *esp = PHYS_BASE;
51
Lab 3::PB::Preparatory Steps (2)
• Before continuing, explain why you have
"KERNEL PANIC" after you have removed "-12".
What is wrong and why did the program work
before?
52
Lab 3::PB::Preparatory Steps (3)
• The user program with arguments should be called
with '...' from the Pintos command line:
• pintos --qemu -- run ‘nasty_program arg1 arg2
arg3’
• When the user program with arguments is called
from exec(), you have to call it like this:
• exec("nasty_program arg1 arg2 arg3")
• The implementation of all these things have to be
done in start_process()
53
pid = process ID
tid = thread ID
Lab 3::PB::Exec
Add your implementation of exec() functionalities
into process_execute() and process_start() in process.c
process_execute() {
start_process() {
…
loading – DONE!
Your Exec() system
tid = thread_create
initialization – DONE!
call in syscall.c {
putting program
…
…
generate pid from tid;
arguments into stack
f-eax = process_execute
wait until start_process();
is a PART B;
}
return pid or -1
signal to process_execute
}
}
pid = -1, if the program cannot load or run for any reason.
Use an array or a list to keep track of pid:s.
pid might equal tid, because we have only one thread per process.
Limit the number of user programs (t.ex. 64 or 128).
Lab 3::PB::Implementation (1)
• STEP 1. Parse the string:
• Use strtok_r(), prototyped in lib/string.h
• Read comments in lib/string.c or man
page (run man strtok_r)
• Limit the number of arguments (for simplicity)
• STEP 2. Set up the stack:
• Necessary details about setting up the stack for
this task you can find in Program Startup
Details section of Pintos documentation. 55
Lab 3::PB::Implementation (2)
• nasty_program arg1 arg2 arg3
• After parsing: nasty_program, arg1,
arg2, arg3
• Place the words at the top of the stack
• Align to 4-byte-words, add 0’s
• Reference the words through the pointers
(pointers should point to the addresses of the
words in the stack)
• Put the pointers to the stack (followed with
NULL pointer)
56
Lab 3::PB::Implementation (2)
• Put the address of the first pointer to the stack
• Put the number of words to the stack (the
number of arguments + 1)
• Put “faked” return address to the stack (e.g.
NULL). This is needed in order to meet x86
conventions for program arguments, even
though this return address will not be used.
• So, what we get if we assume that
PHYS_BASE is 0xc0000000 …
57
Address
Name
Type
word-align
Data
arg3\0
arg2\0
arg1\0
nasty_program\0
0
29 --> 32
0xbffffffb
argv[3][...]
0xbffffff6
argv[2][...]
0xbffffff1
argv[1][...]
0xbfffffe3
argv[0][...]
0xbfffffe0
0xbfffffdc
0xbfffffd8
0xbfffffd4
0xbfffffd0
0xbfffffcc
0xbfffffc8
argv[4]
argv[3]
argv[2]
argv[1]
argv[0]
argv
0
0xbffffffc
0xbffffff8
0xbffffff4
0xbfffffe6
0xbfffffd0
char *
char *
char *
char *
char *
char **
0xbfffffc4
argc
4
0
0xbfffffc0
return address
char[5]
char[5]
char[5]
char[14]
uint24_t
int
58
void (*) ()
Lab 3::Test (1)
• Now you are ready for a complete check!
• Run gmake check from userprog/build
• The following tests should pass:
• 1) Argument passing when executing:
args-none, args-single, args-multiple,
args-many, args-dbl-space
2) Different exec-tests:
exec-once, exec-arg, exec-multiple,
exec-missing, exec-bad-ptr
3) Wait-tests:
wait-simple, wait-twice, wait-killed,
59
wait-bad-pid
Lab 3::Test (2)
• Many of the other checks in gmake check belong to the second
lab. If any of them fails, then it means that something is wrong
with your implementation of Lab 2:
• sc-bad-sp, sc-bad-arg, sc-boundary,
sc-boundary-2, halt, exit, create-normal,
create-empty, create-null, create-bad-ptr,
create-long, create-exists, create-bound,
open-normal, open-missing, open-boundary,
open-empty, open-null, open-bad-ptr,
open-twice, close-normal, close-stdin,
close-stdout, close-bad-fd, read-bad-ptr,
read-boundary, read-zero, read-stdout,
read-bad-fd, write-normal, write-bad-ptr,
write-boundary, write-zero, write-stdin,
60
write-bad-fd
Lab 2 & Lab 3
– Lab 2 – 13th of February
– Lab 3 – 5th of March
– …
Discussion about Lab 4
+ Repetition of Lab 3 – next lesson…
61