Laboratory work in TDDI04
Introduction to Pintos
Assignments 00, 0, 1, 2
Viacheslav Izosimov
2008-01-17
viaiz@ida.liu.se
Motivation
• Operating systems are nearly
everywhere
• Crucial piece of software
• Perform concurrent
tasks
• Synchronization
• C is the main
language for real
operating systems!
Motivation
• Nearly the first experience
with a relatively large piece of
software
• Direct jump from 1000 to
1000000 can be too difficult!
1000000-5000000 LOC
5000-10000 LOC
up to 1000 LOC
Your programs so far.
Pintos.
Outline
•
•
•
•
•
•
General questions
Introduction to Pintos
General description of labs
Lab rules
General algorithm for completing the labs
Lab 0
– Setting up the program environment
– Debugging of Pintos
• Lab 1
– Synchronization
– Bounded buffer
• Lab 2 (selected parts)
– System calls
– Memory issues
Real systems.
General questions
Quiz:
Labs:
Groups of five students
Sign up and take your quiz today!
Groups of two students
Sign up as soon as possible!
(deadline 2008-01-24)
Answer on the questions
until the next lesson!
Work jointly!
No cheating!
Lab lessons:
Mandatory – participation list…
5
You are not allowed to work
alone!
General questions
Quiz example
Your names!
Q1: Give an example of
Pintos recipe (2p)
A question
Your answer here!!!
Obtained points: all questions should
be answered, at least 70% of them
correctly
One copy for each group!
Sign up on the table during
the break. Once 5 students
have signed up, you can
take the quiz.
6
Introduction to Pintos
• Shift from Nachos (1990s) to Pintos (2004)
–
–
–
–
Pintos is the most up-to-date “toy operating” system
Can boot as a real operating system
Code is not as “messy” as in Nachos
Pure C implementation
Pintos
Install on Sun machines during Lab #0
Extend during Lab #1, #2, and #3
Introduction to Pintos
•
•
•
•
Runs on x86 machine
QEMU – a computer system emulator is used
UNIX environment
C implementation
(both Kernel
and user programs)
• Not complete…
• Your task!
Introduction to Pintos
User programs
User memory
User stack
Threads
Scheduler
Lab 1
Kernel stack
Synchronization
User-exception handler
primitives
Memory management
Interrupt handler
Timer
File system
General Description of Labs
• Lab 00: “Introduction to C Programming”
– Checks your ability to complete the labs
• Lab 0: “Introduction and installation”
– Introductory lab, where you need to setup your program
environment and try out debugging
• Lab 1: “Synchronization primitives”
– First “real” lab
– First introduction to threads
– Synchronization mechanisms
General Description of Labs
• Lab 2: “System calls”
–
–
–
–
System calls
Single user program
Memory issues
Console
• Lab 3: “Execution, termination and
synchronization of user programs”
–
–
–
–
–
Handling program arguments
Execution of several user programs
Termination of a user program
Synchronization of shared data structures
Wait system call
General Description of Labs
• Lab 4: “File system”
– Synchronization of read-write operations
But before…
Lab Rules
1. Every member of the group participate
equally and every member should be able to
answer questions related to labs
2. Before doing the lab answer on “preparatory
questions” on the lab page and in the
corresponding quiz
3. Understand first, then implement
4. Try to find ”bugs” yourself at first
5. No copying of source code from any sources,
no cheating
6. Pass assignments on time!
Lab Rules
Pass assignments on time to get bonus!
•
•
•
•
•
•
Lab 00, 0 – today!
Lab 1 – 30th of January (+1 point)
Lab 2 – 13th of February (+1 point)
Lab 3 – 5th of March (+1 point)
Lab 4 – 7th of April (+1 point)
•
If got 4 points, +1 extra point! 20% of the exam!
•
FINAL DEADLINE – 21st of May (no points)
General Algorithm to Complete the Labs
1. Answer on all preparatory questions. Note that many questions
are replicated in Quizzes! So, solving quizzes will help!
2. Understand what you are supposed to do (read
instructions carefully)
3. Discuss all the issues with your lab partner to make sure
that he/she also understands (That’s you future help!)
4. Understand the source code, which you are supposed to
understand
5. Begin implementation!
6. Document carefully what you are doing (put date, name
and short description around each new piece of code)
7. Work hard… and not only during lab hours, they are
mostly for asking questions from lab assistants
Lab 00
• You will need to debug a small program “debugthis.c”
with DDD after compiling it with GCC (not CC!)
• Create a linked list
• Learn how to browse the source code
• If you are not able to complete the lab, then you should
focus on C before you proceed further or even consider to
take the course next year. No formal pass/fail or other
regulations though…
Lab 0
• Set up the program environment (don’t forget!)
• module initadd
~TDDI04/labs2008/modules/pintos
• module add ~TDDI04/labs2008/modules/pintos
• Check if you have correct version of GCC and GDB
module list
• GCC should be 3.4.x and you will know if GDB
version is correct after completing Lab 00
• Copy Pintos to your directory
• gzip -cd ~TDDI04/labs2008/pintos_ida.tar.gz |
tar xvf -
Lab 0
• You will have the following structure of directories
• pintos/src
–
–
–
–
–
–
–
–
–
–
•
devices
examples
filesys
lib
misc
tests
threads
userprog
utils
vm
Lab 0
•Compile and build Pintos
• cd pintos/src/threads
• gmake
•Test if Pintos works
• cd build
• pintos --qemu -- run alarm-multiple
Lab 0
•Debugging (from build)
pintos --qemu --gdb
•
•
-- run testname
ddd --gdb --debugger pintos-gdb kernel.o&
–Note that only breakpoints can be inserted and only “Continue”
command of DDD works since Pintos is already running. Other
commands would terminate Pintos!
•Other commands:
•pintos --qemu --gdb --dport=port_number -- run testname
•Port number is 1234 + k, where k is your group number in WebReg
•ddd --gdb --debugger pintos-gdb kernel.o&
• target remote localhost:port_number
Lab 1
• First introduction to threads
• Synchronization mechanisms
– Semaphores
– Locks
– Conditions
Example with Threads
A system contains several threads running in parallel
Thread is a “basic unit of CPU utilization”
§Thread ID
§A program counter
§A register set
§A stack
Thread 1
Thread 2
Thread execution sequence
22
Example with Threads
void SimpleThread(void * which)
{
int num;
for (num = 0; num < 5; num++) {
printf("*** thread %d looped %d times\n", (int)which, num);
thread_yield();
Entering SimpleTest
}
*** thread 0 looped 0 times
}
void SimpleThreadTest(void)
{
char *t_name = "forked thread";
printf("Entering SimpleTest");
*** thread 1 looped 0 times
*** thread 0 looped 1 times
*** thread 1 looped 1 times
…
thread_create(t_name, PRI_MIN, SimpleThread, (void *)1);
}
SimpleThread((void *)0);
23
Critical Section Problem
• Critical Section: A set of instructions, operating on
shared data or resources, that should be executed
by a single process without interruption
– Atomicity of execution
– Mutual exclusion: At most one process should
be allowed to operate inside at any time
– Consistency: inconsistent intermediate states of
shared data not visible to other processes outside
• General structure, with structured control flow:
– Entry of critical section C
– … critical section C: operation on shared data
24
– Exit of critical section C
Semaphores
• Semaphore S:
– shared integer variable
– two atomic operations to modify S:
P() and V()
Example: At most five passengers in the car
• P (S):
•
while S <= 0 do Sleep(currentThread)
•
S--;
• // Critical section
• V (S):
•
WakeUp (sleepingThread)
25
•
S++;
Semaphores
• // P(S)
• void sema_down (struct semaphore *sema) {
•
enum intr_level old_level;
•
…
•
old_level = intr_disable ();
•
while (sema->value == 0) {
•
list_push_back (&sema->waiters, &thread_current ()->elem);
•
thread_block ();
•
}
•
sema->value--;
•
intr_set_level (old_level);
26
• }
Semaphores
• // V(S)
• void sema_up (struct semaphore *sema) {
•
enum intr_level old_level;
•
…
•
old_level = intr_disable ();
•
if (!list_empty (&sema->waiters))
•
thread_unblock (list_entry (list_pop_front (&sema->waiters),
struct thread, elem));
•
sema->value++;
Good for handling simple
•
intr_set_level (old_level);
• }
synchronization
problems
27
Locks
• Binary semaphore (e.g. “taken” or “not taken”)
• Only the owner can release the lock
• Advantage: No other thread can enter the
critical section!
• Essential for security, for example, access to
the shared data, arrays, lists, and etc.
• Problem: deadlock situations
28
Conditions
•
•
•
•
•
Queuing mechanism on top of Locks
Helps to manage the Locks
Prevents deadlock situations
Locks are often used together with Conditions
Locks/Conditions are good for handling
complex synchronization problems
• Attention: never use a Condition with an
empty Lock!
29
Example
Lab 1::Part B
30
Lab 1
• Your task is to write the code for
BoundedBuffer data structure
• In a bounded buffer you must handle two
conditions: the case of an empty buffer and a
full buffer
• Provide two alternative implementations:
– with Semaphores
– with Locks/Conditions
31
Lab 1
• Understand the source code in
– threads/synch.[h|c]
– threads/synchlist.[h|c]
– threads/boundedbuffer.[h|c]
– contains a skeleton of Bounded Buffer.
– src/tests/threads/threadtest.c
– tests your Bounded Buffer
implementations
Lab 1::BoundedBuffer
• An array with a fixed number of available
positions to store data in
• Used for communication between threads
–A thread produces data and puts it in the buffer
–A thread consumes data by reading data from buffer
Lab 1::BoundedBuffer
• How does it work?
• Look at it as a circular buffer
• Treat the buffer as an array and three indexes,
in, out, and count
Synchronization!
Lab 1::BoundedBuffer
• What happens if a consumer tries to read data
from an empty buffer?
• What happens if a producer tries to put data in a
full buffer?
• Note! The buffer is a shared resource
Lab 1::BoundedBuffer
• Using of BoundedBuffer (from threadtest.c)
• struct BoundedBuffer bounded_buffer[2];
• bb_write(&bounded_buffer[buf],(int) p);
• struct Data *data = (struct Data*)
bb_read(&bounded_buffer[buf]);
Lab 1::Test
• To test your implementation use threadtest.c.
• Prints 50 messages that are sent from a set of producers to
a set of consumers through the BoundedBuffer that you
have implemented
• Do not modify this program!
• You can run Pintos from threads/build directory as
• pintos --qemu -- run threadtest
• Run Pintos several times to test your implementation to
see if you do not get into a deadlock and if all the
messages appear correctly
Lab 2
•
•
•
•
System calls
Single user program
Memory issues
Console
Lab 2 :: Systems Calls At High Level
• System calls:
– communication between the user program and the
kernel
– functions, called from the user program and
performed by the kernel
– computers often use interrupts to accomplish that
switch from user code to system code
Lab 2:: Systems Calls At High Level
user_program.c
bool flag;
flag = create(“file.txt”, 1000);
Calling
Return result back
corresponding
to the user program
function in
pintos/src/lib/user/syscall.c
System
call
wrapper
the wrapper
bool create (const char *file, unsigned initial_size) {
return syscall2 (SYS_CREATE, file, initial_size);
}
Calling
corresponding
exception in Pintos kernel
pintos/src/userprog/syscall.c
the kernel
Return result back
to the wrapper
User exception handler
// Your code here!
Lab 2::What to Implement?
• You will need to implement:
– create - creates a file.
– open - opens a file.
– close - closes a file.
– read - reads from a file or the console (the keyboard).
– write - writes to a file or the console (the monitor).
– halt - halts the processor.
– exit - Terminates a program and deallocates resources
occupied by the program, for example, closes all files
opened by the program.
Lab 2::Useful Files
• You have to have a look at:
– pintos/src/lib/user/syscall[.h|.c] – the wrapper…
– userprog/syscall[.h|.c] – Your implementation of system
calls!!!
– threads/interrupt.[h|c] – important structures!!!
– lib/syscall-nr.h – system call constants
–filesys/filesys.[h|c] Pintos file system implementation…
– examples/lab2test.c – the “official” test program for this lab
– filesys/file.[h|c] – useful functions for operations with files.
Things which you don’t find in “filesys”, you find here
–userprog/process.c Look up what is here and tell me…
Lab 2::How Do I Start? (1)
• STEP 1
• Understand what the user program is
• Look into “examples” directory and a couple of
the user programs, especially “halt.c”
• Look into Makefile in this directory to understand
how the user programs are compiled
• Compile the user programs by “gmake”
• STEP 2
• Prepare Pintos disk
• Copy one or two user programs to the disk
Lab 2::How Do I Start? (2)
• STEP 2 (Continuation) Dealing with Pintos disk
• In “userprog/build”: pintos-mkdisk fs.dsk 2
– This will create a file fs.dsk with a 2MB simulated disk in
the directory.
• Format: pintos --qemu -- -f -q.
• Copy user programs:
pintos --qemu -p programname -- -q
with new name
pintos --qemu -p programname -a newname -- -q
Lab 2::How Do I Start? (3)
• STEP 2 (Continuation) Dealing with Pintos disk
• Example:
• pintos --qemu -p ../../examples/halt
-a severe_student_program -- -q
• To copy a file from the disk:
pintos --qemu -g programname -- -q
• or
pintos --qemu -g programname -a newname -- -q
* -p = put, -g = get
• To run: pintos
• Example:
--qemu -- run programname
• pintos --qemu -- run severe_student_program
Lab 2::How Do I Start? (4)
• STEP 3
• into userprog/process.c, find setup_stack()
– *esp = PHYS_BASE;
– change to *esp = PHYS_BASE - 12;
• STEP 4
• Make sure that you understand the problems which may
arise if you access (in the kernel) data stored in the user
memory using the pointers provided by the user program
as system call arguments.
• (First, make sure that you understand what this sentence
means…)
Lab 2::Shutdown Example
•
•
•
•
•
•
halt.c
#include "userprog/syscall.h“
#include <stdio.h>
#include <syscall.h>
#include <syscall-nr.h>
int
#include "threads/interrupt.h“
"threads/thread.h“
main (void){ #include
static void syscall_handler (struct intr_frame *);
voidsyscall_init (void) {
halt ();
intr_register_int (0x30, 3, INTR_ON,
}
syscall_handler, "syscall");
userprog/syscall.c
}
static void
syscall_handler (struct intr_frame *f UNUSE
printf ("system call!\n");
thread_exit ();
}
system call!
48
Lab 2::Shutdown Example
•
•
•
•
•
•
#include "userprog/syscall.h“
halt.c
#include <stdio.h>
#include <syscall-nr.h>
#include <syscall.h>
#include "threads/interrupt.h“
int
#include "threads/thread.h“
static void syscall_handler (struct intr_frame *);
main (void){
voidsyscall_init (void) {
intr_register_int (0x30, 3, INTR_ON,
halt ();
syscall_handler, "syscall");
}
}
static void
syscall_handler (struct intr_frame *f UNUSED)
userprog/syscall.c printf ("system call!\n");
get a name of the system callthread_exit ();
! f->esp – stack pointer
SYS_HALT from the stack}
use power_off()
Lab 2::Useful Files for Shutdown
• You have to have a look at:
– pintos/src/lib/user/syscall[.h|.c] – the wrapper…
– userprog/syscall[.h|.c] – Your implementation of system
calls!!!
– threads/interrupt.[h|c] – important structures!!!
– lib/syscall-nr.h – system call constants
–filesys/filesys.[h|c] Pintos file system implementation…
– examples/lab2test.c – the “official” test program for this lab
– filesys/file.[h|c] – useful functions for operations with files.
Things which you don’t find in “filesys”, you find here
–userprog/process.c Look up what is here and tell me…
Pintos Stack (1)
51
W
hy
”?
4
“
Pintos Stack (2)
f->esp
f->esp + 4
f->esp + 8
f->esp + 12
…
52
Lab 2::Create Example
• bool create (const char *file, unsigned
initial_size)
• Example: create(“file.txt”, 1000);
How to get them?
Answer: f->esp
Hint:
… note that, in order to get
How to return a value?
a string, you will need get a void pointer
Answer: f->eax
from esp and then get a char pointer to which
points that void pointer. The char pointer will point to the first
element of the string …
f->esp
54
Lab 2::Create Example
• To implement the system calls, which operate
with files, look at “filesys/filesys.[h|c]” and
“filesys/file.[h|c]”.
• Everything is already done there for you! Just
call the functions…
So simple? Not yet…
Memory issues!!!
All the pointers on the variables,
which you get from the user program,
must be validated!
Problem 1: If the pointer above PHYS_BASE,
It points to Kernel memory! UNSAFE!
57
Memory Issues in Pintos
Kernel VM
Physical Memory
Kernel process
PHYS_BASE
User process
Page directory
If no entry?
a)Kill user
b) Handle page fault
Lab 2::Memory Issues in Pintos
Kernel VM
Physical Memory
Kernel process
PHYS_BASE
Check if the pointer is below
User process
Page directory
Check if the pointer is in the page directory
If no entry?
a)Kill user
b) Handle page fault
Conclusions
•
•
•
•
•
Introduction to Pintos and labs in general
Labs 00, 0, and 1
Lab 2 (selected parts)
TO DO: Prepare Quiz until the next lesson
Deadline for Lab #1 is 30th of January
60