cs4411 – Operating Systems Practicum
Supplementary lecture 2
September 30, 2011
Zhiyuan Teo
Today’s lecture
•Administrative Information
•Common mistakes on Project 1
•Project 2 FAQ
•Technical supplement 1
•Discussion
Administrative Information
•Project 1 has been graded – please review comments
and make amendments.
- Regrade requests: through CMS or in person.
•Project 2 deadline has been extended to 11.59pm
5th October.
Project 1: general
•We prize correctness, but also elegance.
•Don’t need provide as much abstraction as in data
structure courses.
•Comments and README.
•zip files and file content.
- don’t create an extra folder to hold the content.
- submit only changed source files, in addition to README.
- don’t tar and rezip.
- don’t encrypt!
Project 1: general
•Not checking for errors.
- malloc()
- queue_new(), queue_delete(), etc.
- asserts are not checks!
- should not exit unless there is no way for even the user to recover.
•Excessive debug output.
•Indentation and code readability.
Common mistakes: queues
•queue_append/prepend
- forgetting to set next or prev pointers.
•queue_delete/dequeue
- bugs when deleting from head/tail: pointers not updated or illegal access.
- not returning -1 when item to delete isn’t found in queue.
- not freeing up queue node after updating list.
- forgetting to update head/tail pointers.
•queue_iterate
- not checking the return value from function pointer.
- short-circuit evaluation.
Common mistakes: semaphores
•semaphore_create vs semaphore_initialize
- creating the blocked queue in semaphore_initialize.
•semaphore_V
- relying on a blind dequeue or queue length to unblock threads.
•semaphore_destroy
- forgetting to free up the blocked queue.
•TAS locks
- yielding instead of spinning.
- homemade TAS code.
- non-atomic clears.
Common mistakes: minithreads
•minithread_fork/create
- reimplementing create in fork (by cutting and pasting code).
•minithread_yield
- self-switching.
•minithread_stop
- beware of the semantics.
- comments for minithread_stop indicate that it is used for blocking threads.
- not to be used in place of minithread_yield, even if scheduling results are
correct.
Common mistakes: minithreads
•cleanup handler
- directly starting the cleanup thread through minithread_start.
- forcing the cleanup thread to run immediately (via context switch).
- forcing the next yield to run the cleanup thread.
•cleanup thread
- blindly calling dequeue before semaphore_P.
- freeing multiple items per semaphore_V called on it.
•global wait queue
- replicating functionality already achieved in semaphores.
- inefficient to delete.
Project 2 FAQ
•malloc and library calls
- all library calls are safe: interrupts are automatically disabled upon calling.
interrupts will be restored to its original state (enabled/disabled) after the call.
•units of time
- PERIOD is defined as 50 ms, which is 50000 as a constant.
- alarm and wakeup delays are specified in milliseconds.
- you have to convert units; don’t blindly subtract PERIOD.
•irregular/random clock interrupts
- this is normal.
- be careful of introducing heisenbugs because of your debug statements.
When to disable interrupts
•When you need to do something that must be done
atomically.
•Typically manipulations on shared data structures.
- data structures that can be accessed by multiple threads ‘simultaneously’.
- eg. modifying the cleanup queue, ready queue, alarm list.
•Trivial way of achieving correctness: disable interrupts
for everything.
-Why is this a bad idea?
Semaphores revisited
•typical sem_P code:
while (TAS(&lock) == 1);
sem->counter--;
if (sem->counter < 0)
{
append thread to blocked queue
atomically unlock and stop
}
else
{
atomic_clear(&lock);
}
Semaphores revisited
•typical sem_V code:
while (TAS(&lock) == 1);
sem->counter++;
if (sem->counter <= 0)
{
take one thread from blocked queue
start the thread
}
atomic_clear(&lock);
Semaphore design considerations
•Interrupts are not disabled in sem_P and sem_V.
•…but semantics must still be correct even if used in an
interrupt-enabled environment.
•Semaphores should also be safely usable from
places where interrupts are disabled.
•Why not just disable interrupts inside sem_P/V?
Semaphore use in user threads
•Interrupts can arrive at any time.
•If interrupts arrive while a TAS lock is held:
- another thread that tries to acquire the TAS lock will spin until its time quanta
is exhausted.
- thread holding the TAS lock will eventually regain control and make progress.
- progress ensures other threads can eventually get the TAS lock.
semaphore_P
semaphore_V
while (TAS(&lock) == 1);
while (TAS(&lock) == 1);
sem->counter--;
if (sem->counter < 0)
{
append thread to blocked queue
atomically unlock and stop
}
else
atomic_clear(&lock);
sem->counter++;
if (sem->counter <= 0)
{
take one thread from blocked queue
start the thread
}
atomic_clear(&lock);
Semaphore use in the kernel
•Typically used to block some thread and wake it up
on some condition.
- minithread_sleep_with_timeout()
- wake up the thread after the elapsed time.
•Sleep with timeout requires calling sem_P on the TCB’s
sleep semaphore. Done in user space.
•Waking up requires calling sem_V on that sleep
semaphore. Done in kernel space with interrupts
disabled.
struct tcb {
int tid;
…
semaphore_t sleep_sem;
};
An unfortunate interleaving
•What if user calls sleep_with_timeout(0) ?
- sem_P is called, and thread blocks itself.
•What if sem_P was interrupted just after placing
thread on blocked queue but before clearing TAS lock?
user calls sleep_with_timeout(0)…
…clock handler tries to wake that thread up
while (TAS(&lock) == 1);
while (TAS(&lock) == 1);
sem->counter--;
if (sem->counter < 0)
{
append thread to blocked queue
sem->counter++;but interrupts are disabled
if (sem->counter
<= clock
0) handler!
in the
{
take one thread from blocked queue
start the thread
}
clock interrupt!
atomically unlock and stop
}
else
atomic_clear(&lock);
atomic_clear(&lock);
Solution
•Disable interrupts for sem_P and sem_V.
- atomicity: sem_P will be done with everything before an interrupt can
possibly arrive.
- if interrupt arrives, acquisition of TAS lock is guaranteed in kernel space.
“You should disable interrupts first before making
calls to semaphore_P from within your kernel code.”
-project 2 slides
•What about sem_V?
- sem_V is called from interrupt handler.
- interrupts are already disabled in the handler.
When is this applicable?
•If semaphore will be used in portions of your kernel
where interrupts are disabled.
- right now: only the sleep semaphore.
•What about cleanup semaphore?
- Cleanup semaphore is not signaled from any place where interrupts are
disabled.
•Cleanup code should only disable interrupts while
accessing the cleanup queue, not for semaphore
signaling.
Why spin on TAS? Why not yield?
•Seems logical to yield if TAS lock cannot be acquired.
- behavior would still be correct in user threads.
semaphore_P
semaphore_V
while (TAS(&lock) == 1)
1);yield();
while (TAS(&lock) == 1)
1);yield();
sem->counter--;
if (sem->counter < 0)
{
append thread to blocked queue
atomically unlock and stop
}
else
atomic_clear(&lock);
sem->counter++;
if (sem->counter <= 0)
{
take one thread from blocked queue
start the thread
}
atomic_clear(&lock);
Why spin on TAS? Why not yield?
•But interrupts are enabled after the yield.
- not a problem in user threads, since they should run with interrupts enabled.
- presents problems in the kernel where you want interrupts to stay disabled
when using semaphores.
semaphore_P
semaphore_V
while (TAS(&lock) == 1) yield();
while (TAS(&lock) == 1) yield();
sem->counter--;
if (sem->counter < 0)
{
append thread to blocked queue
atomically unlock and stop
}
else
atomic_clear(&lock);
sem->counter++;
if (sem->counter <= 0)
{
take one thread from blocked queue
start the thread
}
atomic_clear(&lock);
No longer safe from interrupts;
interrupt handler can be interrupted.
How a problem may arise
•Suppose interrupts are not disabled before calling
sem_P/V in the kernel.
1. Suppose clock handler was invoked to service a clock
interrupt. It runs with interrupts disabled.
clock handler
void clock_handler() {
disable_interrupts();
while(sleeping_threads > 0) {
if (time_to_wake_thread)
sem_V(&tcb->sleep_semaphore);
}
restore_interrupt_level();
}
How a problem may arise
2. Clock handler is unable to acquire TAS lock, so it
yields.
semaphore_V()
while (TAS(&lock) == 1) yield();
sem->counter++;
if (sem->counter <= 0)
{
take one thread from blocked queue
start the thread
}
atomic_clear(&lock);
How a problem may arise
3. Control returns after yield. Interrupts are now enabled.
Suppose TAS lock succeeds (previous thread gave it up).
semaphore_V()
while (TAS(&lock) == 1) yield();
sem->counter++;
if (sem->counter <= 0)
{
take one thread from blocked queue
start the thread
}
atomic_clear(&lock);
How a problem may arise
4. A network interrupt arrives and calls sem_V because
it wants to wake up the thread.
semaphore_V()
network handler
while (TAS(&lock) == 1) yield();
void network_handler() {
network interrupt!
sem->counter++;
if (sem->counter <= 0)
{
take one thread from blocked queue
start the thread
}
atomic_clear(&lock);
disable_interrupts();
sem_V(&net_tcb->sleep_semaphore);
restore_interrupt_level();
}
How a problem may arise
5. Network handler yields because it cannot acquire TAS
lock.
No amount of yielding will allow progress in this scenario. Why?
Nested interrupts
0x7fffffff
Thread A
Clock interrupt handler
•An interrupt
•Network
interrupt
is like handler
an
is
unscheduled
waiting for clock
function
handler
call.to
finish, but this is impossible.
Network interrupt handler
•Only way to do this is for
network interrupt handler to
return.
•Yielding does not unwind the
call stack.
Program text, heap, etc.
0x00000000
Q & A for Project 2
Technical supplement roadmap
•How pointer casts work.
•How the stack works; function calls and context switches;
anatomy of buffer overflow & stack smashing exploits.
•x86 history and a (very short) introduction to x86
assembly.
•None of these are examinable nor strictly needed for
4410/4411 but are good to know as OS designers.
Pointer casting
•A pointer has two components:
- starting address of the data being pointed to.
- size of data type associated with this memory location.
int a = 5;
int *x = &a;
0x00001000
0
0
x = 0x00001004
*x = 0x00000005
pointer x: data starts from 0x00001004 and spans 4 bytes
0
0
0
0
0
0x00001004
5
0
0
0
0
0
0
0
0
Pointer casting
•Casting the pointer changes only the way the memory is
interpreted.
•It does not change the memory contents nor the address
stored in the pointer.
x
*x
y
*y
int
a = 5;
int *x = &a;
char *y;
y = (char*) x;
0x00001000
0
0
=
=
=
=
0x00001004
0x00000005
0x00001004
0x00
pointer y:
x: data starts from 0x00001004 and spans 41 bytes
byte
0
0
0
0
0
0x00001004
5
0
0
0
0
0
0
0
0
Pointer casting
•We can change the values from the casted pointer
if we like.
•No array bounds check in C!
x
*x
y
*y
int
a = 5;
int *x = &a;
char *y = (char*) x;
*y
= 1;
y[2]
= 3;
=
=
=
=
0x00001004
0x01000305
0x00001004
0x01
0x00001000
0
0
0
0
0
1
0
0
3
0x00001004
5
0
0
0
0
0
0
0
0
Structures
base struct
struct queueable {
struct queueable* next;
struct queueable* prev;
};
‘derived’ struct
struct tcb {
struct queueable reserved;
int tid;
semaphore_t sleep_sem;
};
struct queueable* queue_node;
struct tcb*
idle_tcb;
pointer queue_node: data starts from 0x00001000 and spans 8 bytes (struct is aligned)
0
0
0x00001000
next
0
0
0
0
prev
0
0
0
0
0
0
0
0
0
0
Structures
base struct
struct queueable {
struct queueable* next;
struct queueable* prev;
};
‘derived’ struct
struct tcb {
struct queueable reserved;
int tid;
semaphore_t sleep_sem;
};
struct queueable* queue_node;
struct tcb*
idle_tcb;
pointer idle_tcb: data starts from 0x00001000 and spans 16 bytes (struct is aligned)
0
0
0x00001000
0
0
0
reserved
0
0
0
0
0
0
tid
0
0
0
0
sleep_sem
0
Common questions on casting
•Casting from a derived struct pointer to the base struct
pointer is safe.
- any operation that works on data members of the base struct will not write
outside of the space allocated for the derived struct.
struct queue_node* node = (struct queue_node*) idle_tcb;
node->prev = 0xdead; node->next = 0xbeef;
pointer idle_tcb:
otherdata
fields
starts
in the
from
tcb 0x00001000
struct (tid and
and
sleep_semaphore)
spans 16 bytes (struct
are not
is aligned)
affected.
d
c
e
a
af
d
e
b
e
a
b
e
ef
0
b
0
a
0
a
0
d
0f
0
0
0
d
pointer queue_node: data starts from 0x00001000 and spans 8 bytes.
manipulation of next and prev fields in queue_node will only affect the first 8 bytes
of the tcb structure.
Common questions on casting
•What about casting from base struct to derived struct?
- unless you are very sure you will not touch data fields other than those
in the base struct.
- but if that is the case, why did you perform the cast?
struct tcb* some_tcb = (struct tcb*) queue_node;
some_tcb->reserved.next = 0xdead;
some_tcb->reserved.prev = 0xbeef;
some_tcb->tid = 1;
•Writing outside of known bounds may cause
unpredictable behavior.
- may overwrite another variable.
- may overwrite stack (and thus modify return address; a common exploit).
- usually just crashes.
void* pointers
•Review: a pointer has two components.
- starting address of the data being pointed to.
- size of data type associated with this memory location.
•void* pointers are not associated with any data type.
- has only a starting address.
- no known size, therefore not possible to tell where data ends in memory.
- malloc() is a good example.
•casting to void* from other pointer types.
- automatic, no need to prepend (void*).
- other pointers can easily cast to void*; just ignore the size of the data type.
•casting to other pointer types from void*.
- explicit cast required.
- destination data type supplies the size of the data type.
Q&A