Outline
CS4233 Network Programming
z
Beginning of UNIX programming
z
UNIX Programming
z
z
UNIX APIs
z
Chen-Lung Chan
Department of Computer Science
National Tsing Hua University
z
z
z
z
z
z
Compiler
z
Assembler
z
Assembly code
Linker
Fully-resolved object
code (machine code)
z
z
Loader
Executable image
Create/wait a thread
Thread synchronization
C Compilers
Source code
Object code
(machine code)
File I/O
Process management
Signal handling
Interprocess communications
POSIX threads
z
Compilers: The Big picture
gcc
make
gcc, cc
Ex1: build an execution file “a.out” from hw1.c
$ gcc hw1.c
Ex2: build an execution file “hw1” from hw1.c
$ gcc –o hw1 hw1.c
Ex3: link with a library (/usr/lib/libxnet.so)
$ gcc –o hw1 hw1.c –lxnet
Ex4: compile an object file (hw1.c ⇒ hw1.o)
$ gcc –c hw1.c
Exercise
z Write
a “Hello World” UNIX console
program.
make (1/3)
z
make [-f makefile][option] target
z
z
$ gcc –o hello hello.c
z $ ./hello
Hello World!!
z
z
A tool to update files derived from other files
The default files for make
are ./makefile, ./Makefile, ./s.makefile
Use the –f option to specify some other file
z
z
The makefile has three components
z
z
z
make (2/3)
z
Macros:
z
z
z
string1 = string2
Ex:
CC = gcc
CFLAG = -I../include
Target rules:
z
z
Target: [prerequisite…]
<tab> command
Ex:
hw1: myprog1.c myprog2.c myprog3.c
<tab> $(CC) –o hw1 $(CFLAG) myprog1.c myprog2.c myprog3.c
$ make –f makefile1
Macros: define constants
Target rules: specify how targets are made
Inference rules: specify how targets can be made,
implicitly.
make (3/3)
z
Reference
z
Ex:
CC = gcc
hw: main.o util.o
$(CC) –o hw main.o util.o
util.o: util.h util.c
$(CC) –c util.c
main.o: main.c
$(CC) –c main.c
clean:
@rm *.o *~
z
http://www.study-area.org/cyril/opentools/opentools/makefile.html
Portability
z Standards
Byte Order
z
z
Source code portability: ANSI/ISO C
z UNIX standards: POSIX, open group
z Internet engineering task force (IETF)
z
z 32
bit vs 64 bit
z Byte order
z
Standard programming language
z
z
z
z
z
High-order byte is
stored at lowest
address
Outline
z
Beginning of UNIX programming
z
z
z
z
z
z
z
gcc
make
UNIX APIs
z
Example: POSIX.1
Auto-configuration mechanisms
Programmer discipline
Low-order byte is
stored at lowest
address
Big-Endian
z
Example: ANSI/ISO C
Standard libraries
Standard API to operating system
z
z
Little endian vs big endian
Improve Source Code Portability
z
Little-Endian
File I/O
Process management
Signal handling
Interprocess communications
POSIX threads
z
z
Create/wait a thread
Thread synchronization
UNIX File System
z inode:
a segment of data in a filesystem
that describes a file, including how to find
the rest of the file in the system
z File descriptor: a non-negative integer,
with a per-process mapping to an open
file description
z Open file description: an OS internal
data-structure, shareable between
processes
File Descriptors (2/2)
File Descriptors (1/2)
z
Each open file is associated with an open file
description.
z
z
Each process has a (logical) array of references to
open file descriptions.
Logical indices into this array are file descriptors.
z
z
These integer values are used to identify the files for I/O
operations.
The file descriptor 0 is reserved for standard input,
the file descriptor 1 for standard output, and the file
descriptor 2 for the standard error.
Direct File I/O (1/2)
z Functions
z
open(), creat(), read(), write(), lseek(),
close(), dup()
z When
to use direct file I/O?
Do not want to view the data as characters
z Want greater efficiency
z The extra (stream) layer of buffering causes
us problems with synchronization
z
Direct File I/O (2/2)
z
z
z
int open(const char *pathname, int flags)
int open(const char *pathname, int flags, mode_t
mode)
int creat(const char *pathname, mode_t mode)
z
z
z
z
z
z
z
z
z
write to a file descriptor
z
reposition read/write file offset
z
z
close a file descriptor
Process Management
z Environment
variables
z
status
z Process ID
z Functions
fork()
z exit()
z wait() and waitpid()
z execv()
Create/wait a thread
Thread synchronization
Environment Variables
z Exit
z
File I/O
Process management
Signal handling
Interprocess communications
POSIX threads
z
int close(int fd)
gcc
make
UNIX APIs
z
read from a file descriptor
off_t lseek(int fildes, off_t offset, int whence)
Beginning of UNIX programming
z
ssize_t write(int fd, const void *buf, size_t count)
z
z
z
ssize_t read(int fd, void *buf, size_t count)
z
z
open and possibly create a file or device
Outline
A set of environment variables is associated
with each process.
z
z
z
Ex:
$ EDITOR=/usr/bin/joe
$ export EDITOR
These values are passed to the process when
it is executed.
A process can access its environment
variables via getenv().
Exit Status
z
Each process (except the system's initial
process) has a parent process
z
When a process terminates, its parent can find out
what caused the termination via a status value.
z
z
z
The termination status contains a short integer exit status
code.
The function exit() allows a process to terminate itself and
specify the exit status code that it wants to return to the
parent.
z Each
process has a unique identifier, of
signed arithmetic type pid_t.
z The process ID of the current process
can be obtained by calling the function
getpid().
z ps - report process status
When a command is run from a shell, the exit
status code is returned to the shell, which allows it
to be used in job control statements.
fork()
z Create
a new process by duplicating the
context of the calling process.
z The calling process is called the parent,
and the new process the child.
z The return value of fork() distinguishes
the two processes.
Child: 0
z Parent: child process ID, or -1 for error
z
Process ID
exit() and waitpid()
z
exit(int status)
z
z
z
z
z
Clean up the process (for example, close all files)
Tell its parent that it is dying (SIGCHLD)
Tell child processes that it is dying (SIGHUP)
status can be accessed by the parent
waitpid(pid_t pid, int *stat_loc, int options)
z
waitpid() suspends the calling process until one of
its children changes state.
z
z
Return the pid and status of the child process
Ex: waitpid(-1, &status, WCONTINUED);
What Happens to a Process Whose
Parent Does not Wait for It?
z
z
z
The process becomes a zombie in UNIX
terminology.
Zombies stay in the system until they are
waited for.
If a parent terminates without waiting for a
child, the child becomes an orphan and is
adopted by the system init process which has
process ID equal to 1.
z
Comparison of a foreground process and a daemon
process … cont’d
z
Foreground process (will not return to the shell)
z
z
z
Daemon
z
z
$ ./run
test!!
test!!
…
Daemon (run in the background mode, return to the shell
immediately)
z
$ ./run
$
test!!
test!!
…
A collection of background processes that perform a particular
system task.
Comparison of a foreground process and a daemon
process
z
The init process periodically waits for children so
eventually orphan zombies are removed.
Daemon (2/4)
z
Daemon (1/4)
run.c:
int main()
{
while (1)
{
printf(“test!!\n”);
sleep(10);
}
}
Daemon (3/4)
z How
to develop a daemon?
if (fork())
exit(0);
z setsid(): run a program in a new session
z Then, the parent process will terminate
immediately, and the child process
becomes a daemon process.
z
Daemon (4/4)
z
run_daemon.c:
execv()
z
int main()
{
if (fork())
return 0;
setsid();
while (1)
{
printf(“test!!\n”);
sleep(10);
}
}
Outline
z
Beginning of UNIX programming
z
z
z
z
z
z
z
File I/O
Process management
Signal handling
Interprocess communications
POSIX threads
z
z
Create/wait a thread
Thread synchronization
Execute a command
z
Wipes out most of the context
z
z
z
z
The file descriptor table is kept
We can manipulate the I/O of the command by
manipulating the file descriptor table.
Anything after this system call will not be executed
if the system call is successful.
Ex:
char *argv[3] = {“ps”, “-e”, NULL};
execv(“/bin/ps”, argv);
Signals
z
z
UNIX APIs
z
z
gcc
make
exec family system calls
Software interrupts
A notification to a process that an event has
occurred
z
z
Signals can be sent by:
z
z
z
usually the process doesn’t know ahead of time
exactly when a signal will occur
a process to another process (or to itself)
the kernel to a process
Every signal has a name specified in
<signal.h>.
Signals Generation (1/2)
z Some
examples of how signals are
generated
z
z
Signals Generation (2/2)
z Some
examples of how signals are
generated … cont’d
The kill command can be used to send signals.
Certain terminal characters generate signals
z an interrupt character (<ctrl>-c or Delete)
generates a signal called SIGINT
z a quit character (<ctrl>-backslash)
z
z
z
z
z
terminates a process
generates a signal called SIGQUIT to process and
generates a core image of the process
The core image can be used for debugging
z
SIGHUP
z
z
The terminal disconnects from the system
Ask a daemon to read configuration files again
z
SIGCHLD
z
SIGKILL
z
z
z
z
z
z
A child process changes its status (ex: terminate)
A process can provided a function that is
called whenever a specific type of signal
occurs.
z
Destroy the process
kill -9 <pid>
Software termination
The default signal sent by the kill command
z
SIGINT
z
SIGALRM
z
z
When user presses the interrupt key (Ctrl-C)
Alarm clock
SIGURG signal is generated when some urgent
data arrives on a socket
Handle a Signal (1/2)
SIGTERM
z
floating point arithmetic errors generate a signal
called SIGFPE
Certain software conditions can generate
signals
z
z
Some of the Signals
Certain hardware conditions generate
signals
z
This function, called a signal handler, can do
whatever the process wants to handle the condition.
(“catch” the signal)
A process can choose to ignore the signal. All
signals, except SIGKILL, can be ignored.
z
SIGKILL is special, it guarantees the system
administrator a way of terminating any process
Handle a Signal (2/2)
zA
process can allow the default to
happen.
z
How to Specify the Signal Handler?
z
Use signal() system call
i.e. #include <signal.h>
z signal() is a function that returns a pointer to a function that
returns an integer.
z The function argument specifies the address of the function
that doesn’t return anything.
z There are two special cases that provide special values for
the function argument
Normally, a process is terminated on receipt
of a signal with certain signals generating a
core image of the process in the current
working directory.
z
z
z
ANSI signal()
z Syntax:
#include <signal.h>
void (*signal(int sig, void (*disp)(int)))(int)
z
Semantics
sig - signal (defined in signal.h)
z disp - SIG_IGN, SIG_DFL or the address of a
signal handler
z Handler may be erased after one invocation
SIG_DFL to specify the signal is to be handled in the default way
SIG_IGN to specify that the signal is to be ignored.
If the signal handler returns, the process that received
the signal continues where it was interrupted.
Example
z
Catch SIGTERM
z
Ex:
void destroy(int sig)
{
fprintf(stderr, "destroy!!\n");
}
z
Use the kill command to send SIGTERM to the process
z
z
Simple techniques for avoiding this can lead to
incorrect results.
int main()
{
signal(SIGTERM, destroy);
sleep(60);
}
kill()
Outline
z Send
a signal to a process
#include <signal.h>
z #include <sys/types.h>
z int kill(pid_t pid, int signo)
z
Beginning of UNIX programming
z
z
z
z
UNIX APIs
z
z
z
z
z
z
z
module
Communication of
modules in a
process
z
module
z
z
z
File I/O
Process management
Signal handling
Interprocess communications
POSIX threads
z
IPC on a Single process
gcc
make
Create/wait a thread
Thread synchronization
IPC on a Single Computer
Process A
Process B
global variables
function calls
parameters
result
Kernel
IPC on Two Computers
Process A
Several Different Methods of IPC
Process B
DOS (Distributed OS)
Kernel
Kernel
Shared Memory (1/2)
z
shmget() - get shared memory segment identifier
z
int shmget(key_t key, size_t size, int shmflg)
z
ex: a 4-byte shared memory with key = 99887
z
z
z
z
z
(Software Interrupt)
z Pipes and FIFOs
z Message Queues
z Semaphores
z Shared Memory
z Sockets
z RPC (Remote Procedure Calls)
z Mutex and Conditional Variables
Shared Memory (2/2)
z
Ex: allocate/release a block of shared memory
(for an int variable)
z
shmget((key_t)99887, 4, SHM_R | SHM_W | IPC_CREAT); // create
shmget((key_t)99887, 4, SHM_R | SHM_W);
// open
Other operations
z
z Signals
shmat() - attaches the shared memory
shmdt() - detaches from the shared memory
z
z
z
ipcs - provide information on ipc facilities
ipcrm - remove a message queue, semaphore set or shared
memory id
int shmid = shmget(33537, sizeof(int), SHM_R | SHM_W |
IPC_CREAT);
Attach:
z
int *shmptr = (int *)shmat(shmid, (char *)0, 0);
z
Access: (directly access the shm via the pointer)
z
Detach:
z
Release:
Related commands
z
Allocate:
z
z
z
*shmptr = 1234;
shmdt( (void *)shmptr);
shmctl(shmid, IPC_RMID, NULL);
Semaphores
z is
a synchronization primitive
z is a form of IPC, semaphore is not used
for exchanging large amounts of data, as
is pipe; but are intended to let multiple
processes synchronize their operations.
Semaphores in System V (1/3)
z
Get set of semaphores
z
int semget(key_t key, int nsems, int semflg)
z
z
z
z
z
Semaphore operations … cont’d
z
z
struct sembuf {
ushort_t sem_num;
short
sem_op;
short
sem_flg;
}
z
struct sembuf sems[1];
// … initialize sems[1]
semop(semid, sems, 1);
Ex1: wait the semaphore
sems[0].sem_num = 0;
sems[0].sem_op = -1;
sems[0].sem_flg = 0;
semop(semid, sems, 1);
/* semaphore # */
/* semaphore operation */
/* operation flags */
User semaphore template for semop() system calls
Semaphores in System V (3/3)
z
Semaphore operations … cont’d
z
int semop(int semid, struct sembuf *sops, size_t
nsops)
z
semget((key_t)99667, 1, SEM_R | SEM_A | IPC_CREAT); // create
semget((key_t)99667, 1, SEM_R | SEM_A);
// open
Semaphore operations
z
Semaphores in System V (2/3)
Ex: a semaphore with key = 99667
z
Semaphore control operations
z
z
z
Ex2: signal the semaphore
sems[0].sem_num = 0;
sems[0].sem_op = 1;
sems[0].sem_flg = 0;
semop(semid, sems, 1);
int semctl(int semid, int semnum, int cmd, ...)
Ex: release a semaphore
semctl(semid, 0, IPC_RMID);
Notes: the default value of a semaphore is 0.
z
z
Wait for such a semaphore will cause the program being blocked.
Use semctl() or semop() to set the value of a semaphore to a positive
value before waiting for it.
Summary
z Use
ipcs to check if IPC resources are
created successfully.
z Use ipcrm to remove IPC resources
which are not released successfully.
z Choose a IPC key which does not
conflict with others.
z
Based on your student number
z Be
z
A thread is an independent sequence of
execution of program code inside a process.
A thread is often called a lightweight process.
z
But it is NOT a process
z
z
Beginning of UNIX programming
z
z
z
z
z
z
z
z
A thread has less
z
Program counter, a stack, and a set of registers, etc.
File I/O
Process management
Signal handling
Interprocess communications
POSIX threads
z
z
Create/wait a thread
Thread synchronization
Threads (2/2)
z
Threads share
z
z
It is something smaller than a process.
Memory, instruction, program counter, stack pointer,
registers, open file descriptor, etc.
gcc
make
UNIX APIs
z
z
Process instructions, most data, open files, signal
handlers and signal dispositions, current working
directory, user and group ID
Threads do not share
The process context includes
z
z
z
careful while using pointers!
Threads (1/2)
z
Outline
Thread ID, set of registers (including pc – program
counter – and sp – stack pointer), stack, errno,
signal mask, priority
POSIX standard: pthreads
z
Specifies the API and the way other POSIX
interfaces deal with threads
Threads vs Processes
z Advantages
of threads
It is less expensive to create a thread.
z Inter-thread communication is much simpler
and cheaper than inter-process
communication.
z
z Thread
Basic Thread Functions (1/2)
z
Pthread library
z
Thread creation
z
int pthread_create(pthread_t *thread, const pthread_attr_t *attr,
void *(*start_routine)(void *), void * arg)
z Thread attributes can be modified
z
z
programming can be dangerous
z
z
Wait for a thread to terminate
int pthread_join(pthread_t thread, void **value_ptr)
z Note that the thread ID, and not a pointer to it, is given as the first
argument.
z The return value of the function executed by the thread is stored in
*value_ptr.
z
Ex: Create a thread and wait for its termination
z
void* thread_proc(void *pv)
{
…
}
pthread_t th;
pthread_create(&th, NULL, thread_proc, NULL);
…
pthread_join(th, NULL);
z
z
Use NULL for default
Returns 0 if successful, and an error number if it fails
New thread executes the function given by start_routine
z
It is easy to make mistakes.
z Some system functions may break with
threads.
z
Basic Thread Functions (2/2)
Append “-lpthread” while building a pthread project
void* thread_proc(void *pv)
arg is the argument to this function
The thread ID is stored in *thread
Some More Thread Functions
z
pthread_t pthread_self(void)
z
z
void pthread_exit(void *value_ptr)
z
z
Get ID of the current thread
Exits a thread
int pthread_detach(pthread_t tid)
z
Make a thread detachable
z
z
This guarantees that the memory resources consumed by
the thread will be freed immediately when the thread
terminates.
This prevents other threads from synchronizing on the
termination of the thread using pthread_join().
Mutual Exclusion
z
z
z
int pthread_mutex_lock(pthread_mutex_t *mutex)
int pthread_mutex_unlock(pthread_mutex_t *mutex)
int pthread_mutex_init(pthread_mutex_t *mutex,
const pthread_mutex-attr_t *mutexattr)
z
z
z
Mutex pointer and mutex attribute pointer arguments
int pthread_mutex_destroy(pthread_mutex_t *mutex)
Ex:
z
pthread_mutex_t mutex;
pthread_mutex_init(&mutex, NULL);
pthread_mutex_lock(&mutex);
// … critical section
pthread_mutex_unlock(&mutex);
pthread_mutex_destroy(&mutex);
Condition Variables
z
int pthread_cond_wait(pthread_cond_t *restrict cond,
pthread_mutex_t *restrict mutex)
z
z
z
int pthread_cond_signal(pthread_cond_t *cond)
z
z
z
z
Atomically releases mutex and blocks on cond
On return, mutex is locked and owned by the calling thread
Unblocks one of the threads blocked on cond
The unblocked threads contend for their mutex
int pthread_cond_init(pthread_cond_t *cond,
pthread_condattr_t *cond_attr)
int pthread_cond_destroy(pthread_cond_t *cond)