NATIONAL UNIVERSITY OF SINGAPORE
School of Business
Department of Decision Sciences
BDC 5102
Stochastic Operations Research Models
Lecturer
:
Assoc. Prof Ou Jihong
Session
:
Semester II, 2007/08
Prerequisites
A course in probability and statistics (using calculus) such as ST2334 Probability and
Statistics.
Course Objectives
This module introduces students to some of the main stochastic models used in
engineering and management science applications: discrete-time Markov chains, Poisson
processes, birth and death processes and other continuous Markov chains, renewal
reward processes, Markov decision problems.
Course Outline
1. Probability Extra:
 Conditional probability
 Poisson distribution, exponential distribution
2. Markov Chains
 Discrete time Markov chains
Chapman-Kolmogorov equations
Classification of states
Long run properties
First passage times
 Continuous time Markov chains
Classification of states
Steady state probabilities
 Birth-and-death process
3. Models of Queues
 Queues with exponential arrivals and services
 Queues with general arrivals and services
 Networks of queues
4. Renewal Processes




Renewal processes
Forward and backward recurrence time
Equilibrium distribution and inspection paradox
Elementary renewal theorem, Blackwell’s renewal theorem,
and Key renewal theorem
5. Markov Decision Problems
 Dynamic programming formulation
 Hamilton-Jacobi condition
 Queueing optimization problems
Main Text
S. Ross, Introduction to Probability Models, 8th Edition, Academic Press.
Assessment Methods
Assessment will be based on the following:
Homework Assignment
20%
Mid-term
30%
Final Exam
50%