F79SP Stochastic Processes
Lecturers: S. Foss
Aims
To introduce fundamental stochastic processes which are useful in insurance, investment and
stochastic modelling, and to develop techniques and methods for analysing the long term
behaviour of these processes.
Summary
In this course, we develop methods for modelling systems or quantities which change randomly
with time. Specifically, the evolution of the system is described by a collection {Xt} of random
variables, where Xt denotes the state of the system at time t.
Discrete-time processes studied include (renewal processes and) Markov chains. In particular we
consider branching processes, random walk processes, and more general countable state-space
chains.
Continuous-time processes studied include Poisson and compound Poisson processes;
continuous time Markov processes; population, queueing and risk models.
Reading
Useful reference books are:





Grinstead and Snell. Introduction to Probability. American Mathematical Society:
http://www.dartmouth.edu/~chance/teaching_aids/books_articles/probability_book/amsb
ook.mac.pdf
P. Bremaud (1997). An Introduction to Probabilistic Modeling. Springer.
P. Bremaud (1999). Markov Chains. Springer.
J. R. Norris (1998). Markov Chains. Cambridge University Press.
G. R. Grimmett & D. R. Stirzaker (2001) Probability and Random Processes, 3rd ed.
Oxford University Press.
Recommended textbooks on background matters:





D. Stirzaker (1999). Probability and Random Variables: a beginner's guide, Cambridge
University Press.
K.L. Chung and F. Aitsahlia (2003). Elementary Probability Theory, Springer-Verlag.
G. Grimmett & D. Welsh (1990). Probability: an Introduction, Oxford University Press.
S. M. Ross (1988). A First Course in Probability, 3rd edition, Macmillan.
D. Williams (2001). Weighing the Odds: A Course in Probability and Statistics,
Cambridge University Press.
Assessment
This course will be assessed by a 2-hour examination at the end of the year. It is synoptically
linked with F79SU Survival Models.
Help
If you have any problems or questions regarding the course, you are encouraged to contact the
lecturer.
Course web page
Further information and course materials are available at
http://www.ma.hw.ac.uk/~takis/stochproc08/ and on Vision.
Detailed syllabus
























Review of independence
Sequences of random variables and the Markov property
Review of matrix algebra
Review of summation notation and other useful concepts.
Using the Markov property
Absorbing Markov chains with finite state space:
o Computing probability of absorption
o Computing expected time to absorption
First-step (backwards) equations
Basic examples: Mouse and cheese, drunkard's walk, Ehrenfest chain, genetic models,
gambling chains, etc.
Stationarity problem for finite space chains
Tricks on the computation of the stationary distribution (fluxes, reversibility, symmetry)
State classification
Periodicity
Convergence to stationarity
The strong theorem (law) of large numbers for sequences of i.i.d. random variables
Frequencies
Simple random walk (SRW) in the integers
Markov chains with infinite but countable state space
Recurrence and transience
Examples: Discrete option pricing, branching processes, insurance loss, queueing for
service, waiting for a bus, etc.--as time permits
Basic relations between exponential, gamma, Poisson and uniform distributions
Simple point processes, Poisson and compound Poisson processes
Continuous time Markov processes
Numerical solution of the Kolmogorov Forward Equations for a time-inhomogeneous
Markov process
Examples