COMPUTATIONAL METHODS IN STATISTICS
STA 4102/5106: Spring 2002
Instructor: Dr. Anuj Srivastava (Room OSB 210B/106D, 644-8832)
http://stat.fsu.edu/~anuj
Location: Room 205 OSB, MWF 10:10-11:00am
Course Objective:
To gain an understanding of the techniques and ideas used in implementing mathematical
formulations on computers, with focus on common statistical approaches. The main
computational tool will be Matlab.
Prerequisites: Probability theory (discrete and continuous random variables), linear algebra, and real
analysis.
Texts:
1. Elements of Statistical Computing: Numerical Computation, Ronald A. Thisted (Chapman
& Hall, 88)
2. Numerical Analysis: A Practical Approach, M. J. Maron (Macmillan Publishing
Company, 82).
3. Matrix Computations, Golub and VanLoan (Johns Hopkins University Press, 96).
4. Simulation by Sheldon Ross, Second Edition, Academic Press, 1997.
5. Monte Carlo Statistical Methods by C. P. Robert and G. Castella, Springer Text in
Statistics, 1999.
6. Random Number Generation and Monte Carlo Methods by James Gentle
Topics Covered:
1. Numerical Analysis: Floating point arithmetic and error analysis.
2. Numerical Linear Algebra: Multiple regression analysis, orthogonalization by
Householder transformations.
3. Nonlinear Methods: root finding, numerical optimization.
4. Numerical Integration: Quadrature integration, Newton-Cotes’ method, Composite rules
5. Random Number Generators: modular arithmetic, linear congruential generators
combination generators
6. Monte-Carlo methods for Integration:
a. General MC formulation: sample mean and variance
b. Importance sampling (ex: Cauchy), optimal choice of sampling density
c. Variance reduction techniques: antithetic variables, control variates, variance
reduction by conditioning, importance sampling via twisted simulations.
7. Markov chain Monte-Carlo (MCMC) methods:
a. Introduction to stochastic processes (ex: Poisson counting process, Random Walk
and its limiting case), Markov process, stationarity, homogeneous Markov
process,
b. Finite-state case: Markov chain, transition matrix, Perron-Frobenious theorem,
irreducibility and aperiodicity, ergodic result
c. Countable-state case: recurrence, basic limit theorem of Markov chains, positive
recurrent, ergodicity
d. Metropolis-Hastings: general algorithm, detailed balance condition, independent
Metropolis-Hastings, random walk Metropolis-Hastings.
e. Gibbs Sampler: general algorithm, bivariate Gibbs sampler, completion Gibbs
sampler (ex: truncated normal), slice sampler.
Grading Policy:
50% homework, 20% mid-term, 25% final and 5% class participation and quizzes
(grading will be relative)
Homework:
It will be assigned Wednesday every week and will be due the next Wednesday. Late
assignments will not be accepted. The assignment with lowest grade will be dropped. Please
write clear and detailed answers to the homework problems. If a problem involves writing a
program, submit a copy of the code with the solution. Provide illustrative outputs of your
programs to accompany the homework solutions.
Attendance Policy:
It is required to attend all the classes. It is the student’s responsibility to understand the
material covered in the class during any absence.
Academic Honor System:
“The Academic Honor System of The Florida State University is based on the premise that
each student has the responsibility to: 1) Uphold the highest standards of academic integrity in
the student’s work, 2) Refuse to tolerate violations of academic integrity in the academic
community, and 3) Foster a high sense of integrity and social responsibility on the part of
University community.”
Please note that violations of this Academic Honor System will not be tolerated in this class.
Specifically, incidents of plagiarism of any type or referring to any unauthorized material
during examinations will be rigorously pursued by this instructor. Before submitting any work
for this class, please read the “Academic Honor System” in its entirety (as found in the FSU
General Bulletin and in the FSU Student Handbook) and ask the instructor to clarify any of its
expectations that you do not understand).