BTRY/STSCI 6520: Statistical Computing
MWF 1:25-2:15 Upson 205
Instructor: Giles Hooker, (gjh27@cornell.edu); 1186 Comstock Hall
Office Hours: 2:15 – 4:15 Wednesdays
Co-requisites: ORIE 6700/STSCI 6730 or equivalents and at least one course in
probability.
Expected proficiencies: programing in R, probability, multivariable calculus, real
analysis.
Description
This course is designed to provide students with an introduction to statistical computing.
The class will cover the basics of \ numerical methods for optimization and linear algebra
and their application to statistical estimation, generating random variables, bootstrap,
jackknife and permutation methods, Markov Chain Monte Carlo methods, Bayesian
inference and computing with latent variables.
Assessment will be based on four assignments and a take-home exam.
Learning Outcomes
Students will be able to
- Design, analyze and implement specialized optimization methods for statistical
problems
- Design, analyze and implement methods for numerical linear algebra
- Design, analyze and implement methods for generating random numbers
- Implement and design resampling methods for statistical inference
- Research, implement and present advanced methods from the literature in
numerical analysis as it relates to statistics.
Class times: Monday/Wednesday/Friday 1:25pm-2:15pm, Upson 205
Assessment: 4 homeworks (15% each), take-home final (40%)
Texts:
John Monahan, “Numerical Methods of Statistics”, Cambridge
Approximate Syllabus (probably over-ambitious):
- Computer architecture and digital arithmetic (Week 1)
o Computer components and manipulation
o Binary arithmetic and representation of numbers
o Floating point representation
- Numerical precision and error (Week 2)
o Round-off error accumulation
o Over-flow and under-flow
o Condition of numerical algorithms
o How to improve numerical precision
- Numerical linear algebra and linear regression 1: exact methods (Weeks 3 and 4)
o Gaussian elimination and QR decompositions
o Gramm-Schmidt orthogonalization
o Cholesky decomposition
o Numerical Eigen problems and principal components analysis
- Numerical optimization/zero-finding, nonlinear regression and generalized
linear models (Week 5)
o Maximum likelihood and nonlinear optimization.
o Bisection/golden section search
o Steepest descent/ascent methods and stopping criteria
o Conjugate gradient methods and variations
o Numerical linear algebra II: conjugate gradients
- Statistical Interlude: the Bootstrap, subsampling and permutation methods
(Week 6)
o Permutation tests; implementation and scope of application.
o Non-parametric bootstrap
o Parametric bootstrap and other variants.
o Jackknife estimates.
o Subsampling methods and U-statistics.
- Gradient free optimization and specialized methods (Week 7)
o Nelder-Meade simplex methods
o Monte Carlo search and simulated annealing
o Iteratively-reweighted least squares
o The EM algorithm
- Constrained Optimization (Weeks 8)
o Linear programming,
o Quadratic programming and the LASSO
o General constrained optimization problems
- More Monte Carlo: Generating Random Variables (Week 9)
o Inverse transform
o Accept-reject methods and importance sampling
o Simulating stochastic processes
o Markov Chain Monte Carlo
- Monte Carlo Integration (Week 10 and 11)
o Monte Carlo integration (and comparison to numerical integration)
o Antithetic Sampling
o Stratified Sampling
o Inferential methods
o Bayesian inference
- Latent Variables and Statistical Computing (Week 12)
o Random effects models
o Simulation
o EM algorithm
o Bayesian inference.
- Nonparametric Estimators (Week 13)
o Kernel methods for smoothing and density estimation
o Basis expansions: B-splines, Fourier series and wavelets
o Trees, radial basis functions and other topics in machine learning
Possible additional topics: parallel computing, large-scale data, sequential Monte
Carlo, optimizing stochastic objectives, Gaussian processes.
Academic Integrity
As per university policy, all syllabi should contain some reference to
the Cornell Academic Integrity Code. Violations are dealt with
seriously, so please read it:
http://www.cuinfo.cornell.edu/Academic/AIC.html
Accommodations for Students with Disabilities:
In compliance with Cornell University policy and equal access laws,
Professor Hooker is available to discuss appropriate academic
accommodations that may be required for students with documented
disabilities. Except for unusual circumstances, requests for academic
accommodations should be made during the first three weeks of the
semester. Students are strongly encouraged to register with Student
Disability Services to verify their eligibility for appropriate
accommodations.