ISEN 636 Large-Scale Stochastic Optimization
INSTRUCTOR
L. Ntaimo
239C Zachry Engineering Center
979 862-4066
ntaimo@tamu.edu
http://ise.tamu.edu/people/faculty/Ntaimo/
OFFICE HOURS
TBA and by appointment (open door policy)
COURSE DESCRIPTION
Introduction to models, theory and computational methods for large-scale stochastic programming.
Methods include decomposition-coordination algorithms for large-scale mathematical programming such
as Benders, regularized Benders, Dantzig-Wolf, L-shaped and statistically motivated decomposition
methods. Applications, theory and practical algorithm implementation and computational experimentation
will be emphasized.
PREREQUISITES
ISEN 622 – Linear Programming
STAT 610 or equivalent – Probability and Statistics
C/C++ Programming “competence”
COURSE OBJECTIVES
This is an introductory course to stochastic programming. The aim of the course is to introduce students
to optimal decision-making problems with data uncertainty. The field of stochastic programming is
currently developing rapidly with contributions from many disciplines such as operations research,
mathematics, and probability. Stochastic programming has a wide range of applications especially in
science and engineering such as manufacturing, transportation, telecommunications, electricity power
generation, health care, agriculture/forestry, finance, etc. The course will cover a broad overview of the
applications, basic theory, and decomposition methods of this vibrant field. Emphasis in this course is
placed on both theory and practical algorithm implementation and tools for solving difficult stochastic
programming problems. This is a first course in stochastic programming and is suitable for students with
knowledge of linear programming, elementary analysis, probability, and C/C++ programming. The
programming skills are needed for algorithm implementation using state-of-the-art optimization software
CPLEX. This course has a research level orientation and as such, students will be required to review
literature on stochastic programming and conduct computational experiments with stochastic programs.
TEXTBOOK AND ADDITIONAL COURSE MATERIAL
No course textbook is required. Course notes and other reading material will be provided and posted on
eLearning or the course website. However, you are encouraged to get a copy of a stochastic programming
textbook for your reference.
References:
1. J.R. Birge and F. Louveaux, Introduction to Stochastic Programming, 1st Edition, Duxbury
Press, Belmont, CA, 2003. ISBN 0 534 35964 7.
2. A. Ruszczynski and A. Shapiro (Eds.), Stochastic Programming. Handbooks in Operations
Research and Management Science Volume 10. New York, NY, 2003. ISBN 0 444 50854 6
3. R.K. Martin, Large Scale Linear and Integer Optimization: A Unified Approach, Kluwer, 1999.
4. A. Shapiro, D. Dentcheva and A. Ruszczynski, Lectures on Stochastic Programming:
Modeling and Theory, to be published by SIAM, Philadelphia, 2009.
1
5. ILOG CPLEX 11.0, User’s Manual and Reference Manual, ILOG, S.A., 2005,
DETAILED TOPICAL OUTLINE
The times listed are approximate.
1. Introduction to Stochastic Optimization and Applications
1.1 Decision-making in an uncertain world
-applications, decision stages, decision trees, decision-making models
1.2 Probability spaces and random variables
1.3 From linear programming to stochastic linear programming
Week 1-2, class-notes, Ch.1-2 in Birge and Louveaux, 1997.
2. Stochastic Programming Models
4.1 Two-stage recourse models
4.2 Chance-constrained models
4.3 Multi-stage models
Week 3, Ch. 3 in Birge and Louveaux, 1997; Ch. 1 Ruszczynski and Shapiro, 2003.
3. Deterministic Large-Scale Decomposition Methods
2.1 Basic theory - convexity, separation, convergence
2.2 Benders decomposition
2.3 Regularized Benders
2.4 Dantzig-Wolfe decomposition
Week 4-5, class-notes, Ch. 10-11 in Martin, 1999
4. Computational Experimentation
3.1 Empirical testing
3.2 SMPS format, sparse matrix representation, etc.
3.3 Introduction to the CPLEX callable library
Week 6, class-notes
5. Stochastic Linear Programming Methods
5.1 The L-shaped method
5.2 Multi-cut, bunching and other efficiencies
5.3 Regularized version
Week 7-8, Ch. 5 in Birge and Louveaux, 1997; Ch. 3 Ruszczynski and Shapiro, 2003.
6. The Value of Information and the Stochastic Solution
6.1 Expected value of perfect information
6.2 The value of the stochastic solution
6.3 Basic relationships
6.4 Bounds
Week 9, Ch. 4 in Birge and Louveaux, 1997
7. Sample-Based Methods
7.1 General results
7.2 Sampling in the L-shaped method
7.3 Basic stochastic decomposition
Week 10-11, Chapter 10 in Birge and Louveaux, 1997; Ch. 6 Ruszczynski and Shapiro, 2003.
8. Introduction to Stochastic Integer Programming (if time permits)
8.1 Integer L-shaped
8.2 Simple integer recourse
8.3 Binary first-stage
8.4 New approaches
Week 12, Ch. 8 in Birge and Louveaux, 1997; Ch. 4 Ruszczynski and Shapiro, 2003.
2
SEMESTER INDIVIDUAL “MINI” PROJECTS
The semester individual “mini” projects will involve homework problems, implementing large-scale
decomposition algorithms using C/C++ and the CPLEX Callable Library, and performing computational
experiments on a given set of instances from the literature. A report detailing the algorithm steps,
computational results, and sample code (soft copy) will be required.
Project 1 - Benders algorithm (Week 3)
Project 2 - Regularized Benders algorithm (Week 5)
Project 3 - L-shaped algorithm (Week 6)
Project 4 - Binary first-stage method (Week 11)
SEMESTER “TEAM” PROJECT
Students will form semester teams comprising of 1-2 students. Each student will model an interesting
real-life practical problem(s) of their choice using stochastic programming, implement an extension(s) of
the algorithm(s) from the “mini” projects, conduct computational experiments to solve large-scale
instances of the problem (may be randomly generated) using the implemented code, and perform a
solution analysis. Finally, each team will write a project report and give a class presentation the last week
of class. Students are encouraged to work on problems from their thesis/dissertation area.
CLASS SCHEDULE
Lectures: MWF 10:20AM – 11:10PM.
Classroom: Zachry 104A.
GRADING
Since this is an elective course, your grade for the course will be based on the level and quality of your
participation in the course. There will be a midterm and a final take-home exam. The exams may involve
performing some computational tests using your code in addition to solving some theoretical problems.
The exams will constitute 50% of the grade while the other 50% will come from the homework and mini
projects (25%) assigned during the semester and the semester project (25%). Grades will be assigned as
follows: A (90-100%), B (80-89.9%), C (70-79.9%), D (60 – 69.9%), F (< 60%).
TEACHING ASSISTANT (TA)
Eric Beier, Ph.D. Student
Office: 301B Zachry
Office hours: TR 10:00-11:00 or by appointment
Phone: 847-8507 (Systems Lab - 304 Zachry)
Email: ebeier@tamu.edu
COURSE WEB SITE: http://ise.tamu.edu/ISEN689-601
PROFESSIONAL COMPONENT
This course provides fundamental concepts, theory and procedures for modeling, solving and analyzing
problems using stochastic programming. The knowledge learned in this course is used to develop a
semester research project or the basis for a potential thesis/dissertation project.
DISABILITIES ACT
The Americans with Disabilities Act (ADA) is a federal anti-discrimination statute that provides
comprehensive civil rights protection for persons with disabilities. Among other things, this
legislation requires that all students with disabilities be guaranteed a learning environment that
provides for reasonable accommodation of their disabilities. If you believe you have a disability
requiring an accommodation, please contact the Disability Services in Room B118 of the Cain Hall
Building or call 845-1637, email disability@tamu.edu, http://disability.tamu.edu
3
ACADEMIC INTEGRITY:
“An Aggie does not lie, cheat, or steal or tolerate those who do.” It is the responsibility of students
and instructors to help maintain scholastic integrity at the university by refusing to participate in or
tolerate dishonesty. (http://www.tamu.edu/aggiehonor)
PROGRAM OUTCOMES
A. An ability to apply knowledge of mathematics, science, and engineering.
Specialized knowledge and analytical procedures are developed to enhance the decision-making process under
uncertainty using stochastic programming. This includes the following: (1) formulate a problem in technical terms;
(2) determine and implement the appropriate modeling approach; (3) choose the appropriate solution method and
apply feedback to improve the system
B. An ability to design and conduct experiments, as well as analyze and interpret data.
Students are required to do several mini projects in which they implement several decomposition methods in
stochastic programming and perform computational experiments on real-life and randomly generated problem
instances. This will also include analysis and interpretation of the problems’ optimal solutions. The students are also
taught how to use state-of-the-art optimization software (CPLEX callable library).
C. An ability to function on multi-disciplinary teams.
Students are required to do a semester research project in teams. In order to promote the team approach, teams of 12 students are allowed for the semester project. The requirements for the semester project are as follows: (1)
implement an extension(s) of the algorithms learned in class; (2) model a real-life problem involving uncertainty in
the data (students are encouraged to find a topic based on their thesis/dissertation problem); (3) provide solutions to
the problem using the implemented algorithm(s); (4) performing computational experiments to find optimal
solutions to the problem; (5) provide recommendations via a project report and presentation at the end of the
semester. Some test instances and C/C++ code “fragments” will be provided by the instructor.
E. An ability to identify, formulate, and solve engineering problems.
ISEN 636 emphasizes on the importance of the following in industrial engineering: (1) identify a real-world
problem; (2) determine whether or not stochastic programming is an appropriate modeling framework; (3) formulate
the problem as a stochastic program; (4) solve the problem for its optimal solution; (5) interpret the models'
solutions and infer solutions to the real-world problem.
G. An ability to communicate effectively.
Students are encouraged to engage in group discussions in class concerning the subject matter. The semester team
project requires a report to be reviewed and graded. Each report may be reviewed and specific suggestions and
recommendations made. This will require each team to revise it effectively to reflect those improvements identified
in the review process. Each team makes a project presentation with specific areas presented by designated members
of the group. The presentation takes approximately 20 minutes per team, including questions (about 5 minutes).
I. A recognition of the need for, and an ability to engage in life-long learning.
Throughout the development of the class, a great deal of lecture time is devoted to discussing real-life problems
where stochastic programming is an appropriate tool. This allows the students to recognize the importance of being
life-long learners, particularly since most of the real-life problems are open-ended.
K. An ability to use the techniques, skills, and modern engineering tools necessary for engineering practice.
The following techniques, skills and tools are studied and used: C/C++, CPLEX Callable Library.
Summary: The student should be able to formulate, devise, and implement algorithms to solve stochastic
programming problems
PREPARED BY: Dr. Lewis Ntaimo
DATE: AUG, 2007, Revs: AUG 2009
4