2WB08 – Stochastic processes
Why are you here?
This is a Masters-level course in stochastic processes. As such, you will be introduced to several
advanced ideas and principles that can prove quite challenging at first. However, regardless whether
you plan to go to industry after finishing your degree or if you are planning on further studies, you
will certainly use material from this class.
The course will prove challenging. It is a Masters-level course, and experience tells us that most
students do not appreciate the difference between their experience so far and this new level. In
particular, you may expect the following difficulties:



You will be taught in a (most probably) foreign language (to you). Although your English may
be excellent, the language barrier will still exist.
The course pace will be fast. At Masters level, a lot of the teaching responsibility is shifted to
the students. Students should do suggested homework on their own time and without any
further prompting from the lecturer.
As a result of the previous point, study after every lesson at home and revise the material
before the next lecture. Waiting until the Christmas break to revise all material (for the first
time) is not a smart strategy.
Instructors:
Dr. Maria Vlasiou (first quarter)
Department of Mathematics
and Computer Science,
Eindhoven University of Technology,
P.O. Box 513,
5600 MB Eindhoven,
The Netherlands
Dr. Francesca Nardi (second quarter)
email: m.vlasiou@tue.nl
office: (+31) 40 247 5185
fax: (+31) 40 243 6685
email: f.r.nardi@tue.nl
office: (+31) 40 247 4232
fax: (+31) 40 243 6685
Location: MF 4.092
Location: MF 4.076
Office hours: to be announced
Office hours: to be announced
(same mailing address)
Course material:
A First Course in Stochastic Processes, 2nd Edition
Authors: Samuel Karlin & Howard Taylor
ISBN 10: 0-12-398552-8, ISBN 13: 978-0-12-398552-1
Tentative class schedule (to be updated weekly):
1. Week 1
2. Week 2
Pages 26 – 33, 167 – 183 and additional examples
Pages 184 – 197 and additional examples and exercises
3. Week 3
4. Week 4
5. Week 5
6. Week 6
7. Week 7
8. Week 8
9. Week 9
10. Week 10
11. Week 11
12. Week 12
13. Week 13
14. Week 14
15. Week 15
16. 17/1/13
Pages 197 – 204, 218 – 221 and additional exercises on cumulative
RPs and ON-OFF RPs
CLT for RPs (p. 208); Exercises on renewal reward processes and on
CLT for RPs. Martingales 238 – 248 and few additional examples
Pages 248 – 262
Pages 262 – 267, 278 – 287
Applications b, c, and f of section 6 (pages 287 – 297). An additional
exercise (proof of the SLLN when an m.g.f. exists.) Exercises
(elementary 9, 10, 11, 12, 13, 18)
No classes scheduled unless a revision is desired by the students
Branching process. Examples: Chapter 2 pg. 54 – 55 and Chapter 8,
section 1, pg. 392 – 393. Definition and properties: Chapter 8, section
2, 394 – 395. Generating function Relations for BP and Extinction
Probabilities: Chapter 8, section 2 – -3, pg. 395 – 399.
Branching process. Extinction Probabilities and examples: Chapter 8,
section 3 – 4, pg. 399 – 404. Branching processes and martingales:
notes and Chapter 6 pg 321 – 325. Introduction to Random Walk.
Examples: Chapter 2 pg. 48 – 52. Absorption probabilities example
gambler’s ruin: Chapter 3, section 3, pg. 92 – 94. Random Walk
Chapter 3, section 7, pg. 106 – 108.
Random walk, Duality property, properties.
Random walk, Exchangeable random variables, De Finetti theorem
(without proof), Using Martingales to Analyze Random Walks.
Applications to G/G/1 queues and ruin problems.
Introduction and definition of Brownian motion. Joint probabilities
for Brownian motion. Continuity of paths and the Maximum variable.
Chapter 7 pg. 340 – 348
Chapter 7 pg. 348 – 357
Finishing BM and additional exercises
No classes scheduled unless a revision is desired by the students
Exams and Grading:
There will be one homework (HW) set during the first quarter, worth 1 point of the final
score, and one set of exercises in class (MT) during the second quarter, worth an additional
1 point of the final score. These two sets can improve your final score, provided that you
score at least 50% in the final exam (F), worth 9 point of the final score. As a result, 11
points are achievable. The bonus will count for the retrial exam too.
In Excel language: Final Grade = IF(F>4.5, MIN(F + HW + MT, 10), F)
The dates of the examinations (MT, final, retrial) will be announced later.
Useful tips:


All announcements will be posted in OASE.
Attendance is not obligatory but it is highly recommended.

Students are expected to:
o contribute to a positive, respectful, and engaged academic environment
inside and outside the classroom;
o appear regularly for class meetings in a timely fashion;
o appear at a mutually convenient appointment for official matters of academic
concern;
o full attendance at examination, midterms, presentations, with the exception
of formal pre-approved excused absences or emergency situations;
o be prepared for class, appearing with appropriate materials and having
completed assigned readings and homework;
o full engagement within the classroom, including meaningful focus during
lectures, appropriate and relevant questions, and class participation;
o report early on difficulties with the material, the lecturer or any other
difficulties that arise so that we can fix things in a timely fashion;
o act with integrity and to adhere to the formal university policies (read: no
cheating!)