Prof. J. Karhunen
T-61.5130 Machine Learning and Neural Networks
Prof. J. Karhunen
T-61.5130 Machine Learning and Neural Networks
Background
• Code T-61.5130, 5 credit points.
T-61.5130 Machine Learning and
Neural Networks (5 cr)
• Can be taken as a part of graduate studies, too.
• The course is lectured from 26th October to 1st December.
• The course and all its materials are completely in English.
General information on the course
Autumn 2015
• The course will change considerably from the year 2014 with the
adoption of a new textbook.
• This is the second middle one of the three machine learning courses
in our department.
Prof. Juha Karhunen
• This course is fairly independent of the two other machine learning
courses in our department:
https://mycourses.aalto.fi/
– T-61.3050 Machine Learning: Basic Principles.
Aalto University School of Science, Espoo, Finland
Prof. J. Karhunen
1
T-61.5130 Machine Learning and Neural Networks
– T-61.5140 Machine Learning: Advanced Probabilistic Methods.
Aalto University School of Science, Espoo, Finland
Prof. J. Karhunen
T-61.5130 Machine Learning and Neural Networks
Thursday 29th October in lecture hall T1.
• However, the participants should know differential and integral
calculus, matrix algebra, and basic probability theory.
• The last 13th lecture deals with Deep learning which is currently a
hot topic in machine learning and neural networks.
• More information on the course can be found on its homepage
https://mycourses.aalto.fi/.
• There will be no exercises corresponding to this lecture.
• The lecture slides will be available on the home pages of the course
before each lecture.
Lectures
• Both in full color versions for viewing, and black-and-white versions
suitable for printing (4 slides per A4 page).
• The lectures will be given on:
– Mondays 12:15–14:00 o’clock in the lecture hall T1 in the
Computer Science and Engineering (CSE) building.
• Please note that each set of the lecture slides covers one topic.
– Tuesdays 14:15–16:00 o’clock also in the lecture hall T1.
• The number of slides of each lecture varies quite a lot.
• The teaching period II covers this year weeks 44-49 from 26th
October to 4th December.
• In practice, presenting the slides covering each topic may take less
or more than one oral lecture of 2 × 45 minutes.
• Furthermore, there is an extra lecture in place of exercises on
• If there is not enough time to discuss all the matters, you can read
Aalto University School of Science, Espoo, Finland
2
3
Aalto University School of Science, Espoo, Finland
4
Prof. J. Karhunen
T-61.5130 Machine Learning and Neural Networks
yourself the rest from the lecture slides.
Prof. J. Karhunen
T-61.5130 Machine Learning and Neural Networks
• The solutions will also be available on the home page of the course
for viewing and printing.
• The lecturer is Prof. Juha Karhunen.
• Home page: http://users.ics.aalto.fi/juha/
• The course assistant is Dr. Mark van Heeswijk.
• His room is B327 in the CSE building (Tietotekniikan talo) in
Otaniemi, Espoo.
• His email is mark.van.heeswijk@aalto.fi, or you can meet him in the
exercises or in his room B325 in the CSE building.
• Contact preferably during or after the lectures, or by email:
juha.karhunen@aalto.fi
• The exercise times are:
– Thursdays 10:15–12 o’clock in the lecture hall T1 in CSE
building.
• Or if necessary by Tel. 0400–817 276, or visiting his room.
– Fridays 12:15–14 o’clock in the lecture hall T1.
Exercises
• The first exercises will be held on Friday 30th October.
• Exercise problems will be available on the home page of the course
before the exercises will be held.
• There are no exercises on Thursday 29th October, because the
first quite introductory lecture does not provide material for them.
• In the exercises, the course assistant presents the solutions of the
problems given.
Aalto University School of Science, Espoo, Finland
Prof. J. Karhunen
• Otherwise, the exercises will be held regularly with no exceptions in
5
T-61.5130 Machine Learning and Neural Networks
their times and places.
Aalto University School of Science, Espoo, Finland
Prof. J. Karhunen
T-61.5130 Machine Learning and Neural Networks
• And of course also the examination.
• There is one exercise corresponding to each set of lecture slides
except for the first lecture.
• The assistant responsible for the computer assignments is Mudassar
Abbas.
• In addition to standard problems, the exercises include some
computer simulations.
• His email address is mudassar.abbas@aalto.fi, Tel. 050-430 4773,
and room B310.
• For demonstrating the performance of methods and algorithms.
• There are 5 different computer assignments assigned to the
participants according to the last digit of their study book number.
• The exercise problems and their solutions will be the same as in
autumn 2012 when they were improved and updated.
• The deadline for returning them is February 1, 2016.
Computer assignment
• This late deadline will not disturb your preparation to the
examinations.
• The course includes also a computer assignment (small project) for
each participant.
• The marks given on the computer assigments are either “accepted”
or “rejected”.
• To pass the course, one must perform acceptably the computer
assignment allocated to him or her.
Aalto University School of Science, Espoo, Finland
6
• The course assistant shows which parts of the rejected computer
7
Aalto University School of Science, Espoo, Finland
8
Prof. J. Karhunen
T-61.5130 Machine Learning and Neural Networks
assignments must be corrected, and contacts you if necessary.
Prof. J. Karhunen
T-61.5130 Machine Learning and Neural Networks
• It is now K-L. Du and M. Swamy, “Neural Networks and
Statistical Learning”, Springer 2014.
• The rejected assignments must be revised until they have been
accepted.
• This book will be complemented by material from other sources.
• Therefore the total mark (grade) given to you on passing the course
is determined by the mark of the examination.
• The book has more than 800 pages, and deals with quite many
topics.
• The computer assignments will be the same as earlier.
• Then some old assignments were replaced by new ones.
• It includes many topics that do not belong to our course, and we
shall skip the corresponding parts and chapters.
• Send the report on your computer assignment as an attachment of
an email to the course assistant.
• Even though the book has more than 800 pages, it deals many
matters somewhat superficially.
• You can buy the textbook with less than 100 US dollars if you wish.
Course materials
• All the course materials will be in English.
• Several topics are discussed on lecture slides only.
• The main textbook used as background material in our course will
change from the previous year.
• Lecture slides and exercise solutions are the central materials in this
course.
Aalto University School of Science, Espoo, Finland
Prof. J. Karhunen
9
T-61.5130 Machine Learning and Neural Networks
• You are responsible for printing the materials provided on the home
pages of the course to your disposal if necessary.
Prof. J. Karhunen
10
T-61.5130 Machine Learning and Neural Networks
2. For lecture 8: Chapter 6, “Support Vector Machines”, from the
textbook S. Haykin, “Neural Networks - A Comprehensive
Foundation”, Prentice-Hall 1998, pp. 318-350.
• Including the lecture slides, exercise sheets, solutions of the
exercises, and examination requirements.
3. For lecture 9: A. Hyvärinen and E. Oja, “Independent
component analysis: algorithms and applications”, Neural
Networks, vol. 13, 2000, pp. 411-430.
Additional materials on the home page of the course
4. For lecture 11: Chapter 13, “Temporal Processing Using
Feedforward Neural Networks”, from the textbook S. Haykin,
“Neural Networks - A Comprehensive Foundation”, Prentice-Hall
1998, pp. 635-663.
• Some lectures are based strongly on written materials other than
Du’s and Swamy’s book.
• These materials are available on the home page of the course, where
you should be able to read and/or print them.
Examinations
• These materials are:
1. For later part of lecture 5: Journal paper G.-B. Huang, Q.-Y.
Zhu, and C.-H. Siew, “Extreme learning machine: Theory and
applications”, Neurocomputing, vol. 70, 2006, pp. 489-501.
Aalto University School of Science, Espoo, Finland
Aalto University School of Science, Espoo, Finland
• Note that according to our rules you must register to the
examination(s) via the Oodi web pages at least one week
before the examination.
11
Aalto University School of Science, Espoo, Finland
12
Prof. J. Karhunen
T-61.5130 Machine Learning and Neural Networks
• Otherwise you cannot participate in the examination.
Prof. J. Karhunen
T-61.5130 Machine Learning and Neural Networks
pages of the course.
• On the other hand if you register to the examination but are not
present in it, you should get the mark (grade) zero on it.
• Also the examination results will appear there.
• Please enroll to the course using the Oodi web pages.
• Exact examination requirements will be available on the home page
of the course later on.
• If this is not possible by email to the lecturer.
Planned contents of the course
• The first examination is on Friday 11th December 9:00-12:00 in the
lecture hall TU2 in the TUAS house.
• The course changes somewhat from the previous years.
• The second examination is on Monday 15th February 17:00-20:00 in
the lecture hall T1.
• Some old stuff is removed, and the emphasis is laid on successful
methods which perform well in practice.
• And the third examination will be on Thursday 26th May
13:00-16:00 in the lecture hall T1.
• The following matters will be discussed in this course:
– Lecture 1: Introduction to neural networks.
Notices and enrollment
– Lecture 2: Simple neural models.
– Lecture 3: Preprocessing of data, principal component analysis.
• Announcements and news on the course will appear on the home
Aalto University School of Science, Espoo, Finland
Prof. J. Karhunen
13
T-61.5130 Machine Learning and Neural Networks
– Lecture 4: Multilayer perceptron (MLP) networks and
backpropagation learning algorithm.
– Lecture 5: Levenberg-Marquard algorithm and Extreme learning
machine.
– Lecture 6: Model assessment and selection: generalization,
validation, and regularization.
– Lecture 7: Radial-basis function networks.
– Lecture 8: Support vector machines.
– Lectures 9: Independent component analysis.
– Lecture 10: Self-organizing maps.
– Lecture 11: Processing of temporal information in neural
networks.
– Lecture 12: Combining classifiers and models.
– Lecture 13: Deep learning and networks.
Aalto University School of Science, Espoo, Finland
15
Aalto University School of Science, Espoo, Finland
14