Algorithm Design and Analysis (ADA)
242-535, Semester 1 2015-2016
0. Preliminaries
Please ask
questions
Who I am:
Andrew Davison
WiG Lab
ad@fivedots.coe.psu.ac.th
• Objective
o to give some background on the course
242-535 ADA: 0. Preliminaries
1
Overview
1. What is an Algorithm?
2. Meeting Times / Locations
3. Workload
4. Exercises
5. Course Materials
6. Books
7. Videos
8. Web Sites
242-535 ADA: 0. Preliminaries
2
1. What is a Algorithm?
• An algorithm is a finite set of unambiguous
instructions for solving a problem.
o An algorithm is correct if on all legitimate inputs, it outputs
the right answer in a finite amount of time
• Can be expressed as
o
o
o
o
pseudocode
flow charts
text in a natural language (e.g. English)
computer code
242-535 ADA: 0. Preliminaries
3
Algorithm Design
The theoretical study of how to solve computational
problems
• sorting a list of numbers
• finding a shortest route on a map
• scheduling when to work on homework
• answering web search queries
• and so on...
242-535 ADA: 0. Preliminaries
4
The Importance of Algorithms
• Their impact is broad and far-reaching.
o Internet. Web search, packet routing, distributed file
sharing, ...
o Biology. Human genome project, protein folding, ...
o Computers. Circuit layout, file system, compilers, ...
o Computer graphics. Movies, video games, virtual reality, ...
o Security. Cell phones, e-commerce, voting machines, ...
o Multimedia. MP3, JPG, DivX, HDTV, face recognition, ...
o Social networks. Recommendations, news feeds,
advertisements, ...
o Physics. N-body simulation, particle collision simulation, ...
242-535 ADA: 0. Preliminaries
5
The Top 10 Algorithms of the 20th Century
• Ten algorithms having "the greatest influence on the
development and practice of science and
engineering in the 20th century".
o Dongarra and Sullivan
Top Ten Algorithms of the Century
Computing in Science and Engineering
January/February 2000
o Barry Cipra
The Best of the 20th Century: Editors Name Top 10
Algorithms
SIAM News
Volume 33, Number 4, May 2000
• http://www.siam.org/pdf/news/637.pdf
242-535 ADA: 0. Preliminaries
6
What are the Top 10?
• 1946: The Metropolis (Monte Carlo) Algorithm.
Uses random processes to find answers to problems
that are too complicated to solve exactly.
• 1947: Simplex Method for Linear Programming.
A fast technique for maximizing or minimizing a linear
function of several variables, applicable to planning
and decision-making.
• 1950: Krylov Subspace Iteration Method.
A technique for rapidly solving the linear equations
that are common in scientific computation.
242-535 ADA: 0. Preliminaries
7
• 1951: The Decompositional Approach to Matrix
Computations. A collection of techniques for
numerical linear algebra.
• 1957: The Fortran Optimizing Compiler.
• 1959: QR Algorithm for Computing Eigenvalues.
A crucial matrix operation made swift and
practical. Application areas include computer
vision, vibration analysis, data analysis.
• 1962: Quicksort Algorithm.
242-535 ADA: 0. Preliminaries
We will look at this.
8
• 1965: Fast Fourier Transform (FFT). It breaks down
waveforms (like sound) into periodic components.
Used in many different areas (e.g. digital signal
processing , solving partial differential equations,
fast multiplication of large integers.)
• 1977: Integer Relation Detection. A fast method for
finding simple equations that explain collections of
data.
• 1987: Fast Multipole Method. Deals with the
complexity of n-body calculations. It is applied in
problems ranging from celestial mechanics to
protein folding.
242-535 ADA: 0. Preliminaries
9
Some Algorithm Types
•
•
•
•
•
•
•
Simple recursive algorithms
Divide and conquer
Backtracking
Dynamic programming
Greedy algorithms
Brute force
Randomized algorithms
242-535 ADA: 0. Preliminaries
10
Simple Recursive
• A simple recursive algorithm:
o Non-recursive base case
o Recurs with a simpler subproblem
• Examples:
o Count the number of occurrence of an element in a tree
o Test if a value occurs in a list
o Fibonnaci number calculation
• Several of the other algorithm types use recursion in
more complex ways
242-535 ADA: 0. Preliminaries
11
11
Fibonnaci numbers
• The Fibonacci sequence:
o 1, 1, 2, 3, 5, 8, 13, 21, 33, ...
• Find the nth Fibonacci number:
fib(int n)
if (n <= 1)
return 1;
else return fib(n-1) + fib(n-2);
242-535 ADA: 0. Preliminaries
12
• Execution of fib(5):
242-535 ADA: 0. Preliminaries
lots of repeated work,
that only gets worse
for bigger n
13
Divide and Conquer
•
A divide and conquer algorithm :
o Divide the problem into smaller subproblems
o Combine the solutions to the subproblems into a solution
to the original problem
•
Examples:
o
o
o
quicksort
merge sort
binary search
242-535 ADA: 0. Preliminaries
14
Backtracking
• Backtracking algorithms use a depth-first recursive
search
o involves choice (non-determinism)
o backtracking means "go back to where you came from"
• Examples:
o search a maze
o color a map with no
more than four colors
242-535 ADA: 0. Preliminaries
15
Dynamic Programming
• A dynamic programming algorithm remembers past
results and uses them to find new results.
o multiple solutions exist; find the “best” one (the optimal one)
o the problen contains overlapping (repeated) subproblems
• solutions are stored and reused
242-535 ADA: 0. Preliminaries
16
Fibonacci numbers Again
• Finding the nth Fibonacci number involves lots of
repeated work (overlapping subproblems) that can
be avoided using memorization:
242-535 ADA: 0. Preliminaries
17
17
Greedy algorithms
• An optimization problem: find the best solution
• A “greedy algorithm” sometimes works well for
optimization problems, and is easy to code
• At each step:
o use the best solution you can get right now, without regard for
future steps
o You hope that by choosing a local optimum at each step,
you will end up with a globally optimum final solution
242-535 ADA: 0. Preliminaries
18
18
Example: Counting Money
• Count out some money using the fewest possible
notes and coins
• A greedy algorithm will take the largest possible
note or coin at each step
• Example: count out $6.39 using:
•
•
•
•
•
a $5 bill
a $1 bill
a 25¢ coin
a 10¢ coin
four 1¢ coins
242-535 ADA: 0. Preliminaries
// to make $6
// to make $6.25
// to make $6.35
// to make $6.39
19
Greedy Algorithms Can 'Fail'
• “Krons” money come in 1, 7, and 10 coins
• Count out 15 krons:
o A 10 kron piece
o Five 1 kron pieces, for a total of 15 krons
• This requires 6 coins, but a better solution is two 7
kron pieces and one 1 kron piece (3 coins)
• The greedy algorithm 'fails' because its final
solution is not the best (not globally optimal)
242-535 ADA: 0. Preliminaries
20
Brute Force
• A brute force algorithm tries all possibilities until a
satisfactory solution is found.
• Often, brute force algorithms require exponential
running time (very large time, so very slow)
• Various heuristics and optimizations can be used
o heuristic means “a rule of thumb”
o look for sub-optimal solutions (not the best), which can be
calculated more quickly than the best one
242-535 ADA: 0. Preliminaries
21
Randomized Algorithms
• A randomized algorithm uses a random number to
make a choice during the computation
o faster than calculating a choice
o the choice may be just as good
• Examples:
o Quicksort randomly chooses a pivot
o Factor a large number by
choosing random numbers
as possible divisors
242-535 ADA: 0. Preliminaries
22
Analysis of Algorithms
The theoretical study of algorithm performance and
resource usage.
Performance isn't the only important things for code:
• modularity
• user-friendliness
• correctness
• programmer time
• maintainability
• simplicity
• functionality
• extensibility
• robustness
• reliability
242-535 ADA: 0. Preliminaries
This subject isn't about
these things.
23
Why study Algorithm Analysis?
• It help us to understand algorithm scalability.
• Performance often draws the line between what
is feasible and what is impossible.
• Algorithmic mathematics provides a precise way to
talk about program behavior.
• The lessons of program performance generalize
to other computing resources.
242-535 ADA: 0. Preliminaries
24
Course Structure
• Mathematical induction, running time of
programs; growth of functions
• Divide-and-conquer; comparison and linear sorts
• Dynamic programming
• Greedy algorithms
• Elementary graph algorithms, minimum spanning
trees, shortest path problems, maximum flow
• String matching
• Computational geometry
• NP completeness; approximation algorithms
242-535 ADA: 0. Preliminaries
25
2. Meeting Times / Locations
• Monday
Friday
10:30 – 12:00
9:00 – 10:30
R101
R101
• I want to change these times to be
3 classes/week, each of 1 hour.
• Tell me your preferences.
242-535 ADA: 0. Preliminaries
26
3. Workload
• Mid-term exam:
35%
(2 hours)
45%
(3 hours)
o week 8
• Final exam:
o weeks 17-18
• Two exercises:
20% (2*10)
• weeks 7-8 and weeks 15-16
242-535 ADA: 0. Preliminaries
27
Non-Attendence Penalty
• I may take registration at the start of a class.
• If someone is not there, they lose 1%
(unless they have a good excuse).
• A maximum of 10% can be lost
o deducted from your final mark
242-535 ADA: 0. Preliminaries
28
4. Exercises
• The two exercises are worth a total of 20%
(each worth 10%).
• They will be maths problems and/or
algorithms to design/write.
242-535 ADA: 0. Preliminaries
continued
29
• Planned exercise times (which may change):
o ex. 1 in weeks 7-8
o ex. 2 in weeks 15-16
• Cheating will result in 0 marks.
o YOU HAVE BEEN WARNED!!
242-535 ADA: 0. Preliminaries
30
5. Course Materials
• All the handouts (and other materials) will be
placed on-line at
http://fivedots.coe.psu.ac.th/
Software.coe/242-535_ADA/
• Print 6 slides-per-page, grayscale, and bring to
class.
242-535 ADA: 0. Preliminaries
31
6. Books
• Introduction to Algorithms
Thomas Cormen, Charles Leiserson, Ronald
Rivest, Clifford Stein
McGraw Hill, 2003, 2nd edition
o mathematical, advanced, the standard text
o now up to version 3 (MIT)
o lots of resources online;
see video section
242-535 ADA: 0. Preliminaries
continued
32
• Algorithms
Robert Sedgewick, Kevin Wayne
Addison-Wesley, 2011, 4th ed.
o implementation (Java) and theory
o intermediate level
• Data Structures and Algorithms in Java
Robert Lafore
Sams Publishing, 2002, 2nd ed.
o Java examples; old
o basic level; not much analysis
242-535 ADA: 0. Preliminaries
33
Fun Overviews
• Algorithms Unlocked
Thomas H. Cormen
MIT Press, March 2013
• Nine Algorithms that Changed the Future
John MacCormick
Princeton University Press, 2011
o http://users.dickinson.edu/~jmac/9algorithms/
• search engine indexing, pagerank, public key
cryptography, error-correcting codes, pattern
recognition, data compression, databases,
digital signatures, computablity
242-535 ADA: 0. Preliminaries
34
• Algorithmic Puzzles
Anany Levitin, Maria Levitin
Oxford University Press, , 2011
• Algorithmics: The Spirit of
Computing
David Harel, Yishai Feldman
Addison-Wesley; 3 ed., 2004
(and Springer, 2012)
o http://www.wisdom.weizmann.ac.il/
~harel/algorithmics.html
242-535 ADA: 0. Preliminaries
35
• The New Turing Omnibus: Sixty-Six
Excursions in Computer Science
A. K. Dewdney
Holt, 1993
o 66 short article; e.g. detecting primes, noncomputable
functions, self-replicating computers, fractals, genetic
algorithms, Newton-Raphson Method, viruses
242-535 ADA: 0. Preliminaries
36
7. Videos
• MIT 6.046J / 18.410J Intro. to Algorithms, Fall 2005
o http://ocw.mit.edu/6-046JF05
• original course website for Cormen book
o http://videolectures.net/mit6046jf05_introduction_algorithms/
• video and slides side-by-side
o http://www.catonmat.net/
category/introduction-to-algorithms
• notes taken while
watching the videos
242-535 ADA: 0. Preliminaries
37
• Hi-tech Trek
Royal Institution Christmas Lectures 2008
o A hi-tech trek through the world of computer science by
Professor Chris Bishop; aimed at school kids.
• Lecture 1: Breaking the speed limit
• Lecture 2: Chips with everything
• Lecture 3: Ghost in the machine
• Lecture 4: Untangling the web
• Lecture 5: Digital intelligence
o http://richannel.org/christmas-lectures/
2008/
o I have copies on a DVD
242-535 ADA: 0. Preliminaries
38
8. Web Sites
• Algorithm Tutorials
o http://community.topcoder.com/tc?module=Static&
d1=tutorials&d2=alg_index
• Algorithmy
o http://en.algoritmy.net/
• brief explanations and code
• Algorithmist
o http://algorithmist.com/index.php/Main_Page
• explanations and code
• Wikipaedia page for an algorithm
o e.g. http://en.wikipedia.org/wiki/Quicksort
242-535 ADA: 0. Preliminaries
39