COMP3251
Lecture 1: Introduction
Welcome to COMP3251
Instructor:
- HUANG, Zhiyi (zhiyi@cs.hku.hk)
- 423 Chow Yei Ching Building
- Consultation Hours: Tuesday, 4:00pm to 5:00pm
- Other time slots may be available by appointment
Tutors:
- SIU, Chak Fung Je (je siu1@hku.hk)
- HU, Qi (hooch@connect.hku.hk)
- LOU, Yuke (louyuke@connect.hku.hk)
- Consultation Hours: TBD
2
ff
ff
COMP3251
Zhiyi Huang
What are algorithms?
An algorithm is a set of precise step-by-step rules for
solving a speci c computational problem.
Examples: Multiplying two numbers, say, 1234 and 5678.
In primary school, we learnt this as
“the correct way” of calculating the
multiplication of two numbers.
In this course:
- How good is this algorithm?
- Can we do better?
⨉
5 6 7 8
1 2 3 4
2
1 7
1 1 3
5 6 7
2 7 1 2
0 3 4
5 6
8
7 0 0 6 6 5 2
4
fi
COMP3251
Zhiyi Huang
An alternative multiplication algorithm
Examples: Multiplying two numbers, say, 1234 and 5678.
1) Partition (1234) into two numbers (12) and (34).
2) Partition (5678) into two numbers (56) and (78).
3) Multiply (12) and (56) and get (672).
4) Multiply (34) and (78) and get (2652).
5) Multiply (12)+(34)=(46) and (56)+(78)=(134) and get (6164).
6) Subtract the rst two numbers from the third one and get
(6164)-(672)-(2652)=(2840).
7) Sum up (6720000) (1st number padded with four 0’s),
(2652) (2nd number), and (284000) (4th number padded
with two 0’s), and get
(6720000) + (2652) + (284000) = (7006652).
5
fi
COMP3251
Zhiyi Huang
An alternative multiplication algorithm
Examples: Multiplying two numbers, say, 1234 and 5678.
1) Partition (1234) into two numbers (12) and (34).
2) Partition (5678) into two numbers (56) and (78).
3)
(12)that
andyou
(56)cannot
and get
(672).
It isMultiply
expected
make
any sense of the
algorithm
thisand
point,
than(2652).
the nal answer is correct.
4)
Multiplyat(34)
(78)other
and get
•5) After
a few
classes, youand
will(56)+(78)=(134)
be able to make
sense
of the
Multiply
(12)+(34)=(46)
and
get (6164).
alternative multiplication algorithm.
6) Subtract the
• By the end of the semester, you will be able to design
algorithms like this one (and other fancy techniques too).
7) Sum up (6720000) (1st number padded with four 0’s),
(2652) (2nd number), and (284000) (4th number padded
with two 0’s), and get
(6720000)
6
fi
fi
COMP3251
Zhiyi Huang
Why study algorithms?
Useful for all areas in CS
networking
database
machine learning
computer vision
COMP3251
8
Zhiyi Huang
Driving force behind tech innovations
Page Rank
COMP3251
9
Zhiyi Huang
Driving force behind tech innovations
Google Map Driving Direction
COMP3251
10
Zhiyi Huang
Driving force behind tech innovations
Online Advertisement
COMP3251
11
Zhiyi Huang
Online Advertisement is Big!
COMP3251
12
Zhiyi Huang
Novel “lens” for areas outside CS
Auctions
COMP3251
Markets
13
Zhiyi Huang
What you will get from this course?
• Become a better programmer
• Sharpen your analytical and mathematical skills
• Ace the algorithm questions in your job interviews
• Think “algorithmically”
COMP3251
14
Zhiyi Huang
Textbook
Lecture notes:
- Self-contained
- Available on Moodle before each lecture
Textbook (required supplementary readings):
- Algorithms
S. Dasgupta, C. H. Papadimitriou, and U. Vazirani
Other resources (optional):
- Algorithm Design
E. Tardos and J. Kleinberg
example
- Introduction to Algorithms
T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein
- Online course by Tim Roughgarden
COMP3251
15
Zhiyi Huang
Course Topics
Algorithm design paradigms:
- Divide and conquer (DPV Ch. 2)
- Greedy (DPV Ch. 5)
- Dynamic programming (DPV Ch. 6)
Graph algorithms:
- Graph decomposition (DPV Ch. 3)
- Shortest path (DPV Ch. 4)
- Max ow (DPV Ch. 7.2, supplementary materials will be
available on Moodle)
Complexity theory:
- NP-completeness and reduction (DPV Ch. 8)
fl
COMP3251
16
Zhiyi Huang
Assessment
• Homework assignments (40%)
- 5 assignments
- Count best 4 out of 5
- No late submission: If you happen to be extremely busy
before one of the deadlines, that’s your worst 1 out of 5
- About 4 problems in each assignment
• Midterm (10%)
- 90 min, semi-open book (2 sheets of A4), in class
- Most likely in the week after the reading week
• Final (50%)
- 3 hours, 4-5 questions, semi-open book (2 sheets of A4)
COMP3251
17
Zhiyi Huang
Collaboration Policy for Homework Assignments
YES: Discuss with friends, acknowledge them,
write your own solutions.
NO: Copy solutions word-for-word or with minor modi cations
(from your friends, from the Internet, from past-year, etc.)
Note: I strongly recommend that you try to complete the
homework assignments on your own.
18
Zhiyi Huang
fi
COMP3251
3251 (Regular class) vs. 3252 (Advanced Class)
• COMP3251 focuses on
- How basic algorithms work
- Applications of algorithms
• COMP3252 emphasizes more on
- Problem solving skills
- Mathematical proofs
COMP3251
19
Zhiyi Huang
Rest of this lecture:
Review of basic running time analysis
Basics of running time analysis
Goal: Given an algorithm A, upper bound its running time.
Example: We may want to prove that for any input of size n,
algorithm A runs for at most 5 n2 + 7 n + 8 steps.
But:
• It is often very di cult to get a precise bound.
• The precise bounds are often meaningless.
- We generally count the number of steps in a pseudo-code
speci cation of an algorithm.
- The number of steps will be di erent if we implement the
algorithm in di erent high-level programming languages.
21
ff
ffi
ff
fi
COMP3251
Zhiyi Huang
Big-O notation
Goal: Given an algorithm A, upper bound its running time.
The main mathematical tool: Asymptotic (rough) analysis for
comparing non-negative functions, say, f(n) and g(n).
Intuitively
Notation
Read as
De nition
f(n) “is smaller than” g(n)
f(n) = O(g(n))
f(n) is order g(n)
limn→∞ f(n)/g(n) ≤ c for some constant c
f(n) ≤ c g(n) when n is large enough
It is a rough analysis because:
• we do not mind what c is (as long as it is a constant);
• we do not insist the inequality holds for all n,
but it should hold for all large enough n.
fi
COMP3251
22
Zhiyi Huang
Asymptotic bounds for some common functions
Polynomial:
For any polynomial p(n) = ad nd + ad-1 nd-1 + … + a1 n + a0
where ad > 0, and ai ≥ 0 for d > i ≥ 0, we have p(n) = O(nd).
Why?
- ni ≤ nd for any integer 0 ≤ i < d
- So p(n) ≤ (ad + ad-1 + … + a1 + a0) nd
- In fact, we can show that p(n) ≤ (ad + 1) nd for su ciently
large n, but the above inequality is good enough.
23
Zhiyi Huang
ffi
COMP3251
Asymptotic bounds for some common functions
Logarithmic vs. polynomial:
For any b > 1 and any d > 0, we have logb n = O(nd).
n0.2
ln n
COMP3251
24
Zhiyi Huang
Asymptotic bounds for some common functions
Logarithmic vs. polynomial:
For any b > 1 and any d > 0, we have logb n = O(nd).
Polynomial vs. exponential:
For any r > 1 and any d > 0, we have nd = O(r n).
COMP3251
25
Zhiyi Huang