Chapter 1
Introduction
Mathematics Review
Sections 1.1, 1.2
1
Outline
• Why does program execution time for large inputs
matter?
• Basic mathematical background
2
Selection Problem
• Find the kth largest number from a group of N
numbers
– How would you solve this?
• Algorithm 1:
– Sort the N numbers and pick the kth one.
– Easy to implement, but
• Sorting requires many comparisons
• Much work comparing elements having no chance to be at
position K
– To sort N elements, need N log2(N) comparisons/swaps in
general
– An unsorted data set with 10,000,000 elements and 1,000
swaps/sec will take around 2 hours
3
Selection Problem (Cont’d)
• Algorithm 2: (Better)
– Sort first K elements in the array.
– Then insert elements (K+1) to N, discarding the smallest
element each time.
– Then pick the kth element.
• What if N=10 million and K=5,000,000?
– How long do you think this would take?
• Both algorithms are impractical.
• A better algorithm can solve this in a second!
4
Mathematics Review
•
•
•
•
•
Exponents
Logarithms
Series
Modular arithmetic
Proof techniques
5
Exponents
X A X B  X A B
XA
A B

X
XB
X 
A B
 X AB
X N  X N  2 X N  X 2N
2 N  2 N  2 N 1
6
Logarithms
• All logarithms are to the base 2, unless otherwise
specified
• Definition 1.1
– XA = B if and only if logxB = A
7
Properties of logarithms
• Theorem 1.1 (base-change)
–
log A B 
log C B
;
log C A
A, B, C > 0, A  1
• Proof: Let X = logCB, Y = logCA, and Z = logAB
–
–
–
–
Then: CX = B, CY = A, and AZ = B
Implies: B = CX = (CY)Z = CYZ
Hence: X = YZ
And: Z = X/Y
8
Logarithms (contd.)
• Theorem 1.2
– log(AB) = log A + log B;
A, B > 0
• Proof: (Assume base 2)
– Let: X = log A, Y = log B, and Z = log AB
– Hence: 2X = A, 2Y = B, and 2Z = AB
– Combining the above: 2Z = AB = 2X2Y = 2(X+Y)
– Implies: X + Y = Z
9
Logarithms (contd.)
• More results
–
–
–
–
–
–
–
log(A/B) = log A – log B
log(AB) = B log A
log X < X for all X > 0
log 1 = 0
log2 2 = 1
log2 1024 = 10
log2 1,048,576 = 20
10
Geometric Series
N
•
2
•
A N 1  1
A 

A 1
i 0
i
 2 N 1  1
i 0
N
•
i
N
If 0 < A < 1,
• If N ->

i
A
 
i 0
1
1 A

, we have
 Ai 
i 0
1
1 A
• How??
11
Arithmetic Series
•
N ( N  1) N 2
i


2
2
i 1
N
• How about 2+5+8+…+(3k-1) ?
12
Modular Arithmetic
• We say that A is congruent to B modulo N
– Written as A  B (mod N)
– If N divides (A-B)
• In other words, the remainder is the same if either A
or B are divided by N
• E.g.
–
81  61  1
(mod 10)
• Similar to equality, if A  B (mod N), then
– A  C  B  C (mod N), and AD  BD (mod N)
13
Proof techniques
• Two common proof techniques in data structure and
algorithm analysis (and in CS, in general)
– Proof by induction
– Proof by contradiction
– Another common technique
• Proof a statement false with a counterexample
14
Proof by Induction
• Given a theorem
• First prove a base case
– Show the theorem is true for some small degenerate values
• Next assume an inductive hypothesis
– Assume the theorem is true for all cases up to some limit k
• Then prove that the theorem holds for the next value
(k+1)
15
Proof by Induction - example
• Fibonacci Series
– F0 = 1, F1 = 1, Fi = F(i-1) + F(i-2), for i>1
• Show that
– Fi < (5/3)i,for i>0
• Base case:
– F1 = 1 < 5/3
– F2 = 2 < (5/3)2=25/9
• Inductive Hypothesis
– Assuming: Fi < (5/3) i , i = 1, 2, ..., k
16
Proof by Induction - example (contd.)
• Now prove that Fk+1 < (5/3)k+1
• From definition of Fibonacci Sequence
– Fk+1 = Fk + Fk-1
• Using inductive hypothesis
–
Fk+1 < (5/3) k + (5/3) k-1
= (5/3) k+1 [ 3/5 + (3/5)2]
= (5/3) k+1 [24/25]
< (5/3) k+1
17
Other types of proofs
• Disprove with a counter-example
– The statement Fk  k 2 is false in the Fibonacci series
– Proof: F11 = 144 > 112 = 121
• Proof by contradiction
–
–
–
–
Initially assume that the theorem is false
Then show that some known property would be false as well.
Example: “There is an infinite number of prime numbers”
Proof:
• Assume the theorem is false (so there are only finite prime)
• Let P1, P2, ..., Pk be all the primes in increasing order.
• Let N = P1P2   Pk + 1,N is > Pk , so it is not a prime
• But it is also not divisible by any of the listed primes,
contradicting the factorization of integers into primes.
• We reach a contradiction
18