COMP250:
Mid-termExamReview
Lecture19
JérômeWaldispühl
SchoolofComputerScience
McGillUniversity
Review-Recursivealgorithms
•  Towritearecursivealgorithm:
–  Findhowtheproblemcanbebrokenupinoneormore
smallerproblemsofthesamenature
–  Rememberthebasecase!
•  Usually,beUerrunningVmesareobtainedwhenthe
sizeofthesubproblemsareapproximatelyequal
–  power(a,n)=a*power(a,n-1)⇒O(n)
–  power(a,n)=(power(a,n/2))2⇒O(logn)
•  Fibonnaci,BinarySearch,MergeSort…
MergeSort
4
Divide
4
3
2
3
4
2
3
3
1
2
4
1
1
Merge
1
1
2
3
4
2
Recursivealgorithms&
inducVons
•  YoucanseeanalgorithmasafuncVon.
•  Toprovethecorrectnessofarecursivealgorithm,wecan
useasimilartechniquetotheoneusedtoprove
mathemaVcalinducVons.
•  Butweshouldalsoprovethatouralgorithmterminates!
InducVonproofs
•  ToprovethataproposiVonP(n)holdsforalln≥a:
–  Basecase:
ProvethatP(a)holds
–  Induc0onsteponn:
InducVonHypothesis:AssumeP(n)holds
ProvethatI.H.impliesthatP(n+1)holds
GeneralizedinducVonproofs
ToprovethataproposiVonP(n)holdsforalln≥a:
•  Basecase:
ProvethatP(a),P(a+1)...(asmanyasneeded)hold
•  Induc0onsteponn:
Induc&onHypothesis:AssumeP(k)holdsforallk≤n
ProvethatinducVonhypothesisimpliesthatP(n+1)holds
InducVonproofs:Example
Claim:forallposiVveintegersk,n,wehave(1+k)n≥1+kn.
Proof:
•  Basecase:n=1,(1+k)1≥1+k
•  InducVonhypothesis:forallkandagivenn,wehavewe
have(1+k)n≥1+kn
•  InducVonstep(showthatthenitistrueforn+1):
(1+k)n+1=(1+k)(1+k)n
≥(1+k)(1+kn)
=1+kn+k+k2n
=1+k(n+1)+(1+k)
≥1+k(n+1)
Provingalgorithms
AF(int x) {
f = 2;
while ( x > 1 ) {
if ( x % f == 0 ) {
System.out.println(f);
x = x / f;
} else {
f++;
}
}
}
IteraVvealgorithm
⇒ Loopinvariant
FM(int[] A, int l, int r) {
if (l<r) {
m=⎣(l+r)/2⎦
return max(
FM(A,l,m),
FM(A,m+1,r));
} else {
return A[l];
}
Recursivealgorithm
⇒ InducVonproof
Provingalgorithms
FM(int[] A, int l, int r) {
if (l<r) {
m=⎣(l+r)/2⎦
return max(FM(A,l,m),FM(A,m+1,r));
} else { return A[l]; }
Pre-condiVon:0≤l≤r≤A.length
Post-condiVon:returnmaxvaluestoredinA[l:r]
Basecase:themaxofanarrayofsize1isthevaluestored.
InducVonHypothesis:foralllandrsuchthat|r-l|≤n,FM(A,l,r)
returnsthemaxinA[l:r]
•  InducVonstep:forall|r-l|=n+1,then
FM(A,l,r)=max(FM(A,l,m),FM(A,m+1,r))
•
•
•
•
Provingalgorithms
FM(int[] A, int l, int r) {
if (l<r) {
m=⎣(l+r)/2⎦
return max(FM(A,l,m),FM(A,m+1,r));
} else { return A[l]; }
•  InducVonHypothesis:foralllandrsuchthat|r-l|≤n,FM(A,l,r)
returnsthemaxinA[l:r]
•  InducVonstep:Assume|r-l|=n+1.
FM(A,l,r)=max(FM(A,l,m),FM(A,m+1,r))
(I.H.)FM(A,l,m)&FM(A,m+1,r)returnmaxofA[l,m]&A[m+1,r]
SincemaxofA[l,r]isinA[l,m]orA[m+1,r],thenFM(A,l,r)returns
maxofA[l:r]
•  TerminaVon:Ateveryrecursivecall,|r-l|decreasesstrictlyunVl
|r-l|=0,whichisthebasecase.
LoopInvariants
AF(int x) {
f = 2;
while ( x > 1 ) {
if ( x % f == 0 ) {
System.out.println(f);
x = x / f;
} else { f++; }
}
}
Invariant:
x=thenumberthatremainstobefactor
andwehaveremovedallfactors<f
Loopinvariant
•  Ini0aliza0on:xcontainsallfactors
•  Maintenance:Atthebeginningoftheloopweremoved
allfactors<f.AmeroneiteraVonoftheloop,iffisa
factorofx,thenweremoveitanddonotincreasef.
Otherwise,wecansafelyincrementfandxissuchthat
weremovedallfactors≤f.
•  Termina0on:Thenumberthatremainstobefactored=1
andwehaveremovedallfactorsincludingthelargest
factor(I.e.:wehavefoundallthefactors)
Discussion
•  DifficultyistoidenVfyagoodloopinvariant.Theproofis
usuallysimpler(iniValizaVon,maintenance,terminaVon).
•  Aloopinvariantcanbeanypredicateaslongasithelpsto
provethattheloophelpstosolvetheproblem.
•  Withrecursivealgorithm,idenVfyingthepropertywewant
toproveiseasierbecauseitisthepost-condiVon.
Invariantvs.InducVon
IteraVvealgorithm
Recursivealgorithm
Loopinvariant
⇔
InducVonhypothesis
IniValizaVon
⇔
Basecase
Maintenance
⇔
InducVonstep
TerminaVon
⇔
TerminaVon
RunningVme
•  PrimiVveoperaVons
–  RunningVmeisconstant,indep.ofproblemsize
•
•
•
•
•
•
Assigningavaluetoavariable
Callingamethod;returningfromamethod
ArithmeVcoperaVons,comparisons
Indexingintoanarray
Followingobjectreference
CondiVonals
•  RunningVme≡NumberofprimiVveoperaVons
•  Loops:SumtherunningVmeofeachiteraVon
•  findMin,inserVonSort
Recurrences
•  Forrecursivealgorithms,weexpressthe
runningVmeT(n)foraninputofsizenasa
funcVonofT(a)forsomea<n
•  Example:
–  Binarysearch:T(n)=T(n/2)+a
–  MergeSort:T(n)=2T(n/2)+cn
Solvingrecurrences
•  Solvingrecurrence≡giveexplicitformulaforT(n)
•  SubsVtuVonmethod:
–  ReplaceoccurrencesofT()bytheirvalue
–  RepeatunVlpaUernemerges
•  ProvebyinducVonthatguessiscorrect
Example
SoluVonofT(n)=2T(n/2)+cn,T(1)=0forn≥0apowerof2?
T(n)=2(2T(n/4)+cn/2)+cn
=22T(n/4)+2cn
=22(2T(n/8)+cn/4)+2cn
=23T(n/8)+3cn
…
=2kT(n/2k)+kcn
Wehaven=2k,thusk=log2(n) and we replace k in T(n).
T(n)=2log(n)T(1)+log2(n)cn=cnlog(n)
ThenprovebyinducVon...
Proof by induction
Proposition. If T (n) satisfies the following recurrence, then T (n) = n log2 n.
T(n) =
0
if n = 1
2 T (n / 2) + n
otherwise
assuming n
is a power of 2
Pf 2. [by induction on n]
・Base case: when n = 1, T(1) = 0.
・Inductive hypothesis: assume T(n) =
・Goal: show that T(2n) = 2n log2 (2n).
n log2 n.
T(2n) = 2 T(n) + 2n
= 2 n log2 n + 2n
= 2 n (log2 (2n) – 1) + 2n
= 2 n log2 (2n). ▪
10
Big-OhnotaVon
g(n)isO(f(n))iffthereexistconstantscandn0such
thatg(n)≤cf(n)foralln≥n0
•  f(n)isΩ(g(n))iffg(n)isO(f(n))
•  f(n)isΘ(g(n))ifff(n)isO(g(n))andf(n)isΩ(g(n))
Example
ShowthatFib(n)isO(2n)
Proof:WewillshowbyinducVonthatforalln≥0,Fib(n)<2n
•  Basecase:n=0,Fib(0)=0<20=1;n=1,Fib(1)=1<21=2;
•  InducVonhypothesis:forallk≤ nFib(k)<2k.Fib(n+1)?
•  InducVonstep:
Fib(n+1)=Fib(n)+Fib(n-1)
<2n+2n-1
<2n+2n=2n+1
UsingthepreviousdefiniVon,wesetc=1andn0=0,then
Fib(n)isO(2n).
Example2
•  Previously,weusedtherecursiveformulaT(n)=2T(n/2)+n,
withT(1)=0,tocharacterizethenumberofoperaVonsof
MergeSort.
•  WeshowedthatT(n)=nlog2(n)
•  Then,MergeSortrunsinO(nlog2(n))
Big-OhnotaVon
Hierarchy of big-Oh classes
Big-OhnotaVon
Simplification rules
•  If f1 (n) ∈ O(g(n)) and f2 (n) ∈ O(g(n)) then
f1 (n) + f2 (n) ∈ O(g(n))
•  If
f1 (n) ∈ O(g(n)) then k. f1 (n) ∈ O(g(n))
•  If
f1 (n) ∈ O(g(n)) and
f2 (n) ∈ O(h(n)) then
f1 (n). f2 (n) ∈ O(g(n).h(n))
…
LogidenVVes
•  log(ab)=log(a)+log(b)
•  log(an)=nlog(a)
•  loga(n)=logb(n)/logb(a)
•  alogb(n)=nlogb(a)
RunningVmeofaForloop
for (i=1; i<N; i=i*2) { … }
Whatistherunning&meofthisloop?
ValueofiamerkiteraVons:2k
Wehavei<N=>2k<N=>k<log2(N).
Thereislessthanlog2(N)iteraVons.
TherunningVmeofthisloopisO(log(n)).
#operaVons:O(log(n))O(1)=O(log(n))
Something
runningin
O(1)
AbstractDataTypes
•  ImplementaVon:Arraysorlinked-lists
•  BasicoperaVons
–  getFirst(),get(n),getLast()
–  removeFirst(),removeLast(),remove(o)
–  addFirst(o),addLast(o),add(o)
–  empty(),size()
•  Advantagesanddisadvantagesoverarrays
•  Stack,Queues,deques,rotaVngarrays
OperaVonsondequeswithArray
•  Queue() {
L = new Array[N];
head = tail = -1;
}
•  isEmpty() {
if ((head == -1)&&(tail==-1)) return true;
else return false;
}
•  isFull() {
if ((head - tail % N) == 1) return true;
else return false;
}
OperaVonsondequeswithArray
•  Enqueue(o) throw Exception {
if ( isFull() ) { throw new Exception(“Full stack”) }
if ( isEmpty() ) { head = tail = 0; }
else { tail = ( ( tail + 1 ) % N ); }
L[tail] = o;
}
•  Dequeue() {
if ( isEmpty() ) { throw new Exception(“Empty stack”) }
Object o = L[head];
if (((head – tail) % N) = 1) { head = tail = -1; }
else { head = ( ( head + 1 ) % N ); }
return o;
}