Amortized Analysis


It is used to analyze the average time required to perform each operation
involved in a sequence of data structure operation.
Probability is NOT involved. It analyzes the average performance of each
operation in the worst case.
The aggregate method

Compute the worst-cast time T (n ) of a sequence of n operations. The amortized
cost for each operation is T (n) / n .

Ex. Stack operations
PUSH ( S , X )
. POP (S )
MULTIPOP ( S , K )


The worst-case time for each operation is O(n) at the first glance.
However, the worst-case time for the sequence is also O(n), because the total #
of push() and pop() is n. therefore, the amortized cost for each stack operation is
O(1).

Ex. Incrementing a binary counter.
Starting form 0, we increments a binary counter for n times.
The worst-case time of INCREMENT is O(k) k=logn,
The least significant bit A0 is flipped n times
A1 is flipped n / 2 …..
A2 …..
n / 4 …...

An
…..
1
times
_______________________________________________
2n = (n)
Therefore, the amortized cost of INCREMENT is (1) .
.
The Accounting Method


Assign different changes to different operations. The difference between a
change and actual cost is credit.
The total credit in the data structure never becomes negative.
E.g., Stack operations
 Actual cost
 PUSH 1
 POP 1


 MULTIPOP
Amortized cost
 PUSH 2
min(k, s)
 POP 0
 MULTIPOP 0
The credit is the number of plates on the stack, which is always non-negative.
E.g., Binary counter
 Actual running time: the number of bits flipped.
 Amortized cost:

 Set a bit to 1: 2
 Set a bit to 0: 0
 Each INCREMENT sets only one bit to 1. Therefore, its amortized cost is 2.
The credit is the number of 1’s in the counter, which is never negative.
Ex. 18.2-1, 18.2-3.
The Potential Method

Define a potential function  that maps the target data structure to a value,
called potential.
The amortized cost operation i is
Cˆ  Ci   ( Di )   ( Di 1 ) ,
where C i is the cost of i ’th operation.
Therefore
n
n
 Cˆ   C
i 1
i
i 1
i
  ( Dn )   ( D0 ) .
To ensure the total amortized cost is an upper bound of the total cost, we
required that  ( Di )   ( D0 ) i  0

Ex. Stack Operations
 (D )  # of objects in stack D.
Thus,  ( D)  0   ( D0 )
The amortized cost of PUSH is
Ci   ( Di )   ( Di 1 )  1  1  2
The amortized cost of POP is
Ci   ( Di )   ( Di 1 )  1  1  0
The amortized cost of MULTIPOP is
Ci   ( Di )   ( Di 1 )  k 'k '  0

Ex. Incrementing a binary counter
 (D )  the # of 1’s in the counter D.
Suppose an INCREMENT resets t bits.
Therefore Ci  t  1 # of 1’s becomes bi 1  t  1
 ( Di )   ( Di 1 )  bi 1  t  1  bi  1  1  t
Cˆ i  t  1  (1  t )  2
Suppose initially there are b0 1’s
n
n
C   2  b
i 1
If
i
i 1
n
 b0  2n  bn  b0
b0  n , the total cost is (n)
NP-Completeness
What is an abstract problem?
A binary relation on a set of problem instances and a set of problem
solutions.
Problem instance
solution

Def. A problem is tractable if there exists an algorithm that solves it in
polynomial time in RAM machine

Def. A decision problem is the one that returns yes/no. If an optimization
problem can be solved efficiently, so can the corresponding decision
problem.

F.g The shortest path problem :
The optimization problem :<G,u,v,  >
PATH  The decision problem:<G,u,v,k>
 Does there exist a path between u and v where length is at most
k?


A computer algorithm that solves an abstract decision problem takes an
encoding of a problem instance as input. An encoding is an instance of
{0,1}*
We say an algorithm solves a problem in time O(T (n)) if when given a
problem instance i of length n ,the algorithm can provide the solution in
at most O(T (n)) time.

Def. A problem is polynomial-time solvable if there exists an algorithm to
solve it in at most O(n k ) time for some constant k

We assume binary encoding

Def A language L is a subset of  *  {0,1}*


empty string
empty language
L  * L
L1 * L2  {x1 * x 2 : x1  L1 and　x2  L2 }

Def A decision problem Q can be represented as a language
L  {x  * : Q( x)  1}
e.g.
The decision problem PATH has the following corresponding language
PATH { G, u, v, k : G (V , E ) is an undirected graph,
u, v  V , k  0 is an integer and there exists a path from u to v in
G where length is at most k }

Def. An algorithm A accepts a string x if A( x)  1
A( x)  0
……………
rejects …..

Def. The language accepted by an algorithm A is the set
L  {x  * : A( x)  1}

Def. A language L is decided by an algorithm A if every binary string is either
accepted or rejected by the algorithm.

Def. A language L is accepted in polynomial time by an algorithm A if for any
length-n string x  L , the algorithm accepts x in time O(n k ) for some
constant k

Def. P  {L  {0,1}* : there exists and algorithm A that decides L in
polynomial time }

Ex. Suppose there are n vertices in G, what is the input size of an instance of
PATH?
*assume we use adjacency matrix for representing G.
 E.g. 36.1-2
 E.g. 36.1-7 (complement) p=co-p

Def. An Hamiltonian cycle of an undirected graph G=(V, E) is a simple cycle that
contains each vertex in V, see Fig 36.1(a)
The formal language:
HAM-CYCLE= { G : G is a Hamiltonian graph }
Brute-force approach:
 Enumerate all possible permutations of V.
 Verify if any permutation a simple cycle.
Let n  G
The # of vertices is O ( n )
The # of permutations is ( n )! (2 n )
Therefore, this algorithm is NOT polynomial time

Def. A verification algorithm is a 2-argument A, where one argument is an
ordinary input string x and the other is a binary string y called certificate.
We say A verifies an input string x if there exists a certificate y . s.t.
A( x, y )  1 The language verified by A is L  x  {0,1}* : there exists
y  {0,1}　s.t. A( x, y )  1}
Intuition behind the verification algorithm:
A verification algorithm uses a certificate y to prove that x  L. In
HAM-CYCLE, x is a graph and y is a list of vertices.

The verification algorithm of HAM-CYCLE takes polynomial time
O ( n * n)

Def. A language L  NP iff there exists a polynomial time verification
algorithm A and constant C s.t. L  {x  {0,1}* : there exists a certificate y
w/ y  a x s.t. A( x, y )  1}

Theorem
P  NP
Proof:
Suppose L  P ,and A is the polynomial time algorithm that decides L
We can construct another verification A( x, y ) as follows:
<1> Execute A(x )
<2> if A( x)  1 return 1, and A( x)  0 return 0
regardless of the 2’nd parameter y

Def. Co-NP= {L : L  NP}

Ex.36.2-1, 36.2-2, 36.2-3, 36.2-8
Reducibility

Def. A language L1 is polynomial-time reducible to a language L2 denoted
L1  p L2 , if there exists a polynomial time computable function f:
{0,1}*  {0,1}* s.t. for all x  {0,1}* , x  L1 iff f ( x)  L2
The algorithm F that computes f is called the reduction algorithm.
Fig 36.3,36.4

Def. A language L is NP-Complete if
1. L  NP　,and
2. L   p L for every L  NP

Def. A language L is NP-hard if L   p L for every L  NP .]


If any NP-complete language  P then P  NP
If we can find the first NP-Complete problem L1 , then to prove a given
problem L2 to be a NP-hard, we only need to show L1  p L2

Def. The circuit satisfiability problem (The 1st NP-Complete Given a
Boolean circuit composed of problem).AND, OR, and NOT gates,
does there exist a truth assignment that causes the output of the circuit
to be 1. If so, this circuit is called satisfiable.

See Fig 36.6
CIRCUIT-SAT={<C>:C is a satisfiable Boolean combinational circuit}
 Lemma CIRCUIT-SAT is NP
Proof.
We can construct an algorithm A that takes an input circuit C and a truth
assignment t (certificate) as the input. Note that the size of truth assignment
is no more than linear to the size of c. A was t to compute c.
 Lemma CIRCUIT-SAT is NP-hard
Proof
Let L be any language in NP. Thus there must exist an algorithm A that verifies L
in polynomial time. Suppose algorithm A runs in T (n ) time on length-n
input. Obviously T (n)  O(n k ) for some constant k .We can construct the
reduction algorithm F as follows: The input of F include an instance of L,
x ,an certificate y ,the encoding of A, the memory and the registers.
Besides, let M be the combinational circuit that implements computer
hardware for one step execution. Thus w / T ( n) sequential concatenation
of M, the output of A appears at some place in memory.
When we are inquired whether an input x  L ,we can construct a
corresponding circuit as F, w / all input determined except y . In their
case,if F is satisfially, them x must be verified by A.
We next need to show F runs in O(n k ), n  x
F constitutes:
A : constant
x:n
y : O( n k )
M : O(n k ),（w /　O(n k ) input )
# of M : O(n k )

Lemma:
If L is a language s.t. L   p L for some L  NPC
Then L is NP-hard. Moreover, if
L  NP then L  NPC
A Boolean formula consists of
1. Boolean variables
2. Boolean connectives: ,, , , 
3. parentheses
A Boolean formula is a satisfiable formula if there exists a truth assignment
that satisfies their formula
SAT= {  :  is a satisfiable Boolean formula}

e.g.
  (( x1  x2 )  ((x1  x3 )  x4 ))  x2
For x1  0, x2  0, x3  1, x4  1, it is satisfiable
Therefore,   SAT
Suppose the formula  has n variables and   O(n k ) for some k
Theorem SAT  NPC


Lemma 1 SAT  NP
The verification algorithm uses a truth assignment as the certificate. It verifies
the formula by evaluating the expression to 1. This task is easily doable in
polynomial time.

Lemma 2 SAT  NP-hard
We only need to show CIRCUIT_SAI  p SAT
If seems straight forward to convert to a circuit to a Boolean variable.
However , this reduction may not be polynomial due to the shared subformulas.
A more clever way has to be porposed.


Each gate is replaced by output-wire  conjunction
Each wire is assigned a variable x
See the mapping of Fig 36.8 in P.942
It is easy to see that the circuit C is satisfiable iffthe formula  is satisfiable.