Algorithmic Problems
Zeph Grunschlag
1
Agenda
Viewing algorithms as languages
Algorithmic Decision Problems

ATM, ACFG , AREX, ETM, … etc.
2
Knowing the Unknowable
Arrived at important juncture in course. Now
we will study one of the fundamental
questions in Computer Science:
What is the ultimate power of
computational machinery?
Of equal importance in human quest to
understand self:
What are the impenetrable limits of
human knowledge?
3
Knowing the Unknowable
Accepting the Church-Turing thesis implies that
these are the same question! Church-Turing
thesis is usually recognized as applying to
human computations, and therefore human
thought processes.
Amazing Fact: It is possible to give precise
examples of problems which are beyond the
reach of mechanical computation as well as
human intellect. We in fact “know the
unknowable.”
4
Decision Problems
A decision problem is a set of yes/no
questions, each called an instance. For the
decision problem to be interesting, the set of
questions should be infinite.
EG: “Does z equal the sum of x and y ?” is a
good example. A computer program that can
give the correct solution for any possible x,y,z
intuitively must have figured out how to add
numbers.
EG: “Does 8 equal the sum of 3 and 5?” is a
bad example. Could easily write program for
this instance without knowing how to add! 5
Decision Problems
Q: How about the problem “Does God
Exist?” Can a computer solve this
problem?
6
Decision Problems
A: In the way defined above, the answer is
yes! We give two computer programs, both
of which accept the input “Does God exist?”
A(String question){
return “No!”
}
B(String question){
return “Yes!”
}
One of these programs is correct! Therefore,
that computer program has solved the
problem of deciding if God exists!
Q: Any misgivings?
7
Decision Problems
A: Many possible misgivings. Here’s one:
“Solution” to existence of God surely not what
is meant by solving the problem. We
expected the solution to be arrived at by
careful consideration of physical evidence, or
by theological arguments, or some other
intricate process.
CS viewpoint: Any algorithm which works only
on a finite set of inputs is considered
meaningless, as could always program a lookup table which gave correct solutions without
8
using any insight.
Decision Problems
So to solve the deity existence problem would
have to give a more general solution. EG:
Given: A complete description of a physical
universe and a complete description of a
deity.
Decide: Does the deity necessarily exist for the
given universe?
Now there are infinitely many possible inputs,
so cannot just use a look-up table and this is
a meaningful problem, though hopeless…
9
Decision Problems as
Languages
To study what it means for a decision problem to
be solvable, need a method of standardizing
such problems and making them easily
understood by computers. Can’t just come
over to a computer and ask “Does 2 + 2 = 4?”
and even if could, dealing with voice
recognition, natural language processing, etc.
takes us far afield to topics irrelevant to
answering “is the problem solvable?”
Solution: represent each instance by a string:
Thus instance is the string “Does 2 + 2 = 4?” or
more simply “2+2=4”.
Q: Define solvability language theoretically? 10
Decision Problems as
Languages
A: A decision problem P is solvable if its encoding
by strings admits a computer program which
inputs such encodings, always halts, and
always outputs the correct answer.
Furthermore, the set of encodings should be
computable in its own right.
Language theoretically: Let S be the alphabet in
which the encoded instances of P exist. Let E
be the set of encoded instance and let L be the
set corresponding to the YES instances.1 Then
P is said to be decidable (or algorithmically
solvable or computable) if L is decidable.
11
Decision Problems
Sensible Encodings
Some common sense is needed: The
encoding scheme should be sensible but
there is no way to define what sensible
means.
EG: “Is p a prime number?”


Encoding scheme which results in the
positive instances L = {2,3,5,7,…} is
sensible.
Encoding scheme which results in the
positive instances L = {1,4,6,8,9,10,…} is
not!
12
Decision Problems
Sensible Encodings
Sensible encoding of sensible decision problem
would never result in an undecidable
encoding set E but condition still needed so
don’t solve impossible problems unwittingly.
EG: “Does a given Java program with no
infinite loops print out “I’m not loopy!” on
some input?”
Natural encoding set E is over Unicode
alphabet, and is just the set of Java programs
with no infinite loops. But we’ll see that E is
itself undecidable so second part becomes
13
meaningless.
Decision Problems
Basic Problems
Before tackling fundamental decision problems of
CS, start introspectively by considering if there
are algorithms for…
…telling if a given string is accepted by a given
language of a fixed type C such as DFA’s,
NFA’s, PDA’s etc. Acceptance Problem AC
…telling if a given language of type is C empty.
Emptiness Problem EC
…telling if two given languages of the same
type C are equal. Equivalence Problem EQC
Each in turn, is encoded by a language.
14
Decision Problems
Encoding Example
Let’s see how ADFA may be encoded.
ADFA is the problem of deciding
Given: A DFA M and a string x.
Decide: Does M accept x ?
The language L of interest is the language of
encodings of pairs (M, x ) where M accepts x.
Encoding is denoted using the angled braces
notation “<M, x >”. In practice, will never
really work with encodings as too unwieldy.
However, for once it’s worth to see how this
might actually be carried out.
15
Decision Problems
Encoding Example
Consider the DFA:
a
0,1
0,1
b
Q: How can you represent this by a
string?
16
Decision Problems
Encoding Example
A: DFA’s are 5-tuples,
so just write down the
5 tuple, spelling out all
details. Therefore
<M > =
a
0,1
0,1
b
({a,b},{0,1},{(a,0,b),(a,1,b),(b,0,a),(b,1,a)},a,{a})
17
Decision Problems
Encoding Example
A: DFA’s are 5-tuples,
so just write down the
5 tuple, spelling out all
details. Therefore
<M > =
a
0,1
0,1
b
({a,b},{0,1},{(a,0,b),(a,1,b),(b,0,a),(b,1,a)},a,{a})
Q
18
Decision Problems
Encoding Example
A: DFA’s are 5-tuples,
so just write down the
5 tuple, spelling out all
details. Therefore
<M > =
a
0,1
0,1
b
({a,b},{0,1},{(a,0,b),(a,1,b),(b,0,a),(b,1,a)},a,{a})
S
19
Decision Problems
Encoding Example
A: DFA’s are 5-tuples,
so just write down the
5 tuple, spelling out all
details. Therefore
<M > =
a
0,1
0,1
b
({a,b},{0,1},{(a,0,b),(a,1,b),(b,0,a),(b,1,a)},a,{a})
d
20
Decision Problems
Encoding Example
A: DFA’s are 5-tuples,
so just write down the
5 tuple, spelling out all
details. Therefore
<M > =
a
0,1
0,1
b
({a,b},{0,1},{(a,0,b),(a,1,b),(b,0,a),(b,1,a)},a,{a})
q0
21
Decision Problems
Encoding Example
A: DFA’s are 5-tuples,
so just write down the
5 tuple, spelling out all
details. Therefore
<M > =
a
0,1
0,1
b
({a,b},{0,1},{(a,0,b),(a,1,b),(b,0,a),(b,1,a)},a,{a})
F
22
Decision Problems
Encoding Example
A: DFA’s are 5-tuples,
so just write down the
5 tuple, spelling out all
details. Therefore
<M > =
a
0,1
0,1
b
({a,b},{0,1},{(a,0,b),(a,1,b),(b,0,a),(b,1,a)},a,{a})
Encoding <M,w > amounts to just adding
the comma and the input string w.
Q: What’s the alphabet of encoding lang. 23E ?
Decision Problems
Encoding Example
A: TOO BIG! The number of letters in
alphabet and number of states is
unbounded. Using different characters for
each state/letter would require an infinite
alphabet if want to be able to encode
every possible DFA! Instead, should fix
some alphabet (e.g. binary or ASCII) and
refer to states/letters using numbers.
Thus the state a is denoted by 1, since it’s
the first state, the state b by 2 and so on…
24
Decision Problems
What You Really Do!
After going to all this effort to define decision
problems in terms of languages, Turing
machines, encodings and so on, in practice,
we use none of the above. The point was to
argue that in principle this could be done, and
remind ourselves that this ought to be
achievable, when wondering whether a not
purported pseudocode is really an algorithm.
Sipser’s approach tries to maintain the veneer
of this approach. I find this to be artificial at
times so will just work directly with the
natural data structure inherited by the
25
problem.
ADFA
For example, the easiest way to view a DFA is
as a labeled graph. Encoding the DFA by a
string and trying to apply some graph
theoretic algorithm to the representative
string is overly burdensome. You basically
have to recreate the whole graph on the TM
tape, which in principle can be done, but in
practice is silly, especially since the starting
point was a graph!
Q: How do you solve ADFA ?
26
ADFA
A: Just follow the path labeled by the input
string from the start state. Accept if end was
in F. More precisely.
DFAaccept(DFA M, String x1 x2 x3 … xn )
State q = q0 //q0 defined by M’s 5-tuple
for(i = 1 to n)
q = d(q,xi) // d defined by M’s 5-tuple
if(q  F)
// F defined by M’s 5-tuple
return ACCEPT
else
return REJECT
27
ANFA
Q: How about ANFA?
28
ANFA
A: Just determinize first and use solution to
ADFA.
NFAaccept(NFA N, String x1 x2 x3 … xn )
DFA M = determinize(N) //Subset Constr’n
return DFAaccept(M, x1 x2 x3 … xn )
Q1: What’s the running time?
Q2: What’s not ideal about this algorithm?
Q3: What’s the fix?
29
ANFA
A1: O(2|N|+n) since determinizing might
involve all the subsets of the states of N so is
exponential, but once have determinized,
simulating the DFA is O(n).
Q2: The problem is the exponential running
time.
Q3: We already know the fix. It’s just the
game “Determinize” which keeps track of
active state without explicitly constructing all
the subsets. Running time of O(|N |2 ·n)
since each update of the active states may
involve looking at O(|N |) active states each
of which may activate O(|N |) other states. 30
ANFA
NFAaccept2(NFA N, String x1 x2 x3 … xn )
StateSet S = {q0 } // store S as bit string
for(i = 1 to n)
S = d(S,xi) // since NFA, d outputs a set
if(S and F are not disjoint)
return ACCEPT
else
return REJECT
31
EDFA
Let’s now focus on emptiness problem.
We can use the pumping lemma to
solve the emptiness problem: Suppose
we know what the pumping number p
is for our DFA. Then if we found no
accepted strings of length < p we could
be sure that the DFA accepts no strings.
Q: Why is this true?
32
EDFA
A: Suppose language not empty. Since
checked that all strings of length < p
are rejected, smallest accepted string s
must be of length  p. The string s is
pumpable, so can be pumped down
obtaining a shorter string s’, which
contradicts the minimality of s !
This prompts the following algorithm:
33
EDFA
DFAempty( DFA M )
integer p = |Q |
for (all strings x over S with length < p)
if( DFAaccepts(M,x) == ACCEPT )
return NONEMPTY
//only got here if nothing was accepted
return EMPTY
Q: Running time? Improvements?
34
EDFA
A:
Running time: O(|S||Q|) –exponential.
Improvements: Just see if any accept state
can be reached from start state by
performing a BFS or DFS as follows:
35
EDFA
DFAempty2( DFA M )
State q = q0
Stack S
// Initialize a LIFO stack
Set V = 
// Set of visited states
S.push(q0)
while(S  )
q = S.pop()
if (q F ) return NONEMPTY
V = V  {q}
for (States q’ satisfying qq’ ) // an edge
if (q’ V ) S.push(q’ )
return EMPTY
// Only got here if F unreachable
Q: Is this a BFS or DFS?
36
EDFA
A: This is a DFS algorithm since stack is
used. Queue would give BFS.
37
ACFG
Acceptance problem is also solvable. There’s a
rather horrible algorithm involving Chomsky
Normal Form. Later we’ll see a much
improved polynomial time algorithm (must
have one, otherwise compilers would be
useless!)
Chomsky normal form gives the following fact:
LEMMA: Suppose a grammar G is in Chomsky
normal form. Let x be in L(G ), and let n be
the length of x. Then x is generated by a
derivation of length 2n-1, when n > 0.
38
ACFG
LEMMA: Suppose a grammar G is in Chomsky
normal form. Let x be in L(G ), and let n be
the length of x. Then x is generated by a
derivation of length 2n-1, when n > 0.
Proof. The first n-1 productions are of the form
A  BC and get us to the correct length. The
last n production are unit terminating A  a
and derive x.
This gives rise to the following algorithm:
39
ACFG
CFGaccept( CFG G, String x=x1 x2 x3 … xn )
CFG G’ = ChomskyNormalForm(G )
for (all derivations from start variable
of G’ of length  2n +1)
if (derivation resulted in x)
return ACCEPT
return REJECT
Q1: Why does this work for x = e?
Q2: Running time in terms of n ?
40
ACFG
A1: Works for x = e because of “ 2n+1”
clause.
A2: Exponential. If we consider the
blow-up of CNF conversion, this is a
truly horrendous algorithm.
41
Blackboard Exercises
1. EQDFA
2. ECFG
3. CARDDFA
42