Spring 2016
CISG 5115
Theory of Computation
Chapter 0
Introduction
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
Some slides are in courtesy of Prof. Xinhua Zhuang at Dept. of CECS at University of Missouri-Columbia
1
Overview
• Fundamental mathematical models of computation
•
•
•
•
what can and cannot be computed?
what computational models?
how quickly?
how much memory?
• Examples
• a compiler detecting the presence of infinite loops?
• a program checking whether or not two programs
compute the same function?
• a program determining the shortest program to compute
a given function?
2
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
Motivation and Overview
• Assess the difficulty of a given problem
• Measure the quality of a design
• Two major tasks for the theory of computation
• devise a universal model of computation
• characterize problems as solvable, efficiently solvable,
simply solvable, etc.
• Tools for practice
• grammars: programming languages
• finite automata: string searching, pattern matching
3
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
Automata, Computability, and
Complexity
• What are the fundamental capabilities and limitations of
computers?
• Automata theory
• definitions and properties of mathematical models of
computation
• finite automaton, push-down automaton, Turing machine
• Computability theory
• distinguish solvable problems from unsolvable ones
• Complexity theory
• what makes problems computationally hard?
• examples:
• sorting problems vs. scheduling problems
• games: tic-tac-toe, checker, chess, go
Spring 2016 CISG 5115
Donghyun (David) Kim
4
Department of Mathematics and Physics
North Carolina Central University
History
• Late 19th to early 20th century
• questions about the nature of computing in the context
of pure mathematics
• rigor and formalism in mathematical arguments
• some great mathematicians: Hilbert, Cauchy, Gödel
• 1930s and 1940s
• several universal models of computation (Church-Turing
thesis)
• Gödel and Kleen: the theory of partial recursive functions
• Alonzo Church: lambda calculus
• Alan Turing: Turing machine
5
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
Recent Development
• 1950s and 1960s
• from computability to complexity
• computing Pioneers: Alan Turing, John von Neumann
• Stephen Cook and Richard Karp: NP-complete problems
• Recent years
• alternative models of computation
• parallel computing
• quantum computing
• DNA computing
• proof theory
• randomized computing
Spring 2016 CISG 5115
Donghyun (David) Kim
6
Department of Mathematics and Physics
North Carolina Central University
Terminologies
• A set: a non-repeating, unordered collection of objects
(elements, members)
A = {C, Fortran, Lisp, C++, Java}
• Empty set ∅
• Subset: A ⊆ B, every element of A is an element of B
• Cardinality of a set: the number of elements in it
• Set operations
• union, intersection, complement, difference
• Power set of a set A: 2 A, the set of all subsets of A
2 {2,3,5}  {,}
• Cartesian (cross) product of two sets A and B:
{( a, b) | a  A, b  B}
Spring 2016 CISG 5115
Donghyun (David) Kim
7
Department of Mathematics and Physics
North Carolina Central University
Sequences
• A sequence: a list of objects in some order
{a, x, 2}
• Tuples : a sequence of k elements is a k-tuple.
• Repetition is allowed.
8
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
Relations
• Let A and B be sets. A relation R from A to B is a
subset A  B.
• Given an ordered pair ( x, y )  A  B, x is related to y
by R, written x R y if and only if ( x, y )  R. The set A is
called the domain of R and the set B is called its codomain.
• Ex. A  {a, b, c}
B  {x, y}
R  {( a, x), (b, y ), (a, y )}
9
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
n-ary Relation
• A more formal way to refer to the kind of relation defined in
the previous slide is to call it a binary relation because it is a
subset of a Cartesian product of two sets.
A  {a, b, c}, B  {x, y}
R  {( a, x), (b, y ), (a, y )}  A  B
• An n-ary relation to be a subset of a Cartesian product of n
sets, where n is any integer greater than or equal to two.
• Roughly, an n-ary relation to be a subset of a Cartesian
product of n sets, where n is any integer greater than or
equal to two.
Spring 2016 CISG 5115
Donghyun (David) Kim
10
Department of Mathematics and Physics
North Carolina Central University
Reflexive Relation
• Informally, a relation R is
• Reflexive: Each element is related to itself.
4
11
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
Symmetric Relation
• Informally, a relation R is
• Symmetric: If any one element is related to any other
element, then the second element is related to the first.
8
6
Spring 2016 CISG 5115
Donghyun (David) Kim
12
Department of Mathematics and Physics
North Carolina Central University
Transitive Relation
• Informally, a relation R is
• Transitive: If any one element is related to a second and
that second element is related to a third, then the first
element is related to the third.
8
7
Spring 2016 CISG 5115
Donghyun (David) Kim
5
13
Department of Mathematics and Physics
North Carolina Central University
Equivalence Relation
• Let A be a set and R a relation on A. R is an equivalence
relation if, and only if, R is reflexive, symmetric, and
transitive.
• Example
Q) Let X be the set of all nonempty subset of {1, 2, 3}.
Define a relation R on X as follows: For all A and B in X,
A
RB  the least element of A equals the least element of B.
Prove that R is an equivalence relation on X.
Show R is reflexive, symmetric, and transitive!
Spring 2016 CISG 5115
Donghyun (David) Kim
14
Department of Mathematics and Physics
North Carolina Central University
Function
• A function f from a set X to a set Y, denoted f : X  Y
is a relation from X, the domain, to Y, the co-domain,
that satisfies following two properties:
• 1) every element in X is related to some element in Y
• 2) no element in X is related to more than one element in
f
f
f
X 
Y.
Y
Y
Y

X 

X 
x
y
z
a
b
c
d
(Arrow Diagram)
Spring 2016 CISG 5115
Donghyun (David) Kim
x
y
z
a
b
c
d
x
y
z
a
b
c
d
15
Department of Mathematics and Physics
North Carolina Central University
FunctionX – cont’
Y


f
x
y
z
Range:
{a,c}
a
b
c
d
• The set of values of f taken together is called the range
of f or the image of x under f. Followings are same
• range of f
• Image of x under f
• { y  Y | y  f ( x), forsome x  X }
Spring 2016 CISG 5115
Donghyun (David) Kim
16
Department of Mathematics and Physics
North Carolina Central University
Function – cont’
• Predicate (property) : a function whose range is
{True, False}.
• k-ary function: a function with k arguments.
Input: {a1,a2 ,,ak }, a k-tuple
• unary function if k = 1
• binary function if k = 2
17
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
Graphs
• A graph consists of a finite set of vertices with lines
connecting some of them (edges).
• Undirected graph
•
•
•
•
degree of a node: the number of edges
path: a sequence of nodes connected by edges
cycle: a path starts and ends in the same node
tree: no cycle
• Directed graph:
• in-degree and out-degree
• directed path
• directed acyclic graph (DAG)
Spring 2016 CISG 5115
Donghyun (David) Kim
18
Department of Mathematics and Physics
North Carolina Central University
Strings
• Strings of characters
• Alphabet : any finite set,  and 
  {0,1,2,3, x, y}
• A string over an alphabet: a finite sequence of symbols
from the alphabet
w  x1211 y3
• Length of a string w :| w |
• Empty string: 
19
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
Strings and Languages
• Reverse of w  01101 : w R  10110
• Substring
• Concatenation
w  w1w2 w3 v  v1v2
 wv  w1w2 w3 v1v2
• Lexicographic ordering of strings
  {0 , 1}
 , 0,1, 00, 01, 10, 11, 000, 
• Language : a set of strings
20
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
Languages
• Language : a set of strings
{00, 010, 111, 10001}
• Finite / infinite
• Kleene closure of alphabet A : A*, the set of all strings
of all lengths over A (arbitrary length)
{0}*  { , 0 , 00 , 000 , 0000 , }
{00 1}*  { , 1, 00 , 001, 0000 , }
• A language L over the alphabet A is a subset of A*
L  A*
21
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
Proofs
• Proof: a convincing logical argument that a statement
is true.
• convincing in an absolute sense
• Methods of proof
•
•
•
•
The pigeonhole principle
Proof by construction
Proof by contradiction
Proof by induction
22
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
The Pigeonhole Principle
• If there are n pigeonholes, n + 1 or more pigeons, and
every pigeon occupies a hole, then some hole must
have at least two pigeons.
• Example
• To show a 9-color pie chart on a 8-color monitor, two
parts must have the same color.
Spring 2016 CISG 5115
Donghyun (David) Kim
23
Department of Mathematics and Physics
North Carolina Central University
Proof by Construction
• Prove a particular type of objects exists by
constructing the object.
• Example
• For each even number n > 2, there exists a 3-regular
graph (no loop, no multiple edges) with n nodes.
• Proof: construct a graph G = (V, E) with n nodes as follow.
V  {0 ,1,  , n  1}
E  {{i, i  1} | for0  i  n  2}  {{n  1,0}}
 {{i, i  n / 2} | for0  i  n / 2  1}
24
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
Proof by Contradiction
• Assume a theorem is false and then show that this
assumption leads to a false consequence.
• Example
• The language L  {00 1}*contains strings with an even
number of 1's.
• Proof: Assume L contains only strings with an odd
number of 1's. However, L contains the string 11, which
contradicts with the assumption.
25
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
Proof by Induction
• A proof by induction has a predicate P,
• a basis - k , P ( k ) is true
• an induction hypothesis - for some n  k , P(k ), P(k  1), , P(n)
are true.
• an inductive step - P(n  1) is true given the induction
hypothesis.
• Example
• n  0, S (n)   nj1 j  n(n 1) / 2.
• Proof:
• Basis - S (0 )  0 .
• Induction hypothesis - S (k )  k (k  1) / 2 for k  0 ,1,  , n.
26
• Inductive step - S (n  1)  S (n)  n  1
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University