Lecture # 1

• Instructor: Dr. Mudasser Naseer mnaseer@ciitahore.edu.pk
Cabin # 1, Faculty Room C7
• Lectures: Sec-B: Wed 1630, Thu 1800 hrs.
Sec-C: Wed 1800, Thu 1630 hrs.
•Office Hrs: Tue & Thu 1400 – 1600 hrs
(or by appointment)
•Prerequisite: CSC102 - Discrete Structures

Text Book: i) J. E. Hopcroft, R. Motwani, & J. D. Ullman
Introduction to Automata Theory,
Languages, and Computation, Third Edition,
Pearson, 2008.
ii) Denial I. A. Cohen Introduction to Computer
Theory, First Edition, John Wiley & Sons, 1986.
Instruments:
There will be 2~3 assignments, 4~5 quizzes,
Weights: Assignments
Quizzes
S-I
10%
15%
10%
S-II
Final Exam
15%
50%

Study and mathematically model various abstract computing machines that serve as models for computations and examine the relationship between these automata and formal languages. Regular expressions, NFAs. Core concepts of Regular
Languages and Finite Automata; Decidability for
Regular Languages; Non-regular Languages; Contextfree Languages and Pushdown Automata; Decidability for Context-free Languages; Non-context-free
Languages; Turing Machines and Their Languages are important part of the course. Transducers (automata with output).

Lect.#
1
2
3
6
7
8
4
5
9
Topics/Contents
Introduction to Automata theory, Its background, Mathematical Preliminaries,
Sets, Functions, Relations, Graphs, Proof Techniques
Formal Languages, Introduction to defining languages, alphabet, language, word, null string, length of a string, reverse of a string, Palindrome, Kleene closure.
Defining languages recursively, Recursive definitions of languages like EVEN,
INTEGERS, POSITIVE, POLYNOMIALS, ARITHMETIC EXPRESSIONS
Theorems relating to recursive definitions and their proofs.
Formal definition of Regular Expressions, Defining languages with regular expressions, Languages associated with regular expressions.
Equality of Regular Expressions, Introducing the language EVEN-EVEN.
More examples related to regular expressions.
Introducing Finite Automata., Defining languages using Finite Automata.
Constructing Finite Automata for different languages.
Recognizing the language defined by the given Finite Automata.
10 More examples related to Finite Automata.
11 Sessional I

Lect.#
Topics/Contents
12 Transition Graphs with examples, Generalized Transition Graphs, Nondeterminism in case of Transition Graphs.
13 Non-deterministic FA’s. Differences between FA, TG and NFA.
14 Finite Automata with output, machines and Mealy machines with examples. 1’s Complement machine, Increment machine.
15 Transducers as models of sequential circuits, Theorems for Converting
Moore machines into Mealy machines and vice versa.
16 Regular Languages, Closure properties (i.e. , Concatenation and Kleene closure) of Regular Languages with examples. Complements and
Intersections of Regular Languages, Theorems relating to regular languages and the related examples.
17 Decidability, decision procedure, Blue-paint method, Effective decision procedure to prove whether two given RE’s or FA’s are equivalent.
18 Non-Regular Languages, The pumping Lemma, Examples relating to
Pumping Lemma.
19 Myhill-Nerode theorem, Related Examples, Quotient Languages.
20 Context-Free Grammars, CFG’s for Regular Languages with examples.
21 CFG’s for non-regular languages, CFG’s of PALINDROME, EQUAL and
EVEN-EVEN languages, Backus-Naur Form.
22 Parse Trees, Examples relating to Parse Trees, Lukasiewicz notation, Prefix and Postfix notations and their evaluation.
23 Sessional II

Lect.# Topics/Contents
24 Ambiguous and Unambiguous CFG’s, Total language tree. Regular
Grammars, Semi-word, Word, Working String, Converting FA’s into
CFG’s.
25 Regular Grammars, Constructing Transition Graphs from Regular
Grammars. Killing null productions.
26 Killing unit productions, Chomsky Normal form with examples, Left most derivations.
27 Pushdown Automata, Constructing PDA’s for FA’s, Pushdown stack.
Nondeterministic PDA.
28 Examples related with PDA, PDA for Odd Palindrome, Even Palindrome,
PalindromeX.
29 Proving CFG = PDA with examples. Constructing PDA from CFG in CNF and making trace table for any word.
30 Context Free Languages, their closure properties, , Concatenation and
Kleene Closures using CFG’s and PDA’s.
31 Top down parsing, Bottom up parsing with examples, Turing machines,
Examples of Turing Machines with trace tables, Converting FA’s into
Turing machines.
32 The subprogram Insert, The subprogram Delete, Constructing Turing machine for the language EQUAL using Insert subprogram.

• Automaton = A self-operating machine or mechanism (Dictionary definition), plural is
Automata.
• Automata = abstract computing devices
• Automata theory = the study of abstract machines (or more appropriately, abstract
'mathematical' machines or systems, and the computational problems that can be solved using these machines.
• Mathematical models of computation
• Finite automata
• Push-down automata
• Turing machines

• 1930s : Alan Turing defined machines more powerful than any in existence, or even any that we could imagine – Goal was to establish the boundary between what was and was not computable.
• 1940s/150s : In an attempt to model “Brain function” researchers defined finite state machines.
• Late 1950s : Linguist Noam Chomsky began the study of Formal Grammars.
• 1960s : A convergence of all this into a formal theory of computer science, with very deep philosophical implications as well as practical applications (compilers, web searching, hardware,
A.I., algorithm design, software engineering,…)

CPU memory
Courtesy Costas Busch - RPI 10

temporary memory
CPU
Program memory input memory output memory
Courtesy Costas Busch - RPI 11

f ( x )
 x
3 temporary memory input memory
CPU
Program memory compute x
 x compute x
2  x
Courtesy Costas Busch - RPI output memory
12

f ( x )
 x
3 temporary memory input memory x

2
CPU
Program memory compute x
 x compute x
2  x
Courtesy Costas Busch - RPI output memory
13

temporary memory z

2 * 2

4 f ( x )
 z * 2

8 f ( x )
 x
3 input memory x

2
CPU
Program memory compute x
 x compute x
2  x
Courtesy Costas Busch - RPI output memory
14

temporary memory z

2 * 2

4 f ( x )
 z * 2

8 f ( x )
 x
3 input memory x

2
CPU
Program memory compute x
 x compute x
2  x
Courtesy Costas Busch - RPI f ( x )

8 output memory
15

temporary memory
Automaton input memory
CPU output memory
Program memory
Courtesy Costas Busch - RPI 16

Automata are distinguished by the temporary memory
• Finite Automata: no temporary memory
• Pushdown Automata: stack
• Turing Machines: random access memory
Courtesy Costas Busch - RPI 17

temporary memory
Finite
Automaton
Example: Vending Machines
(small computing power)
Courtesy Costas Busch - RPI input memory output memory
18

Stack Push, Pop
Pushdown
Automaton input memory output memory
Example: Compilers for Programming Languages
(medium computing power)
Courtesy Costas Busch - RPI 19

Random Access Memory
Turing
Machine input memory output memory
Examples: Any Algorithm
(highest computing power)
Courtesy Costas Busch - RPI 20

Finite
Automata
Less power
Pushdown
Automata
Turing
Machine
Solve more
More power computational problems
Courtesy Costas Busch - RPI 21

• Sets
• Functions
• Relations
• Graphs
• Proof Techniques
Courtesy Costas Busch - RPI 22

A set is a collection of elements
A

{ 1 , 2 , 3 }
B

{ train , bus , bicycle , airplane }
We write
1

A ship

B
Courtesy Costas Busch - RPI 23

C = { a, b, c, d, e, f, g, h, i, j, k }
C = { a, b, …, k }
S = { 2, 4, 6, … }
S = { j : j > 0, and j = 2k for some k>0 }
S = { j : j is nonnegative and even }
Courtesy Costas Busch - RPI 24

A = { 1, 2, 3, 4, 5 }
6
7
10
1
2 3
4 5
9
Universal Set: all possible elements
U = { 1 , … , 10 }
Courtesy Costas Busch - RPI
8
25

• Union
A = { 1, 2, 3 } B = { 2, 3, 4, 5}
A B
4
A U B = { 1, 2, 3, 4, 5 }
1
2
3
5
• Intersection
A B = { 2, 3 }
• Difference
A - B = { 1 }
B - A = { 4, 5 }
1
2
3
Venn diagrams
Courtesy Costas Busch - RPI 26

• Complement
Universal set = {1, …, 7}
A = { 1, 2, 3 } A = { 4, 5, 6, 7}
4
A
1
A
2
3
5
7
6
A = A
Courtesy Costas Busch - RPI 27

{ even integers } = { odd integers }
Integers
1
3 odd
2 even
0
4
6
7
5
Courtesy Costas Busch - RPI 28

Courtesy Costas Busch - RPI 29

= { }
S U = S
= Universal Set
S = S
- S =
Courtesy Costas Busch - RPI 30

A = { 1, 2, 3} B = { 1, 2, 3, 4, 5 }
A B
Proper Subset: A B
A
B
Courtesy Costas Busch - RPI 31

A = { 1, 2, 3 } B = { 5, 6}
A B
Courtesy Costas Busch - RPI 32

• For finite sets
A = { 2, 5, 7 }
|A| = 3
(set size)
Courtesy Costas Busch - RPI 33

A powerset is a set of sets
S = { a, b, c }
Powerset of S = the set of all the subsets of S
2 S = { , {a}, {b}, {c}, {a, b}, {a, c}, {b, c}, {a, b, c} }
Observation: | 2 S | = 2 |S| ( 8 = 2 3 )
Courtesy Costas Busch - RPI 34

A = { 2, 4 } B = { 2, 3, 5 }
A X B = { (2, 2), (2, 3), (2, 5), ( 4, 2), (4, 3), (4, 5) }
|A X B| = |A| |B|
Generalizes to more than two sets
A X B X … X Z
Courtesy Costas Busch - RPI 35

5
4
domain range
A
1
2
3 f(1) = a b
B a c f : A -> B
If A = domain then f is a total function otherwise f is a partial function
Courtesy Costas Busch - RPI 36

Let A & B be sets. A binary relation “R” from A to B
R = {(x
1
, y
1
), (x
2
, y
2
), (x
3
, y
3
), …} x

A y i

B
R ⊆ A x B x i
R y i to denote
( a , b )

R e. g. if R = ‘>’: 2 > 1, 3 > 2, 3 > 1
Courtesy Costas Busch - RPI 37

• Reflexive: x R x
• Symmetric: x R y y R x
• Transitive: x R y and y R z x R z
Example: R = ‘=‘
• x = x
• x = y y = x
• x = y and y = z x = z
Courtesy Costas Busch - RPI 38

Given an equivalence relation R on a set A and an element , the equivalence class of set    y | ( , )

R
 x is the
Example:
R = { (1, 1), (2, 2), (1, 2), (2, 1),
(3, 3), (4, 4), (3, 4), (4, 3) }
Equivalence class of 1 = {1, 2}
Equivalence class of 3 = {3, 4}
Courtesy Costas Busch - RPI 39

A directed graph e node a b d c
• Nodes (Vertices)
V = { a, b, c, d, e }
• Edges
E = { (a,b), (b,c), (b,e),(c,a), (c,e), (d,c), (e,b), (e,d) }
Courtesy Costas Busch - RPI 40

a
2
6
1 b
3 e
5
6 c
2 d
Courtesy Costas Busch - RPI 41

e b a d c
Walk is a sequence of adjacent edges
(e, d), (d, c), (c, a)
Courtesy Costas Busch - RPI 42

e b a d c
Path is a walk where no edge is repeated
Simple path: no node is repeated
Courtesy Costas Busch - RPI 43

a
3 base b
2
1 c e d
Cycle: a walk from a node (base) to itself
Simple cycle: only the base node is repeated
Courtesy Costas Busch - RPI 44

a
4
8
7 b
5 6
3 c base e
1
2 d
A cycle that contains each edge once
Courtesy Costas Busch - RPI 45

a
5
4 b base e
1
2 d
3 c
A simple cycle that contains all nodes
Courtesy Costas Busch - RPI 46

e b a d c origin
Courtesy Costas Busch - RPI 47

(c, a)
(c, e) a
b e c origin d
Courtesy Costas Busch - RPI 48

b
(c, a)
(c, a), (a, b)
(c, e)
(c, e), (e, b)
(c, e), (e, d) a
Courtesy Costas Busch - RPI e c origin d
49

e b a
(c, a)
(c, a), (a, b)
(c, a), (a, b), (b, e)
(c, e)
(c, e), (e, b)
(c, e), (e, d) c origin
Courtesy Costas Busch - RPI d
50

e b
(c, a)
(c, a), (a, b) a
(c, a), (a, b), (b, e)
(c, a), (a, b), (b, e), (e,d)
(c, e)
(c, e), (e, b)
(c, e), (e, d) c origin
Courtesy Costas Busch - RPI d
51

root
parent leaf
Trees have no cycles
Courtesy Costas Busch - RPI child
52

leaf root
Level 0
Level 1
Level 2
Height 3
Level 3
Courtesy Costas Busch - RPI 53

Courtesy Costas Busch - RPI 54

• Proof by induction
• Proof by contradiction
Courtesy Costas Busch - RPI 55

We have statements P
1
, P
2
, P
3
, …
If we know
• for some b that P
1
, P
2
, …, P b are true
• for any k >= b that
P
1
, P
2
, …, P k imply P k+1
Then
Every P i is true
Courtesy Costas Busch - RPI 56

• Inductive basis
Find P
1
, P
2
, …, P b which are true
• Inductive hypothesis
Let’s assume P
1
, P
2
, …, P k are true, for any k >= b
• Inductive step
Show that P k+1 is true
Courtesy Costas Busch - RPI 57

Theorem: A binary tree of height n has at most 2 n leaves.
Proof by induction: let L(i) be the maximum number of leaves of any subtree at height i
Courtesy Costas Busch - RPI 58

We want to show: L(i) <= 2 i
• Inductive basis
L(0) = 1 (the root node)
• Inductive hypothesis
Let’s assume L(i) <= 2 i for all i = 0, 1, …, k
• Induction step we need to show that L(k + 1) <= 2 k+1
Courtesy Costas Busch - RPI 59

height k k+1
From Inductive hypothesis: L(k) <= 2 k
Courtesy Costas Busch - RPI 60

height k k+1
L(k) <= 2 k
L(k+1) <= 2 * L(k) <= 2 * 2 k = 2 k+1
(we add at most two nodes for every leaf of level k)
Courtesy Costas Busch - RPI 61

Recursion is another thing
Example of recursive function: f(n) = f(n-1) + f(n-2) f(0) = 1, f(1) = 1
Courtesy Costas Busch - RPI 62

We want to prove that a statement P is true
• we assume that P is false
• then we arrive at an incorrect conclusion
• therefore, statement P must be true
Courtesy Costas Busch - RPI 63

Theorem:
2
is not rational
Proof:
Assume by contradiction that it is rational
2
= n/m n and m have no common factors
We will show that this is impossible
Courtesy Costas Busch - RPI 64

2
= n/m 2 m 2 = n 2
Therefore, n 2 is even n is even n = 2 k
2 m 2 = 4k 2 m 2 = 2k 2
Thus, m and n have common factor 2
Courtesy Costas Busch - RPI m is even m = 2 p
65