Chapter 2
Finite Automata
and the
Languages They
Accept
1
Copyright © 2011 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Finite Automata: Examples and
Definitions
• A finite automaton is a simple type of computer
– Its output is limited to “yes” or “no”
– It has very primitive memory capabilities
• Any computer that answers yes or no acts as a
language acceptor
• For this chapter, consider that:
– The input comes in the form of a string of individual
input symbols
– The computer gives an answer for the current string
(the string of symbols that have been read so far)
Introduction to Computation
2
Finite Automata: Examples and
Definitions (cont’d.)
• A finite automaton (FA) or finite state machine is
always in one of a finite number of states
• At each step it makes a move that depends only on
the state it’s currently in and the input symbol it gets
• The move is to enter a particular state (possibly the
same as the one it was already in)
• States are either accepting or nonaccepting
– Entering an accepting state means answering “yes”
– Entering a nonaccepting state means “no”
• A FA has an initial state, which is an accepting state if
and only if the language the FA accepts includes 
Introduction to Computation
3
Finite Automata: Examples and
Definitions (cont’d.)
• An FA can be described by the set of states, the input
alphabet, the initial state, the set of accepting states,
and a transition function
– This can be described by a diagram or a table of values
– In a diagram, states are represented by circles,
transitions by arrows labeled with input symbols, and
accepting states by double circles
p
a
Introduction to Computation
q
4
Finite Automata: Examples and
Definitions (cont’d.)
• This FA accepts the language of strings that end in aa
– The three states represent strings that end with no a’s,
one a, and two a’s, respectively
– From each state, if the input is anything but an a, go
back to the initial state, because now the current string
doesn’t end with a
Introduction to Computation
5
Finite Automata: Examples and
Definitions (cont’d.)
• This FA accepts the strings containing b and not aa
– The idea is to go to a permanently-non-accepting
state if you ever read two a’s in a row
– Go to an accepting state if you see a b (and haven’t
read two a’s), leave it when you see anything else
Introduction to Computation
6
Finite Automata: Examples and
Definitions (cont’d.)
• This FA accepts strings that contain abbaab
• What do we do when a prefix of abbaab has been
read but the next symbol doesn’t match?
– Go back to the state representing the longest prefix of
abbaab at the end of the new current string
– If we’ve read abba and the next symbol is b, go to q2,
because ab is the longest prefix at the end of abbab
Introduction to Computation
7
Finite Automata: Examples and
Definitions (cont’d.)
• A FA that accepts binary
representation of integers divisible
by 3
– States 0, 1, and 2 represent the
current “remainder”
– The initial state is non-accepting: at
least one bit is required
– Leading zeros are prohibited
– Transitions represent
multiplication by two, then
addition of the input bit
Introduction to Computation
8
Finite Automata: Examples and
Definitions (cont’d.)
• FAs are ideally suited for lexical analysis, the first
stage in compiling a computer program
• A lexical analyzer takes a string of characters and
provides a string of “tokens”
• Tokens have a simple structure: e.g., “41.3”, “main”
• The next slide shows an FA that accepts tokens for a
simple language based on C
– The only tokens are identifiers, semicolons, =, aa, and
numeric literals; tokens are separated by spaces
– Accepting states represent scanned tokens; each
accepting state represents a category of token
Introduction to Computation
9
Finite Automata: Examples and
Definitions (cont’d.)
• D is any digit
• L is a lowercase letter
other than a
• M is D or L
• N is D or L or a
•  is a space
• All transitions not
shown explicitly go to
an error state and stay
there
Introduction to Computation
10
Finite Automata: Examples and
Definitions (cont’d.)
• Definition: a finite automaton is a 5-tuple
M = (Q, , q0, A, ), where:
–
–
–
–
–
Q is a finite set of states
 is a finite input alphabet
q0  Q is the initial state
A  Q is the set of accepting states
 : Q    Q is the transition function
• From state q the machine will move to state (q, ) if
it receives input symbol 
Introduction to Computation
11
Finite Automata: Examples and
Definitions (cont’d.)
• The notation *(q, x) describes the state the FA is in
after starting in state q and receiving input string x.
The extended transition function * : Q  *  Q is
defined recursively as follows:
– For every q  Q, *(q, ) = q
– For every q  Q, every y  *, and every   ,
*(q, y) = (*(q, y), )
This just says that if you already know how to process
the string y, then to process one additional symbol ,
you use the ordinary transition function  starting
from the state *(q, y)
Introduction to Computation
12
Finite Automata: Examples and Definitions
(cont’d.)
• Suppose we wanted to evaluate *(q0, baa) in the
example on slide 7. This is easy using the diagram
(just follow the arrows), but let’s use the recursive
formula on the previous slide, to see how it works
• *(q0, baa) = (*(q0, ba), a) = ((*(q0, b), a), a)
= ((*(q0, b), a), a)
= (((*(q0, ), b), a), a)
= (((q0, b), a), a) = ((q0, a), a)
= (q1, a) = q1
• We had to look at the diagram only in the last 3 steps,
to get the values of 
Introduction to Computation
13
Finite Automata: Examples and
Definitions (cont’d.)
• Definition:
– Let M=(Q, , q0, A, ) be an FA, and let x  *. Then x is
accepted by M if *(q0, x)  A and rejected otherwise
• The language accepted by M is
L(M) = {x  * | x is accepted by M}
Introduction to Computation
14
Accepting the Union, Intersection, or
Difference of Two Languages
• Suppose that L1 and L2 are languages over 
– Given an FA that accepts L1 and another that accepts
L2, we can construct one that accepts L1 ∪ L2
– The same approach works for intersection and
difference as well
– The idea is to construct an FA that executes both of the
original FAs at the same time
– This works because if x  *, then knowing whether
x  L1 and whether x  L2 is enough to determine
whether x  L1 ∪ L2
Introduction to Computation
15
Accepting the Union, Intersection, or
Difference of Two Languages (cont’d.)
• Theorem: Suppose M1=(Q1, , q1, A1, 1) and
M2=(Q2, , q2, A2, 2) are FAs accepting L1 and L2. Let
M=(Q, , q0, A, ) be defined as follows:
– Q = Q1  Q2
– q0 = (q1, q2)
– ((p, q), ) = (1(p, ), 2(q, ))
• Then, if :
– A = {(p, q) | p  A1 or q  A2}, M accepts L1 ∪ L2
– A = {(p, q) | p  A1 and q  A2}, M accepts L1 ∩ L2
– A = {(p, q) | p  A1 and q ∉ A2}, M accepts L1 - L2
Introduction to Computation
16
Accepting the Union, Intersection, or
Difference of Two Languages (cont’d.)
• Given two machines, create the Cartesian product of
the state sets, and draw the necessary transitions
Introduction to Computation
17
Accepting the Union, Intersection, or
Difference of Two Languages (cont’d.)
• Simplify the resulting machine, if possible, and
designate the appropriate accepting states
• The machine below accepts the union of the two
languages
Introduction to Computation
18
Accepting the Union, Intersection, or
Difference of Two Languages (cont’d.)
• For the intersection, we can
simplify further, and we
end up with the machine on
the right
• The simplification involved
turning states CP, CQ, and
CR into a single state (none
of them was accepting, and
there was no way to leave
them)
Introduction to Computation
19
Distinguishing One String from Another
• Any three-state FA, such as the one that accepts
the strings ending in aa, ignores, or “forgets”, a
lot of information
– aba and aabbabbabaaaba lead to the same state; there
is no way for the FA to remember which string has
been seen
– aba and ab, however, lead to different states; the
essential difference is that one ends with a and the
other doesn’t
– aba and ab are distinguishable with respect to the
language accepted by the FA; there is at least one
string z (such as a) so that abaz is in the language (i.e.,
is accepted) and abz is not, or vice versa
Introduction to Computation
20
Distinguishing One String from Another
(cont’d.)
• Definition:
– If L is a language over the alphabet , and x and y are
strings in *, then x and y are distinguishable with
respect to L, or L-distinguishable, if there is a string
z  * such that either xz  L and yz  L, or xz  L and
yz  L
– A string having this property is said to distinguish x
and y with respect to L
– Equivalently, x and y are L-distinguishable if L/x ≠ L/y,
where L/x = {z   * | xz  L}
Introduction to Computation
21
Distinguishing One String from Another
(cont’d.)
• Theorem: Suppose M=(Q, , q0, A, ) is an FA
accepting L  *
– If x and y are two strings in * that are
L-distinguishable, then *(q0, x) ≠ *(q0, y)
– For every n ≥ 2, if there is a set of n pairwise
L-distinguishable strings in *, then Q must contain at
least n states
– This shows why we need at least three states in any
machine that accepts the language L of strings ending
in aa: {, a, aa} contains 3 pairwise L-distinguishable
strings
Introduction to Computation
22
Distinguishing One String from Another
(cont’d.)
• To create an FA to accept L = {aa, aab}*{b}, we notice
first that  is not in L, b is in L, a is not, and  and a
are L-distinguishable (for example, b  L, ab  L)
– We need at least the states in the first diagram
– L contains b but nothing else that begins with b, so we
add a state s to take care of illegal prefixes
– If the input starts with aa we, need to leave state p
because a and aa are L-distinguishable; create state t
Introduction to Computation
23
Distinguishing One String from Another
(cont’d.)
• (t, b) must be accepting, because
aab  L; call that new state u
• Let (u,b) be r, because aabb is in L
but not a prefix of any other string
in L
• States t and u can be thought of as
representing the end of an
occurrence of aa or aab; if the next
symbol is a it’s the start of a new
occurrence, so go back to p
• The result is shown here
Introduction to Computation
24
The Pumping Lemma
• Suppose that M=(Q, S, q0, A, δ) is an FA accepting L
and that it has n states
– If it accepts a string x such that |x| ≥ n, then by the time n
symbols have been read, M must have entered some state
more than once; i.e., there must be two different prefixes u
and uv such that
δ *(q0,u)= δ *(q0,uv)
Introduction to Computation
25
The Pumping Lemma (cont’d.)
• This implies that there are many more strings in L,
because we can traverse the loop v any number of
times (including leaving it out altogether)
• In other words, all of the strings uviw for i ≥ 0 are in L
• This fact is known as the Pumping Lemma for
Regular Languages
Introduction to Computation
26
The Pumping Lemma (cont’d.)
• Theorem: Suppose L is a language over 
• If L is accepted by the FA M=(Q, , q0, A, ), then there
is an integer n so that for every x in L satisfying
|x| ≥ n, there are three strings u, v, and w such that
x = uvw and
– |uv| ≤ n
– |v| > 0 (i.e. v ≠ )
– For every i ≥ 0, the string uviw belongs to L
• The way we found n was to take the number of states
in an FA accepting L. In many applications we don’t
need to know this, only that there is such an n
Introduction to Computation
27
The Pumping Lemma (cont’d.)
• The most common application of the pumping lemma
is to show that a language cannot be accepted by an
FA, because it doesn’t have the properties that the
pumping lemma says are required for every language
that can be.
• The proof is by contradiction. We suppose that the
language can be accepted by an FA, and we let n be
the integer in the pumping lemma
• Then we choose a string x with |x| ≥ n to which we
can apply the lemma so as to get a contradiction
Introduction to Computation
28
The Pumping Lemma (cont’d.)
• Let L be the language AnBn = {aibi | i ≥ 0}; let us prove
that it cannot be accepted by an FA
– Suppose, for the sake of contradiction, that L is
accepted by an FA; let n be as in the pumping lemma
– Choose x = anbn; then x  L and |x| ≥ n
– Therefore, by the pumping lemma, there are strings
u, v, and w such that x = uvw and the 3 conditions hold
– Because |uv| ≤ n and x starts with n a’s, all the symbols
in u and v are a’s; therefore, v = ak for some k > 0
– uvvw  L, so an+kbn  L. This is our contradiction, and
we conclude that L cannot be accepted by an FA
Introduction to Computation
29
The Pumping Lemma (cont’d.)
• Let’s show L = {a
i2
| i ≥ 0} is not accepted by an FA
– Suppose L is accepted by an FA, and let n be the integer
in the pumping lemma
2
– Choose x = an
– x = uvw, where 0 < |v| ≤ n
– Then n2 = |uvw| < |uv2w| = n2 + |v| ≤ n2 + n < (n+1)2
– This is a contradiction, because |uv2w| must be i2 for
some integer i (because uv2w  L), but there is no
integer i whose square is strictly between n2 and
(n+1)2
Introduction to Computation
30
The Pumping Lemma (cont’d.)
• There are other languages that are not accepted by
any FA, among them:
– Balanced, the set of balanced strings of parentheses
– Expr, the language of simple algebraic expressions
– The set L of legal C programs
• In all three examples, because of the nature of these
languages, a proof using the pumping lemma might
look a lot like the proof for AnBn, our first example
Introduction to Computation
31
The Pumping Lemma (cont’d.)
• We can formulate several “decision problems”
involving the language L accepted by an FA
– The membership problem (Given x, is x  L?)
– Given an n-state FA M, is the language L(M) empty?
• It follows from the pumping lemma that this an be
solved by looking at all possible strings of length 0 to
n -1; if none of those is accepted, the language is empty
– Given an n-state FA M, is L(M) infinite?
• The pumping lemma implies that the language is infinite
if and only if at least one of the strings with length from
n to 2n -1 is accepted
Introduction to Computation
32
How to Build a Simple Computer Using
Equivalence Classes
• Consider M, an FA accepting the language of strings
ending in aa
• We’ve shown that three states are needed; why is this
enough?
– We can confirm that M really does accept L by showing
that if x and y are two strings that cause M to end in the
same state, then M doesn’t need to distinguish them,
because they are not L-distinguishable
– The three states of M correspond to three sets of
strings: those not ending in a, those ending in a but not
aa, and those ending in aa
Introduction to Computation
33
How to Build a Simple Computer Using
Equivalence Classes (cont’d.)
• No two strings in any one of these sets are Ldistinguishable, and a string in one of these sets is
distinguishable from a string in any other one
• These two facts say that the three sets are the
equivalence classes for a certain equivalence relation
• Definition: For a language L  {a,b}*, we define the
relation IL on * as follows: For x, y,  *, x IL y if and
only if x and y are L-indistinguishable
• IL is an equivalence relation and we can start building
an FA accepting L by using the induced equivalence
classes as its states
Introduction to Computation
34
How to Build a Simple Computer Using
Equivalence Classes (cont’d.)
• The initial state should be [], because when we start
we have received no input
• The accepting state should be the equivalence class of
strings ending in aa, since that’s the language we
want to accept
• The transitions are defined in a natural way:
– Take any element x of one class, and consider xa or xb
– The new string is in some equivalence class
– The a-transition or b-transition from [x] simply goes to
that class
Introduction to Computation
35
How to Build a Simple Computer Using
Equivalence Classes (cont’d.)
• Theorem (Myhill-Nerode): L  * can be accepted by
an FA if and only if the set QL of equivalence classes of
the relation IL on * is finite
• If the set QL is finite, then the finite automaton
ML = (QL, , q0, A, ) accepts L, where:
– q0 = []
– A = {q  QL | q  L},
– For every x  * and every   , ([x], ) = [x ]
• ML has the fewest states of any FA accepting L
Introduction to Computation
36
How to Build a Simple Computer Using
Equivalence Classes (cont’d.)
• It is often easier to construct an FA directly than to
determine the set of equivalence classes
• The above theorem serves to answer the question of
how much a computer accepting a language L needs
to remember about the current string x: only its
equivalence class
• Identifying the equivalence classes, if we already
have an FA accepting L, is not too hard
– For each state q, we define Lq = {x  * | *(q0, x) = q}
– Every Lq is a subset of some equivalence class of IL (is
the whole class if the FA has as few states as possible)
Introduction to Computation
37
How to Build a Simple Computer Using
Equivalence Classes (cont’d.)
• The Myhill-Nerode Theorem provides a way of
showing a language cannot be accepted by an FA
(and it might apply even if the pumping lemma
doesn’t, since it’s an if-and-only-if result)
• Consider the equivalence classes of IL for L = AnBn
– for i ≠ j, ai and aj are L-distinguishable, because
aibi  L and ajbi  L
– This implies that there are an infinite number of
equivalence classes, and thus there can be no FA
accepting L
Introduction to Computation
38
Minimizing the Number of States in a
Finite Automaton
• Suppose M = (Q, , q0, A, ) accepts L  *
• Define Lq = { x  * | *(q0, x) = q}
– The first step in minimizing the number of states
is to remove every state q for which Lq = , along
with transitions from these states; removing them
has no effect on the language
– Now define  on Q: p  q means that strings in Lp
are L-indistinguishable from strings in Lq
– This is the same as saying Lp and Lq are subsets of
the same equivalence class of IL
Introduction to Computation
39
Minimizing the Number of States in a
Finite Automaton (cont’d.)
• Two strings x and y are L-distinguishable if, for some
string z, exactly one of xz, yz is in L
• Therefore, p ≢ q if, for some string z, exactly one of
the states *(p, z), *(q, z) is in A
• Define SM as the set of unordered pairs (p, q) of
distinct states satisfying p ≢ q
• A systematic way of finding SM is this:
– If exactly one of p, q is in A, then (p, q)  SM
– For every pair of states r and s, and every symbol , if
((r,), (s,))  SM, then (r, s)  SM
Introduction to Computation
40
Minimizing the Number of States in a
Finite Automaton (cont’d.)
• An algorithm to identify the pairs (p, q) with p ≢ q :
– List all unordered pairs (p, q) of distinct states.
– Make a sequence of passes through these pairs:
– On the first pass, mark each pair (p, q) so that exactly
one of the two states is in A
– On each subsequent pass, and each unmarked pair
(r, s), if (r, ) = p and (s, ) = q for some   , and
(p, q) is marked, then mark (r, s)
– After a pass in which no pairs are marked, stop
– The marked pairs are the pairs (p, q) for which p ≢ q
41
Minimizing the Number of States in a
Finite Automaton (cont’d.)
• When the algorithm terminates, the unmarked pairs
(p, q) represent two states that can be combined into
one.
• Make one final pass through the states. The first
state represents a state in the new minimal FA.
Every subsequent state q determines a new state
only if (p, q) is marked for every p considered
previously
• Once we have the states in the new minimum-state
FA, determining the transitions is straightforward
42