1 Kleene’s theorem
CHAPTER 1 INTRODUCTION
1:
This theorem was proved by kleene in 1956.
We introduced three separate ways of defining a language: generation by regular expression, acceptance by finite automata, and acceptance by transition graph. in this book we present a theorem proved by Kleene in 1956.which says that if a language can be defined by any one of these three ways, then it can also be defined by the other two ways.
Any language that can be defined by a regular expression, or a finite automaton, or a transition graph can be defined by all 3 methods.
All of these three methods of defining language are equivalent.
If we were trying to prove the mathematical theorem that the set of all ZAPS, the set of all
ZEPS and the set of all ZIPS are same. To do this, we need three parts. In part 1,we show that all ZAPS are ZEPS.In part 2,we shall show that all ZEPS are ZIPS.Finaly in part three ,we shall show that all ZIPS are ZAPS. Taken together, these three parts will establish the equivalence of the three sets:
[ZAPS ZEPS ZIPS ZAPS]=[ZAPS=ZEPS=ZIPS]
This is the fundamental theorem of finite automata.


CHAPTER
1
INTRODUCTION

2 Kleene’s theorem

CHAPTER 2 PROOF(KLEENE’S THEOREM PART I)

3 Kleene’s theorem
2
As we know that A finite automata is a collection of three things:
1.
A finite set of states, one of which is designed as the initial state, called start state, and some
(may be none) of which are designed as final states.
2.
An alphabet ∑ of possible input letters.
3.
A finite set of transition that tell for each state and for each letter of the input alphabet which state to go to next.
Clause three in the definition means that every edge labelled by some string of only one letter or strings of letters.
We also know that a transition graph is a collection of three things:
1.
A finite set of states at least one of which is start state and some of which are designed as final state.
2.
An alphabet ∑ of possible input letters.
3.
A finite set of transition(edges,labellesls)that show how to go from some states to another, based on reading specified substring of input letters(possibly even the null string ).
Clause three in the definition means that every edge labelled by some string or strings of letters. Not necessarily only one letter. Transition graph were invented by John Mayhill in 1957.
It is extremely important for us to notice that every FA is also a TG.This means that any picture that represents an FA can be interpreted as a picture of a TG.Of course, not every TG satisfies the definition of an FA. Since every FA can be viewed as a TG that accepts the same language therefore, any language that has been defined by a finite automaton has already been defined by transition graph. But let see some examples that proof this statement true.
Example
CHAPTER
2
PFOOF(KLEENE THEOREM PART I)
Let we have a language L={ ^ } then the FA that accept this language is

4 Kleene’s theorem and the TG that accept this language is
So both FA and TG accept this language.
Example
Let we have a language L={baa} then both FA and TG accept this language
A FA that accept this language is
A TG that accept this language is
Example
The FA below accepts exactly the two strings baa and ab.
CHAPTER 2 PROOF (KLEENE’S THEOREM PART I)

5 Kleene’s theorem
The TG below accept exactly the two strings baa and ab.
When an edge is labelled with the string ^ it means can take a ride it offers free(without consuming any letters from the input string).Remember that we do not have to follow that edges, but we can if we want to.
Example
Let we have a regular expression (a+b)*b.
Language of this regular expression is all words ending on b.
A FA that accept this language is
And the same language also accepted by a TG.TG that accept this language is
CHAPTER 2 PROOF(KLEENE’S THEOREM PART I)

6 Kleene’s theorem
Hence it is proved that every language that can be defined by finite automaton can also be defined by a transition graph.
CHAPTER 3 PROOF(KLEENE’S THEOREM PART II)

7 Kleene’s theorem
3
To prove part II of the theorem, an algorithm consisting of different steps, is explained showing how a
RE can be obtained corresponding to the given TG. For this purpose the notion of TG is changed to that of GTG i.e. the labels of transitions are corresponding REs.Basically this algorithm converts the given TG to GTG with one initial state alone with a single loop, or one initial state connected with one final state by a single transition edge. The label of the loop or the transition edge will be the required RE.

Step 1
If a TG has more than one start states, then introduce a new start state connecting the new state to the old start states by the transitions labelled by Λ and make the old start states the non -start states.
This step can be shown by the following example
Example
CHAPTER 3 PROOF(KLEENE’S THEOREM PART II)

8 Kleene’s theorem
The above TG can be converted to

Step 2:
If a TG has more than one final states, then introduce a new final state, connecting the old final states to the new final state by the transitions labelled by Λ.This step can be shown by the previous example of
TG, where the step 1 has already been processed
Example
The above TG can be converted to
CHAPTER 3 PROOF(KLEENE’S THEOREM PART II)

9 Kleene’s theorem
If a state has two (more than one) incoming transition edges labelled by the corresponding RE ,from the same state (including the possibility of loops at a state), then replace all these transition edges with a single transition edge labelled by the sum of corresponding RE.
This step can be shown by a part of TG in the following example
Example
CHAPTER
3
PROOF(KLEENE’S THEOREM PART II)

10 Kleene’s theorem

If three states in a TG, are connected in sequence then eliminate the middle state and connect the first state with the third by a single transition (include the possibility of circuit as well) labelled by the RE which is the concatenation of corresponding two REs in the existing sequence. This step can be shown by a part of TG in the following example
To eliminate state 5 the above can be reduced to
Consider the following example containing a circuit
Example
Consider the part of a TG, containing a circuit at a state, as shown below
CHAPTER 3 PROOF(KLEENE’S THEOREM PART II)

11 Kleene’s theorem
To eliminate state 4 the above TG can be reduced to
CHAPTER 3 PROOF(KLEENE’S THEOREM PART II)

12 Kleene’s theorem
Note:
It is to be noted that to determine the RE corresponding to a certain TG, four steps have been discussed.
This process can be explained by the following particular examples of TGs
Example
Consider the following TG
To have single final state, the above TG can be reduced to the following
To eliminate states 2 and 3, the above TG can be reduced to the following
CHAPTER
3
PROOF(KLEENE’S THEOREM PART II)

13 Kleene’s theorem
To eliminate state 1 the above TG can be reduced to the following
Hence the required RE is (ab+ba)(aa+b)*(aaa+bba)
Example
Consider the following TG
To have single initial and single final state the above TG can be reduced to the following
CHAPTER
3
PROOF(KLEENE’S THEOREM PART II)

14 Kleene’s theorem
To obtain single transition edge between 1 and 3; 2 and 4, the above can be reduced to the following
To eliminate states 1,2,3 and 4, the above TG can be reduced to the following TG
CHAPTER
3
PROOF(KLEENE’S THEOREM PART II)

15 Kleene’s theorem
To connect the initial state with the final state by single transition edge, the above TG can be reduced to the Following
Hence the required RE is (b+aa)b*+(a+bb)a*
Example
Consider the following TG, accepting EVEN-EVEN language
It is to be noted that since the initial state of this TG is final as well and there is no other final state, so to obtain a TG with single initial and single final state, an additional initial and a final state are introduced as shown in the following TG
To eliminate state 2, the above TG may be reduced to the following
CHAPTER
3
PROOF(KLEENE’S THEOREM PART II)

16 Kleene’s theorem
To have single loop at state 1, the above TG may be reduced to the following
To eliminate state 1, the above TG may be reduced to the following
Hence the required RE is (aa+bb+(ab+ba)(aa+bb)*(ab+ba))*
CHAPTER 3 PROOF(KLEENE’S THEOREM PART II)

17 Kleene’s theorem

Using PROOF(KLEENE’S THEOREM PART II) convert the following TG in to RE.
1.
Let us suppose a TG that has some state(called state x) inside it (not the – or + state)that has more than one loop circling back to itself:
Where r
1 , r
2 , r
3 are all regular expressions or simple string .In this case we replace the three loops by one loop labelled with a regular expression.
The meaning here is that from state x we can read any one string from the input that fits the regular expression r
1
+ r
2
+r
3 and return to the same state.
2.
Suppose two states are connected by more than one edge going in the same direction:
We can replace this with a single edge that is labelled with a regular expression
CHAPTER
3
PROOF(KLEENE’S THEOREM PART II)

18 Kleene’s theorem
3.
Suppose the following TG:
We can convert this TG into the following RE by using bypass and state elimination operation.
4.
Suppose the following TG:
This TG becomes
5. Suppose the following TG:
The above TG becomes
CHAPTER 3 PROOF(KLEENE’S THEOREM PART II)

19 Kleene’s theorem
6.
Suppose the following TG:
The following RE defines the same language that TG defines.
7.
Suppose the following TG:
This becomes
8.
Suppose the following TG:
CHAPTER
3
PROOF(KLEENE’S THEOREM PART II)

20 Kleene’s theorem
It becomes
9.
Suppose the following TG:
In this TG the incoming states are 1 and 3 and the outgoing state is only 1.Therefore, we must add edges connecting state 3 to 1 and 1 to 1.so the machine becomes
CHAPTER 4 PROOF(KLEENE’S THEOREM PART III)

21 Kleene’s theorem
4
If the language can be expressed by a RE then there exists an FA accepting the language. As the regular expression is obtained applying addition, concatenation and closure on the letters of an alphabet and the Null string, so while building the RE, sometimes, the corresponding FA may be built easily, as shown in the following examples
Example
Consider the language, defined over Σ = {a ,b}, consisting of only b, then this language may be accepted by the following FA. which shows that this FA helps in building an FA accepting only one letter
Example
Consider the language, defined over Σ = {a ,b}, consisting of only Λ , then this language may be accepted by the following FA.
CHAPTER
4
PROOF(KLEENE’S THEOREM PART III)

22 Kleene’s theorem
As, if r1 and r2 are regular expressions then their sum, concatenation and closure are also regular expressions, so an FA can be built for any regular expression if the methods can be developed for building the FAs corresponding to the sum, concatenation and closure of the regular expressions along with their FAs. These three methods are explained in the following discussion
Using the FAs corresponding to r1 and r2 an FA can be built, corresponding to r1+ r2. This method can be developed considering the following examples
Example
Let r1 = (a+b)*b defines L1 and the FA1
And r2 = (a+b )*aa(a+b )* defines L2 and FA
Let FA3 be an FA corresponding to r1+ r2, then the initial state of FA3 must correspond to the initial state of FA1 and the initial state of FA2. Since the language corresponding to r1+ r2 is the union of
CHAPTER
4
PROOF(KLEENE’S THEOREM PART III)

23 Kleene’s theorem corresponding languages L1 and L2, consists of the strings belonging to L1or L2 or both, therefore a final state of FA3 must correspond to a final state of FA1 or FA2 or both.
Since, in general, FA3 will be different from both FA1 and FA2, so the labels of the states of FA3 may be supposed to be z1,z2, z3, …, where z1 is supposed to be the initial state. Since z1 corresponds to the states x1 or y1, so there will be two transitions separately for each letter read at z1. It will give two possibilities of states either z1 or different from z1. This process may be expressed in the following transition table for all possible states of FA
3
CHAPTER 4 PROOF(KLEENE’S THEOREM PART III)

24 Kleene’s theorem
RE corresponding to the above FA may be r1+r2 = (a+b)*b + (a+b)*aa(a+b)*.
FA1 contains two states while FA2 contains three states. Hence the total number of possible combinations of states of FA1 and FA2, in sequence, will be six. For each combination the transitions for both a and b can be determined, but using the method in the example, number of states of FA3 was reduced to five.
Example
Let r1 = (a+b)*a and the corresponding FA1 be also r2 = (a+b)((a+b)(a+b))* or ((a+b)(a+b))*(a+b) and FA2 be
FA corresponding to r1+r2 can be determined as
CHAPTER
4
PROOF(KLEENE’S THEOREM PART III)

25 Kleene’s theorem
Using the FAs corresponding to r1 and r2, an FA can be built, corresponding to r1r2. This method can be developed considering the following examples
Example
Let r1 = (a+b)*b defines L1 and FA1 be and r2 = (a+b )*aa (a+b )* defines L2 and FA2 be
CHAPTER 4 PROOF(KLEENE’S THEOREM PART III)

26 Kleene’s theorem
Let FA3 be an FA corresponding to r1r2, then the initial state of FA3 must correspond to the initial state of FA1 and the final state of FA3 must correspond to the final state of FA2.Since the language corresponding to r1r2 is the concatenation of corresponding languages L1 and L2, consists of the strings obtained, concatenating the strings of L
1 to those of L
2
, therefore the moment a final state of first FA is entered, the possibility of the initial state of second FA will be included as well.
Since, in general, FA3 will be different from both FA1 and FA2, so the labels of the states of FA3 may be supposed to be z1,z2, z3, …, where z1 stands for the initial state. Since z1 corresponds to the states x1, so there will be two transitions separately for each letter read at z1. It will give two possibilities of states which correspond to either z1 or different from z1. This process may be expressed in the following transition table for all possible states of FA3
Hence the required FA will be as follows
CHAPTER 4 PROOF(KLEENE’S THEOREM PART III)

27 Kleene’s theorem
Example
Let r1 = ((a+b)(a+b))* and the corresponding FA1 be also r2 = (a+b)((a+b)(a+b))* or ((a+b)(a+b))*(a+b) and FA2 be
FA corresponding to r1r2 can be determined as
CHAPTER
4
PROOF(KLEENE’S THEOREM PART III)

28 Kleene’s theorem
Hence the required FA will be as follows
Building an FA corresponding to r*, using the FA corresponding to r. It is to be noted that if the given FA already accepts the language expressed by the closure of certain RE, then the given FA is the required FA. However the method, in other cases, can be developed considering the following examples Closure of an FA, is same as concatenation of an FA with itself, except that the initial state of the required FA is a final state as well. Here the initial state of given FA, corresponds to the initial state of required FA and a non final state of the required FA as well.
Example
Let r = (a+b)*b and the corresponding FA be then the FA corresponding to r* may be determined as under
CHAPTER 4 PROOF(KLEENE’S THEOREM PART III)

29 Kleene’s theorem
The corresponding transition diagram may be as under
Example
Let r = (a+b)*aa(a+b)* and the corresponding FA be then the FA corresponding to r* may be determined as under
CHAPTER
4
PROOF(KLEENE’S THEOREM PART III)

30 Kleene’s theorem
The corresponding transition diagram may be
Example
Consider the following FA, accepting the language of strings with b as second letter then the FA corresponding to r* may be determined as under
CHAPTER
4
PROOF(KLEENE’S THEOREM PART III)

31 Kleene’s theorem
If at then the FA corresponding to r* may be determined as under he initial state of the given FA, there is either a loop or an incoming transition edge, the initial state corresponds to the final state and a non final state as well, of the required FA, otherwise the initial state of given FA will only correspond to a single state of the required FA (i.e. the initial state which is final as well).
CHAPTER 5 NONDETERMINISTIC FINITE AUTOMATA

32 Kleene’s theorem
5
An NFA is a TG with a unique start state and a property of having single letter as label of transitions. An
NFA is a collection of three things
1.
Finite many states with one initial and some final states
2.
Finite set of input letters, say, Σ = {a, b, c}
3.
Finite set of transitions, showing where to move if a letter is input at certain state ( Λ is not a valid transition), there may be more than one transition for certain letters and there may not be any transition for certain letters
It may be observed, from the definition of NFA, that the string is supposed to be accepted, if there exists at least one successful path, otherwise rejected. It is to be noted that an NFA can be considered to be an intermediate structure between FA and TG. The examples of NFAs can be found in the following
Example
It is to be noted that the above NFA accepts the language consisting of a and ab.
Example
CHAPTER 5 NONDETERMINISTIC FINITE AUTOMATA

33 Kleene’s theorem
It is to be noted that the above NFA accepts the language of strings, defined over Σ = {a , b}, containing aa.
Note
It is to be noted that NFA helps to eliminate a loop at certain state of an FA. This process is done converting the loop into a circuit. But during this process the FA remains no longer FA and is converted to a corresponding
NFA, which is shown in the following example.
Example
Consider a part of the following FA with an alphabet Σ = {a ,b,c,d}
To eliminate the loop at state 7, the corresponding NFA may be as follows
Definition of NFA is different from that of FA since

A FA must have from each state an arc labelled with each letter of alphabet, while NFA does not.

A FA is deterministic, while a NFA may be nondetermisic.

An NFA can have repeated labels from any single state.

NFA allows for human choice to become a factor in selecting a way to process an input string.
CHAPTER
5
NONDETERMINISTIC FINITE AUTOMATA

34 Kleene’s theorem
Definition of NFA is different from that of TG since

A TG can have arcs labelled with substrings of letters while a NFA has arcs labelled with only letters.

A TG can have arcs labelled with ^ while a NFA cannot.

A TG can have more than one start state while a NFA can only have one.
It is to be noted that according to the Keene’s theorem, if a language can be accepted by an FA , then there exists a TG accepting that language. Since, an NFA is a TG as well, therefore there exists an NFA accepting the language accepted by the given FA. In this case these FA and NFA are said to be equivalent to each others.
.

Every FA is an NFA since we can consider an FA to be an NFA without the extra possible features.

Every NFA is a TG.

Kleene’s theorem states that every TG has an equivalent FA.
Following are the examples of FAs to be converted to the equivalent NFAs.
Example
Consider the following FA corresponding to (a+b)*b
CHAPTER
5
NONDETERMINISTIC FINITE AUTOMATA

35 Kleene’s theorem
Can the structure of above NFA be compared with the corresponding RE ?
 NFA useful because i.
Applications in artificial intelligence (AI). ii.
Given two FA’s for two languages with regular expressions r1 and r2, it is easy to construct an NFA to accept language corresponding to regular expression r1 + r2. iii.
This works when neither of the original FA’s has any arcs going into their original initial states. iv.
If one or both of the original FA’s has an arc going into its original initial state, the newly constructed FA for the language corresponding to regular expression r1 + r2 may be incorrect. This is because the new FA may process part of the word on one of the original FA’s and then process the rest of the word on the other FA, and then incorrectly accept the word.