Formal Languages and
Theory of Computation
Taj Mahal in India
Wen-Hsiang Tsai
蔡文祥 講座教授
2011/9
1
Chapter 0
Introduction
Opera House in
Sidney, Australia
2
Outline

0.0 Reasons for Taking This Course

0.1 General Concepts

0.2 Problems Studied in Theory of Computation

0.3 Fields Related to Theory of Computation

0.4 Applications of Theory of Computation

0.5 Brief History of Theory of Computation

0.6 Textbooks and Course Grading
Chapter. 0 – p. 3
0.0 Reasons for Taking This Course

Philosophy of computer science (CS)

Indispensable for complete study of CS

Broadening views in all fields of CS

Enhancing problem solving capability

Basic training for undergraduate students

Background course for graduate students

Qualification exam topic for Ph. D. students.
Chapter. 0 – p. 4
0.1 General Concepts

Course title --“Formal Languages and Theory of
Computation” (正規語言與計算理論)

Scope of course:
 Formal
languages
 Automata theory
 Computability
 Computational complexity
Chapter. 0 – p. 5
0.1 General Concepts

Formal Languages -- Definition
---
Study of “strings of symbols,” including their
 descriptions,
properties,
 generations,
recognitions,
 compiling,
applications, …
Chapter. 0 – p. 6
0.1 General Concepts

Formal Languages -- Concepts
(1/2) ---
Strings of symbols include those of
 natural
languages,
 computer
 discrete
languages,
signals, …
Chapter. 0 – p. 7
0.1 General Concepts

Formal Languages -- Concepts
(2/2) ---
 Results
of study of automatic processing of
symbolic information
 Parallel
to numerical mathematical
processing
 Similar
to the role of set theory (集合論) in
mathematics
Chapter. 0 – p. 8
0.1 General Concepts

Automata Theory --A
question ---
 Do
you know how a vending machine
works? Can you design one?
Vending machine
room seen in
Hokkaido, Japan
2004
Chapter. 0 – p. 9
0.1 General Concepts

Automata Theory -- An
example (1/3) ---
How to design a vending machine?
 Use a finite automaton!
Chapter. 0 – p. 10
0.1 General Concepts

Automata Theory -- An
example (2/3) ---
 Assumptions
(for simplicity):

Only NT 5-dollar and 10-dollar coins are used.

Only drinks all of 20 dollars are sold.
Chapter. 0 – p. 11
0.1 General Concepts

Automata Theory --
An example (3/3) --Requiring “memory” called “states” for the
design.
$10
$5
start
$0
$5
$10

$15
$5
$10
$5 $10
$10
$20
$5
transition diagram
Notes: see next page.
Chapter. 0 – p. 12
0.1 General Concepts

Automata Theory --
Notes for the last diagram:
 $10

start

$20
$5 is returned as “output” (not shown)
$0
“Start state”
“Final state”
Chapter. 0 – p. 13
0.1 General Concepts

Automata Theory -- Definition
---
Study of dynamic behaviors of “discreteparameter information systems” in form of
“abstract computing devices” or “machines.”
Chapter. 0 – p. 14
0.1 General Concepts

Automata Theory -- Examples
of discrete-parameter
information systems -- digital
systems, nerve systems, languages
 information
transmission systems
 human-environment
interactions, …
Chapter. 0 – p. 15
0.1 General Concepts

Automata Theory -- Three
major models of automata ---
 generator
 acceptor
--- with output and without input
--- with input and without output
 transducer
--- both with input and with output
Yes
or
No
trigger
signal
generator
acceptor
transducer
Chapter. 0 – p. 16
0.1 General Concepts

Automata Theory -- Examples
---
 generator

---
“natural language” grammar
(generating “sentences” spoken by people)

reception robot
(speaking organized words and sentences)

context-free grammar
(generating strings of symbols) ***
Reception robot
--- Expo 2005
(*** abstract models studied in this course)
Chapter. 0 – p. 17
0.1 General Concepts

Automata Theory -- Examples
 acceptor

-----
digital lock
(accepting digits)

lexical analyzer
(recognizing computer language keywords)

finite automaton
Digital lock
(accepting valid strings of symbols) ***
(*** abstract models studied in this course)
Chapter. 0 – p. 18
0.1 General Concepts

Automata Theory -- Examples
---
 transducer

---
Interpreter
(translating natural languages)

Compiler
(translating high-level languages into
machine codes)

Turing machine
(transforming strings of symbols) ***
(*** abstract models studied in this course)
“Interpreter”
雙面翻譯
(a movie)
Chapter. 0 – p. 19
0.1 General Concepts

Computability -- Definition
---
Study of problem solving capabilities of
computational models.
Chapter. 0 – p. 20
0.1 General Concepts

Computability -- Problem
types based on resources ---
 Impossible
problems
 Possible-with-unlimited-resources-but-
impossible-with-limited-resources problems
 Possible-with-limited-resources
problems
Chapter. 0 – p. 21
0.1 General Concepts

Computability -- Problem
types based on time ---
 Undecidable
 Intractable
 Tractable
problems
problems
problems
Chapter. 0 – p. 22
0.1 General Concepts

Computability -- Advantage
---
Studies of computability help us not to
waste time on “unsolvable problems”
already investigated before.
Chapter. 0 – p. 23
0.1 General Concepts

Computational complexity -- Definition
---
Study of “efficiency” of problem solving.
Chapter. 0 – p. 24
0.1 General Concepts

Computational complexity -- Concepts
 To
(1/2) ---
unify comparison, an abstract model is
needed as the machine for executing
problem solutions.
 Usually
the most famous “Turing machine”
(an automaton) is used.
Chapter. 0 – p. 25
0.1 General Concepts

Computational complexity -- Concepts
 Turing
(2/2) ---
machine, though simple, has been
proved to be able to simulate any problem
solving steps (“algorithms”) designed by
human beings!
 Turing
machine is the foundation for
development of modern computation theory!
Chapter. 0 – p. 26
0.2 Problems Studied in Theory of
Computation

Essential problems studied -- “What
are the fundamental capabilities and
limitations of computers?”
 What
can a computer do at all? ---
studied in the domain of Computability!
 What
can a computer do efficiently? --- studied
in the domain of Computational complexity!
Chapter. 0 – p. 27
0.2 Problems Studied in Theory of
Computation

A basic concept -- A Turing
machine conceptually is no
more than a pen and a paper with
several reasoning rules!
？
？
=
=
Turing
machine
Chapter. 0 – p. 28
0.3 Fields Related to Scope of
Theory of Computation
Fields
Related theory
Compiling theory
Formal languages
Switching circuit theory
Automata theory
Algorithm analysis
Computational complexity
Natural language processing
Formal languages
Syntactic pattern recognition
Formal languages
Programming languages
Formal languages
Artificial intelligence
Formal languages and automata theory
Neural networks
Automata theory
Chapter. 0 – p. 29
0.4 Applications of Theory of
Computation

Applications -- Text
analysis ---
 text
search (文辭搜尋)
 text
editing (文句編輯)
 Compiler
design (編譯器設計)
 lexical
analysis (辭彙分析)
 parser
generation (剖析器產生)
Chapter. 0 – p. 30
0.4 Applications of Theory of
Computation

Applications -- Language
design
 programming
language design (程式語言設計)
 document description language design (公文
描述語言設計)

e.g., HTML, XML, …
 picture

language design (圖畫語言設計)
e.g., SVG, VHML, …
 special
language design (特別語言設計)
Chapter. 0 – p. 31
0.4 Applications of Theory of
Computation

Applications -- Digital
system design
 computer
design (計算機設計)
 special digital system design (特殊數位系統
設計)
 Protocol
modeling and verification (網路
協定模式化與驗証)
 Expert system design(專家系統設計)
 Cryptography(密碼學) …
Chapter. 0 – p. 32
0.4 Applications of Theory of
Computation
 A specific application --- special
digital system design
 Mainly
based on automata theory ---
using finite automata or “sequential
machines” (finite automata both with input
and with output)
 Some
specific examples of digital
systems -- Elevators,
vending machines, traffic signal
control systems …
Chapter. 0 – p. 33
0.5 Brief History of Theory of
Computation

At beginning of 19th century,
mathematicians believe:
Axiomatization of mathematics
(數學公理化)
 That
is,
it is possible to decide the truth of any
mathematical statement thru use of
inference rules and axiom systems.
Chapter. 0 – p. 34
0.5 Brief History of Theory of
Computation

In 1931, Kurt Gödel proposed the
incompleteness theorem which says:
the truth of certain mathematical
statements cannot be proved by the
use of mathematical axioms.
 That
is,
some mathematical problems cannot
be solved by algorithms.
Chapter. 0 – p. 35
0.5 Brief History of Theory of
Computation

From then, the ideal of axiomatization
of mathematics was broken.
 Then,
mathematicians started to
 think
the scope of algorithm capabilities,
 design
 study
theoretical computational machines, &
their capability limitations.
Chapter. 0 – p. 36
0.5 Brief History of Theory of
Computation

Many theories were proposed, but the
corresponding machines could not be
manufactured at that time.
Chapter. 0 – p. 37
0.5 Brief History of Theory of
Computation

In 1936, Turing proposed the model
of
universal algorithm machine,
which later was called
Turing machine
mentioned before.
Chapter. 0 – p. 38
0.5 Brief History of Theory of
Computation

Some similar ideas were proposed by
other scholars, including
Stephen C. Kleene,
Alonzo Church,
Emil Post, etc.,
in different forms of models.
Chapter. 0 – p. 39
0.5 Brief History of Theory of
Computation

In 1943, neural physiologists
Warren S. McCulloch and
Walter Pitts
developed finite-state systems to
simulate neural networks in biological
systems.

They are pioneers of automata theory.
Chapter. 0 – p. 40
0.5 Brief History of Theory of
Computation

In late 1940’s，Von Neumann
proposed the idea of
stored program
for computer models.
Chapter. 0 – p. 41
0.5 Brief History of Theory of
Computation

In 1951, a real computer following the
idea of “stored program” was
constructed:
UNIVAC I,
the world's first commercially available
computer (manufactured by EckertMauchly Computer Company) .
Chapter. 0 – p. 42
0.5 Brief History of Theory of
Computation

In late 1950’s, linguist N. Chomsky
proposed a mathematical model for
grammars of natural languages.

In 1956, he proposed further the
concept of context-free grammar,
which may be used for defining
computer languages.
Chapter. 0 – p. 43
0.5 Brief History of Theory of
Computation

In 1956, Kleene proposed the
concept of finite automaton for
simulating neural networks proposed
by McCulloch and Pitts.

He also proposed regular expressions
to describe strings of symbols, and
proved them equivalent to finite
automata.
Chapter. 0 – p. 44
0.5 Brief History of Theory of
Computation

In 1959 and 1960, John W. Backus &
Peter Naur proposed sequentially a
special expression for computer
language grammars, called later
Backus-Naur form (BNF) .
 It
was used to describe computer
language ALGOL-60, leading to
development of compiling theory.
Chapter. 0 – p. 45
0.5 Brief History of Theory of
Computation

In 1969, Stephen A. Cook found that
computation problems may be
separated into tractable and intractable
ones.
 Intractable
problems are also called NPhard problems.
 Such problems cannot be solved by
computers except very small instances
(with only small-sized inputs).
Chapter. 0 – p. 46
0.6 Textbooks and Course Grading

Textbook:


Hopcroft, Motwani, Ullman, Introduction to
Automata Theory, Languages, and
Computation (3rd Edition), Addison-Wesley,
Boston, MA, USA, 2007
Reference textbook:

蔡文祥, 正規語言與計算理論 (Formal
Languages & The Theory of Computation)(尚未
出版書籍；中文) (參考用)
Chapter. 0 – p. 47
0.6 Textbooks and Course Grading


Course grading:

Homeworks
33.33%

mid-term exam 33.33%

final exam
33.33%
Website for downloading course material
and quizzes :
http://www.cis.nctu.edu.tw/~whtsai/
Chapter. 0 – p. 48