Problems, Problem Spaces
and Search
DeSiaMore
DeSiaMore
Powered by DeSiaMore
1
Contents
•
•
•
•
•
•
•
•
•
•
•
Defining the problem as a State Space Search
Production Systems
Control Strategies
Breadth First Search
Depth First Search
Heuristic Search
Problem Characteristics
Is the Problem Decomposable?
Can Solution Steps be ignored or undone?
Production system characteristics
Issues in the design of search programs
DeSiaMore
Powered by DeSiaMore
2
To build a system to solve a
problem
1. Define the problem precisely
2. Analyse the problem
3. Isolate and represent the task knowledge
that is necessary to solve the problem
4. Choose the best problem-solving
techniques and apply it to the particular
problem.
DeSiaMore
Powered by DeSiaMore
3
Defining the problem as State
Space Search
• The state space representation forms the basis of most
of the AI methods.
• Its structure corresponds to the structure of problem
solving in two important ways:
– It allows for a formal definition of a problem as the need to
convert some given situation into some desired situation using a
set of permissible operations.
– It permits us to define the process of solving a particular problem
as a combination of known techniques (each represented as a
rule defining a single step in the space) and search, the general
technique of exploring the space to try to find some path from
current state to a goal state.
– Search is a very important process in the solution of hard
problems for which no more direct techniques are available.
DeSiaMore
Powered by DeSiaMore
4
Example: Playing Chess
• To build a program that could “play chess”, we
could first have to specify the starting position of
the chess board, the rules that define the legal
moves, and the board positions that represent a
win for one side or the other.
• In addition, we must make explicit the previously
implicit goal of not only playing the legal game of
chess but also winning the game, if possible,
DeSiaMore
Powered by DeSiaMore
5
Playing chess
•
•
•
•
The starting position can be described as an 8by 8 array where each
position contains a symbol for appropriate piece.
We can define as our goal the check mate position.
The legal moves provide the way of getting from initial state to a goal state.
They can be described easily as a set of rules consisting of two parts:
– A left side that serves as a pattern to be matched against the current board
position.
– And a right side that describes the change to be made to reflect the move
•
•
However, this approach leads to large number of rules 10120 board positions
!!
Using so many rules poses problems such as:
– No person could ever supply a complete set of such rules.
– No program could easily handle all those rules. Just storing so many rules poses
serious difficulties.
DeSiaMore
Powered by DeSiaMore
6
Defining chess problem as State
Space search
• We need to write the rules describing the legal
moves in as general a way as possible.
• For example:
– White pawn at Square( file e, rank 2) AND Square(
File e, rank 3) is empty AND Square(file e, rank 4) is
empty, then move the pawn from Square( file e, rank
2) to Square( file e, rank 4).
• In general, the more succintly we can describe
the rules we need, the less work we will have to
do to provide them and more efficient the
program.
DeSiaMore
Powered by DeSiaMore
7
Water Jug Problem
• The state space for this problem can be
described as the set of ordered pairs of
integers (x,y) such that x = 0, 1,2, 3 or 4
and y = 0,1,2 or 3; x represents the
number of gallons of water in the 4-gallon
jug and y represents the quantity of water
in 3-gallon jug
• The start state is (0,0)
• The goal state is (2,n)
DeSiaMore
Powered by DeSiaMore
8
Production rules for Water Jug
Problem
•
The operators to be used to solve the
problem can be described as follows:
Sl No
Current state
Next State
Descritpion
1
(x,y) if x < 4
(4,y)
Fill the 4 gallon jug
2
(x,y) if y <3
(x,3)
Fill the 3 gallon jug
3
(x,y) if x > 0
(x-d, y)
Pour some water out of the 4
gallon jug
4
(x,y) if y > 0
(x, y-d)
Pour some water out of the 3gallon jug
5
(x,y) if x>0
(0, y)
Empty the 4 gallon jug
6
(x,y) if y >0
(x,0)
Empty the 3 gallon jug on the
ground
7
(x,y) if x+y >= 4 and y >0
(4, y-(4-x))
Pour water from the 3 –
gallon jug into the 4 –gallon
jug until the 4-gallon jug is full
DeSiaMore
Powered by DeSiaMore
9
Production rules
8
(x, y) if x+y >= 3 and x>0
(x-(3-y), 3)
Pour water from the 4-gallon jug
into the 3-gallon jug until the 3gallon jug is full
9
(x, y) if x+y <=4 and y>0
(x+y, 0)
Pour all the water from the 3gallon jug into the 4-gallon jug
10
(x, y) if x+y <= 3 and x>0
(0, x+y)
Pour all the water from the 4gallon jug into the 3-gallon jug
11
(0,2)
(2,0)
Pour the 2 gallons from 3-gallon
jug into the 4-gallon jug
12
(2,y)
(0,y)
Empty the 2 gallons in the 4gallon jug on the ground
DeSiaMore
Powered by DeSiaMore
10
To solve the water jug problem
•
•
•
Required a control structure that
loops through a simple cycle in
which some rule whose left side
matches the current state is
chosen, the appropriate change to
the state is made as described in
the corresponding right side, and
the resulting state is checked to
see if it corresponds to goal state.
One solution to the water jug
problem
Shortest such sequence will have
a impact on the choice of
appropriate mechanism to guide
the search for solution.
DeSiaMore
Gallons in the
4-gallon jug
Gallons in the
3-gallon jug
Rule applied
0
0
2
0
3
9
3
0
2
3
3
7
4
2
5 or 12
0
2
9 0r 11
2
Powered by DeSiaMore
0
11
Formal Description of the problem
1. Define a state space that contains all the
possible configurations of the relevant objects.
2. Specify one or more states within that space
that describe possible situations from which
the problem solving process may start ( initial
state)
3. Specify one or more states that would be
acceptable as solutions to the problem. ( goal
states)
4. Specify a set of rules that describe the actions
( operations) available.
DeSiaMore
Powered by DeSiaMore
12
Production Systems
A production system consists of:
• A set of rules, each consisting of a left side that determines the
applicability of the rule and a right side that describes the operation
to be performed if that rule is applied.
• One or more knowledge/databases that contain whatever
information is appropriate for the particular task. Some parts of the
database may be permanent, while other parts of it may pertain only
to the solution of the current problem.
• A control strategy that specifies the order in which the rules will be
compared to the database and a way of resolving the conflicts that
arise when several rules match at once.
• A rule applier
DeSiaMore
Powered by DeSiaMore
13
Production system
Inorder to solve a problem:
• We must first reduce it to one for which a precise
statement can be given. This can be done by
defining the problem’s state space ( start and
goal states) and a set of operators for moving
that space.
• The problem can then be solved by searching
for a path through the space from an initial state
to a goal state.
• The process of solving the problem can usefully
be modelled as a production system.
DeSiaMore
Powered by DeSiaMore
14
Control Strategies
• How to decide which rule to apply next
during the process of searching for a
solution to a problem?
• The two requirements of good control
strategy are that
– it should cause motion.
– It should be systematic
DeSiaMore
Powered by DeSiaMore
15
Breadth First Search
•
Algorithm:
1.
2.
Create a variable called NODE-LIST and set it to
initial state
Until a goal state is found or NODE-LIST is empty
do
a.
b.
Remove the first element from NODE-LIST and call it E. If
NODE-LIST was empty, quit
For each way that each rule can match the state described
in E do:
i. Apply the rule to generate a new state
ii. If the new state is a goal state, quit and return this state
iii. Otherwise, add the new state to the end of NODE-LIST
DeSiaMore
Powered by DeSiaMore
16
BFS Tree for Water Jug problem
(0,0)
(4,0)
(4,3)
DeSiaMore
(0,3)
(0,0)
(1,3)
(4,3)
Powered by DeSiaMore
(0,0)
(3,0)
17
Algorithm: Depth First Search
1. If the initial state is a goal state, quit and
return success
2. Otherwise, do the following until success
or failure is signaled:
a. Generate a successor, E, of initial state. If
there are no more successors, signal failure.
b. Call Depth-First Search, with E as the initial
state
c. If success is returned, signal success.
Otherwise continue in this loop.
DeSiaMore
Powered by DeSiaMore
18
Backtracking
• In this search, we pursue a singal branch of the tree until
it yields a solution or until a decision to terminate the
path is made.
• It makes sense to terminate a path if it reaches deadend, produces a previous state. In such a state
backtracking occurs
• Chronological Backtracking: Order in which steps are
undone depends only on the temporal sequence in
which steps were initially made.
• Specifically most recent step is always the first to be
undone.
• This is also simple backtracking.
DeSiaMore
Powered by DeSiaMore
19
Advantages of Depth-First Search
• DFS requires less memory since only the
nodes on the current path are stored.
• By chance, DFS may find a solution
without examining much of the search
space at all.
DeSiaMore
Powered by DeSiaMore
20
Advantages of BFS
• BFS will not get trapped exploring a blind
alley.
• If there is a solution, BFS is guarnateed to
find it.
• If there are multiple solutions, then a
minimal solution will be found.
DeSiaMore
Powered by DeSiaMore
21
TSP
• A simple motion causing and systematic control structure
could solve this problem.
• Simply explore all possible paths in the tree and return
the shortest path.
• If there are N cities, then number of different paths
among them is 1.2….(N-1) or (N-1)!
• The time to examine single path is proportional to N
• So the total time required to perform this search is
proportional to N!
• For 10 cities, 10! = 3,628,800
• This phenomenon is called Combinatorial explosion.
DeSiaMore
Powered by DeSiaMore
22
Branch and Bound
• Begin generating complete paths, keeping track
of the shortest path found so far.
• Give up exploring any path as soon as its partial
length becomes greater than the shortest path
found so far.
• Using this algorithm, we are guaranteed to find
the shortest path.
• It still requires exponential time.
• The time it saves depends on the order in which
paths are explored.
DeSiaMore
Powered by DeSiaMore
23
Heuristic Search
• A Heuristic is a technique that improves the efficiency of a search
process, possibly by sacrificing claims of completeness.
• Heuristics are like tour guides
• They are good to the extent that they point in generally interesting
directions;
• They are bad to the extent that they may miss points of interest to
particular individuals.
• On the average they improve the quality of the paths that are
explored.
• Using Heuristics, we can hope to get good ( though possibly
nonoptimal ) solutions to hard problems such asa TSP in non
exponential time.
• There are good general purpose heuristics that are useful in a wide
variety of problem domains.
• Special purpose heuristics exploit domain specific knowledge
DeSiaMore
Powered by DeSiaMore
24
Nearest Neighbour Heuristic
•
•
It works by selecting locally superior alternative at
each step.
Applying to TSP:
1.
2.
3.
–
–
Arbitrarily select a starting city
To select the next city, look at all cities not yet visited and
select the one closest to the current city. Go to next step.
Repeat step 2 until all cities have been visited.
This procedure executes in time proportional to N2
It is possible to prove an upper bound on the error it
incurs. This provides reassurance that one is not
paying too high a price in accuracy for speed.
DeSiaMore
Powered by DeSiaMore
25
Heuristic Function
• This is a function that maps from problem state
descriptions to measures of desirsability, usually
represented as numbers.
– Which aspects of the problem state are considered,
– how those aspects are evaluated, and
– the weights given to individual aspects are chosen in such a way
that
• the value of the heuristic function at a given node in the
search process gives as good an estimate as possible of
whether that node is on the desired path to a solution.
• Well designed heuristic functions can play an important
part in efficiently guiding a search process toward a
solution.
DeSiaMore
Powered by DeSiaMore
26
Example Simple Heuristic functions
• Chess : The material advantage of our
side over opponent.
• TSP: the sum of distances so far
• Tic-Tac-Toe: 1 for each row in which we
could win and in we already have one
piece plus 2 for each such row in we have
two pieces
DeSiaMore
Powered by DeSiaMore
27
Problem Characteristics
• Inorder to choose the most appropriate method for a
particular problem, it is necessary to analyze the
problem along several key dimensions:
– Is the problem decomposable into a set of independent smaller
or easier subproblems?
– Can solution steps be ignored or at least undone if they prove
unwise?
– Is the problem’s universe predictable?
– Is a good solution to the problem obvious without comparison to
all other possible solutions?
– Is the desired solution a state of the world or a path to a state?
– Is a large amount of knowledge absolutely required to solve the
problem or is knowledge important only to constrain the search?
– Can a computer that is simply given the problem return the
solution or will the solution of the problem require interaction
between the computer and a person?
DeSiaMore
Powered by DeSiaMore
28
Is the problem Decomposable?
• Whether the problem can be decomposed
into smaller problems?
• Using the technique of problem
decomposition, we can often solve very
large problems easily.
DeSiaMore
Powered by DeSiaMore
29
Blocks World Problem
• Following operators
are available:
• CLEAR(x) [ block x
has nothing on it]->
ON(x, Table)
• CLEAR(x) and
CLEAR(y) ->
ON(x,y) [ put x on y]
Start: ON(C,A)
C
A
B
B
C
Goal:
ON(B,C) and
ON(A,B)
ON(B,C) and ON(A,B)
ON(B,C)
ON(B,C)
DeSiaMore
A
Powered by DeSiaMore
ON(A,B)
CLEAR(A)
CLEAR(A)
ON(A,B)
30
ON(A,B)
Can Solution Steps be ignored or
undone?
• Suppose we are trying to prove a math theorem.We can prove a
lemma. If we find the lemma is not of any help, we can still continue.
• 8-puzzle problem
• Chess: A move cannot be taken back.
• Important classes of problems:
– Ignorable ( theorem proving)
– Recoverable ( 8-puzzle)
– Irrecoverable ( Chess)
• The recoverability of a problem plays an important role in
determining the complexity of the control structure necessary for the
problem’s solution.
– Ignorable problems can be solved using a simple control structure that
never backtracks
– Recoverable problems can be solved by a slightly more complicated
control strategy that does sometimes make mistakes
– Irrecoverable problems will need to be solved by systems that expends
a great deal of effort making each decision since decision must be final.
DeSiaMore
Powered by DeSiaMore
31
Is the universe Predictable?
• Certain Outcome ( ex: 8-puzzle)
• Uncertain Outcome ( ex: Bridge, controlling a
robot arm)
• For solving certain outcome problems, open
loop approach ( without feedback) will work fine.
• For uncertain-outcome problems, planning can
at best generate a sequence of operators that
has a good probability of leading to a solution.
We need to allow for a process of plan revision
to take place.
DeSiaMore
Powered by DeSiaMore
32
Is a good solution absolute or
relative?
• Any path problem
• Best path problem
• Any path problems can often be solved in
a reasonable amount of time by using
heuristics that suggest good paths to
explore.
• Best path problems are computationally
harder.
DeSiaMore
Powered by DeSiaMore
33
Is the solution a state or a path?
• Examples:
– Finding a consistent interpretation for the sentence
“The bank president ate a dish of pasta salad with the
fork”. We need to find the interpretation but not the
record of the processing.
– Water jug : Here it is not sufficient to report that we
have solved , but the path that we found to the state
(2,0). Thus the a statement of a solution to this
problem must be a sequence of operations ( Plan)
that produces the final state.
DeSiaMore
Powered by DeSiaMore
34
What is the role of knowledge?
• Two examples:
– Chess: Knowledge is required to constrain the search for a solution
– Newspaper story understanding: Lot of knowledge is required even to
be able to recognize a solution.
• Consider a problem of scanning daily newspapers to decide which
are supporting the democrats and which are supporting the
republicans in some election. We need lots of knowledge to answer
such questions as:
– The names of the candidates in each party
– The facts that if the major thing you want to see done is have taxes
lowered, you are probably supporting the republicans
– The fact that if the major thing you want to see done is improved
education for minority students, you are probably supporting the
democrats.
– etc
DeSiaMore
Powered by DeSiaMore
35
Does the task require Interaction
with a person?
• The programs require intermediate
interaction with people for additional inputs
and to provided reassurance to the user.
• There are two types of programs:
– Solitary
– Conversational:
• Decision on using one of these
approaches will be important in the choice
of problem solving method.
DeSiaMore
Powered by DeSiaMore
36
Problem Classification
• There are several broad classes into
which the problems fall. These classes
can each be associated with generic
control strategy that is appropriate for
solving the problems:
– Classification : ex: medical diagnostics,
diagnosis of faults in mechanical devices
– Propose and Refine: ex: design and planning
DeSiaMore
Powered by DeSiaMore
37
Production System Characteristics
1.
Can production systems, like problems, be described by a set of
characteristics that shed some light on how they can easily be
implemented?
If so, what relationships are there between problem types and the
types of production systems best suited to solving the problems?
Classes of Production systems:
2.
•
–
–
–
–
DeSiaMore
Monotonic Production System: the application of a rule never
prevents the later application of another rule that could also have
been applied at the time the first rule was selected.
Non-Monotonic Production system
Partially commutative Production system: property that if application
of a particular sequence of rules transforms state x to state y, then
permutation of those rules allowable, also transforms state x into
state y.
Commutative Production system
Powered by DeSiaMore
38
Monotonic Production Systems
• Production system in which the application
of a rule never prevents the later
application of another rule that could also
have been applied at the time the first rule
was applied.
• i.e., rules are independent.
DeSiaMore
Powered by DeSiaMore
39
Commutative Production system
• A partially Commutative production system
has a property that if the application of a
particular sequence of rules transform
state x into state y, then any permutation
of those rules that is allowable, also
transforms state x into state y.
• A Commutative production system is a
production system that is both monotonic
and partially commutative.
DeSiaMore
Powered by DeSiaMore
40
Partially Commutative, Monotonic
• These production systems are useful for solving
ignorable problems.
• Example: Theorem Proving
• They can be implemented without the ability to backtrack
to previous states when it is discovered that an incorrect
path has been followed.
• This often results in a considerable increase in
efficiency, particularly because since the database will
never have to be restored, It is not necessary to keep
track of where in the search process every change was
made.
• They are good for problems where things do not change;
new things get created.
DeSiaMore
Powered by DeSiaMore
41
Non Monotonic, Partially
Commutative
• Useful for problems in which changes occur but
can be reversed and in which order of
operations is not critical.
• Example: Robot Navigation, 8-puzzle, blocks
world
• Suppose the robot has the following ops: go
North (N), go East (E), go South (S), go West
(W). To reach its goal, it does not matter
whether the robot executes the N-N-E or N-E-N.
DeSiaMore
Powered by DeSiaMore
42
Not partially Commutative
• Problems in which irreversible change occurs
• Example: chemical synthesis
• The ops can be :Add chemical x to the pot, Change the temperature
to t degrees.
• These ops may cause irreversible changes to the potion being
brewed.
• The order in which they are performed can be very important in
determining the final output.
• (X+y) +z is not the same as (z+y) +x
• Non partially commutative production systems are less likely to
produce the same node many times in search process.
• When dealing with ones that describe irreversible processes, it is
partially important to make correct decisions the first time, although
if the universe is predictable, planning can be used to make that less
important.
DeSiaMore
Powered by DeSiaMore
43
Non
DeSiaMore
Powered by DeSiaMore
44
Four Categories of Production
System
Monotonic
NonMonotonic
Partially
Commutative
Theorem
proving
Robot
Navigation
Not Partially
Commutative
Chemical
Synthesis
Bridge
DeSiaMore
Powered by DeSiaMore
45
Issues in the design of search
programs
• The direction in which to conduct the
search ( forward versus backward
reasoning).
• How to select applicable rules ( Matching)
• How to represent each node of the search
process ( knowledge representation
problem)
DeSiaMore
Powered by DeSiaMore
46
Summary
•
Four steps for designing a program to
solve a problem:
1. Define the problem precisely
2. Analyse the problem
3. Identify and represent the knowledge
required by the task
4. Choose one or more techniques for problem
solving and apply those techniques to the
problem.
DeSiaMore
Powered by DeSiaMore
47