Introduction
I Environments
1.1
Fully vs. Partially Observable
1.2.
Single Agent vs. Multi Agent
1.3.
Deterministic vs. Stochastic
1.4.
Episodic vs. Sequential
1.5. Static vs. Dynamic
1.6.
Discrete vs. Continuous
1.7
Known vs. Unknown
II Agents
2.1 Simple Reflex Agent
2.2
Model-based Reflex Agent
2.3
Goal-based Agent
2.4
Utility-based Agent
2.5
Learning Agent
2.6
Other types of Agents
III Logic
3.1
Propositional Logic
3.2
Forward and Backward Chaining
3.3
First-Order Logic
IV Agent performance
V Future of Artificial Intelligence
Bibliography
1
Introduction
What is artificial intelligence (AI)? Where can it be used? Why should we
use artificial intelligence? The technology of using artificial intelligence has been
discovered many years ago when humans were in need of making faster decisions
or performing some tasks much faster than they were able to. The field of AI was
presented as research at a conference on the campus of Dartmouth College in 1956.
AI techniques are very various and there are too many of them to list.
Instead, providing a few examples of the advantages of using AI will be helpful.
An expert system that asks patient questions about the health and then detects a
possible diagnosis is not complex but at the same time creative example of using
AI in medicine. Another example of using AI in our life is a credit card application
processing system. This system takes the information from an applicant including
but not limited to1: employment status, annual salary, marital status, and credit
score.
I have shown only two examples of using AI, but these two examples were
chosen from completely independent and non-related areas. It is easy to note that
AI can be applied to everything where humans operate. As life goes by, everything
becomes computerized and takes less time: the decision making systems make
decisions, the robots to space to gather the information; but all these systems or
robots are not fully computerized, sometimes the system or the agent is not sure
which decision is the most appropriate to the problem. The human help is needed
in such cases. On the other hand, it is good that the robots, at this point, are not
able to fully substitute humans. At the same time, new ways of thinking and
constructing more complex problems occur. One of them is Machine Learning
(ML) – is the study [AI 61] of computer algorithms that improve automatically
through experience. Machine learning can be divided into two areas:

Supervised learning
This type of ML includes both classification and numerical regression.
Classification is used to determine what category something belongs
in, after seeing a number of examples of things from various
categories. Regression is the attempt to produce a function that
describes the relationship between inputs and outputs and predicts
how the outputs should change as the inputs change [AI 64].
2
 Unsupervised learning
Unsupervised learning is the ability to find patterns in a stream of
input.
The other very “hot” direction of research in the field of AI is Natural
Language Processing (NLP). NLP gives machines [AI 70] the ability to read and
understand the languages that humans speak. Many MLP applications are found
very useful in the industry. A powerful NLP system would enable natural language
user interfaces and the acquisition of knowledge directly from human-written
sources. Some obvious applications of NLP include text mining and machine
translation. A common method or processing and extracting meaning from natural
language is through semantic indexing. Increases in processing speeds and the drop
in the cost of data storage makes indexing large volumes of abstractions of the
users input much more efficient.
Any system, including AI systems, needs an agent which will be operating
in such system. There exist different types of agents operating in various
environments. I will illustrate how to design agents, how different the
environments are constructed, and provide an example of different types of agents
operating in the same environment to evaluate the agents’ performance.
3
I Environments
Let’s begin with specifying a task environment, and then illustrate the
process and provide some examples. The task environment can be described by
four components. These components are as following: Performance, Environment,
Actuators and Sensors. For compactness, we will use the common abbreviation:
PEAS.
The range of task environments that exists in AI is wide. We can, however,
identify a relatively small number of dimensions along which task environments
can be categorized. Table 1 provides some examples of agent types and their PEAS
descriptions. Chapter II will provide with detailed explanations of types of agents.
As we defined before, PEAS states for Performance, Environment, Actuators
and sensor. The example provided in Table 1 will be helpful. Let’s now consider
each of the components:
 Performance
Performance is a function that measures the quality of the actions the
agent executed.
 Environment
The environment is an area in which the agent operates. The types of
different environments will be described later in this chapter.
 Actuators
Actuators are the set of devices that the agent can use to perform
actions.
 Sensors
Sensors allow the agent to collect the sequence of percepts that will be
used for deliberating on the next action.
4
Agent Type
Performance
measure
Environment
Actuators
Sensors
Medical
diagnosis system
Healthy patient,
reduced costs,
time savings
Patient, hospital,
staff
Display of
questions, tests,
diagnoses,
medicine,
referrals
Keyboard entry
of symptoms,
findings,
patient’s answers
Satellite image
analysis system
Correct image
categorization
Downlink from
satellite
Display of scene
categorization
Color pixel
arrays
Part-picking
robot
Percentage of
parts in correct
bins
Conveyor belt
with parts
Jointed arm and
hand
Camera, joint
angle sensors
Refinery
controller
Purity, yield,
safety
Refinery,
operators
Valves, pumps,
beaters, displays
Temperature,
pressure,
chemical sensors
Interactive Math
tutor
Student’s score
on exam
Students, exam
sheets
Display of
exercises,
suggestions,
corrections
Keyboard Entry
Taxi driver
Speed,
comfortable ride,
maximizing
profits
Roads, traffic,
pedestrians,
passengers,
traffic lights
Steering wheel,
accelerator,
gears, breaks,
horn
Camera,
speedometer,
odometer, GPS,
petroleum tank
Train operator
Speed, stops,
maximizing
profits
Rails, traffic,
traffic lights,
passengers
Control unit,
traffic lights,
breaks,
accelerator
Camera, GPS,
speedometer,
petroleum tank
Table 1. Examples of agent types and their PEAS descriptions
I find very helpful providing an example of some environments and there
descriptions. This will help to understand the properties of environments with the
same agents that were described in Table 1. These examples will be provided in
Table 2.
5
Task
environment
Observable
Agents
Deterministic
Episodic
Static
Medical
Diagnosis
System
Partially
Single
Stochastic
Sequential
Dynamic Continuous
Satellite
Image
Analysis
System
Fully
Single
Deterministic
Episodic
Semi
Part-picking
robot
Partially
Single
Stochastic
Episodic
Dynamic Continuous
Refinery
controller
Partially
Single
Stochastic
Sequential
Dynamic Continuous
Interactive
Math Tutor
Partially
Multi
Stochastic
Sequential
Dynamic Discrete
Taxi driver
Partially
Multi
Stochastic
Sequential
Dynamic Continuous
Train operator Partially
Multi
Stochastic
Sequential
Dynamic Continuous
Table 2. Properties of the environment
6
Discrete
Continuous
1.1 Fully observable vs. Partially observable
If an agent's sensors give it access to the complete state of the environment
at each point in time, then we say that the task environment is fully observable. A
task environment is effectively fully observable if the sensors detect all aspects that
are relevant to the choice of action; relevance, in turn, depends on the performance
measure. Fully observable environments are convenient because the agent need not
maintain any internal state to keep track of the world. An environment might be
partially observable because of noisy and inaccurate sensors or because parts of the
state are simply missing from the sensor data—for example, a train operator is in
the partially observable environment because the agent simply doesn’t know if
there are any oncoming trains in one mile. Another example, XXX. It’s important
to note that if the agent does not have any sensors, then environment is
unobservable.
7
1.2 Single vs. Multi agent
The difference between single and multi agent environments is clear and
simple. For example, an agent sorting the cards is considered to be operating in the
single agent environment until there is a different agent operating simultaneously;
then these two agents operate in the multi agent environment. There are, however,
some issues with this definition. The main issue is what is to be considered as an
agent what is to be considered as not an agent. Does an agent A (taxi driver for
instance) have to treat an object B (another vehicle) as an agent or can it be treated
merely as an object behaving according to the laws of physics, analogous to waves
at the beach? The key distinction is whether B’s behavior is best described as
maximizing a performance measure whose value depends of agent A’s behavior.
Multi agent environments are much more complicated comparing to single
agent environments. In some multi agent environments, when the agent is
designed, his behavior may be randomly generated in order to avoid any
predictability in behavior. Hence randomized behavior in this case will be
considered rational. As an example, an agent solving a puzzle is considered to
operate in a single agent environment. An agent playing chess would be an
example of the agent operating in the multi agent environment because chess is a
competitive multi agent environment (according to the definition provided above).
8
1.3 Deterministic vs. Stochastic
If the next state of the environment is fully determined by the action
executed by the agent and the current state, then the environment is considered
deterministic; otherwise, the environment is considered stochastic. It’s easy to
mention that if the agent is operating in the fully observable, deterministic
environment, he doesn’t have to worry about uncertainty. However, if the
environment is partially observable, it could still be stochastic. The majority of real
cases are so complex, so it’s very difficult to keep track of all the unobserved
aspects, hence this environment is stochastic.
9
1.4 Episodic vs. Sequential
Based on the definition, it is easy to distinguish between the episodic and
sequential environments. If the agent’s experience is divided into an amount of
episodes, then this is an episodic environment. In each of these episodes the agent
receives a percept and then executes an action. Roughly speaking, the next episode
is not based on the actions taken in the previous episode. Many tasks are episodic;
it’s difficult to find a sequential task, although these exist as well. It’s important to
note that the current decision could affect all future decisions. Episodic
environments are simpler than sequential environments because the agent does not
need to think ahead. Sometimes the sequential environments are also called nonepisodic environments. For example, a mail sorting system is an episodic
environment, but chess game is a sequential game.
10
1.5 Static vs. Dynamic
If the environment is most likely to change while the agent is operating in
the environment, then this environment is dynamic for that agent; otherwise, it is
static. The static, unlike dynamic environments, are easy to deal with because the
agent does not have to look at how the environment has changed since the last
action has been executed. On the other hands, the dynamic environments are very
complicated when they are continuously asking the agent what it wants to do; if it
hasn’t decided what to do yet. For example, crossword puzzles are static, while a
train driver is dynamic.
11
1.6 Discrete vs. Continuous
The definition discrete/continuous applies to the state, but mostly to the way
time is handled, and the how distinction is applied to the percepts and actions. For
example, chess game is has a finite number of states, has a discrete set of percepts
and actions. Unlike chess, the train operator is a continuous-state problem: the
locations of the trains are distributed through a range of continuous values. The
train operator actions are also continuous.
12
1.7 Known vs. Unknown
This definition is not very common but it’s used sometimes. Roughly
speaking, this distinction is not used for describing an environment in which the
agent(s) is/are operating but it’s used to the agent itself. To be more appropriate, I
use this definition to the agent’s state of knowledge about the laws of physics of
the environment. If the environment is unknown, the agent has to learn about the
laws of this environment; such as how likely if whether environment will be
change if action A is executed. Unlike in an unknown environment, in known
environment agent is expected to know the outcomes for all the actions that he can
execute. The last definition about unknown environment may confuse and refer to
the definition of the observable and partially observable environments. It is
possible for a known environment to be partially observable – for example in one
of the card games where the probability of the outcomes is not known upfront.
Likewise, an unknown environment can be observable – in a new video game, the
screen my show the entire scene or the world, but the games may not yet be aware
of which buttons do what.
It’s expected to understand that the most complicated case would be when
the environment is: partially observable, multi agent, stochastic, sequential,
dynamic, continuous and unknown. To conclude this chapter, I provide an example
of the most complicated case: taxi driving is the hardest in this case pretending the
driver is operating in the unfamiliar area (then the environment will be considered
unknown, although the rules of road are known to the driver).
13
II Agents
An intelligent agent (IA) is an autonomous entity which observes through
sensors and acts upon an environment using actuators and directs its activity
towards achieving goals. Intelligent agents may also learn or use knowledge to
achieve their goals. They may be very simple or very complex depending on the
architecture: a reflex agent such as a vacuum cleaner is an intelligent agent, as is a
human being, as is a community of human beings working together towards a goal.
Intelligent agents are often described schematically as an abstract functional
system similar to a computer program. The scheme of an agent is illustrated in
Figure 1.
AGENT
Percepts
ENVIRONMENT
text
AGENT FUNCTION(S)
Actions
Figure 1. Agent interacts with environment.
It is obvious that the agents have to interact with the environment. The
agents interact with the environment through sensors and actuators2. The agent’s
behavior is managed by the agent’s function. On the other hand, the agent’s
function can be implemented by the agent program. The agent function is an
abstract description; the agent program is a concrete implementation which runs on
the hardware.
In order for the agent to be successful, it has to be some sort of rational.
What is rationality? What is it applicable to? A rational agent is an agent that
executes an action which leads this agent to maximize a reward or an outcome of
this action. To understand the term rationality, I will provide an example:
“If the agent is choosing between two actions to execute: action A and B. Both
actions will lead the agent to a reward, but the reward for executing action A is 5,
but the reward for executing action B is 10. In this case a rational agent will
choose to execute action B due to a higher reward which this action leads to.”
14
Now that we have learnt what a rational agent is, we have to be careful in
distinguishing between two terms rationality and omniscience. Thus, an omniscient
agent is the one which knows what the outcomes of an action is beforehand,
although omniscience is only a theoretical term because it is impossible in reality.
Intelligent agents are often described schematically as an abstract functional
system similar to a computer program. There is a close similarity between the
terms intelligent agent and software agents. The software agents have some
intelligence that is used in solving a problem. For example, autonomous programs
used for operator assistance or text mining can be described as intelligent agents.
Intelligent agents must be able to set goals [56AI] and achieve them. The agents
must have a representation of the state of the world and be able to make predictions
about how their actions will change the world. When there are more than just one
agent operating in the environment the cooperation and [60 AI] collaboration of
many agents are needed to achieve a goal.
Since the problem of defining the intelligent agent has been very popular in
the past, there have been many definitions of intelligent agents. AI systems should
exhibit the following characteristics [2]:
 Accommodate new problem solving rules incrementally;
 Adapt online and in real time;
 Be able to analyze itself in terms of behavior, error and success;
 Learn and improve through interaction with the environment
(embodiment);
 Learn quickly from large amounts of data;
 Have memory-based exemplar storage and retrieval capacities;
 Have parameters to represent short and long term memory, age,
forgetting, etc.
Mathematically speaking, an agent may be defined as following [1]:
Agent = Architecture + Program,
the program has to be appropriate for the type of agent defined. For example, if the
program decides to execute an action taking a photo, then the architecture should
have a camera to perform this action. In order to be able to design agent programs,
we have to understand that there is only one input to the program which is obtained
15
by the agent from the environment in which the agent is operating. This input is a
percept from the environment, there is nothing else available for the agent. Let’s
consider the following pseudo code for understanding the agent program better:
function A returns ACTION
persistent: percepts
table (table of actions)
action ⃪ LOOKUP (percepts, table)
return action
A simple agent program can be defined mathematically as an agent function which
transforms every possible percepts sequence to a possible action the agent can
execute:
The agent function and the program agent are different terms. While the agent
function is an abstract concept, the program agent translates a percept into an
action.
16
2.1 Simple Reflex Agent
The simple reflex agents have the simplest structure, no memory and can’t keep
track of the environment. Thus, a simple reflex agent will execute an action based
only on the current percept that it received from the environment. The conditionaction rules have to be defined in order for the agent to execute a correct action.
For example, agent controlling air temperature in the room gets a perception that
the temperature is more than 85F, then, according to already preset rules, it will
activate air conditioning system to cool down. Opposite, if the agent gets a percept
that the temperature in the room is below 60F, it will, according to the conditionaction rules activate a heating system. It’s clear that the agent does not carry the
information about the previous percept or the action it has executed because it
simply does not matter in this case. It is clear that since the agent decides which
action to perform only on the basis of the predefined rules, it may run into
unpredictable loop when performing in not fully observable environments (such
environments will be defined in the second chapter). The structure of the simple
reflex agent is shown in Figure 1. The agent program is shown below:
function SIMPLE REFLEX AGENT returns Action
persistent: rules, a set of rules
State ⃪ Interpret-Input (percept)
Rule ⃪ Rule-Matching (state, rules)
Action ⃪ Rule.Action
return Action
Sensors
What is the world
like at the moment?
Set of rules
What action should
be executed?
Actuators
Figure 1. Structure of simple reflex agent
17
2.2. Model-based reflex agent
A model-based agent is a little more complicatedly structured than a simple
reflex agent. The model-based agent has an internal state and can operate in
partially observable environments. This internal state depends on the percept
history and thereby reflects at least some of the unobserved aspects of the current
state. For the state to keep up-to-date information in it, it has to be updated
periodically. The state is complete when it carries two types of knowledge in it:
 Information about the world - this information is typically related to
how the world changes in time;
 Information about the actions’ impacts - this information is typically
related to how the agent’s actions influence and change the world.
This knowledge represents a model of the world in which the agent is operating.
The information of the current state is generated by the old internal state and
current percept which depend on the world’s description by the agent. Figure 2
shows a structure of the model-based reflex agent. The agent program is illustrated
below:
function MODEL BASED REFLEX AGENT(percept) returns Action
persistent: state, model, rules, action
State ⃪ UPDATE-STATE (state, action, percept, model)
Rule
⃪ RULE-MATCHING (state, rules)
Action
⃪ Rule.Action
returns Action
Sensors
AGENT
ENVIRONMENT
State
How the world
changes
What is the world
like at the moment?
What my actions do
What action should
be executed?
Set of rules
Actuators
Figure 2. Structure of model-based reflex agent
18
The new internal state description is defined by the function UPDATE-STATE
and does not have, which is less obvious, to define “what the world is like at the
moment.”
19
2.3 Goal-based agent
Knowing about the current state of the environment may not be enough in
some cases to decide what to do. Goal-based agent is the agent which has a goal
which influences the agent’s decision. Generally speaking the goal-based agent is
the same as the model-based agent but also has a goal. Agent program can combine
these two facts (goal and model) to decide which action to execute to obtain a goal.
Figure 3 illustrates the structure of the goal-based agent (sometimes this agent may
be called model-based, goal-based agent).
ENVIRONMENT
State
How the world
changes
What my actions do
Goals
What is the world
like at the moment?
Sensors
How the world
changes if action is
executed
What action should
be executed?
Actuators
Figure 3. Structure of goal-based agent
While in some domains (e.g., chess) it is rather straightforward to specify the
goal test, in others, it is not as obvious and requires considerable thought. In
general, the cognitive processes having to do with goal selection and goal
specification in humans and animals are poorly understood. Consequently, the
design of autonomous systems that elect, prioritize, and update their goals is
largely an unsolved problem in AI. We need to specify precisely all of the
primitive actions (including their preconditions and their expected effects on the
environment) that are sufficient to achieve the goal. Early AI systems assumed that
given an action (also called an operator) and a description of the current state of the
world, the action completely specifies the conditions under which the action can be
applied to the current world as well as the exact state of the world after the action
is executed in the current world. Actions were viewed as atomic, discrete events
that can be thought of as occurring at an instant in time. Thus, the world is in a
state before the action is executed, and it makes a transition to a different state
(specified implicitly or explicitly by the effects of the action) once the action is
20
executed. For example, if “Mary is in class” and she performs the action “go
home” she will end up “at home.” Given this representation of action, we have no
way of describing a point in time when Mary is neither in class nor at home (i.e.,
she is on her way home).
It is also typical to assume that no two actions are executed simultaneously.
It is generally assumed that the world itself has no dynamics of its own besides the
aspects that are explicitly modeled by the agent. While these assumptions are quite
reasonable for manipulating symbolic worlds" of the sort encountered in
mathematical problem-solving, they can be problematic for agents that need to
cope with dynamic, uncertain, real-world environments.
[http://www.cs.iastate.edu/~cs572/handout2.pdf].
The main difference between the goal-based and model-based agents is
decision making, the goal-based agent considers the future – “What will happen if
I execute this action?” and “What action will make me happy?” The way the agent
designed, this information is not represented explicitly, because the rules map
directly from percepts to actions. The goal-based agent may seem less efficient,
opposite, it is more flexible than the model-based agent because of the knowledge
representation.
21
2.4 Utility-based agent
The fact of having goals may not be enough in some environments to
achieve the high results and generate high-quality behavior. Goal-based agents
only distinguish between goal states and non-goal states. It is possible to define a
measure of how desirable a particular state is. This measure can be obtained
through the use of a utility function which maps a state to a measure of the utility
of the state. A more general performance measure should allow a comparison of
different world states according to exactly how happy they would make the agent.
The term utility can be used to describe how "happy" the agent is.
A rational utility-based agent chooses the action that maximizes the expected
utility of the action outcomes- that is, the agent expects to derive, on average,
given the probabilities and utilities of each outcome. A utility-based agent has to
model and keep track of its environment, tasks that have involved a great deal of
research on perception, representation, reasoning, and learning. [AI 6]
Utility is a function that maps a state into a real number, which describes a degree
of happiness. Hence the utility-based agent has many advantages in terms of
flexibility and learning. Technically speaking, a rational utility-based agent
chooses the action which will maximize the expected utility of the action outcomes
– that is, the utility the agent expects to derive, on average, given the probabilities
and utilities of each outcome. Figure 4 illustrates the structure of the utility-based
agent.
Sensors
AGENT
State
How the world
changes
What my actions do
Utility
ENVIRONMENT
What is the world
like at the moment?
How the world
changes if action is
executed
How happy will I be
in this state?
What action should
be executed?
Actuators
Figure 4. Structure of utility-based agent
It is useful to note that Dynamic Bayesian Networks (DBN) are very useful
in constructing utility-based agents. Bayesian Network (BN) is a probabilistic
22
graphical model that represents a set of random variables and their conditional
dependencies via a directed acyclic graph. Dynamic Bayesian Network is Bayesian
Network which relates variables to each other over adjacent time steps. This is
often called a Two-Timeslice BN because it says that at any point in time T, the
value of a variable can be calculated from the internal regressors and the
immediate prior value (time T-1). DBNs are common in robotics, and have shown
potential for a wide range of data mining applications. DBNs have a large variety
of usage including but not limited to: bioinformatics, speech recognition, quoting
insurance rates etc. Example of such network is shown in Figure 5.
Figure 5. Dynamic Bayesian Network
23
2.5 Learning agent
A learning agent can be broken down in four components, as shown in
Figure 6. There are two very important components of learning agent: the learning
element and the performance element. The learning element is mainly responsible
for making improvements, and the performance element is responsible for
selecting external actions. The learning element uses feedback from the “critic” on
how the agent is doing and determines how the performance element should be
modified to do better in the future.
Learning has an advantage that it allows the agents to initially operate in unknown
environments and to become more competent than its initial knowledge alone
might allow. The most important distinction is between the "learning element",
which is responsible for making improvements, and the "performance element",
which is responsible for selecting external actions.
The learning element uses feedback from the "critic" on how the agent is doing and
determines how the performance element should be modified to do better in the
future. The performance element is what we have previously considered to be the
entire agent: it takes in percepts and decides on actions.
The last component of the learning agent is the "problem generator". It is
responsible for suggesting actions that will lead to new and informative
experiences.
Permormance standard
Sensors
Critic
feedback
Element of learning
Performance
Element
Problem generator
Actuators
Figure 6. Structure of learning agent
24
ENVIRONMENT
2.6 Other types of Agents
The agents that have been previously described in this chapter are the most
commonly used agents in the field of AI. Nevertheless, in order to provide a
complete picture about the intelligent agents, these types of AI agents have to be
mentioned. Here is a list of some agents:
 Decision Agents;
 Neural Network based Agents;
 Processing Agents;
 Spatial Agents;
 World Agents;
 Believable agents - An agent exhibiting a personality via the use of an
artificial character (the agent is embedded) for the interaction;
 Physical Agents - A physical agent is an entity which percepts through
sensors and acts through actuators;
 Temporal Agents - A temporal agent may use time based stored information
to offer instructions or data acts to a computer program or human being and
takes program inputs percepts to adjust its next behaviors.
Neural networks are widely used in many AI applications? What are the
neural networks and why are they popular? An Artificial Neural Network (ANN)
is an information processing paradigm that is inspired by the way biological
nervous systems, such as the brain, process information. The key element of this
paradigm is the novel structure of the information processing system. It is
composed of a large number of highly interconnected processing elements
(neurones) working in unison to solve specific problems. ANNs, like people, learn
by example. ANN is configured for a specific application, such as pattern
recognition or data classification, through a learning process. Learning in
biological systems involves adjustments to the synaptic connections that exist
between the neurones. This is true of ANNs as well.
25
Neural networks, with their remarkable ability to derive meaning from
complicated or imprecise data, can be used to extract patterns and detect trends that
are too complex to be noticed by either humans or other computer techniques. A
trained neural network can be thought of as an "expert" in the category of
information it has been given to analyze. This expert can then be used to provide
projections given new situations of interest and answer "what if" questions. Other
advantages include:
 Adaptive learning: An ability to learn how to do tasks based on the data
given for training or initial experience.
 Self-Organization: ANN can create its own organization or representation of
the information it receives during learning time.
 Real Time Operation: ANN computations may be carried out in parallel, and
special hardware devices are being designed and manufactured which take
advantage of this capability.
 Fault Tolerance via Redundant Information Coding: Partial destruction of a
network leads to the corresponding degradation of performance. However,
some network capabilities may be retained even with major network
damage.
Figure 7 demonstrates how Artificial Neural Network is constructed.
Figure 7. Artificial Neural Network
26
III Logic
Logic is the science of reasoning. Logic helps to decide whether some
statement takes on the value true or false. A popular example of a syllogism (it is a
kind of logical arguments in which one proposition is inferred from two or more
premises) was given by Aristotle:
1. All men are mortal
2. Socrates is a man
3. Therefore, Socrates is mortal
A logical proof is a list of statements. Each statement in the proof is either an
assumption or has been proven to follow from earlier statements in the proof. A
proof shows that one statement, the conclusion, follows from the assumptions. One
can, for example, prove that “Aristotle is mortal” follows from “Aristotle is a man”
and “All men are mortal”.
Logic is widely used by computers; it’s used in the form of algorithms.
Algorithm is sort of a list of things that has to be done by the computer; it tells
computer what to do and when. Logic is used in mathematics as well. People who
study math create proofs that use logic to show that math facts are correct. The
way that mathematicians use logic is mathematical logic.
Every type of logic must contain its syntax and semantics. The sentences
that are used in the logic must follow the syntax rules. For example, “k + l=7” – is
a correct and formed sentences while “kl9+=” is not. The semantics of the logic
defines the truth of each sentence with respect to each possible world. The
semantics would specify that “k + l = 7” is true in case when k = 6 and l = 2 and
when k = 2 and l = 3 is false. In standard logic (excluding fuzzy logic which is not
discussed here) the sentences can only be true of false (it can’t be anything in
between).
27
3.1 Propositional Logic
Propositional logic is a very simple, but at the same time powerful tool of
expressing sentences. As any other logic, propositional logic has semantics and
syntax.
The syntax of propositional logic defines which sentences are legal and
which are not legal. The atomic sentences consist of a single propositional symbol.
Each such symbol stands for a proposition that can be only true of false. There are
two proposition symbols – true is the always true-proposition and false is the only
false-proposition. Complex sentences are constructed from simpler sentences that
use logical operators located between the sentences. There are five connectives that
are commonly used:
 ¬ (not). Any sentence that has ¬ in front means that the negative value of
this sentence should be considered. For example, ¬ False = True.
 ∧ (and). Any sentence which has ∧, such is Black ∧ White is called a
conjunction. For example, True ∧ False = False (no matter what the first
conjunct is, as long as there is at least one False in a series of conjuncts,
the value will be False.
 ∨ (or). Any sentence which has ∨, such as Black ∨ White is called a
disjunction. For examples, True ∨ False = True (no matter what
disjuncts are used in this expression, as long as one of them is True, the
whole expression will be True).
 ⇒ (implies). Any sentence which has ⇒, such as (Black ∧ White) ⇒
MultiColor. For example, (True ∧ False) ⇒ True. Please note, the
implication symbol can also be written as →.
 ⇔ (if and only if [iff]). Any sentence which has ⇔, such as
Black ⇔ ¬ White. For examples, (True ∧ False ) ⇔ ¬(True ∨ False). Please
note, the iff symbol can also be written as ↔.
28
In math, values of False and True stand for 0 and 1 respectively. Table 3 has
examples (based on Boolean Logic) of dealing with logic operator listed
above.
A
B
¬A, ¬B
A∧B
A∨B
A⇒B
A⇔B
0
0
1, 1
0
0
1
1
0
1
1, 0
0
1
1
0
1
0
0, 1
0
1
0
0
1
1
0, 0
1
1
1
1
Table 3. Boolean Logic
Let’s derive some equations which follow from Table 3 that will be helpful in
dealing with Boolean Logic. Boolean Logic has the following properties (not all
the properties are listed below):
 Associativity:
A∨ (B∨C) = (A∨B) ∨ C
A ∧ (B∧C) = (A∧B) ∧C
 Communitativity:
A∧B=B∧A
A∨B=B∨A
 Distributivity:
A ∧ (B∨C) = (A∧B) ∨ (A∧C)
 De Morgan Laws:
 De Morgan Rule 1: (¬A) ∧ (¬B) = ¬(A∨ B)
 De Morgan Rule 2: (¬A) ∨ (¬B) = ¬(A∧ B)
Let’s now focus more on semantics of Propositional Logic. What is semantics?
The semantics defines the rules for determining the truth of a sentence with respect
to a particular model. Moreover, the semantics encompasses all of the following:
29
 The definition or semantic models;
 The relations between different semantic models;
 The relations between different approaches to meaning;
 The relation between computation and the underlying mathematical
structures from fields such as logic, set theory, model theory, category
theory, and others.
In propositional logic, a model simply fixes the truth value – true or false – for
every propositional symbol. For example, if the sentence in the knowledge base
(KB) makes use of the propositional symbols A1, A2, A3, A4, then one possible
model could be:
Model1 = {A1 = true, A2 = false, A3 = true, A4 = true}. Please note, “Model1“is not
the only model that could be constructed. In this case, since A1 – A4 are binary
variables (binary variables can only take on two values, in this case true or false)
and there four variables, then we could create 24=16 different models with four
binary variables. The semantics for propositional logic must specify how to
compute the truth value of any sentence, given in a model, which is done
recursively. All sentences are constructed with atomic sentences and can only have
five connectors between them. Atomic sentences are easy:
 True is true in every model and False is false in every model respectively;
 The truth value of every other propositional symbol must be specified
directly in the model. For example, in the model Model 1 given above, A2 is
false.
According to everything mentioned above and Table 3, we can conclude the
following: (¬A) ∧ (¬B) = ¬ (A∨ B)
 ¬A is true iff P is false in Model1;
 A ∧ B is true iff both A and B are true in Model1;
 A ∨ B is true iff either A or B are true in Model1;
 A ⇒ B is true unless A is true and B is false in Model1;
 A ⇔ B is true iff both A and B are either true of false in Model1.
30
The rules can also be defined and express in the numerical way such as assigning
true (1) and false (0) values to each possible propositional symbol.
Now that the semantics for the propositional logic has been defined, KB has
to be defined for every environment we will be dealing with. The main goal is to
find whether there is such sentence α, that KB⊨ α. There is a number of very
helpful and simple rules that follow the definition of entailment, this rule,
indeed, is very helpful for proofs:
α ≡β iff α ⊨ β and at the same time β⊨ α, α ⊨ β iff (α ⇒ β), α ⊨ β iff (α ∨ ¬β) is
unsatisfiable.
Deriving a proof is not easy, but satisfiable with some requirements. There are
some inference rules that can be applied to derive a proof – a chain of
conclusions that leads to the desired goal. The best rule is Modus Ponens, it
states the following:
α ⇒ β, α
β
The formula above means that, whenever any sentences of the form α ⇒ β and α
are given, then the sentence β can be inferred. For example, if (White ∧ Black) ⇒
MultiColor and (White ∧ Black) are given, then MultiColor can be inferred.
Another useful inference rule is And-Elimination, which states that a
conjunction, any of the conjuncts can be inferred:
α∧β
α
The formula above means that, whenever any sentences of the form (α ∧ β),
either α or β can be inferred. For example, if (White ∧ Black) is given, then
White or Black can be inferred.
The final property of logical systems states that the set of entailed sentences
can only increase when information is added to KB. For any sentences α and β,
the following will hold:
If KB ⊨ α, then KB ∧ β ⊨ α as well.
31
For example, let’s assume that KB contains some information, and β contains
the information that fulfills the information in KB, then it may only help to
derive the conclusion, and cannot in any way invalidate the conclusion.
There are many complete algorithms such as A*, iterative deepening
depth-first search (IDDFS) and others. These algorithms will always find a
solution, if one exists. The common idea of both algorithms is provided below.
A* is a path-finding algorithm which is fairly flexible and can be used in a
wide range of contexts. A* uses a best-first search and finds the smallest path
from the starting node to the goal. At first, A* follows the path and it think is
the shortest, at the same time it keeps all the segments of the path in the
memory. It uses a knowledge-based [A* WIKI LINKS] cost function function of
node x to determine the order in which the search visits nodes in the tree. The
cost function is a sum of two functions:
 The past path-cost function, which is the known distance from the
starting node to the current node x (this function is usually listed as
g(x));
 A future path-cost function, which is an admissible heuristic estimate of
the distance from x to the goal (this function is usually listed as h(x)).
The part f(x) function must not overestimate the distance to the goal. Thus,
h(x) will represent the straight-line distance to the goal, since this distance is
the physically smallest possible path between two nodes. The pseudo-code is
provided below (it consists of two separate functions: constructing and
reconstructing a path):
function A*(start,goal)
closedset := the empty set
openset := {start}
came_from := the empty map
g_score[start] := 0
f_score[start] := g_score[start] + heur_cost_est(start,goal)
while openset is not empty
current := the node in openset having the lowest
f_score[] value
if current = goal
32
return reconstruct_path(came_from, goal)
remove current from openset
add current to closedset
for each neighbor in neighbor_nodes(current)
tentative_g_score := g_score[current] +
dist_between(current,neighbor)
tentative_f_score := tentative_g_score +
heuristic_cost_estimate(neighbor, goal)
if neighbor in closedset and tentative_f_score >=
f_score[neighbor]
continue
if neighbor not in openset or tentative_f_score <
f_score[neighbor]
came_from[neighbor] := current
g_score[neighbor] := tentative_g_score
f_score[neighbor] := tentative_f_score
if neighbor not in openset
add neighbor to openset
return failure
function reconstruct_path(came_from, current_node)
if current_node in came_from
p := reconstruct_path(came_from,
came_from[current_node])
return (p + current_node)
else
return current_node
Iterating deepening depth-first search algorithm [LINKS FROM WIKI] is
a state place search algorithm in which a depth-limited search is run
repeatedly, increasing the depth limit with each iteration until it reaches the
value of the depth of the shallowest goal state. IDDFS is equivalent to BFS, but
it uses much less memory; on each iteration, it visits the nodes in the search
33
tree in the same order as DFS, but the cumulative order in which nodes are
first visited is effectively bread-first. The pseudo-code is provided below (it
consists of two separate functions):
IDDFS(start, goal)
{
depth = 0
while(no solution)
{
visited = [] // <-- Empty List
solution = DLS(start, goal, depth, visited)
depth = depth + 1
}
return solution
}
DLS(node, goal, depth, visited)
{
if ( depth >= 0 )
{
if ( node == goal )
return node
visited.insert(node)
for each child in expand(node)
if (child is not in visited)
DLS(child, goal, depth-1, visited)
}
}
34
Forward and Backward Chaining
Forward chaining is one of the two main and most common used methods of
reasoning when using inference rules and can be described logically [WIKI
FORWARD CHAINING] as repeated application of modus ponens. Forward
chaining is a popular implementation strategy for expert systems, analytics and
production rule systems. The opposite of forward chaining is backward chining,
which will be described later in the section. Forward chaining starts with the
available data and uses inference rules to extract more data until a goal is reached.
Let’s consider an example of how forward chaining works. Assume that the goal is
to conclude the color of a pet named Einstein given that he barks and eats meat,
and the rule base contains the following four rules:
1. If X barks and eats meat- then X is a dog;
2. If X chirps and signs – then X is a canary;
3. If X is a dog – then X is black;
4. If X is a canary – then X is yellow.
Let’s list how forward chaining follows the pattern of a computer as it evaluates
the rules. Assume the following facts:
 Einstein barks;
 Einstein eats meat;
 Tiger eats meat;
 Tiger growls;
 Tiger is yellow.
With forward reasoning, the computer can derive that Einstein is a dog in only
three steps:
1. Einstein barks and Einstein eats meat;
Based on logic, the computer can derive:
2. Einstein barks and eats meat;
Based on rule 1, the computer can derive:
35
3. Einstein is a dog.
The name forward chaining comes from the fact that the computer starts with the
data and reasons its way to the answer, as opposed to backward chaining, which
works the other way around. I the derivation, the rules are used in the reverse order
as compared to backward chaining. Note that the computer doesn’t use any
knowledge about Times, when it computes that Einstein is a dog. Because the data
determines which rules are selected and used, this method is called data-driven
method, in contrast to goal-driven backward chaining interference. The main
advantage of forward chaining when compared to backward chaining is that the
reception of new data can trigger new inferences, which makes the engine better
suited to dynamic situations in which conditions are likely to change. The pseudo
code for forward chaining is illustrated below:
Forward-Chaining
repeat
for every rule do
if antecedents match assertions in the working memory
and consequents would change
the working memory then
Create triggered rule instance
end if
end for
Pick one triggered rule instance, using conflict resolution
strategy if needed, and fire it
until no change in working memory, or no STOP signal
Backward chaining is an inference method that can be described (in lay
terms) as working backward from the goals. It is used an automated theorem
proving, proof assistants and other artificial intelligence applications, but it has
also been observed in primates. Backward chaining is implemented in logic
programming by selective linear definite resolution. Both rules are based on the
modus ponens inference rule. Backward chaining is based on a depth-first search
strategy. It starts with a list of goals and works from the consequent to the
antecedent to see if there are data available that will support any of these
consequents. An inference engine using backward chaining would search the
inference rules [WIKI BACKWARD] until it finds one which has a consequent
that matches a desired goal. If the antecedent of that rule is not known to be true,
36
then it is included in the list of goals. Let’s consider the same example that was
used for forward chaining:
 Einstein barks;
 Einstein eats meat;
 Tiger eats meat;
 Tiger growls;
 Tiger is yellow.
With backward reasoning, the computer can answer the question “Who is a dog?”
in the same quantity of steps (as forward search):
1. ? is a dog;
Based on rule 1, the computer can further derive:
2. ? barks and eats meat;
Based on logic, the computer can derive:
3. ? barks and ? eats meat;
Based on the facts, the computer can derive:
4. Einstein barks and Einstein eats meat.
This derivation will cause the computer to produce Einstein as the
answer to the questions: “Who is a dog?”
Note that the computer did not imply to use any knowledge about the tiger to
compute that Einstein is a dog. The goals always match the affirmed versions of
the consequents of implications and even then, their antecedents are then
considered as the new goals which intimately must match known facts, thus, the
inference rule which is used a modus ponens. The pseudo code for backward
chaining is illustrated below:
Backward-Chaining
if H matches an assertion in working memory then
return true
end if
37
if there is no rule with a consequent that matches H then
ASK USER or ASSUME false
end if
for every rule R with a consequent that matches H do
if for all antecedents A of rule R, we have BackwardChaining(A) = true then
return true
end if
end for
return false
38
3. 3 First-Order Logic
First order logic is a formal system used in many sciences including
computer science. First order logic is very expressive in representing a good deal
of our commonsense knowledge. First-order logic is distinguished from
propositional logic by its use of quantified variables.
While propositional logic simply deals with declarative propositions, firstorder logic additionally covers predicates and quantification. A predicate takes an
entity in the domain as input either True of False. Consider two sentences “Lincoln
is a president” and “Franklin is a president”. In first-order logic, these sentences
are viewed as being unrelated and are denoted, for example, by p and q. However,
the predicate "is a president" occurs in both sentences which have a common
structure of "a is a president". The variable a is instantiated as "Lincoln" in first
sentence and is instantiated as "Franklin" in the second sentence. The use of
predicates, such as "is a president" in this example distinguishes first-order logic
from propositional logic.
Predicates can be compared. For example, the first-order formula “is a is a
president, then a is a citizen”. This formula happens to be a conditional statement
with “a us a president” as hypothesis and “a is a citizen” as conclusion. The truth
of this formula depends on which object is denoted by a, and on the interpretations
of the predicates “is a president” and “is a citizen”. The predicates "is a
philosopher" and "is a scholar" each take a single variable. Predicates can take
several variables. In the first-order sentence "Lincoln is a brother of Franklin", the
predicate "is the teacher of" takes two variables, in this case False, because
Lincoln and Franklin were not brothers.
There are two key parts of first order logic. The syntax determines which
collections of symbols are legal expressions in first-order logic, while semantics
determine the meanings behind these expressions .Unlike any real language, such
as English or Russian, the language of first-order logic is completely formal, so
that is can be mechanically determined whether a given expression is legal. There
are two key types of legal expressions: terms, which intuitively represent objects,
and formulas, which express predicates that can be either True of False. It is
common to divide the symbols of the alphabet into logical symbols, which always
have the same meaning, and non-logical symbols whose meaning varies by
interpretation.
39
There are several logical symbols in the alphabet, which vary by author but
usually include:
 The quantifier symbols ∃ and ∀;
 The logical connectives such as conjunction, disjunction, implication,
and negation (all the connectives are discussed in this chapter earlier);
 Parentheses, brackets, and other punctuation symbols. The choice of
such symbols varies depending on context;
 An infinite set of variables, often denoted by lowercase letter at the end
of the alphabet x, y, z;
 An equality symbol.
The formula listed below provides an example of the expression in the first-order
logic:
To introduce some definitions of the first-order logic, let’s look at the predicates:
 The first predicate is a unary predicate, it only consists of one variable;
 The second predicate is a little more complex, it consists of two variables,
thus it’s called binary predicate;
 The third predicate, like the first one, is a unary predicate.
As any logical language, first-order logic follows the order of operations:
 Negation is evaluated first;
 Conjunction and disjunction are evaluated next;
 Quantifiers are evaluated next;
 Implication is evaluated last.
Let’s now consider a few examples starting with the simplest examples:

means that all presidents are persons;
40

means that President Lincoln has a
residency. Please note that it’s indeed required for the second predicate to be
binary;

means that for every x and y related to
each other as sisters, the same x and y will always be related as siblings.
It’s very important to discuss how the quantifiers are connected to each other. For
example:
is equivalent to
To go one more step further, we can express the following expression “Everyone
likes candies” which also means “There is no one who doesn’t like candies”:
is equivalent to
41
IV Agent Performance
Let’s introduce Wumpus World. Wumpus World [Hunt the Wumpus in The
Best of Creative Computing, Volume 1]consists of object, actions and senses. A
more detailed introduction is listed below:
 Objects
 Wumpus – a beast that eats anyone enters its room;
 Agent – the player that traverses the world in search of gold and
while trying to kill the wumpus;
 Pits – bottomless pit that will trap anyone who enters the room
except the wumpus.

Actions
 Move forward;
 Move left;
 Move right;
 Grab (gold);
 Shoot.
 Senses
 In the square containing the wumpus and in the directly (not
diagonal) adjacent squares, the agent will perceive a Stench;
 In the squares directly adjacent to a pit, the agent will perceive a
Breeze;
 In the square where gold is, the agent will perceive a Glitter.
 When the agent walks into a wall, the agent will perceive a Bump;
 When the wumpus is killed, it emits a Scream;
I have modeled the wumpus environment using Java Language. Table 4 provides
the results of modeling different types of agents in the wumpus world.
42
Agent Type
Score (pts)
Simple reflex agent
140
Model-based reflex agent
240
Goal-based agent
380
Utility based agent
500
43