peer-to-peer and
agent-based computing
Basic Theory of Agency
(Cont’d)
Indistinguishable environments (1)
• Imagine two environmental states e1,e2Env, such
that
– e1 e2 and
– see(e1) = see(e2)
• Such environments are different, but as far as
agents are concerned they are indistinguishable!
– The environments are mapped to the same percept
– The agent would receive the same perceptual
information from different environment states
– Agents cannot tell them apart…
2
Indistinguishable environments (2)
• Environment states consisting of
– Boolean variable temp set to true if the temperature of
the room is above 35o Celsius (otherwise temp is false)
– Boolean variable smoke set to true if smoke has been
detected in the environment (otherwise smoke is false)
• The full set of environment states is
{temp,smoke,temp,smoke,temp,smoke,temp,smoke}
e1
e2
e3
e4
• The see function of a thermostat agent is:
see(e)=
p1 if e =e1 or e =e2
p2 if e =e3 or e =e4
3
Agents with State (1)
• Now, consider agents that record information
about environment state and history:
4
Agents with State (2)
• Let I be the set of all internal states.
• The perception function see is unchanged:
see : E  Per
• The action selection function is now a mapping
action : I  Ac
from internal states to actions.
• An additional function next is introduced, which
maps an internal state and percept to an internal
state:
next : I  Per  I
5
Agent Control Loop
1. Agent starts in some initial internal state i0
2. icur = i0
3. Agent observes the environment state e and generates a percept
see(e)
4. Agent’s internal state is updated via next function
inext = next (icur ,see(e))
5. Action selected by agent is
action(next (icur ,see(e))
This action is then performed
6. Go to 3 with icur = inext
6
Tasks for Agents
• We build agents to carry out tasks for us.
• The task must be specified…
• However, we want to tell agents what to do
without telling them how to do it.
• So, the question is:
– how do we give an agent information about what states
are better than others?
7
Utility Functions over States (1)
• Possibility: associate utilities with individual states
– Agents pursue states that maximise utility.
• A task specification is a function
u:E
which associates a real number with every state.
8
Utility Functions over States (2)
• But what is the value of a run? Is it
–
–
–
–
The minimum utility of all states in a run?
The maximum utility of all states in a run?
The sum of utilities of all states in a run?
The average of utilities of all states in a run?
• Disadvantage:
– It is difficult to specify a long term view when assigning
utilities to individual states!
9
Utilities over Runs
• Another possibility – assign utilities to runs rather
than individual states:
u:R
• This takes an inherently long term view.
• Other variations:
– Incorporate probabilities of different states emerging
• Difficulties with utility-based approaches:
– Where do the numbers come from?
– We don’t think in terms of utilities!
– Hard to formulate tasks in these terms…
10
Utility in the TILEWORLD
• Simulated 2D grid environment with
– Agents
– Tiles
– Obstacles and holes
• Agents can move up, down, left or right
– If agent is next to a tile, agent can push tile.
• Holes have to be filled up with tiles by the agent to
score points.
• Aim: score lots of points!!
• Holes appear and disappear at random…
• Utility function of a run r:
number of holes filled in r
u (r) def= 
number of holes that appeared in r
11
Expected Utility & Optimal Agents
• P (r |Ag,Env ) denotes probability that run r occurs
when agent Ag is placed in environment Env:
 P (r |Ag,Env ) = 1
rR(Ag,Env)
• The optimal agent Agopt in an environment Env is
the one that maximises expected utility:
Agopt = arg max
 u(r)  P(r |Ag,Env)
Ag AG rR(Ag,Env)
(1)
12
Bounded Optimal Agents
• Some agents cannot be implemented on some
computers;
– E.g. a function Ag : RE  Ac may need more memory
than what is available to implement.
• AGm denotes those agents that can be
implemented on machine (computer) m:
AGm = {Ag | Ag AG and Ag can be implemented on m}
• We can replace equation (1) with the following,
which defines the bounded optimal agent Agopt:
Agopt = arg max  u(r)  P(r |Ag,Env) (2)
Ag AGm rR(Ag,Env)
13
Predicate Task Specifications
• A special case of assigning utilities to histories is to
assign 0 (false) or 1 (true) to a run
– If a run is assigned 1, then the agent succeeds on that
run
– Otherwise it fails.
• We call these predicate task specifications
• We denote a predicate task specification as :
 : R  {0,1}
14
Task Environments (1)
• A task environment is a pair Env,, where Env is
an environment, and
 : R  {0,1}
is a predicate over runs.
• Let TE be the set of all task environments.
• A task environment specifies:
– The properties of the system the agent will inhabit
– The criteria by which an agent will be judged to have
either failed or succeeded
15
Task Environments (2)
• R (Ag, Env ) denotes the set of all runs of agent Ag
in environment Env that satisfy :
R (Ag, Env ) = {r | rR (Ag, Env )  (r ) = 1}
• We then say that an agent Ag in task environment
Env,   is successful if
R (Ag, Env ) = R(Ag, Env )
16
The Probability of Success
• Let P (r | Ag, Env ) denote the probability that run r
occurs when agent Ag is placed in environment Env.
• The probability P (|Ag,Env ) that  is satisfied by
Ag in Env would be:
P (|Ag,Env) =rR 
P
(r
|Ag,Env
)
(Ag,Env )

17
Achievement & Maintenance Tasks (1)
• Most common tasks in agent-based computing are
– Achievement tasks: “achieve state of affairs ”
– Maintenance tasks: “maintain state of affairs ”
18
Achievement & Maintenance Tasks (2)
• An achievement task is specified by a set G of
“good” or “goal” states G  E:
– An agent succeeds if it is guaranteed to bring about at
least one of these states
– We do not care which one – they are all considered
equally good
• A maintenance goal is specified by a set B of “bad”
states B  E :
– An agent succeeds in a particular environment if it
manages to avoid all states in B, i.e., it never acts so that
a state in B occurs.
19
Agent Synthesis
• Agent synthesis is automatic programming:
– The goal is to have a program that will take a task
environment and
– Automatically generate an agent that succeeds in this
environment:
syn : TE  (Ag  {})
(Think of  as being like null in Java)
• Common agent synthesis mechanism: genetic
programming
– E.g., in the evolution of simple predator and prey agents.
20
Agent Synthesis (2)
• A synthesis algorithm is sound
– If it returns an agent, this agent succeeds in the task
environment passed as input
– I.e., the algorithm satisfies the condition:
[syn(Env,)=Ag] [R(Ag,Env)=R (Ag,Env)]
• A synthesis algorithm is complete
– if it is guaranteed to return an agent whenever there
exists an agent that will succeed in the task environment
given as input;
– I.e., the algorithm satisfies the condition:
AgAG.[R(Ag,Env)=R (Ag,Env)][syn(Env,)]
21
Suggested Reading
• An Introduction to Multi-Agent Systems, M.
Wooldridge, John Wiley & Sons, 2002. Chapter 2.
Multi-Agent Systems (CS5562) – Wamberto Vasconcelos
22