CS 416
Artificial Intelligence
Lecture 17
Reasoning over Time
Chapter 15
Sampling your way to a solution
As time proceeds, you collect information
• Xt – The variables you cannot observe (at time t)
• Et – the variables you can observe (at time t)
– A particular observation is et
• Xa:b – indicates set of variables from Xa to Xb
Dealing with time
Consider P ( xt | e0:t )
• To construct Bayes Network
– xt depends on et
– et depends on et-1
– et-1 depends on et-2
– … potentially infinite number of parents
• Avoid this by making an assumption!
Markov assumption
The current state depends only on a finite history
of previous states
• First-order Markov process: the current state depends only
on the previous state
Stationarity assumption
Changes in the real world are caused by a
stationary process
• The laws that cause a state variable to change at time t are
exactly the same at all other times
– The variable values may change over time, but the nature
of the system doesn’t change
Models of state transitions
State transition model
•
Sensor model
•
• Evidence variables depend only on the current state
–
• The actual state of the world causes the evidence values
Initial Conditions
Specify a prior probability over the states at time
0
• P(X0)
A complete joint distribution
We know
• Initial conditions of state variables: P (X0)
• Initial observations (evidence variables):
• Transition probabilities:
Therefore we have a complete model
What might we do with our model?
Filtering
• given all evidence to date, compute the belief state of the unobserved variables:
P(Xt | e1:t)
Prediction
• Predict the posterior distribution of a future state: P(Xt+k | e1:t)
Smoothing
• Use recent evidence values as hindsight to predict previous values of the
unobserved variables: P(Xk | e1:t), 0<=k<t
Most likely explanation
• What sequence of states most likely generated the sequence of observations?
argmaxx1:t P(x1:t | e1:t)
Filtering / Prediction
Given filtering up to t, can we predict t+1 from
new evidence at t+1?
Two steps:
• Project state at xt to xt+1 using transition model: P(Xt | Xt-1)
• Update that projection using et+1 and sensor model: P(Et | Xt)
Filtering/Projection
• Project state at xt to xt+1 using transition model: P(Xt | Xt-1)
• Update that projection using et+1 and sensor model: P(Et | Xt)
sensor
model
transition model
must solve because we
don’t know Xt
Filtering/Projection
• Xt+1 is really a function of
e1:t and xt
must solve prediction of Xt+1
• Because we don’t know xt, we
sum across all possible values
(previous values not
useful)
Filtering example
Is it rainingt? Based on observation of umbrellat
• Initial probability, P(R0) = <0.5, 0.5>
• Transition model:
P (Rt+1 | rt) = <0.7, 0.3>
P (Rt+1 | ~rt) = <0.3, 0.7>
• Sensor model:
P (Rt
P (Rt
| ut) = <0.9, 0.1>
| ~ut) = <0.2, 0.8>
Given U1 = TRUE, what is P(R1)?
• First, predict transition from x0 to x1 and update with evidence
Given U1 = TRUE, what is P(R1)?
Predict transition from x0 to x1
• Because we don’t know x0 we have to consider all cases
It wasn’t raining
It was raining
Given U1 = TRUE, what is P(R1)?
Update with evidence
sensor
model
prob. of seeing
umbrella given
it was raining
prob. of seeing
umbrella given
it wasn’t raining
Given U1 and U2 = true, what is P(R2)
We computed R1 in previous steps
First, predict R2 from R1
Given U1 and U2 = true, what is P(R2)
Second, update R2 with evidence
From R1
When queried to solve for Rn
• Use a forward algorithm that recursively solves for Ri for i < n
Prediction
Use evidence 1t to predict state at t+k+1
• For all possible states xt+k consider the transition model to xt+k+1
• For all states xt+k consider the likelihood given e1:t
Prediction
Limits of prediction
• As k increases, a fixed output results – stationary distribution
• The time to reach the stationary distribution – mixing time
Smoothing
P(Xk | e1:t ), 0<=k<t
• Attack this in two parts
– P(Xk | e1:k, ek+1:t)
bk+1:t = P(ek+1:t | Xk)
Bayes
Smoothing
Forward part:
• What is probability Xk given evidence 1k
Backward part:
• What is probability of observing evidence k+1t given Xk
How do we compute the backward part?
Smoothing
Computing the backward part
Whiteboard
Example
Probability r1 given u1 and u2
solved for this
in step one of
forward soln.
Viterbi
Consider finding the most likely path through a
sequence of states given observations
Could enumerate all 25 permutations of five-sequence
rain/~rain options and evaluate P(x1:5 | e1:5)
Viterbi
Could use smoothing to find posterior distribution
for weather at each time step and create path
through most probable – treats each as a single
step, not a sequence!
Viterbi
Specify a final state and find previous states that form most likely path
• Let R5 = true
• Find R4 such that it is on the optimal path to R5. Consider each value of R4
– Evaluate how likely it will lead to R5=true and how easily it is reached
 Find R3 such that it is on optimal path to R4. Consider each value…
Viterbi – Recursive algorithm
Viterbi - Recursive
Viterbi - Recursive
The Viterbi algorithm is just like the filtering
algorithm except for two changes
• Replace f1:t = P(Xt | e1:t)
– with:
• Summation over xt replaced with max over xt
Review
Forward:
Forward/Backward:
Max:
Hidden Markov Models (HMMs)
Represent the state of the world with a single discrete
variable
• If your state has multiple variables, form one variable whose value takes
on all possible tuples of multiple variables
• Let number of states be S
– Transition model is an SxS matrix
 Probability of transitioning from any state to another
– Evidence is an SxS diagonal matrix
 Diagonal consists of likelihood of observation at time t
Kalman Filters
Gauss invented least-squares estimation and
important parts of statistics in 1745
• When he was 18 and trying to understand the revolution of
heavy bodies (by collecting data from telescopes)
Invented by Kalman in 1960
• A means to update predictions of continuous variables given
observations (fast and discrete for computer programs)
– Critical for getting Apollo spacecrafts to insert into orbit
around Moon.