Metrics for real time
probabilistic
processes
Radha Jagadeesan, DePaul University
Vineet Gupta,
Google Inc
Prakash Panangaden, McGill University
Josee Desharnais,
Univ Laval
Outline of talk
Models for real-time probabilistic
processes
 Approximate reasoning for real-time
probabilistic processes

Discrete Time Probabilistic
processes

Labelled Markov Processes
0.5
0.3
For each state s
For each label a
K(s, a, U)
0.2
Each state labelled
with propositional
information
Discrete Time Probabilistic
processes

Markov Decision Processes
For each state s
For each label a
K(s, a, U)
0.5
0.3
0.2
Each state labelled
with numerical rewards
Discrete time probabilistic proceses

+ nondeterminism : label does not
determine probability distribution uniquely.
Real-time probabilistic processes

Add clocks to Markov processes
Each clock runs down at fixed rate r
c(t) = c(0) – r t
Different clocks can have different rates

Generalized SemiMarkov Processes
Probabilistic multi-rate timed automata
Generalized semi-Markov
processes.
{c,d}
Each state labelled
with propositional
Information
s
Each state has a set
of clocks associated
with it.
u
{d,e}
{c}
t
Generalized semi-Markov
processes.
{c,d}
Evolution determined by
generalized states
<state, clock-valuation>
s
<s,c=2, d=1>
u
{d,e}
{c}
t
Transition enabled when a clock
becomes zero
Generalized semi-Markov
processes.
{c,d}
<s,c=2, d=1> Transition enabled in
1 time unit
s
<s,c=0.5,d=1> Transition enabled in
0.5 time unit
u
{d,e}
{c}
t
Clock c
Clock d
Generalized semi-Markov
processes.
{c,d}
Transition determines:
s
a. Probability distribution on next
states
0.2
0.8
b. Probability distribution on
clock values for new clocks
u
{d,e}
{c}
t
Clock c
Clock d
Generalized semi Markov proceses

If distributions are continuous and states are
finite:
Zeno traces have measure 0

Continuity results. If
stochastic processes from <s, > converge
to the stochastic process at <s, >
Equational reasoning
Establishing equality: Coinduction
 Distinguishing states: Modal logics
 Equational and logical views coincide
 Compositional reasoning: ``bisimulation is
a congruence’’

Labelled Markov
Processes
PCTL
Bisimulation [Larsen-Skou,
Desharnais-Panangaden-Edalat]
Markov Decision
Processes
Bisimulation [Givan-Dean-Grieg]
Labelled Concurrent
Markov Chains
PCTL [Hansson-Johnsson]
Labelled Concurrent
Markov chains (with
tau)
PCTLCompleteness: [DesharnaisGupta-Jagadeesan-Panangaden]
Weak bisimulation
[Philippou-Lee-Sokolsky,
Lynch-Segala]
With continuous time
Continuous time Markov CSL [Aziz-Balarin-Braytonchains
Sanwal-Singhal-S.Vincentelli]
Generalized SemiMarkov processes
Stochastic hybrid
systems
Bisimulation,Lumpability
[Hillston, Baier-Katoen-Hermanns]
CSL
Bisimulation:?????
Composition:?????
Alas!
Instability of exact equivalence
Vs
Vs
Problem!

Numbers viewed as coming with an error
estimate.
(eg) Stochastic noise as abstraction
Statistical methods for estimating
numbers
Problem!
Numbers viewed as coming with an error
estimate.
 Reasoning in continuous time and
continuous space is often via discrete
approximations.

eg. Monte-Carlo methods to approximate
probability distributions by a sample.
Idea: Equivalence

metrics
Jou-Smolka, Lincoln-Scedrov-Mitchell-Mitchell
Replace equality of processes by
(pseudo)metric distances between
processes

Quantitative measurement of the
distinction between processes.
Criteria on approximate reasoning
Soundness
 Usability
 Robustness

Criteria on metrics for approximate
reasoning

Soundness

Stability of distance under temporal
evolution: ``Nearby states stay close '‘
through temporal evolution.
``Usability’’ criteria on metrics
Establishing closeness of states:
Coinduction.
 Distinguishing states: Real-valued modal
logics.
 Equational and logical views coincide:
Metrics yield same distances as realvalued modal logics.

``Robustness’’ criterion on
approximate reasoning

The actual numerical values of the
metrics should not matter --``upto uniformities’’.
Uniformities (same)
m(x,y) = |x-y|
m(x,y) = |2x + sinx
-2y – siny|
Uniformities (different)
m(x,y) = |x-y|
Our results
Our results

For Discrete time models:
Labelled Markov processes
Labelled Concurrent Markov chains
Markov decision processes

For continuous time:
Generalized semi-Markov
processes
Results for discrete time models
Logic
Bisimulation
Metrics
(P)CTL(*)
Real-valued
modal logic
Compositionality Congruence
Nonexpansivity
Proofs
Coinduction
Coinduction
Results for continuous time models
Logic
Bisimulation
Metrics
CSL
Real-valued
modal logic
Compositionality ???
???
Proofs
Coinduction
Coinduction
Metrics for discrete
time probablistic
processes
Bisimulation

Fix a Markov chain. Define monotone F
on equivalence relations:
Defining metric: An attempt
Define functional F on metrics.
Metrics on probability measures

Wasserstein-Kantorovich

A way to lift distances from states to a
distances on distributions of states.
Metrics on probability measures
Metrics on probability measures
Example 1: Metrics on probability
measures
Unit measure concentrated at x
Unit measure concentrated at y
m(x,y)
x
y
Example 1: Metrics on probability
measures
Unit measure concentrated at x
Unit measure concentrated at y
m(x,y)
x
y
Example 2: Metrics on probability
measures
Example 2: Metrics on probability
measures
THEN:
Lattice of (pseudo)metrics
Defining metric coinductively
Define functional F on metrics
Desired metric is maximum fixed point of F
Real-valued modal logic
Real-valued modal logic
Tests:
Real-valued modal logic (Boolean)
q
q
Real-valued modal logic
Results

Modal-logic yields the same distance
as the coinductive definition
 However, not upto uniformities since glbs
in lattice of uniformities is not determined
by glbs in lattice of pseudometrics.
Variant definition that works upto
uniformities
Fix c<1. Define functional F on metrics
Desired metric is maximum fixed point of F
Reasoning upto uniformities
For all c<1, get same uniformity
[see Breugel/Mislove/Ouaknine/Worrell]


Variant of earlier real-valued modal logic
incorporating discount factor c
characterizes the metrics
Metrics for real-time
probabilistic
processes
Generalized semi-Markov
processes.
{c,d}
Evolution determined by
generalized states
<state, clock-valuation>
s
: Set of generalized states
u
{d,e}
{c}
t
Clock c
Clock d
Generalized semi-Markov
processes.
{c,d}
Path:
s
Traces((s,c)): Probability distribution
on a set of paths.
u
{d,e}
{c}
t
Clock c
Clock d
Accomodating discontinuities:
cadlag functions
(M,m) a pseudometric space.
cadlag if:
Countably many jumps, in general
Defining metric: An attempt
Define functional F on metrics. (c <1)
traces((s,c)), traces((t,d)) are distributions on
sets of cadlag functions.
What is a metric on cadlag functions???
Metrics on cadlag functions
x
y
are at distance 1 for unequal x,y
Not separable!
Skorohod metrics (J2)
(M,m) a pseudometric space. f,g cadlag with
range M.
Graph(f) = { (t,f(t)) | t \in R+}
Skorohod J2 metric: Hausdorff distance between graphs of f,g
g
f
(t,f(t))
f(t)
g(t)
t
Skorohod J2 metric
(M,m) a pseudometric space. f,g cadlag
Examples of convergence to
Example of convergence
1/2
Example of convergence
1/2
Examples of convergence
1/2
Examples of convergence
1/2
Examples of non-convergence
Jumps are detected!
Non-convergence
Non-convergence
Non-convergence
Non-convergence
Summary of Skorohod J2

A separable metric space on cadlag
functions
Defining metric coinductively
Define functional on 1-bounded pseudometrics (c <1)
a. s, t agree on all propositions
b.
Desired metric: maximum fixpoint of F
Real-valued modal logic
Real-valued modal logic
Real-valued modal logic
h: Lipschitz operator on unit interval
Real-valued modal logic
Real-valued modal logic
Base case for path formulas??
Base case for path formulas
First attempt:
Evaluate state formula F on state
at time t
Problem: Not smooth enough wrt time since
paths have discontinuities
Base case for path formulas
Next attempt:
``Time-smooth’’ evaluation of state
formula F at time t on path
Upper Lipschitz approximation to
at t
evaluated
Real-valued modal logic
Non-convergence
Illustrating Non-convergence
1/2
1/2
Results

For each c<1, modal-logic yields the same
uniformity as the coinductive definition

All c<1 yield the same uniformity. Thus,
construction can be carried out in lattice of
uniformities.
Proof steps
Continuity theorems (Whitt) of GSMPs
yield separable basis
 Finite separability arguments yield closure
ordinal of functional F is omega.
 Duality theory of LP for calculating metric
distances

Results

Approximating quantitative observables:
Expectations of continuous functions are
continuous

Continuous mapping theorems for
establishing continuity of quantitative
observables
Summary

Approximate reasoning for real-time
probabilistic processes
Results for discrete time models
Logic
Bisimulation
Metrics
(P)CTL(*)
Real-valued
modal logic
Compositionality Congruence
Nonexpansivity
Proofs
Coinduction
Coinduction
Results for continuous time models
Logic
Bisimulation
Metrics
CSL
Real-valued
modal logic
Compositionality ???
???
Proofs
Coinduction
Coinduction
Questions?