Approximate reasoning
for probabilistic realtime processes
Radha Jagadeesan DePaul University
Vineet Gupta
Google Inc
Prakash Panangaden McGill University
Outline of talk
Beyond CTMCs to GSMPs
 The curse of real numbers
 Metrics
 Uniformities
 Approximate reasoning

Real-time probabilistic processes

Add clocks to Markov processes
Each clock runs down at fixed rate
Different clocks can have different rates

Generalized Semi Markov Processes:
Probabilistic multi-rate timed automata
Generalized semi-Markov
processes.
{c,d}
Each state is labelled
with propositional
Information
s
Each state has a set
of clocks associated
with it.
u
{d,e}
{c}
t
Generalized semi-Markov
processes.
{c,d}
Evolution determined by
generalized states
<state, clock-valuation>
s
<s,c=2, d=1>
u
{d,e}
{c}
t
Transition enabled when a clock
becomes zero
Generalized semi-Markov
processes.
{c,d}
<s,c=2, d=1> Transition enabled in
1 time unit
s
<s,c=0.5,d=1> Transition enabled in
0.5 time unit
u
{d,e}
{c}
t
Clock c
Clock d
Generalized semi-Markov
processes.
{c,d}
0.2
s
Transition determines:
0.8
a. Probability distribution on next
states
b. Probability distribution on
clock values for new clocks
u
{d,e}
{c}
t c. This need not be exponential.
Clock c
Clock d
Generalized semi Markov
processes

If distributions are continuous and states are
finite:
Zeno traces have measure 0

Continuity results. If
stochastic processes from <s, > converge
to the stochastic process at <s, >
The traditional reasoning paradigm
Establishing equality: Coinduction
 Distinguishing states: HM-type logics
 Logic characterizes the equivalence (often
bisimulation)
 Compositional reasoning: ``bisimulation is
a congruence’’

Labelled Markov
Processes
PCTL
Bisimulation [Larsen-Skou,
Desharnais-Edalat-P]
Markov Decision
Processes
Bisimulation [Givan-Dean-Grieg]
Labelled Concurrent
Markov Chains
PCTL [Hansson-Johnsson]
Labelled Concurrent
Markov chains (with
tau)
PCTLCompleteness: [DesharnaisGupta-Jagadeesan-P]
Weak bisimulation
[Philippou-Lee-Sokolsky,
Lynch-Segala]
With continuous time
Continuous time Markov CSL [Aziz-Balarin-Braytonchains
Sanwal-Singhal-S.Vincentelli]
Generalized SemiMarkov processes
Stochastic hybrid
systems
Bisimulation,Lumpability
[Hillston, Baier-KatoenHermanns,Desharnais-P]
CSL
Bisimulation:?????
Composition:?????
The curse of real numbers:
instability
Vs
Vs
Problem!



Numbers viewed as coming with an error
estimate.
Reasoning in continuous time and continuous
space is often via discrete approximations.
Asking for trouble if we require exact match
Idea: Equivalence

metrics
Jou-Smolka90, DGJP99, …
Replace equality of processes by (pseudo)
metric distances between processes

Quantitative measurement of the
distinction between processes.
Criteria on approximate reasoning
Soundness
 Usability
 Robustness

Criteria on metrics for approximate
reasoning

Soundness

Stability of distance under temporal
evolution: “Nearby states stay close”
through temporal evolution.
``Usability’’ criteria on metrics
Establishing closeness of states:
Coinduction.
 Distinguishing states: Real-valued modal
logics.
 Equational and logical views coincide:
Metrics yield same distances as realvalued modal logics.

``Robustness’’ criterion on
approximate reasoning
The actual numerical values of the
metrics should not matter too much.
 Only the topology matters?
 Our results show that everything is defined
“up to uniformities.’’

What are uniformities?
In topology open sets capture an abstract
notion of “nearness”: continuity,
convergence, compactness, separation …
 In a uniformity one axiomatises the notion
of “almost an equivalence relation”:
uniform continuity, …
 Uniform continuity is not a topological
invariant.

Uniformities: definition
A nonempty collection U of subsets of SxS
such that:
 Every member of U contains
 If X in U then so is
 If X in U, there is a Y s.t. YoY is contained
in X
 Down closed, intersection closed

Two apparently different
Uniformities which are actually the
same
m(x,y) = |x-y|
m(x,y) = |2x + sinx
-2y – siny|
Uniformities (different)
m(x,y) = |x-y|
Our results
Our results
A metric on GSMPs based on
Wasserstein-Kantorovich and Skorohod
 A real-valued modal logic
 Everything defined up to uniformity

Results for discrete time models
Logic
Bisimulation
Metrics
(P)CTL(*)
Real-valued
modal logic
Compositionality Congruence
Nonexpansivity
Proofs
Coinduction
Coinduction
Results for continuous time models
Logic
Bisimulation
Metrics
CSL
Real-valued
modal logic
Compositionality ???
???
Proofs
Coinduction
Coinduction
Metrics for discrete
time probabilistic
processes
Defining metric: An attempt
Define functional F on metrics.
Metrics on probability measures

Wasserstein-Kantorovich

A way to lift distances from states to a
distances on distributions of states.
Metrics on probability measures
Not up to uniformities

If the Wasserstein metric is scaled you get
the same uniformity, but when you
compute the fixed point you get a different
uniformity because the lattice of
uniformities has a different structure (glbs
are different) then the lattice of metrics.
Variant definition that works up to
uniformities
Fix c<1. Define functional F on metrics
Desired metric is maximum fixed point of F
Reasoning up to uniformities
For all c<1 we get same uniformity
[see Breugel/Mislove/Ouaknine/Worrell]

Metrics for real-time
probabilistic
processes
Generalized semi-Markov
processes.
{c,d}
Evolution determined by
generalized states
<state, clock-valuation>
s
: Set of generalized states
u
{d,e}
{c}
t
Clock c
Clock d
The role of paths
In the continuous time case we cannot use
single actions: there is no notion of
“primitive step”
 We have to talk about a “timed path” of
one process matching a “timed path” of
another process.

Generalized semi-Markov
processes.
{c,d}
Path:
s
Traces((s,c)): Probability distribution
on a set of paths.
u
{d,e}
{c}
t
Clock c
Clock d
Accomodating discontinuities:
cadlag functions
(M,m) a pseudometric space.
cadlag if:
Countably many jumps, finitely many jumps
higher than any fixed “h”.
Defining metric: An attempt
Define functional F on metrics. (c <1)
traces((s,c)), traces((t,d)) are distributions on
sets of cadlag functions.
What is a metric on cadlag functions???
Metrics on cadlag functions
x
y
are at distance 1 for unequal x,y
Not separable!
Skorohod’s metrics on cadlag
Skorohod defined 4 metrics on cadlag: J1,J2
M1 and M2 with different convergence
properties.
All these are based on “wiggling” the time.
The M metrics “fill in the jumps”.
The J metrics do not.
Skorohod metric (J2)
(M,m) a pseudometric space. f,g cadlag with
range M.
Graph(f) = { (t,f(t)) | t \in R+}
Skorohod J2 metric: Hausdorff distance between graphs of f,g
g
f
(t,f(t))
f(t)
g(t)
t
Skorohod J2 metric
(M,m) a pseudometric space. f,g cadlag
Examples of convergence to
Example of convergence
1/2
Example of convergence
1/2
Examples of convergence
1/2
Examples of convergence
1/2
Non-convergence in J2:
Sequences of continuous functions cannot converge to
a discontinuous function.
In general, the number of jumps can decrease in the limit,
but they cannot increase.
Non-convergence
Non-convergence
Non-convergence
Non-convergence
Summary of Skorohod J2
A separable metric space on cadlag
functions
 Allows jumps to be nearby
 Allows jumps to decrease in the limit.
 Not complete.

Defining metric coinductively
Define functional on 1-bounded pseudometrics (c <1)
a. s, t agree on all propositions
b.
Desired metric: maximum fixpoint of F
Results



All c<1 yield the same uniformity. Thus,
construction can be carried out in lattice of
uniformities.
Real valued modal logic which gives an
alternate definition of a metric.
For each c<1, modal logic yields the same
uniformity but not the same metric.
Proof steps
Continuity theorems (Whitt) of GSMPs
yield separable basis.
 Finite separability arguments yield the
result that the closure ordinal of the
functional F is omega.
 Duality theory of LP for calculating metric
distances.

Summary
Metric on GSMPs defined up to uniformity.
 Real valued modal logic that gives the
same uniformity.
 Approximating quantitative observables:
Expectations of continuous functions are
continuous.
 Might be worth looking at the M2 metric.

Real-valued modal logic
Real-valued modal logic
Real-valued modal logic
Real-valued modal logic
h: Lipschitz operator on unit interval
Real-valued modal logic
Base case for path formulas??
Base case for path formulas
First attempt:
Evaluate state formula F on state
at time t
Problem: Not smooth enough wrt time since
paths have discontinuities
Base case for path formulas
Next attempt:
``Time-smooth’’ evaluation of state
formula F at time t on path
Upper Lipschitz approximation to
at t
evaluated
Real-valued modal logic
Non-convergence
Illustrating Non-convergence
1/2
1/2