Models for
Control and Verification
Ian Mitchell
Department of Computer Science
The University of British Columbia
research supported by
National Science and Engineering Research Council of Canada
Outline
• Classes of models
–
–
–
–
–
Well-posed models
Difference Equations
Nonlinear Ordinary Differential Equations
Syntax vs semantics
Visualization
• Optimal Control
– Objective and value functions
• Verification: Reachability
– Forward & backward, tubes & sets, maximal & minimal
March 2008
Ian Mitchell (UBC Computer Science)
2
Control & Verification Require Modeling
• Dynamic systems change with time
• We wish to reason about that change
– Control: We seek to guide the evolution to achieve a desired
objective
– Verification: We seek to confirm the evolution will achieve a
desired objective
• Control and verification require prediction of future
evolution
– Prediction is achieved by mathematical models
• System is described by state and time
March 2008
Ian Mitchell (UBC Computer Science)
3
Discrete vs Continuous
• Discrete variable
–
–
–
–
Drawn from a countable domain, typically finite
Often no useful metric other than the discrete metric
Often no consistent ordering
Examples: names of students in this room, rooms in this
building, natural numbers
• Continuous variable
–
–
–
–
March 2008
Drawn from an uncountable domain, but may be bounded
Usually has a continuous metric
Often no consistent ordering
Examples: Real numbers [ 0, 1 ], Rd, SO(3)
Ian Mitchell (UBC Computer Science)
4
Classes of Models for Dynamic Systems
• Discrete time and state
• Continuous time / discrete state
– Discrete event systems
• Discrete time / continuous state
• Continuous time and state
• Markovian assumption
– All information relevant to future
evolution is captured in the state
variable
• Models are deterministic
– Future evolution completely
determined by initial conditions
• Not the only classes of models
March 2008
Ian Mitchell (UBC Computer Science)
5
Well-Posed Models
• Mathematical models may not behave nicely
– May describe impossible evolutions
– May not be easy to apply formal reasoning
• We want to forbid such (eg ignore) models
• Common desirable traits
– There exists a solution for all (or some) time
– The solution is unique
– The solution depends continuously on the data (initial
conditions, dynamics)
March 2008
Ian Mitchell (UBC Computer Science)
6
Difference Equations
• Existence: for all t and x, f(t, x)  ;
• Uniqueness: for all t and x, | f(t, x) | = 1
• Continuous dependence on the data: for all t, x, y
there exists constant  such that
– Only makes sense if state space has a continuous metric
– Sufficient but not necessary
– Might also want to handle mistakes in f
March 2008
Ian Mitchell (UBC Computer Science)
7
Lipschitz Continuity
• Called “Lipschitz continuity” with respect to x (or y)
• Constant  is the “Lipschitz constant”
• Relationship with continuity and differentiability?
Continuity
Lipschitz continuity
no
relation
Differentiability
Differentiability
with bounded derivative
March 2008
Lipschitz continuity
Lipschitz continuity
Ian Mitchell (UBC Computer Science)
8
Lipschitz Continuous Functions
• Which of these functions is Lipschitz continuous?
A
B
C
D
March 2008
Ian Mitchell (UBC Computer Science)
9
Ordinary Differential Equations (ODEs)
• What about second order ODE?
– Newton’s second law: force = (mass)(acceleration)
• Need to reformulate into first order form
– Define new variable z(t) 2 R2*d
• May also be useful to remove dependence on t
– Define new variable y(t) 2 Rd+1
– Called “autonomous system” in mathematics
March 2008
Ian Mitchell (UBC Computer Science)
10
Standard First Order Form
• Q45: What is the equivalent first order form of the
following ODE for the motion of a pendulum?
l
m
March 2008
Ian Mitchell (UBC Computer Science)
11
Standard First Order Form
• Q46: What is the equivalent first order form of the
following high order ordinary differential equation?
March 2008
Ian Mitchell (UBC Computer Science)
12
Well-Posed ODEs
• Consider initial value problem (IVP): x(ti) = xi
• If f is Lipschitz continuous in x for all t 2 [ ti, tf ]
– There exists a unique solution x(t) for t 2 [ ti, tf ] for each xi
such that dx /dt exists and dx/dt = f(t,x)
– For perturbed initial data yi yielding y(t)
– For perturbed dynamics
• Sufficient but not necessary conditions
March 2008
Ian Mitchell (UBC Computer Science)
13
Ill-Posed ODEs
• Why do we care that the ODE is well-posed?
– Theory: much depends on the existence of a unique solution
– Numerics: approximate solution may not be desired solution,
and may not even be near a true solution
March 2008
Ian Mitchell (UBC Computer Science)
14
Syntax vs Semantics
• Syntax: what are legal statements?
– Boolean expression over variable x 2 { 0, 1 } and boolean
expressions f and g: x | 0 | 1 | ¬ f | fg | f + g | f ◦ g | (f)
– Arithmetic expression over x 2 R [ 1 and expressions f and
g: x | –f | f + g | f – g | fg | f / g | f ◦ g | (f)
• Semantics: what do those statements mean?
– Boolean expression “or”
x
0
0
1
1
y
0
1
0
1
x+ 0 1
1
y
– Arithmetic expression 4 + 5 = 9
1
March 2008
Ian Mitchell (UBC Computer Science)
15
Checking a Model
• Well-posed conditions are examples of syntactic
checks: tests applied directly to the model
– Model does not itself evolve, but is a static entity
– Complexity of check depends only on the complexity of the
model
• Alternative: Semantic checks
– Requires understanding the evolving solution
– Complexity of check depends on the complexity of the
solution trajectory
March 2008
Ian Mitchell (UBC Computer Science)
16
Restricted Classes of Model
• Many results in control and verification assume a
restricted class of models
– Permits more checks to be syntactic / static
– May simplify checks of semantic / dynamic
– Example: Are there any syntactically correct but semantically
incorrect boolean expressions?
– Example: Are there any syntactically correct but semantically
incorrect arithmetic expressions other than 0 / 0?
• Our nonlinear ODE and DI models are very general
– Most of what we discuss (beyond well-posedness) will be
semantic / dynamic checks
March 2008
Ian Mitchell (UBC Computer Science)
17
• Most of the visualization of
system evolution will be done
in the phase or state space
(ignore time)
pendulum workspace
phase Space
– Pendulum states angle and
angular velocity
state vs time
Visualization
March 2008
Ian Mitchell (UBC Computer Science)
18
Outline
• Classes of models
–
–
–
–
–
Well-posed models
Difference Equations
Nonlinear Ordinary Differential Equations
Syntax vs semantics
Visualization
• Optimal Control
– Objective and value functions
• Verification: Reachability
– Forward & backward, tubes & sets, maximal & minimal
Dimitri Bertsekas,
Dynamic Programming & Optimal Control,
Athena Scientific (3rd edition 2005)
March 2008
Ian Mitchell (UBC Computer Science)
19
Achieving Desired Behaviours
• We can attempt to control a system when there is a parameter u
of the dynamics (the “control input”) which we can influence
– Time dependent dynamics are possible, but we will mostly deal with
time invariant systems
• Without a control signal specification, system is nondeterministic
– Current state cannot predict unique future evolution
• Control signal may be specified
– Open-loop u(t) or u: R → U
– Feedback, closed-loop u(x(t)) or u: S → U
– Either choice makes the system deterministic again
March 2008
Ian Mitchell (UBC Computer Science)
20
Visualization: Vector Fields
• Introduction of a free control input changes the vector field plot
in the phase space into a field of cones (nondeterministic)
• Feedback control law changes it back into a (static) vector field
• Open loop control law does not
unspecified
input signal
no inputs
(“autonomous” for
control engineers)
March 2008
feedback
input signal
Ian Mitchell (UBC Computer Science)
21
Objective Function
• We distinguish quality of control by an objective / payoff / cost
function, which comes in many different variations
– eg: discrete time discounted with fixed finite horizon tf
– eg: continuous time no discount with target set T
March 2008
Ian Mitchell (UBC Computer Science)
22
Value Function
• Choose input signal to optimize the objective
– Optimize: “cost” is usually minimized, “payoff” is usually maximized
and “objective” may be either
• Value function is the optimal value of the objective function
– May not be achieved for any signal (eg: min should be inf)
• Set of signals U is contentious
– For implementation purposes, we desire restricted classes:
bounded, continuous, piecewise constant
– Unfortunately, theory applies to (and thus can only
guarantee optimality with) very general classes: measurable
March 2008
Ian Mitchell (UBC Computer Science)
23
Example: LQR for Linear Systems
• Much of the “optimal control” literature and most classes focus
(without mentioning it) on linear systems
• Corresponding objective functions are usually quadratic
where A, B, Q, R, Qf are all matrices of appropriate size
• Successful but restricted class of problems
– Not rigorously part of the results to follow (due to a technicality)
March 2008
Ian Mitchell (UBC Computer Science)
24
Outline
• Classes of models
–
–
–
–
–
Well-posed models
Difference Equations
Nonlinear Ordinary Differential Equations
Syntax vs semantics
Visualization
• Optimal Control
– Objective and value functions
• Verification: Reachability
– Forward & backward, tubes & sets, maximal & minimal
Ian Mitchell,
“Comparing Forward and Backward Reachability
as Tools for Safety Analysis,”
Hybrid Systems Computation and Control,
LNCS 4416, Springer-Verlag (2007).
March 2008
Ian Mitchell (UBC Computer Science)
25
Verification: Safety Analysis
• Does there exist a trajectory of system H leading
from a state in initial set I to a state in terminal set T ?
(under some policy for input u(¢))
March 2008
Ian Mitchell (UBC Computer Science)
26
Typical Systems: ODEs
• Common model for continuous state spaces
• Standard existence and uniqueness
March 2008
Ian Mitchell (UBC Computer Science)
27
Working with Sets
• Optimal control works with a single optimal trajectory
• Verification works with sets of trajectories
– Takes a nondeterministic (but not probabilistic) viewpoint
• Basic construct is reachability
– Many versions: forward and backward, sets or tubes
– What should the input do?
• Many related concepts in control theory
– Invariant sets, controlled invariant sets, stability
• Safety is not the only verification goal
– Liveness is a common goal, but often harder to verify
March 2008
Ian Mitchell (UBC Computer Science)
28
Forward Reachability
• Start at initial conditions and compute forward
March 2008
Ian Mitchell (UBC Computer Science)
29
Backward Reachability
• Start at terminal set and compute backwards
March 2008
Ian Mitchell (UBC Computer Science)
30
Exchanging Algorithms
• Algorithms are (mathematically) interchangeable if
system dynamics can be reversed in time
• For example:
• Then
March 2008
Ian Mitchell (UBC Computer Science)
31
Maximal Reachability
• Input signal u(¢) maximizes size of the set or tube
March 2008
Ian Mitchell (UBC Computer Science)
32
Maximal Reachability Definition
March 2008
Ian Mitchell (UBC Computer Science)
33
Maximal Reachability Results
• Reach sets and tubes provide similar information
• The following properties are equivalent
• Any maximal reachability operator can be used to
demonstrate safety for all possible inputs
March 2008
Ian Mitchell (UBC Computer Science)
34
Maximal Reachability Demonstration
Initial and Terminal Sets
System Dynamics
Forward Reach Set Results
March 2008
Ian Mitchell (UBC Computer Science)
35
Maximal Reachability Demonstration
Initial and Terminal Sets
System Dynamics
Forward Reach Tube Results
March 2008
Ian Mitchell (UBC Computer Science)
36
Maximal Reachability Demonstration
Initial and Terminal Sets
System Dynamics
Backward Reach Set Results
March 2008
Ian Mitchell (UBC Computer Science)
37
Maximal Reachability Demonstration
Initial and Terminal Sets
System Dynamics
Backward Reach Tube Results
March 2008
Ian Mitchell (UBC Computer Science)
38
Minimal Reachability
• Input signal u(¢) minimizes size of the set or tube
March 2008
Ian Mitchell (UBC Computer Science)
39
Minimal Reachability Definition
March 2008
Ian Mitchell (UBC Computer Science)
40
Minimal Reachability Results
• Reach tubes provide more information
– Choice of trajectory length t is quantified first for sets but last
for tubes
March 2008
Ian Mitchell (UBC Computer Science)
41
Minimal Reachability Results
• Backward reach tubes are the only minimal
reachability operator that can prove that there exists
an input u(¢) which keeps the system safe
– Basic problem with minimal forward reachability: the state
lying in the terminal set is chosen before the input, while the
state lying in the initial set is chosen after
March 2008
Ian Mitchell (UBC Computer Science)
42
Minimal Reachability Demonstration
Initial and Terminal Sets
System Dynamics
(Correct) Backward Reach Tube Results
March 2008
Ian Mitchell (UBC Computer Science)
43
Minimal Reachability Demonstration
Initial and Terminal Sets
System Dynamics
(Incorrect) Forward Reach Tube Results
March 2008
Ian Mitchell (UBC Computer Science)
44
Models for
Control and Verification
For more information contact
Ian Mitchell
Department of Computer Science
The University of British Columbia
mitchell@cs.ubc.ca
http://www.cs.ubc.ca/~mitchell