Chapter 3 Fundamentals of Lyapunov Theory 3.1 Nonlinear Systems and Equilibrium Points • Nonlinear Systems A nonlinear dynamic system x f x, t f : n×1 nonlinear vector function x: n×1 state vector. n : numbers of states, order of the systems. (3.1) —A solution x(t): (i) a curve in state space as t varies from 0 to ∞, (ii) a state trajectory or a system trajectory. Closed-Loop Dynamics If the plant dynamics is x f x, u, t and some control law has been selected u g x, t then the closed-loop dynamics is x f x, g x, t , t Linear systems: A special class of nonlinear systems —Its dynamics: x A t x A(t): n×n matrix • Autonomous and Non-autonomous systems Linear systems are classified as either timevarying or time-invariant, depending on whether the system matrix A varies with time or not. Autonomous and Nonautonomous Definition 3.1: The nonlinear system (3.1) is said to be autonomous if f does not depend explicitly on time, i.e., if the system’s state equation can be written as x f x Otherwise, the system is called nonautonomous. ---Linear time-invariant (LTI) systems: autonomous. —Linear time varying (LTV) systems: non-autonomous. Equilibrium Points 1. Definition 3.2: A state x* is an equilibrium state (or equilibrium point) of the system if once x(t) is equal to x*, it remains equal to x* for all future time. 2. Example 3.1: The Pendulum Consider the pendulum MR b MgR sin 0 2 (3.5) R: the pendulum’s length M: its mass b: the friction coefficient at the hinge g: the gravity constant. Letting x1 = , x2 = , the corresponding statespace equations is x1 x2 b g x2 x sin x1 2 2 MR R —The equilibrium points: (0 [π], 0) x2 = 0, sin x1 = 0 the pendulum resting exactly at the vertical up and down positions. •Perturbation Dynamics —Let x* (t) be the nominal motion trajectory, corresponding to initial condition x*(0) = xo. — Perturb x(0) = xo + δxo x* f x* x 0 x0 x f ( x) x 0 x0 x0 Figure 3.2 —The motion error due to initial condition perturbation: e(t) = x(t)-x*(t) and e = f(x* + e,t)-f(x*, t) = g(e, t) with e(0) = δxo. represent the perturbation dynamics —g(0, t) = 0: the new dynamic system, with e as state and g in place of f, has an equilibrium point at the origin of the state space. - Studying the deviation of x(t) from x*(t) for the original system study the stability of the perturbation dynamics with respect to the equilibrium point 0. - Perturbation dynamics is non-autonomous, due to the presence of the nominal trajectory x*(t). •Stability and Instability Definition 3.3:The equilibrium state x = 0 is said to be stable (or Lyapunov stable) if , for any R > 0, there exists r > 0, such that if ||x(0)|| < r, then ||x(t)|| < R for all t 0 . Otherwise, the equilibrium point is unstable. Figure 3.3:Concepts of stability Phase Portrait of a Mass-spring System Mass-spring system x x 0 , x(0) x0 x(0) 0 Solution: x(t ) x0 cos t x(t ) x0 sin t Equation of the trajectories: x x x0 2 2 2 Figure 2.1: A mass-spring system and its phase portrait Example 2.5: A satellite control system Figure 2.5:Satellite control using on-off thrusters If u = -U, then 2U c1 2 Example 3.3: Instability of the Van der Pol Oscillator The Van der Pol oscillator x1 x2 x2 x1 1 x12 x2 Origin: an unstable equilibrium point Figure 3.4:Unstable origin of the Van der Pol Oscillator —Observation: system trajectories starting from any non-zero initial states all asymptotically approach a limit cycle. —Illustration: if we choose R to be small enough the circle of radius R fall completely within the closed-curve of the limit cycle, then system trajectories starting near the origin will eventually get out of this circle Definition 3.4:An equilibrium point 0 is asymptotically stable if it is stable, and if, in addition, there exists some r > 0 such that ||x(0)|| < r implies that x(t) 0 as t —Asymptotic stability means that the equilibrium is stable, and that in addition, started close to 0 actually converge to 0 as time t goes to ∞. —marginally stable: An equilibrium point which is Lyapunov stable but not asymptotically stable. Figure 3.3:Concepts of stability —Domain of attraction of the equilibrium point: the largest region such that trajectories initiated at the points in the region eventually converge to the origin. •Example 2.2: A nonlinear second-order system The system x 0.6 x 3x x 0 2 has two singular points: (0, 0) and (-3, 0). - The trajectories move towards the point (0,0) while moving away from the point (-3,0). —State convergence stability. —Example: a simple system studied by Vinograd. — The origin is unstable in the sense of Lyapunov, despite the state convergence. Figure 3.5: State convergence does not imply stability — Calling such a system unstable is quite reasonable, since a curve such as C may be outside the region where the model is valid. — Example: the subsonic and supersonic dynamics of a high-performance aircraft are radically different, while, with the problem under study using subsonic dynamic models, C could be in the supersonic range. In many engineering applications, there is a need to estimate how fast the system trajectory approaches 0. Definition 3.5 :An equilibrium point 0 is exponentially stable if there exists two strictly positive numbers and such that t 0, x t x 0 e t in some ball Br around the origin. λ: rate of convergence —The state vector of an exponentially stable system converges to the origin faster than an exponential function. —Example: x 1 sin x x 2 is exponentially convergent to x = 0 with a rate = 1,because t therefore x t x 0 exp 1 sin 2 x d 0 x t x 0 et —Exponential stability asymptotic stability. — Asymptotic stability exponential stability —Eample: 2 x(0) 1 x x , whose solution is x = 1/(1+t), a function slower that any exponential function e t (with > 0). Definition 3.6: If asymptotic (or exponential) stability holds for any initial states, the equilibrium point is said to be asymptotically (or exponentially) stable in the large. It is also called globally asymptotically (or exponentially) stable. —Examples: —Linear time-invariant (LTI) systems are either asymptotically stable, or marginally stable, or unstable — Asymptotic stability of LTI system is always global and exponential — Instability of LTI system always implies exponential blow-up. — This explains why the refined notions of stability are explicitly needed only for nonlinear systems. 3.3 Linearization and Local Stability Consider the autonomous system x f ( x) assuming ' f(x) C is continuously differentiable and f(0)=0. Then the system dynamics can be written as f x x x 0 x + fh. o. t. (x) where fh. o. t. stands for higher-order terms in x. —Let A denote f x x 0 Then, the system x Ax is called the linearization (or linear approximation) of the original nonlinear system at the equilibrium point 0. —For non-autonomous nonlinear system with a control input u x f ( x, u ) such that f(0,0)=0, we can write f x ( ) ( x0,u0) x f x ( ) ( x0,u0 ) u u fh.o.t ( x,u ) —Let f f A( ) ( x0 ,u 0 ) B ( ) ( x0 ,u 0 ) x u — x Ax Bu : the linearization (or linear approximation) of the original nonlinear system at (x = 0, u = 0). Control law u = u(x) (with u(0) = 0) transforms the non-autonomous system into an autonomous closed-loop system, having x = 0 as an equilibrium point. —Linearly approximating the control law as du u ( ) x0x=Gx dx the closed-loop dynamics can be linearly approximated as x f ( x, u ( x)) ( A BG) x Alternative Linear Approximation —Considering the autonomous closed-loop system x f ( x, u( x)) f1 ( x) and linearizing the function f1with respect to x at its equilibrium point x = 0. Example 3.4:consider the system x1 x x1 cos x2 x2 x2 ( x1 1) x1 x1 sin x2 2 2 —Linear approximation about x = 0: x1 1 x1 0 x2 x1 x2 1 x1 1 x2 x1 x2 —The Linearized system: 1 0 x x 1 1 —Example:consider the system x 4 x5 ( x 2 1)u 0 — Linear approximation about x = 0: x 0 (0 1)u 0 — The linearized system: x u —Control law: u sin x x 3 x cos 2 x — The linearized closed-loop dynamic: x x x 0 Theorem 3.1 (Lyapunov’s linearization method) •If the linearized system is strictly stable (i.e ,if all eigenvalues of A are strictly in the left-half complex plane), then the equilibrium point is asymptotically stable (for the actual nonlinear system). •If the linearized system is unstable (i.e , if at least one eigenvalue of A is strictly in the right-half complex plane), then the equilibrium point is unstable(for the nonlinear system). •If the linearized system is marginally stable (i. e, all eigenvalues of A are in the left-half complex plane, but at least one of them is on the j axis), then one cannot conclude anything from the linear approximation (the equilibrium point may be stable, asymptotically stable, or unstable for the nonlinear system). Example 3.6:Consider the first order system x ax bx 5 —The origin 0 is an equilibrium point — The linearization around 0: x ax —The stability properties of the nonlinear system: a < 0:asymptotically stable; a > 0:unstable; a = 0:cannot tell from linearization. —What shall we do when a=0? 3.4 Lyapunov’s Direct Method Basic idea from Physical Observation If the total energy of a mechanical (or electrical) system is continuously dissipated, then the system, where linear or nonlinear, must eventually settle down to an equilibrium point. –Example: Consider the nonlinear mass-damperspring system: mx b x x k0 x k1 x 3 0 :nonlinear dissipation or damping (k0 x k1x3 ) :nonlinear spring term bx x –Assume the mass is pulled away from the natural length of the spring by a large distance, and then released (i) linearization method does not apply (ii) the linearized system is marginally stable only Figure 3.6:A nonlinear mass-damperspring system –The total mechanical energy of the system : x 1 2 3 V (x) mx (k0 x k1 x )dx 0 2 1 2 1 1 4 2 mx k0 x k1 x (3.14) 2 2 4 – Comparing the definitions of stability and mechanical energy: ‧zero energy at the equilibrium point (x 0, x ‧asymptotic stability the convergence of mechanical energy to zero -- instability is related to the growth of mechanical energy 0) –These relations indicate (i) the mechanical energy indirectly reflects the magnitude of the state vector (ii) the stability properties can be characterized by the variation of the mechanical energy –The rate of energy variation: V (x) m x x (k0 x k1 x 3 ) x x(b x x ) b x 3 shows the energy of the system is continuously dissipated by the damper until x 0 . Definition 3.7: A scalar continuous function V(x) is said to be locally positive definite if V(0) = 0 and, in a ball B R 0 x 0 V (x) 0 If V(0) and the above property holds over the whole state space, then V(x) is said to be globally positive definite. – A function V(x) is negative definite if V(x) is positive definite – V(x) is positive semi-definite if V(x) 0 and V(x)=0 for some x 0 – V(x) is negative semi-definite if (–V(x)) is positive semi-definite. Figure 3.7:Typical shape of a positive definite function V ( x1 , x2 ) Figure 3.8:Interpreting positive definite functions using contour curves –V(x) represents an implicit function of time t. – Assuming that V(x) is differentiable: dV (x) V V V x f(x) dt x x Definition 3.8 If, in a ball B R , the function V(x) is positive definite and has continuous partial derivatives, and if its time derivative along any state trajectory of system x f ( x) is negative semi-definite, i. e., 0 V (x) 0 then V(x) is said to be a Laypunov function for the system x f ( x) . Figure 3.9:Illustrating Definition 3.8 for n= 2. Theorem 3.2 (Lyapunov Theorem for Local Stability) If, in a ball B R , there exists a scalar function V(x) with continuous first partial derivatives such that V(x) is positive definite (locally in B R ). V ( x) is negative semi-definite (locally in B R ). then the equilibrium point 0 is stable. If, actually, the derivative V ( x) is locally negative definite in B R0, then the stability is asymptotic. 0 0 0 Figure 3.11: Illustrating the proof of Theorem 3.2 for n =2 Example 3.7:Local stability A simple pendulum with viscous damping sin 0 Consider a locally positive definite V (x) (1-cos ) 2 2 :total energy of the pendulum V (x) sin - 2 0 –The origin is a stable equilibrium point –Physical insight for V (x) 0 :the damping term absorbs energy. – V is the power dissipated in the pendulum. –However, with this Lyapunov function, one cannot draw conclusions on the asymptotic stability of the system, because V (x) is only negative semi-definite. Figure 3.6:A nonlinear mass-damperspring system –The total mechanical energy of the system : x 1 2 3 V (x) mx (k0 x k1 x )dx 0 2 1 2 1 1 4 2 mx k0 x k1 x (3.14) 2 2 4 –These relations indicate (i) the mechanical energy indirectly reflects the magnitude of the state vector (ii) the stability properties can be characterized by the variation of the mechanical energy –The rate of energy variation: V (x) m x x (k0 x k1 x 3 ) x x(b x x ) b x 3 shows the energy of the system is continuously dissipated by the damper until x 0 . Example 3.8:Asymptotic stability Consider the nonlinear system x1 x1 ( x1 x2 -2 )-4 x1 x2 2 2 2 x2 4 x1 x2 x2 ( x1 x2 -2 ) 2 2 2 Define the positive definite function V ( x1 , x2 ) x1 x2 2 2 – V 2( x12 x22 )( x12 x22 2) –Locally negative definite in B2 x1 , x2 x x2 2 2 1 2 the origin is asymptotically stable. Theorem 3.3 (Lyapunov Theorem for Global Stability ) Assume that there exists a scalar function V of the state x, with continuous first order derivatives such that • V(x) is positive definite • V (x) is negative definite • V (x) as x (V(x) must be radially unbounded) then the equilibrium at the origin is globally asymptotically stable. Reason for Radial Unboundedness –To assure the contour curves V (x) V correspond to closed curves. (why?) If the curves are not closed, it is possible for the state trajectories to drift away from the equilibrium point, even though the state keeps going through contours corresponding to smaller and smaller V' s . Figure 3.12:Motivation of the radial unboundedness condition 3.4.3 Invariant Set Theorems –Motivation: it often happens that V , is only negative semi-definite. –Fortunately, it is still possible to draw conclusions on asymptotic stability, with the help of the powerful invariant set theorems, attributed to La Salle. –Additional feature: To describe convergence to a limit cycle Definition 3.9: A set G is an invariant set for a dynamic system if every system trajectory which starts from a point in G remains in G for all future time. –Example: any equilibrium point. –The domain of attraction of an equilibrium point –The whole state-space –any trajectories of an autonomous system –limit cycles Theorem 3.4(Local Invariant Set Theorem) Consider an autonomous system x f ( x), with f continuous, and let V(x) be a scalar function with continuous first partial derivatives. Assume that • for some l > 0, the region l defined by V (x) l is bounded • V (x) 0 for all x in l Let R be the set of all points within l where V (x) 0 , and M be the largest invariant set in R. Then, every solution x(t) originating in l tends to M as t . Remarks –The word “largest” means M is the union of all invariant sets (e. g., equilibrium points or limit cycles) within R. –If the set R is itself invariant (i. e., if once V 0, then V 0 for all future time), then M = R. –V, although often still referred to as a Lyapunov function, is not required to be positive definite. Figure 3.14:Convergence to the largest invariant set M Figure 3.6:A nonlinear mass-damperspring system Example 3.7:Local stability A simple pendulum with viscous damping sin 0 Consider a locally positive definite V (x) (1-cos ) 2 2 :total energy of the pendulum V (x) sin - 2 0 –The origin is a stable equilibrium point Example 3.12:Limit Cycle Consider the system x1 x2 7 x1 x 4 1 2 x2 10 2 x2 x1 3 x2 x1 2 x2 10 3 5 4 2 Notice first that the set defined by x 2x 10 0 is an invariant set, because 4 1 d ( x14 2 x22 10) dt (4 x110 12 x26 )( x14 2 x22 10) 0 2 2 on the set. The motion on this invariant set is described (equivalently) by either of the equations x1 x2 x2 x13 Therefore, we see that the invariant set actually represents a limit cycle, along which the state vector moves clockwise. Figure 3.15:Convergence to a limit cycle Global Invariant Set theorems The above invariant set theorem can be simply extended to a global result, by requiring the radial unboundedness of the scalar function V rather than the existence of a bounded l Theorem 3.5 (Global Invariant Set Theorem) Consider the autonomous system , with f x function f ( x) continuous, and let V(x) be a scalar with continuous first partial derivatives. Assume that • • V (x) as x (x) 0 V Let R be the set of all points where and M be the largest invariant set inVR. (x)Then 0, all solutions globally asymptotically converge to M as t Example 3.14:A class of second-order nonlinear systems Consider a second-order system x b( x ) c ( x ) 0 where b and c are continuous functions satisfying the sign conditions x b x 0 for x 0 xc x 0 for x 0 –Example systems: mass-damper-spring system, nonlinear R-L-C electrical circuit –If the functions b and c are linear b( x) x, c( x) x , the sign conditions are the necessary and sufficient conditions for the system’s stability (1 0,0 0). 1 0 –Assume b(0) = 0 and c(0) = 0 Figure 3.16:A nonlinear R-L-C circuit – 1 2 x V x c( y )dy 0 2 the sum of the kinetic and potential energy of the mass-damper-spring system – V x x c ( x ) x x b( x ) x c ( x ) c ( x ) x x b( x ) 0 power dissipated in the system – xb( x) 0 only if x 0 – x 0 x c ( x ) which is nonzero as long as x 0. Thus M contains only one point: origin. – The local invariant set theorem the origin is a locally asymptotically stable point. Figure 3.17:The functions b( x ) and c( x ) Furthermore, if the integral x o c( y)dy is unbounded as x , then V is a radially unbounded function and the equilibrium point at the origin is globally asymptotically stable, according to the global invariant set theorem. 3.5 System Analysis Based on Lyapunov’s Direct Method There is no general way of finding Lyapunov functions for nonlinear systems. This is the fundamental drawback of the direct method. Thus, a case by case approach is usually taken. •Definition 3.10 T A square matrix M is symmetric if M M (in other words, if i, j, Mij M ji ). A square matrix M is skew-symmetric if T M - M (i. e., if i, j, Mij M ji ). –Any square n n matrix M MM M-M M 2 2 T T –M is an n n skew symmetric matrix x Mx 0, x T (3.16) –In linear systems, we often use xT Mx as Lyapunov function. –We can always assume, without loss of generality, that M is symmetric. Definition 3.11:A square n n matrix M is positive definite (p. d.) if x 0 x Mx 0 T –A matrix M is positive definite x Mx is a positive definite function. T •Lyapunov Functions for Linear Time-Invariant Systems Consider a linear system x Ax, let the Lyapunov function candidate be V xT Px P: a symmetric positive definite matrix. If V x Px x Px -x Qx T T T A P PA -Q then is the Lyapunov equation. T (3.18) (3.19) Lyapunov Equation –Symmetric matrix Q defined by AT P PA -Q is p. d. the origin is globally asymptotically stable. –This “natural” approach may lead to inconclusive result, i. e., Q may be not positive definite even for stable systems. Example 3.17:consider a second-order linear system whose A matrix is 4 0 A 8 12 If we take P = I, then 4 0 T Q PA A P 4 24 Q is not positive definite don’t know whether the system is stable or not. Thinking in Opposite Direction –To derive a positive definite matrix P from a given positive definite matrix Q, i. e., • choose a positive definite matrix Q • solve for P from the Lyapunov equation • check whether P is p. d • If P is p. d., global asymptotical stability is guaranteed. Theorem 3.6:A necessary and sufficient condition for a LTI system x Ax to be strictly stable is that, for any symmetric p. d. matrix Q, the unique matrix P, solution of the Lyapunov equation (3.19), be symmetric positive definite. Example 3.18:consider again the secondorder system of Example 3.17. Let us take Q = I and denote P by p11 p12 P p p 21 22 where, due to the symmetry of P, p21 p12 Then the Lyapunov equations is p11 p 21 p12 0 4 0 8 p11 p22 8 12 4 12 p21 p12 1 0 p22 0 1 whose solution is p11 5 / 16, p12 p 22 1 / 16 The corresponding matrix 1 5 1 P 16 1 1 is positive definite the linear system is globally asymptotically stable. 3.5.3 The Variable Gradient Method –A formal approach to constructing Lyapunov functions. –Assuming the gradient V of an unknown V –Integrating the assumed gradient V x 0 Vd x x where V [V / x1,........,V / xn ] . T curl conditions Vi V j x j xi where (i, j 1, 2,....., n) V Vi (i,j=1,2,......n) xi for n=2, V1 V2 x2 x1 Procedure for Seeking V •Assume n Vi aij x j , i 1,...., n j 1 where aij •Solve for are coefficients to be determined. aij •Restrict aij so that V is negative semi-definite (at least locally) •Compute V from V •Check whether V is positive definite –It is usually convenient to obtain V by integrating along a path which is parallel to each axis, i. e., V x V1 x1,0,.....,0 dx1 V2 x1 , x2 ,0,.....,0 dx2 ... x1 x2 0 0 Vn x1 , x2 ,....., xn dxn xn 0 THEOREM 9.9 Test for Path Independence Let P and Q have continuous first partial derivatives in an open simple connected region. Then C P dx Q dy is independent of the path C if and only if P Q y x for all (x, y) in the region. Example 3.20:To find a Lyapunov function for x1 2 x1 x2 2 x2 2 x x 2 1 2 and discuss the stability of the e.p. (0,0) Assume V a x a x 1 11 1 12 2 V2 a21x1 a22 x2 The curl equation is V1 V2 or x2 x1 a11 a12 a21 a22 x1 a12 x2 a21 x1 x2 x2 x2 x1 x1 If the coefficients are chosen to be a11 a22 1, a12 a21 0 which leads to V1 x1 V2 x2 then V V x 2 x12 2 x22 1 x1x2 locally negative definite in the region (1 x1 x2 ) 0 Then V x 0 x1dx1 0 x1 x2 x12 x22 x2 dx2 2 positive definite the asymptotic stability –The Lyapunov function obtained by the variable Gradient Method is not unique. (3.22) Taking a11 1, a12 x22 a21 3 x22 , a22 3 we obtain the positive definite function x12 3 2 V x2 x1 x23 2 2 whose derivative is V 2 x12 6 x22 2 x22 ( x1 x2 3x12 x22 ) Which is locally negative definite. 3.5.4 Physically Motivated Lyapunov Functions 1 T T V [q Hq q K p q] 2 •A Simple Convergence Lemma Lemma:If a real function W(t) satisfies the inequality W t W t 0 (3.26) where is a positive real number. Then t W (t ) W (0)e Proof:Let us define a function Z(t) by Z (t ) W W (3.27) t W (t ) W (0)e e 0 then W (t ) W (0)et t ( t ) Z ( )d The above lemma implies that, if W is a nonnegative function, the satisfaction of (3.26) guarantees the exponential convergence of W to zero. 4.5 Lyapunov-Like Analysis Using Barbalat's Lemma 4.5.2 Barbalat's Lemma Lemma 4.2 (Barbalat) If the differentiable function f(t) has a finite limit as t , and if f is uniformly continuous, then f (t ) 0 as t . -- A function g is said to be uniformly continuous on [0, ) if R 0, ( R) 0, t 0, t t1 | g (t ) g (t1 ) | R -- Alternative approach: examine the function's derivative. A differentiable function is uniformly continuous if its derivative is bounded. Using Barbalat's Lemma for Stability Analysis Lemma 4.3 ("Lyapunov-Like Lemma") If a scalar function V(x, t) satisfies the following conditions • V (x, t ) is lower bounded • V (x, t ) is negative semi-definite • V (x, t ) is uniformly continuous in time then V (x, t ) 0 as t . Example 4.13:The closed-loop error dynamics of an adaptive control system for a first-order plant with one unknown parameter : e e w(t ) ew(t ) e , : two states of the closed-loop dynamics, representing tracking error and parameter error. w(t): a bounded continuous function. Consider the lower bounded function 2 2 V e Its derivative is V 2e(e w) 2(ew(t )) 2e 2 0 Then V(t) V(0), which implies V(t) has a finite limit as t goes to infinity, and e and are bounded. But the invariant set theorem cannot be used to conclude the convergence of e, because the dynamics is non-autonomous. The derivative of V : V 4e(e w) This shows that V is bounded, since w is bounded by hypothesis, and e and were shown above to be bounded. Hence, V is uniformly continuous. Application of Barbalat's lemma: e 0 as t Note that, although e converges to zero, the system is not asymptotically stable, because is only guaranteed to be bounded.