Introduction to Simulation - Lecture 12 Methods for Ordinary Differential Equations Jacob White Thanks to Deepak Ramaswamy, Jaime Peraire, Michal Rewienski, and Karen Veroy Outline Initial Value problem examples Signal propagation (circuits with capacitors). Space frame dynamics (struts and masses). Chemical reaction dynamics. Investigate the simple finite-difference methods Forward-Euler, Backward-Euler, Trap Rule. Look at the approximations and algorithms Examine properties experimentally. Analyze Convergence for Forward-Euler Application Problems Signal Transmission in an Integrated Circuit Signal Wire Wire has resistance Wire and ground plane form a capacitor Logic Gate Ground Plane • Metal Wires carry signals from gate to gate. • How long is the signal delayed? SMA-HPC ©2003 MIT Logic Gate Application Problems Signal Transmission in an Integrated Circuit Circuit Model resistor capacitor Constructing the Model • Cut the wire into sections. • Model wire resistance with resistors. • Model wire-plane capacitance with capacitors. SMA-HPC ©2003 MIT Application Problems Oscillations in a Space Frame • What is the oscillation amplitude? SMA-HPC ©2003 MIT Application Problems Oscillations in a Space Frame Simplified Structure Bolts Struts Ground Example Simplified for Illustration Load Application Problems Oscillations in a Space Frame Modeling with Struts, Joints and Point Masses Point Mass Strut Constructing the Model • Replace Metal Beams with Struts. • Replace cargo with point mass. Application Problems Chemical Reaction Dynamics Crucible Reagent Strange green stuff • How fast is product produced? • Does it explode? SMA-HPC ©2003 MIT Signal Transmission in an Integrated Circuit Application Problems v1 iC1 C1 iR1 A 2x2 Example v2 iR2 R2 R1 R3 iR3 iC2 C2 Constitutive Equations dv ic = C c dt 1 iR = vR R Conservation Laws iC1 + iR1 + iR2 = 0 iC2 + iR3 − iR2 = 0 Nodal Equations Yields 2x2 System ⎡C1 ⎢0 ⎣ SMA-HPC ©2003 MIT 1 ⎡1 ⎡ dv1 ⎤ + ⎢ R1 R2 0 ⎤ ⎢ dt ⎥ ⎢ ⎥ = −⎢ ⎥ C2 ⎦ ⎢ dv2 ⎥ 1 ⎢ ⎢ −R ⎢⎣ dt ⎥⎦ 2 ⎣ 1 ⎤ − R2 ⎥ ⎡ v1 ⎤ ⎥⎢ ⎥ 1 1 ⎥ ⎣ v2 ⎦ + ⎥ R3 R2 ⎦ Application Problems Signal Transmission in an Integrated Circuit A 2x2 Example Let C1 = C2 = 1, R1 = R3 = 10, R2 = 1 ⎡C1 ⎢0 ⎣ 1 ⎡1 ⎡ dv1 ⎤ + ⎢R R 0 ⎤ ⎢ dt ⎥ 1 2 ⎢ ⎥ = −⎢ ⎥ C2 ⎦ ⎢ dv2 ⎥ 1 ⎢ − ⎢ R ⎢⎣ dt ⎥⎦ 2 ⎣ dx ⎡ −1.1 1.0 ⎤ =⎢ x ⎥ dt ⎣ 1.0 −1.1⎦ A 1 ⎤ R2 ⎥ ⎡ v1 ⎤ ⎥ 1 1 ⎥ ⎢⎣ v2 ⎥⎦ + R3 R2 ⎥⎦ − Eigenvalues and Eigenvectors ⎡1 −1⎤ ⎡ −0.1 0 ⎤ ⎛ ⎡1 −1⎤ ⎞ A=⎢ ⎜⎢ ⎟ ⎥ ⎢ ⎥ ⎥ −2.1⎦ ⎝ ⎣1 1 ⎦ ⎠ ⎣1 1 ⎦ ⎣ 0 eigenvectors SMA-HPC ©2003 MIT Eigenvalues −1 An Aside on Eigenanalysis dx(t ) Consider an ODE: = Ax(t ), dt Eigendecomposition: x(0) = x0 # # ⎤ ⎡ λ1 0 0 ⎤ ⎡ # ⎡# A = ⎢⎢ E1 E2 En ⎥⎥ ⎢⎢ 0 % 0 ⎥⎥ ⎢⎢ E1 ⎢⎣ # # # ⎥⎦ ⎢⎣ 0 0 λn ⎥⎦ ⎢⎣ # E # E2 # Change of variables: Ey (t ) = x (t ) ⇔ y (t ) = E −1 x (t ) dEy (t ) = AEy (t ), Ey (0) = x0 Substituting: dt ⎡λ1 0 0 ⎤ ⎢ ⎥ −1 dy (t ) −1 Multiply by E : = E AEy (t ) = ⎢ 0 % 0 ⎥ y(t ) ⎢0 0 λ ⎥ dt n ⎦⎥ ⎣⎢ SMA-HPC ©2003 MIT # ⎤ En ⎥⎥ # ⎥⎦ −1 An Aside on Eigenanalysis Continued From last slide: ⎡λ1 0 0 ⎤ dy(t ) = ⎢ 0 % 0 ⎥ dt ⎢⎢ 0 0 λ ⎥⎥ n⎦ ⎣ y(t ) Decoupled Equations! dyi (t ) λi t = λi yi (t ) ⇒ yi (t ) = e y (0) Decoupling: dt dx(t ) Steps for solving = Ax(t ), x(0) = x0 dt 1) Determine E , λ λ1t −1 ⎡ e 0 0 ⎤ 2) Compute y (0) = E x0 ⎢ ⎥ 3) Compute y (t ) = ⎢ 0 % 0 ⎥ y (0) ⎢ 0 0 eλnt ⎥ ⎣ ⎦ 4) x (t ) = Ey (t ) SMA-HPC ©2003 MIT Application Problems Signal Transmission in an Integrated Circuit A 2x2 Example v1 (0) = 1 v2 (0) = 0 Notice two time scale behavior • v1 and v2 come together quickly (fast eigenmode). • v1 and v2 decay to zero slowly (slow eigenmode). SMA-HPC ©2003 MIT Application Problems y = y0 + u fs fm Struts, Joints and point mass example A 2x2 Example Constitutive Equations d 2u fm = M 2 dt Conservation Law fs + fm = 0 y − y0 EAc fs = EAc ∗ u = y0 y0 Define v as velocity (du/dt) to yield a 2x2 System ⎡M ⎢0 ⎣ SMA-HPC ©2003 MIT ⎡ dv ⎤ ⎡ EAc ⎤ ⎢ ⎥ 0 − 0 ⎤ dt ⎡v ⎤ ⎢ ⎥ y0 ⎢ ⎥ ⎢ ⎥=⎢ ⎥ ⎥ ⎣u ⎦ 1 ⎦ ⎢ du ⎥ 0 ⎥⎦ ⎢⎣ dt ⎥⎦ ⎢⎣1 Application Problems Struts, Joints and point mass example A 2x2 Example EAc =1 Let M = 1, y0 ⎡M ⎢0 ⎣ ⎡ dv ⎤ ⎡ EAc ⎤ ⎢ ⎥ − 0 0 ⎤ dt ⎡v ⎤ ⎢ ⎥ y = ⎢ ⎥ 0 ⎥ ⎢⎣u ⎥⎦ 1 ⎥⎦ ⎢ du ⎥ ⎢ 0 ⎥⎦ ⎢⎣ dt ⎥⎦ ⎢⎣1 dx ⎡ 0 −1.0 ⎤ =⎢ x ⎥ 0 ⎦ dt ⎣1.0 A Eigenvalues and Eigenvectors ⎡ −1 −1⎤ ⎡ i 0 ⎤ ⎛ ⎡ −1 −1⎤ ⎞ A=⎢ ⎜⎢ ⎟ ⎥ ⎢ ⎥ ⎥ ⎣ i −i ⎦ ⎣0 −i ⎦ ⎝ ⎣ i −i ⎦ ⎠ eigenvectors SMA-HPC ©2003 MIT Eigenvalues −1 Application Problems Struts, Joints and point mass example A 2x2 Example 1 v (0) = 1 0.5 0 u (0) = 0 -0.5 -1 0 5 10 Note the system has imaginary eigenvalues • Persistent Oscillation • Velocity, v, peaks when displacement, u, is zero. SMA-HPC ©2003 MIT 15 Application Problems Chemical Reaction Example A 2x2 Example Amount of reactant = R, the temperature = T dT = −T + R dt dR = − R + 4T dt SMA-HPC ©2003 MIT More reactant causes temperature to rise, higher temperatures increases heat dissipation causing temperature to fall Higher temperatures raises reaction rates, increased reactant interferes with reaction and slows rate. Application Problems Chemical Reaction Example A 2x2 Example ⎡ dT ⎤ ⎢ dt ⎥ ⎡ −1 1 ⎤ ⎡T ⎤ ⎢ ⎥=⎢ ⎥ ⎢ R⎥ − dR 4 1 ⎦⎣ ⎦ ⎢ ⎥ ⎣ ⎢⎣ dt ⎥⎦ dx ⎡ −1 1 ⎤ =⎢ x ⎥ dt ⎣ 4 −1⎦ A Eigenvalues and Eigenvectors ⎡ 1 −1⎤ ⎡1 0 ⎤ ⎛ ⎡ 1 −1⎤ ⎞ A=⎢ ⎥ ⎢0 −3⎥ ⎜ ⎢ 2 2 ⎥ ⎟ 2 2 ⎣ ⎦⎣ ⎦⎝⎣ ⎦⎠ eigenvectors SMA-HPC ©2003 MIT Eigenvalues −1 Chemical Reaction Example Application Problems A 2x2 Example 12 10 8 6 R (0) = 0 4 T (0) = 1 2 0 0 0.5 1 1.5 2 Note the system has a positive eigenvalue • Solutions grow exponentially with time. SMA-HPC ©2003 MIT 2.5 Basic Concepts Finite Difference Methods ∆t ∆t First - Discretize Time 0 t1 ∆t Second - Represent x(t) using values at ti 3 x̂ 2 4 x̂ 1 x̂ x̂ 0 t1 t2 Third - Approximate t3 tL d the ) x(tlusing dt l l −1 d xˆ − xˆ Example: x (tl ) dt ∆tl SMA-HPC ©2003 MIT t L −1 t L = T t2 l xˆ x(tl ) Approx. sol’n discrete xˆ l +1 − xˆ l or ∆tl +1 Exact sol’n xˆ l 's Finite Difference Methods x slope = d x(tl ) dt ∆ slope = tl tl +1 x(tl +1 ) − x(tl ) ∆t Basic Concepts Forward Euler Approximation x(tl +1 ) − x(tl ) d x(tl ) = A x(tl ) ≅ ∆t dt or x(tl +1 ) ≅ x(tl ) + ∆t A x(tl ) t ∆ = x(tl +1 ) − ( x(tl ) + ∆t A x(tl ) ) SMA-HPC ©2003 MIT Finite Difference Methods Basic Concepts Forward Euler Algorithm x(t1 ) xˆ1 = x(0) + ∆tAx ( 0 ) x x(t2 ) xˆ 2 = xˆ1 + ∆tAxˆ1 # x(t L ) xˆ L = xˆ L −1 + ∆tAxˆ L −1 SMA-HPC ©2003 MIT ∆tAxˆ1 ∆tAx (0) t1 t2 t3 t Basic Concepts Finite Difference Methods x slope = ∆ d x(tl +1 ) dt slope = tl tl +1 x(tl +1 ) − x(tl ) ∆t Backward Euler Approximation x(tl +1 ) − x(tl ) d x(tl +1 ) = A x(tl +1 ) ≅ ∆t dt or x(tl +1 ) ≅ x(tl ) + ∆t A x(tl +1 ) t ∆ = x(tl +1 ) − ( x(tl ) + ∆t A x(tl +1 ) ) SMA-HPC ©2003 MIT Finite Difference Methods Basic Concepts Backward Euler Algorithm Solve with Gaussian Elimination x x(t1 ) xˆ1 = x(0) + ∆tAxˆ1 ⇒ [ I − ∆tA] xˆ1 = x(0) x(t2 ) xˆ 2 = [ I − ∆tA] −1 xˆ1 # x(t L ) xˆ L = [ I − ∆tA] −1 xˆ L −1 SMA-HPC ©2003 MIT ∆tAxˆ 2 ∆tAxˆ1 t1 t2 t Finite Difference Methods 1 d d ( x(tl +1 ) + x(tl )) 2 dt dt 1 = ( Ax(tl +1 ) + Ax(tl )) 2 x(tl +1 ) − x(tl ) ∆t 1 x(tl +1 ) x(tl ) + ∆tA( x(tl +1 ) + x(tl )) 2 Basic Concepts Trapezoidal Rule slope = x d x(tl ) d dt x(tl +1 ) slope = dt ∆ slope = x (tl +1 ) − x (tl ) ∆t t 1 1 ∆ = ( x(tl +1 ) − ∆tAx(tl )) − ( x(tl ) + ∆tAx(tl +1 )) 2 2 SMA-HPC ©2003 MIT Basic Concepts Finite Difference Methods Trapezoidal Rule Algorithm Solve with Gaussian Elimination ∆t x (t1 ) xˆ = x (0) + ( Ax (0) + Axˆ1 ) 2 ∆t ⎤ 1 ⎡ ∆t ⎤ ⎡ ⇒ ⎢I − A⎥ xˆ = ⎢ I + A⎥ x (0) 2 ⎦ 2 ⎦ ⎣ ⎣ 1 ∆t ⎡ 2 x (t2 ) xˆ = ⎢ I − 2 ⎣ # ⎡ x (tL ) xˆ = ⎢ I ⎣ L SMA-HPC ©2003 MIT ∆t − 2 −1 ∆t ⎤ ⎡ A⎥ ⎢ I + 2 ⎦ ⎣ −1 ∆t ⎤ ⎡ A⎥ ⎢ I + 2 ⎦ ⎣ ⎤ A⎥ xˆ1 ⎦ ⎤ A⎥ xˆ L−1 ⎦ x t1 t2 t Finite Difference Methods Basic Concepts Numerical Integration View tl +1 d x (t ) = A x (t ) ⇒ x (tl +1 ) = x (tl ) + ∫ Ax (τ )dτ tl dt tl +1 ∆t ∫tl Ax(τ )dτ ≈ 2 ( Ax(tl ) + Ax(tl ) ) Trap ∆tAx (tl +1 ) BE ∆tAx (tl ) SMA-HPC ©2003 MIT tl tl +1 FE Finite Difference Methods Basic Concepts Summary Trap Rule, Forward-Euler, Backward-Euler Are all one-step methods xˆ l is computed using only xˆ l −1 , not xˆ l − 2 , xˆ l − 3 , etc. Forward-Euler is simplest No equation solution explicit method. Boxcar approximation to integral Backward-Euler is more expensive Equation solution each step implicit method Trapezoidal Rule might be more accurate Equation solution each step implicit method Trapezoidal approximation to integral SMA-HPC ©2003 MIT Finite Difference Methods Numerical Experiments Unstable Reaction 4 ∆t = 0.1 3.5 3 Exact Solution Backward-Euler 2.5 2 Trap rule 1.5 Forward-Euler 1 0.5 0 0.5 1 1.5 2 FE and BE results have larger errors than Trap Rule, and the errors grow with time. SMA-HPC ©2003 MIT Numerical Experiments Finite Difference Methods Unstable Reaction-Error Plots 0 0.4 6 -0.05 0.35 5 Backward Euler 0.3 -0.1 x 10 Trap Rule 4 0.25 3 0.2 2 0.15 1 0.1 0 0.05 -1 -3 -0.15 Forward Euler -0.2 -0.25 -0.3 -0.35 0 1 2 0 0 1 2 -2 0 1 All methods have errors which grow exponentially SMA-HPC ©2003 MIT 2 Numerical Experiments Finite Difference Methods 10 M a x E r r o r 10 10 10 10 10 Unstable Reaction-Convergence 2 Backward-Euler 0 -2 -4 Trap rule Forward-Euler -6 -8 10 -3 10 -2 Timestep 10 -1 10 0 For FE and BE, Error ∝ ∆t For Trap, Error ∝ ( ∆t ) SMA-HPC ©2003 MIT 2 Finite Difference Methods 4 2 Numerical Experiments Oscillating Strut and Mass Forward-Euler ∆t = 0.1 0 -2 -4 -6 Trap rule 0 5 10 Backward-Euler 15 20 25 30 Why does FE result grow, BE result decay and the Trap rule preserve oscillations SMA-HPC ©2003 MIT Finite Difference Methods Numerical Experiments Two timescale RC Circuit small ∆t Backward-Euler Computed Solution large ∆t With Backward-Euler it is easy to use small timesteps for the fast dynamics and then switch to large timesteps for the slow decay SMA-HPC ©2003 MIT Finite Difference Methods Numerical Experiments Two timescale RC Circuit Forward-Euler Computed Solution The Forward-Euler is accurate for small timesteps, but goes unstable when the timestep is enlarged SMA-HPC ©2003 MIT Finite Difference Methods • Convergence Numerical Experiments Summary – Did the computed solution approach the exact solution? – Why did the trap rule approach faster than BE or FE? • Energy Preservation – Why did BE produce a decaying oscillation? – Why did FE produce a growing oscillation? – Why did trap rule maintain oscillation amplitude? • Two timeconstant (stiff) problems – Why did FE go unstable when the timestep increased? We will focus on convergence today SMA-HPC ©2003 MIT Finite Difference Methods Convergence Analysis Convergence Definition Definition: A finite-difference method for solving initial value problems on [0,T] is said to be convergent if given any A and any initial condition max xˆ − x ( l ∆t ) → 0 as ∆t → 0 l ⎡ T ⎤ l∈⎢0, ⎥ ⎣ ∆t ⎦ xˆ l computed with ∆t ∆t l xˆ computed with 2 xexact SMA-HPC ©2003 MIT Convergence Analysis Finite Difference Methods Order-p convergence Definition: A finite-difference method for solving initial value problems on [0,T] is said to be order p convergent if given any A and any initial condition max xˆ − x ( l ∆t ) ≤ C ( ∆t ) l ⎡ T⎤ l∈⎢0, ⎥ ⎣ ∆t ⎦ p for all ∆t less than a given ∆t0 Forward- and Backward-Euler are order 1 convergent Trapezoidal Rule is order 2 convergent SMA-HPC ©2003 MIT Finite Difference Methods Convergence Analysis Two Conditions for Convergence 1) Local Condition: One step errors are small (consistency) Typically verified using Taylor Series 2) Global Condition: The single step errors do not grow too quickly (stability) All one-step methods are stable in this sense. SMA-HPC ©2003 MIT Finite Difference Methods Convergence Analysis Consistency Definition Definition: A one-step method for solving initial value problems on an interval [0,T] is said to be consistent if for any A and any initial condition xˆ − x ( ∆t ) 1 ∆t SMA-HPC ©2003 MIT → 0 as ∆t → 0 Convergence Analysis Finite Difference Methods Consistency for Forward Euler Forward-Euler definition 1 xˆ = x ( 0 ) + ∆tAx ( 0 ) Expanding in t about zero yields τ ∈ [0 , ∆ t ] dx ( 0 ) ( ∆t ) d 2 x (τ ) x(∆t ) = x ( 0 ) + ∆t + dt 2 dt 2 2 d Noting that x(0) = Ax(0) and subtracting dt 2 2 Proves the theorem if ( ∆t ) d x (τ ) derivatives of x are 1 xˆ − x ( ∆t ) ≤ SMA-HPC ©2003 MIT 2 dt 2 bounded Finite Difference Methods Convergence Analysis Convergence Analysis for Forward Euler Forward-Euler definition l +1 l l xˆ = xˆ + ∆tAxˆ Expanding in t about l ∆t yields x ( ( l + 1) ∆t ) = x ( l ∆t ) + ∆tAx ( l ∆t ) + el l where e is the "one-step" error bounded by e ≤ C ( ∆t ) , where C = 0.5 maxτ ∈[0,T ] l SMA-HPC ©2003 MIT 2 d x (τ ) 2 dt 2 Finite Difference Methods Convergence Analysis Convergence Analysis for Forward Euler Continued Subtracting the previous slide equations xˆ l +1 − x ( ( l + 1) ∆t ) = ( I + ∆tA ) ( xˆ − x ( l ∆t ) ) + e l l Define the "Global" error E ≡ x − xˆ ( l ∆t ) l E l +1 l = ( I + ∆tA ) E + e l l Taking norms and using the bound on e E l +1 ≤ ( I + ∆tA ) E + C ( ∆t ) l ≤ (1 + ∆t A SMA-HPC ©2003 MIT ) 2 E + C ( ∆t ) l 2 l Convergence Analysis Finite Difference Methods A helpful bound on difference equations A lemma bounding difference equation solutions If u l +1 Then ≤ (1 + ε ) u + b, u = 0, ε > 0 εl e l u ≤ b ε l 0 l To prove, first write u as a power series and sum l −1 u ≤ ∑ (1 + ε ) l j =0 SMA-HPC ©2003 MIT 1 − (1 + ε ) b= b 1 − (1 + ε ) j j Convergence Analysis Finite Difference Methods A helpful bound on difference equations cont. To finish, note (1 + ε ) ≤ eε ⇒ (1 + ε )l ≤ eε l εl 1 − 1 + ε 1 + ε − 1 ( ) ( ) e l u ≤ b= b≤ b 1 − (1 + ε ) ε ε j j Mapping the global error equation to the lemma E l +1 SMA-HPC ©2003 MIT ⎛ ⎞ 2 l ⎜ ⎟ ≤ 1 + ∆t A E + C ( ∆ t ) ⎜⎜ N ⎟⎟ ε ⎠ ⎝ b Convergence Analysis Finite Difference Methods Back to Forward Euler Convergence analysis. Applying the lemma and cancelling terms ⎛ ⎞ l ∆t A 2 e l l −1 2 ⎜ ⎟ E ≤ 1 + ∆t A E + C ( ∆t ) ≤ C t ∆ ( ) ⎜⎜ N ⎟⎟ ∆t A ε ⎠ ⎝ b Finally noting that l ∆t ≤ T , max l∈[0, L] E ≤ e l SMA-HPC ©2003 MIT AT C ∆t A Convergence Analysis Finite Difference Methods Observations about the forward-Euler analysis. max l∈[0, L] E ≤ e l AT C ∆t A • forward-Euler is order 1 convergent • The bound grows exponentially with time interval • C is related to the solution second derivative • The bound grows exponentially fast with norm(A). SMA-HPC ©2003 MIT Convergence Analysis Finite Difference Methods Exact and forward-Euler(FE) Plots for Unstable Reaction. 12 10 RFE 8 Rexact 6 Texact 4 TFE 2 0 0 0.5 1 1.5 2 Forward-Euler Errors appear to grow with time SMA-HPC ©2003 MIT 2.5 Convergence Analysis Finite Difference Methods 1.2 forward-Euler errors for solving reaction equation. 1 E 0.8 r r 0.6 o0.4 r Rexact-RFE 0.2 Texact - TFE 0 -0.2 0 0.5 1 Time 1.5 2 2.5 Note error grows exponentially with time, as bound predicts SMA-HPC ©2003 MIT Convergence Analysis Finite Difference Methods Exact and forward-Euler(FE) Plots for Circuit. 1 v1exact 0.8 v1FE 0.6 0.4 v2FE 0.2 0 0 v2exact 0.5 1 1.5 2 2.5 3 3.5 4 Forward-Euler Errors don’t always grow with time SMA-HPC ©2003 MIT Convergence Analysis Finite Difference Methods forward-Euler errors for solving circuit equation. 0.03 v1exact - v1FE E 0.02 r r 0.01 o 0 r -0.01 v2exact-v2FE -0.02 -0.03 0 0.5 1 1.5 Time2 2.5 3 3.5 4 Error does not always grow exponentially with time! Bound is conservative SMA-HPC ©2003 MIT Summary Initial Value problem examples Signal propagation (two time scales). Space frame dynamics (oscillator). Chemical reaction dynamics (unstable system). Looked at the simple finite-difference methods Forward-Euler, Backward-Euler, Trap Rule. Look at the approximations and algorithms Experiments generated many questions Analyzed Convergence for Forward-Euler Many more questions to answer, some next time