Change-Point Methods for Multiple Structural Breaks and Regime Switching Models Birte Muhsal, joint work with Claudia Kirch | March 26th, 2012 0 2 4 6 8 WORKSHOP ON RECENT ADVANCES IN CHANGEPOINT ANALYSIS AT THE UNIVERSITY OF WARWICK 100 200 300 400 500 0 100 200 300 400 500 0 4 8 12 0 KIT – University of the State of Baden-Wuerttemberg and National Laboratory of the Helmholtz Association www.kit.edu 0 2 2 4 4 6 6 8 8 Multiple Change-Point Problem 0 100 200 300 400 500 0 100 200 300 400 500 We want to construct: an asymptotic test with level α H0 : no change H1 : at least one structural change a consistent estimator of the number of change-points consistent estimators of the location of change-points. Introduction Multiple Change-Point Location Model Test and Estimation Procedure Birte Muhsal – Change-Point Methods for Multiple Structural Breaks and RSM Simulation Study Conclusion and References 2/21 0 2 2 4 4 6 6 8 8 Multiple Change-Point Problem 0 100 200 300 400 500 0 100 200 300 400 500 We want to construct: an asymptotic test with level α H0 : no change H1 : at least one structural change a consistent estimator of the number of change-points consistent estimators of the location of change-points. Introduction Multiple Change-Point Location Model Test and Estimation Procedure Birte Muhsal – Change-Point Methods for Multiple Structural Breaks and RSM Simulation Study Conclusion and References 2/21 0 2 2 4 4 6 6 8 8 Multiple Change-Point Problem 0 100 200 300 400 500 0 100 200 300 400 500 We want to construct: an asymptotic test with level α H0 : no change H1 : at least one structural change a consistent estimator of the number of change-points consistent estimators of the location of change-points. Introduction Multiple Change-Point Location Model Test and Estimation Procedure Birte Muhsal – Change-Point Methods for Multiple Structural Breaks and RSM Simulation Study Conclusion and References 2/21 Outline 1 Introduction 2 Multiple Change-Point Location Model Classical Multiple Change Point Model Regime Switching Model 3 Test and Estimation Procedure 4 Simulation Study 5 Conclusion and References Introduction Multiple Change-Point Location Model Test and Estimation Procedure Birte Muhsal – Change-Point Methods for Multiple Structural Breaks and RSM Simulation Study Conclusion and References 3/21 2 4 6 8 10 Classical Multiple Change-Point Model 0 100 Xi = q +1 X 200 300 dj I {kj −1 < i ≤ kj } + εi , 400 500 i = 1, ..., n, j =1 with random errors ε1 , ..., εn and unknown change points k1 , . . . , kq with 0 = k0 < k1 ≤ . . . ≤ kq ≤ kq +1 = n, kj = [ϑj n], j = 1, . . . , q, and 0 < ϑ1 ≤ . . . ϑq ≤ 1 number of changes q ∈ N expectations d1 , . . . , dq +1 with di 6= di +1 for i = 1, . . . , q. Introduction Multiple Change-Point Location Model Test and Estimation Procedure Birte Muhsal – Change-Point Methods for Multiple Structural Breaks and RSM Simulation Study Conclusion and References 4/21 Regime Switching Model (n) Xi (n) = dQ (n) + εi , i = 1, ..., n, (n ) i (n) with random errors ε1 , ..., εn , expectations d1 , ..., dK ∈ R with di 6= dj , for i , j = 1, . . . , K a non-observable {1, ..., K }-valued stationary process (n) {Qi : i ∈ N}. (n ) Key feature of {Qi : i ∈ N}: long duration times. Differences to the classical change-point model: both number qn and locations k1 , ..., kqn of structural breaks are random the unbounded number of changes qn . Introduction Multiple Change-Point Location Model Test and Estimation Procedure Birte Muhsal – Change-Point Methods for Multiple Structural Breaks and RSM Simulation Study Conclusion and References 5/21 Regime Switching Model (n) Xi (n) = dQ (n) + εi , i = 1, ..., n, (n ) i (n) with random errors ε1 , ..., εn , expectations d1 , ..., dK ∈ R with di 6= dj , for i , j = 1, . . . , K a non-observable {1, ..., K }-valued stationary process (n) {Qi : i ∈ N}. (n ) Key feature of {Qi : i ∈ N}: long duration times. Differences to the classical change-point model: both number qn and locations k1 , ..., kqn of structural breaks are random the unbounded number of changes qn . Introduction Multiple Change-Point Location Model Test and Estimation Procedure Birte Muhsal – Change-Point Methods for Multiple Structural Breaks and RSM Simulation Study Conclusion and References 5/21 Assumptions on errors Let the errors ε1 , ..., εn be a strictly stationary sequence with (1) E ε1 = 0 , 2 2 0 < σ = E ε1 < ∞, X E |ε1 | 2+ν < ∞ for some ν > 0, |γ(h)| < ∞, where γ(h) = cov(ε0 , εh ), h≥0 2 2 and long run variance τ = σ + 2 X γ(h) < ∞. h>0 (2) Invariance principle: It exists a Wiener process {W (k ), 1 ≤ k ≤ n} such that max G≤k ≤n−G k +G X 1 1 √ εi − (W (k + G) − W (k )) = op (log(n/G))− 2 . 2G τ i =k +1 1 (3) Hájek-Rényi-type moment condition: j X γ E εk ≤ C |j − i + 1|ϕ for some γ ≥ 1, ϕ > 1 and some constant C > 0. k =i Introduction Multiple Change-Point Location Model Test and Estimation Procedure Birte Muhsal – Change-Point Methods for Multiple Structural Breaks and RSM Simulation Study Conclusion and References 6/21 2 4 6 8 MOSUM (Moving Sum) Statistic MOSUM statistic (Hušková and Slabý (2001)) Tn (G) = max Tk ,n (G) G ≤k ≤n −G k X 1 Tk ,n (G) = √ 2G τ Xi − i =k −G+1 with kX +G i =k +1 Xi , where G = G(n) is the bandwidth fulfilling 2 n 2+ν log n G Introduction → 0, n G →∞. Multiple Change-Point Location Model Test and Estimation Procedure Birte Muhsal – Change-Point Methods for Multiple Structural Breaks and RSM Simulation Study Conclusion and References 7/21 2 4 6 8 MOSUM (Moving Sum) Statistic k=G MOSUM statistic (Hušková and Slabý (2001)) Tn (G) = max Tk ,n (G) G ≤k ≤n −G k X 1 Tk ,n (G) = √ 2G τ Xi − i =k −G+1 with kX +G i =k +1 Xi , where G = G(n) is the bandwidth fulfilling 2 n 2+ν log n G Introduction → 0, n G →∞. Multiple Change-Point Location Model Test and Estimation Procedure Birte Muhsal – Change-Point Methods for Multiple Structural Breaks and RSM Simulation Study Conclusion and References 7/21 2 4 6 8 MOSUM (Moving Sum) Statistic k−G+1 k=G k+G MOSUM statistic (Hušková and Slabý (2001)) Tn (G) = max Tk ,n (G) G ≤k ≤n −G k X 1 Tk ,n (G) = √ 2G τ Xi − i =k −G+1 with kX +G i =k +1 Xi , where G = G(n) is the bandwidth fulfilling 2 n 2+ν log n G Introduction → 0, n G →∞. Multiple Change-Point Location Model Test and Estimation Procedure Birte Muhsal – Change-Point Methods for Multiple Structural Breaks and RSM Simulation Study Conclusion and References 7/21 2 4 6 8 MOSUM (Moving Sum) Statistic k−G+1 k−G+1 k=G k=G+1 k+G k+G MOSUM statistic (Hušková and Slabý (2001)) Tn (G) = max Tk ,n (G) G ≤k ≤n −G k X 1 Tk ,n (G) = √ 2G τ Xi − i =k −G+1 with kX +G i =k +1 Xi , where G = G(n) is the bandwidth fulfilling 2 n 2+ν log n G Introduction → 0, n G →∞. Multiple Change-Point Location Model Test and Estimation Procedure Birte Muhsal – Change-Point Methods for Multiple Structural Breaks and RSM Simulation Study Conclusion and References 7/21 2 4 6 8 MOSUM (Moving Sum) Statistic k−G+1 k−G+1 k=G k=G+1 k+G k+G - k−G+1 k=250 k+G MOSUM statistic (Hušková and Slabý (2001)) Tn (G) = max Tk ,n (G) G ≤k ≤n −G k X 1 Tk ,n (G) = √ 2G τ Xi − i =k −G+1 with kX +G i =k +1 Xi , where G = G(n) is the bandwidth fulfilling 2 n 2+ν log n G Introduction → 0, n G →∞. Multiple Change-Point Location Model Test and Estimation Procedure Birte Muhsal – Change-Point Methods for Multiple Structural Breaks and RSM Simulation Study Conclusion and References 7/21 2 4 6 8 MOSUM (Moving Sum) Statistic k−G+1 k−G+1 k=G k=G+1 k+G k+G - k−G+1 k=250 - k+G k−G+1 k=n−G k+G MOSUM statistic (Hušková and Slabý (2001)) Tn (G) = max Tk ,n (G) G ≤k ≤n −G k X 1 Tk ,n (G) = √ 2G τ Xi − i =k −G+1 with kX +G i =k +1 Xi , where G = G(n) is the bandwidth fulfilling 2 n 2+ν log n G Introduction → 0, n G →∞. Multiple Change-Point Location Model Test and Estimation Procedure Birte Muhsal – Change-Point Methods for Multiple Structural Breaks and RSM Simulation Study Conclusion and References 7/21 Asymptotic Distribution Theorem (Hušková and Slabý (2001), Kirch and M. (2012)) Let the assumptions on errors (1)-(2) and bandwidth G hold. Then, under H0 , α(n/G) T k ,n ( G ) max G≤k ≤n−G τ D − β(n/G) − → Γ, where Γ has a Gumbel extreme value distribution, i.e. P (Γ ≤ x ) = exp(−2exp(−x )), and α(x ) = √ 2 log x , β(x ) = 2 log(x ) + critical value: Dn (G; α) = 1 2 log log x − 1 2 log π. β(n/G) − log log √11−α α(n/G) αn → 0, but not too fast. Introduction Multiple Change-Point Location Model Test and Estimation Procedure Birte Muhsal – Change-Point Methods for Multiple Structural Breaks and RSM Simulation Study Conclusion and References 8/21 Asymptotic Distribution Theorem (Hušková and Slabý (2001), Kirch and M. (2012)) Let the assumptions on errors (1)-(2) and bandwidth G hold. Then, under H0 , α(n/G) T k ,n ( G ) max G≤k ≤n−G τ D − β(n/G) − → Γ, where Γ has a Gumbel extreme value distribution, i.e. P (Γ ≤ x ) = exp(−2exp(−x )), and α(x ) = √ 2 log x , β(x ) = 2 log(x ) + critical value: Dn (G; α) = 1 2 log log x − 1 2 log π. β(n/G) − log log √11−α α(n/G) αn → 0, but not too fast. Introduction Multiple Change-Point Location Model Test and Estimation Procedure Birte Muhsal – Change-Point Methods for Multiple Structural Breaks and RSM Simulation Study Conclusion and References 8/21 Assumptions on errors Let the errors ε1 , ..., εn be a strictly stationary sequence with (1) E ε1 = 0 , 2 2 0 < σ = E ε1 < ∞, X E |ε1 | 2+ν < ∞ for some ν > 0, |γ(h)| < ∞, where γ(h) = cov(ε0 , εh ), h≥0 2 2 and long run variance τ = σ + 2 X γ(h) < ∞. h>0 (2) Invariance principle: It exists a Wiener process {W (k ), 1 ≤ k ≤ n} such that max G≤k ≤n−G k +G X 1 1 √ εi − (W (k + G) − W (k )) = op (log(n/G))− 2 . 2G τ i =k +1 1 (3) Hájek-Rényi-type moment condition: j X γ E εk ≤ C |j − i + 1|ϕ for some γ ≥ 1, ϕ > 1 and some constant C > 0. k =i Introduction Multiple Change-Point Location Model Test and Estimation Procedure Birte Muhsal – Change-Point Methods for Multiple Structural Breaks and RSM Simulation Study Conclusion and References 9/21 2 4 6 8 10 12 Change-Point Estimators (Antoch et al. (2000)) 100 200 300 400 500 0 100 200 300 400 500 0 5 10 0 Test statistic: Tn (G) := max G≤k ≤n−G Tk ,n (G)/τ — critical value Dn (G; αn ) Introduction Multiple Change-Point Location Model Test and Estimation Procedure Birte Muhsal – Change-Point Methods for Multiple Structural Breaks and RSM Simulation Study Conclusion and References 10/21 2 4 6 8 10 12 Change-Point Estimators (Antoch et al. (2000)) 100 200 300 400 500 0 100 200 300 400 500 0 5 10 0 Test statistic: Tn (G) := max G≤k ≤n−G Tk ,n (G)/τ — critical value Dn (G; αn ) Introduction Multiple Change-Point Location Model Test and Estimation Procedure Birte Muhsal – Change-Point Methods for Multiple Structural Breaks and RSM Simulation Study Conclusion and References 10/21 2 4 6 8 10 12 Change-Point Estimators (Antoch et al. (2000)) 100 200 300 400 500 0 100 200 300 400 500 0 5 10 0 All pairs of indices vj , wj are chosen such that Tk ,n (G)/τ ≥ Dn (G; αn ) k = vj , ..., wj , Tk ,n (G)/τ < Dn (G; αn ) k = vj − 1, wj + 1 wj − vj > εG . The number of change-points q can be estimated by q̂n , the number of pairs (vj , wj ). The estimator of change-point kj is defined as k̂j := arg max Tk ,n (G)/τ . vj ≤k ≤wj Introduction Multiple Change-Point Location Model Test and Estimation Procedure Birte Muhsal – Change-Point Methods for Multiple Structural Breaks and RSM Simulation Study Conclusion and References 10/21 Consistency of q̂n Classical model: Theorem (Kirch and M. (2012)) Let the assumptions on errors (1)-(2), bandwidth G and level {αn } hold. Furthermore assume lim sup d0 (n)/G = C > 2 with n→∞ d0 (n) := min |kj +1 − kj |. 0≤j ≤qn Then, under H1 , P (q̂n = q ) −→ 1 as n → ∞. Introduction Multiple Change-Point Location Model Test and Estimation Procedure Birte Muhsal – Change-Point Methods for Multiple Structural Breaks and RSM Simulation Study Conclusion and References 11/21 Consistency of q̂n Regime switching model: Theorem (Kirch and M. (2012)) Let the assumptions on errors (1)-(2), bandwidth G and level {αn } hold. Furthermore assume lim sup d0 (n)/G = C > 2 with n→∞ d0 (n) := min |kj +1 − kj |. 0≤j ≤qn Then, under H1 , P (q̂n = q ) −→ 1 as n → ∞. Introduction Multiple Change-Point Location Model Test and Estimation Procedure Birte Muhsal – Change-Point Methods for Multiple Structural Breaks and RSM Simulation Study Conclusion and References 11/21 Consistency of q̂n Regime switching model: Theorem (Kirch and M. (2012)) Let the assumptions on errors (1)-(2), bandwidth G and level {αn } hold. Furthermore assume lim P (d0 (n) > 2G) = 1 n→∞ with d0 (n) := min |kj +1 − kj |. 0≤j ≤qn Then, under H1 , P (q̂n = q n ) −→ 1 as n → ∞. Introduction Multiple Change-Point Location Model Test and Estimation Procedure Birte Muhsal – Change-Point Methods for Multiple Structural Breaks and RSM Simulation Study Conclusion and References 11/21 Assumptions on errors Let the errors ε1 , ..., εn be a strictly stationary sequence with (1) E ε1 = 0 , 2 2 0 < σ = E ε1 < ∞, X E |ε1 | 2+ν < ∞ for some ν > 0, |γ(h)| < ∞, where γ(h) = cov(ε0 , εh ), h≥0 2 2 and long run variance τ = σ + 2 X γ(h) < ∞. h>0 (2) Invariance principle: It exists a Wiener process {W (k ), 1 ≤ k ≤ n} such that max G≤k ≤n−G k +G X 1 1 √ εi − (W (k + G) − W (k )) = op (log(n/G))− 2 . 2G τ i =k +1 1 (3) Hájek-Rényi-type moment condition: j X γ E εk ≤ C |j − i + 1|ϕ for some γ ≥ 1, ϕ > 1 and some constant C > 0. k =i Introduction Multiple Change-Point Location Model Test and Estimation Procedure Birte Muhsal – Change-Point Methods for Multiple Structural Breaks and RSM Simulation Study Conclusion and References 12/21 Consistency of Change-Point Estimators Classical model: Theorem (Kirch and M. (2012)) Let the assumptions on errors (1)-(3), bandwidth G and level {αn } hold. Furthermore assume lim sup d0 (n)/G = C > 2 with n→∞ d0 (n) := min |kj +1 − kj |. 0≤j ≤qn γ and let P (qn > γn ) → 0, where {γn } satisfies γn log(G)G− 2 → 0. With q n := min(qn , q̂n ) we have, under H1 , 2 max |k̂j − kj | = Op (1) .Op γnγ 1≤j ≤q n Introduction Multiple Change-Point Location Model Test and Estimation Procedure Birte Muhsal – Change-Point Methods for Multiple Structural Breaks and RSM Simulation Study Conclusion and References 13/21 Consistency of Change-Point Estimators Regime switching model: Theorem (Kirch and M. (2012)) Let the assumptions on errors (1)-(3), bandwidth G and level {αn } hold. Furthermore assume lim sup d0 (n)/G = C > 2 with n→∞ d0 (n) := min |kj +1 − kj |. 0≤j ≤qn γ and let P (qn > γn ) → 0, where {γn } satisfies γn log(G)G− 2 → 0. With q n := min(qn , q̂n ) we have, under H1 , 2 max |k̂j − kj | = Op (1) .Op γnγ 1≤j ≤q n Introduction Multiple Change-Point Location Model Test and Estimation Procedure Birte Muhsal – Change-Point Methods for Multiple Structural Breaks and RSM Simulation Study Conclusion and References 13/21 Consistency of Change-Point Estimators Regime switching model: Theorem (Kirch and M. (2012)) Let the assumptions on errors (1)-(3), bandwidth G and level {αn } hold. Furthermore assume lim P (d0 (n) > 2G) = 1 n→∞ with d0 (n) := min |kj +1 − kj |. 0≤j ≤qn γ and let P (qn > γn ) → 0, where {γn } satisfies γn log(G)G− 2 → 0. With q n := min(qn , q̂n ) we have, under H1 , 2 max |k̂j − kj | = Op γnγ . 1≤j ≤q n Introduction Multiple Change-Point Location Model Test and Estimation Procedure Birte Muhsal – Change-Point Methods for Multiple Structural Breaks and RSM Simulation Study Conclusion and References 13/21 2 4 6 8 10 12 Variance Estimators (i.i.d. case) 10 0 100 200 300 400 500 0 5 Under H1 the standard variance estimator overestimates the variance. 0 100 200 σ̂n2 300 400 n 1X = (Xi − X n )2 n 500 i =1 Solution: Variance estimator depends on time point k . σ̂k2,n Introduction 1 := 2G k X (Xi − X k −G+1,k )2 + i =k −G +1 Multiple Change-Point Location Model kX +G ! (Xi − X k +1,k +G )2 . i =k +1 Test and Estimation Procedure Birte Muhsal – Change-Point Methods for Multiple Structural Breaks and RSM Simulation Study Conclusion and References 14/21 0 2 4 6 8 Performance of σ̂k2,n 100 200 300 400 500 100 200 300 400 500 100 200 300 400 500 3 4 0 0 1 2 G=50 3 4 0 0 1 2 G=75 0 — σ2 - - - σ̂n2 = n 1X (Xi − X n )2 n i =1 — Introduction σ̂k2,n 1 = 2G k X 2 (Xi − X k −G+1,k ) + i =k −G+1 Multiple Change-Point Location Model kX +G ! 2 (Xi − X k +1,k +G ) i =k +1 Test and Estimation Procedure Birte Muhsal – Change-Point Methods for Multiple Structural Breaks and RSM Simulation Study Conclusion and References 15/21 Asymptotic Results with τ̂k ,n Lemma (Kirch and M. (2012)) If the long run variance estimator τ̂k2,n fulfills 1 max |τ̂k ,n − τ | = op (log(n/G))− 2 under H0 G ≤k ≤n −G and max G ≤k ≤n −G τ̂k ,n = Op (1) under H1 all of the above results remain true. Example: σ̂k2,n in the case of i.i.d. random variables. Introduction Multiple Change-Point Location Model Test and Estimation Procedure Birte Muhsal – Change-Point Methods for Multiple Structural Breaks and RSM Simulation Study Conclusion and References 16/21 Simulation Study Questions: How good is the general performance of the MOSUM procedure? How does the choice of the variance estimator influence the performance of the MOSUM procedure? How does the bandwidth selection influence the performance? Introduction Multiple Change-Point Location Model Test and Estimation Procedure Birte Muhsal – Change-Point Methods for Multiple Structural Breaks and RSM Simulation Study Conclusion and References 17/21 Simulation Study ε1 , . . . , εn i.i.d. with ε1 ∼ N (0, 1) 300 400 500 10 6 0 100 200 300 400 500 4 1 400 500 100 200 300 400 500 100 200 300 400 500 100 200 300 400 500 0 100 200 300 400 500 100 200 300 400 500 100 200 300 400 500 100 200 300 400 500 4 1 200 300 400 500 0 100 200 300 400 500 100 200 300 400 500 100 200 300 400 500 4 4 8 8 0 G= 50 G= 50 0 0 Introduction G= 75 0 Multiple Change-Point Location Model 0 0 0 5 G= 75 5 4 10 0 0 0 −1 0 100 0 0 −1 4 G= 50 0 G= 35 G= 35 0 500 4 3 G= 35 400 8 300 8 200 300 0 0 −1 100 200 G= 25 G= 25 0 100 4 3 G= 25 0 8 200 8 100 2 6 2 6 2 0 2 σ2 10 σk2,n 10 σn2 Test and Estimation Procedure Birte Muhsal – Change-Point Methods for Multiple Structural Breaks and RSM G= 75 0 Simulation Study Conclusion and References 18/21 Simulation Study ε1 , . . . , εn i.i.d. with ε1 ∼ N (0, 1) 100 200 300 400 500 4 0 100 200 300 400 500 0 100 200 300 400 500 100 200 300 400 500 100 200 300 400 500 100 200 300 400 500 100 200 300 400 500 4 3 8 0 0 4 0 0 4 8 σ2 8 σk2,n 8 σn2 1 4 G= 25 G= 25 −1 0 0 G= 25 100 200 300 400 500 0 100 200 300 400 500 0 1 4 G= 35 4 3 8 8 0 G= 35 100 200 300 400 500 100 200 300 400 500 100 200 300 400 500 0 100 200 300 400 500 100 200 300 400 500 100 200 300 400 500 G= 50 G= 50 0 0 Introduction 5 G= 75 0 Multiple Change-Point Location Model 0 0 0 5 G= 75 0 10 4 10 0 0 −1 0 1 4 G= 50 0 4 3 8 0 8 −1 0 0 G= 35 Test and Estimation Procedure Birte Muhsal – Change-Point Methods for Multiple Structural Breaks and RSM G= 75 0 Simulation Study Conclusion and References 19/21 Conclusion We have theoretically justified the use of the MOSUM procedure for both the classical model as well as the regime switching model by analysing the consistency of the change point estimators. In the simulation study the procedure gives good estimates for the change points (as long as the bandwidth G is appropriate) and is additionally easy to implement. Future research: MOSUM procedure for change detection in more general models, i.e. autoregressive or ARMA models. Introduction Multiple Change-Point Location Model Test and Estimation Procedure Birte Muhsal – Change-Point Methods for Multiple Structural Breaks and RSM Simulation Study Conclusion and References 20/21 References Antoch, J. and Hušková, M. and Jarušková, D. (2000). Change point detection. 5th ERS IASC Summer School. Hušková, M. and Slabý, A. (2001). Permutation tests for multiple changes. Kybernetica, 37, 605 – 622. Kirch, C. and Muhsal, B. (2012). Change-point methods for multiple structural breaks and regime switching models. In Preparation. Thank you for your attention! Introduction Multiple Change-Point Location Model Test and Estimation Procedure Birte Muhsal – Change-Point Methods for Multiple Structural Breaks and RSM Simulation Study Conclusion and References 21/21