Jou rna l of System s Science and System s Eng ineering
2002, V o l. 11, N o. 3, pp. 3062312
Sta te Pred iction of Chaotic System Ba sed on ANN M odel
YU E Y i2hong, HAN W en 2x iu
M anag em en t S chool , T ianj in U n iv ersity , T ianj in 300072, C h ina
Abstract: T he cho ice of tim e delay and em bedd ing d im en sion is very im po rtan t to the p hase sp ace recon 2
struction of any chao tic tim e series. In th is p ap er, w e determ ine op tim al tim e delay by com p u ting au toco rre2
lation function of tim e series. O p tim al em bedd ing d im en sion is g iven by m ean s of the relation betw een em 2
bedd ing d im en sion and co rrelation d im en sion of chao tic tim e series. Based on the m ethod s above, w e choo se
ANN m odel to appox im ate the g iven true system. A t the sam e tim e, a new algo rithm is app lied to determ ine
the netw o rk w eigh ts. A t the end of th is p ap er, the theo ry above is dem on strated th rough the research of
tim e series generated by L og istic m ap.
Keywords: chao s; au toco rrelation function; op tim al em bedd ing d im en sion; op tim al tim e delay; L og istic
m ap
0　In troduct ion
Fo r a chao t ic sy stem cha racterized by an one 2d im en siona l t im e series, the appox im a t ing re2
su lt of ANN m odel to it is affected g rea tely by the cho ice of netw o rk inp u t series. T he
conven t iona l m ethod of determ in ing the inp u t series is qua lita t ive ana ly sis, w h ich b ring s
m any sub ject ive facto rs in to sy stem appox im a t ion. It w ill m ake sy stem appox im a t ion m o re
d isto rted. To reso lve th is p rob lem , w e in t roduce the theo ry of chao s in th is p ap er, and
th ree step s a re described: ( i) the recon st ruct ion of sy stem p ha se sp ace; ( ii) the iden t ifica 2
t ion of sy stem chao s; ( iii) the determ ina t ion of ANN m odel.
1　The Pr inc iple and M ethod of Pha se Space Recon struction Technology
Fo r a com p lica ted sy stem described by sta te va riab les { x k } ( k = 1, 2, …, n ) , w ha t w e can
get is a lw ay s a t im e series of one va riab le becau se of so m any rest rict ion s. B y u sing p ha se
sp ace recon st ruct ion techno logy, m u lt id im en siona l info rm a t ion can be revea led from th is
one 2d im en siona l da ta. T heo ry and p ract ice have p roved tha t the recon st ructed p ha se sp ace
can include the dynam ic behaviou r of o rig ina l sy stem very w ell a s long a s the em bedd ing
d im en sion m and the t im e delay Σ a re p rop erly cho sen.
A ssum e a m 2d im en siona l au tonom ou s dynam ic sy stem is described by
Rece ived da te: J u ly 10, 2001
Founda tion item: T he p ro ject is suppo rted by N ational N atu ral Science Foundation of Ch ina (N o. 79970043)
© 1995-2004 Tsinghua Tongfang Optical Disc Co., Ltd. All rights reserved.
YU E Y i2hong et a l.
Sta te P red iction of Chao tic System B a sed on ANN M odel
dx i
= f i ( x 1 , x 2 , …, x m ) , i = 1, 2, …, m
dt
307
( 1)
ba sed on the m ethod of elim ina t ion, fo rm u la ( 1 ) can be t ran sfo rm ed in to fo llow ing equa 2
t ion
x
(m )
(1)
(
= f ( x , x , …, x m -
1)
)
( 2)
fo rm u la ( 2) describes fo llow ing t im e evo lu t ion
( t) , …, x
xθ ( t) = ( x ( t) , x ′
(m - 1)
( t) )
( 3)
m ake fo rm u la ( 3) d iscrete, and sub st itu te d ifference equa t ion fo r deriva t ives of every o rder
in it. W e can get
xθ ( t) = ( x ( t) , x ( t + Σ) , …, x ( t + (m -
1) Σ) )
( 4)
fo rm u la ( 4) is the recon st ructed p ha se sp ace of a sy stem w ith em bedd ing d im en sion m and
t im e delay Σ. In the sen se of d iffeom o rp h ism , th is p ha se sp ace keep s the o rig ina l sy stem ′
s
geom et ric st ructu re, topo log ica l st ructu re and dynam ic behaviou r unchanged. T he m o st
im po rtan t po in t s of p ha se sp ace recon st ruct ion techno logy lie in the determ ina t ion of t im e
delay Σ and em bedd ing d im en sion m . U sua lly, the m va lue can be go t th rough g radua lly
increa sing the em bedd ing d im en sion, and rep ea ted ly com p u t ing co rrela t ion d im en sion o r
the m ax im um L yap unov exponen t un t il it doesn ′
t change w ith the increa se of em bedd ing
d im en sion. T he em bedd ing d im en sion of th is t im e is the op t im a l em bedd ing d im en sion.
T im e delay Σ determ ines the qua lity of o rb it s in p ha se sp ace. If Σ is too la rge, the signa l of
dynam ic sy stem described by t im e series w ill no t be t rue to the o rig ina l, so the co rrela t ion
d im en sion com p u ted from the t im e series is un soundness. O n the o ther hand, if Σ is too lit2
t le, there w ill ex ist au toco rrela t ion in the ob served da ta of sy stem , and the com p u ted co r2
rela t ion d im en sion is a lso d isto rted. In p ract ice, w e can determ ine Σ va lue a s fo llow s:
1) Fo r t im e series {x ( i ) } ( i= 1, 2, …, N ) , it s au toco rrela t ion funct ion est im a te can be
described by
δ = Χ
δg
δ
Θ
1
k
k Χ0 , k = 0, 1, …, N -
( 5)
N - k
δ = 1
Χ
k
∑ (x ( i) - θx ) (x ( i + k ) - xθ ) , k = 0, 1, …, N - 1
w here
N
xθ =
1
N
i= 1
N
∑x ( i)
i= 1
2) Σ is the m in im um t im e delay va lue tha t can elim ina te the au roco rrela t ion in o rig ina l
da ta. Σ shou ld m eet fo llow ing cond it ion:
δ g Φ 0. 1}
Σ = m in{k = 0, 1, …N - 1g g Θ
k
( 6)
2　Iden t if ica t ion of Chaot ic System
T here a re a va riety of criteria o r m ethod s tha t can be u sed to exam ine the chao t ic be2
haviou r of t im e series. T here a re advan tages and d isadvan tages to each of these criteria.
T hese criteria a re: ( i) co rrela t ion d im en sion; ( ii) en t rop y; ( iii) m ax im a l L yap unov expo 2
nen t; ( iv ) Eckm ann 2R uelle cond it ion; (v ) B rock s o r residua l test theo rem ; and ( vi) BD S
sta t ist ic test.
© 1995-2004 Tsinghua Tongfang Optical Disc Co., Ltd. All rights reserved.
308
Jou rna l of System s Science and System s Eng ineering
V o l111 N o 13　In ou r em p irica l ana ly sis, w e u sed the fo llow ing p rocedu re: g iven a list of criteria, the
g iven t im e series is cha racterised a s being chao t ic w ith resp ect to tha t list if:
1) T he leng th of t im e series shou ld m eet the fo rm u la described by
mg
2
( 7)
N > 10
w here m is op t im a l em bedd ing d im en sion.
2) T here ex ist s D tha t con ten t s:
D = lim lim
m →Α Ε→0
lnC m ( Ε)
ln ( Ε)
( 8)
w here
m
C ( Ε) =
1
N - (m - 1) Σ
∑
Η( Ε- g y i - y j g )
( 9)
g y i - y j g is Euclidean d istance betw een any tw o po in t s in the recon st ructed sp ace.
3) T here ex ist s 0< k < + ∞ tha t con ten t s:
m
1
C ( r)
K = lim lim
ln m + 1
m →Α r→0 r
( r)
C
m
w here C ( r ) can be go t from fo rm u la ( 9).
4) T he m ax im um L yap unov exponen t Κm ax > 0.
( 10)
Η( x ) =
(N -
(m - 1) Σ) (N -
1,
x > 0
0,
x Φ 0
(m - 1) Σ - 1)
i, j = 1
3　Appox im a t ing M ethod of System
T here have been m uch study on the a lgo rithm s of B P netw o rk. T heir comm on cha racteris2
t ic lies in the po in t tha t the erro r betw een the rea l ou tp u t s and the exp ected ou tp u t s is re2
duced th rough w eigh t regu la t ion ba sed on g rad ien t m ethod. In fact, there a re tw o defect s
ex ist ing in th is m ethod: ( i) It s dep endence on the in it ia l w eigh t va lues is so st rong tha t, if
the cho ice of in it ia l w eigh t va lues is no t very p rop er, the convergen t sp eed w ill be m uch
slow , o r even no t convergen t; ( ii) G rad ien t m ethod m akes it inevitab le to b ring abou t the
p rob lem of loca l m in im um.
T h is p ap er in t roduces a com p letely new a lgo rithm. It th row s aw ay the conven t iona l
op t im iza t ion th ink ing. Fo r the g iven p a irs of sam p les, w e choo se a g roup of a rb it ra ry free
w eigh t s, and get requested w eigh t s th rough
d irect ly so lving a sy stem of linea r equa t ion.
T h is m ethod overcom es the defect s tha t ex 2
ist in the lea rn ing a lgo rithm of B P netw o rk.
3. 1　A lgor ithm D escr iption
T he ANN m odel cho sen in th is p ap er
ha s th ree layer con st ruct ion a s show n in
F ig 11.
N um ber of nodes:
the st ip u la ted
sym bo ls in the a lgo rithm descrip t ion a re list 2
ed in T ab le 1.
F ig. 1
© 1995-2004 Tsinghua Tongfang Optical Disc Co., Ltd. All rights reserved.
YU E Y i2hong et a l.
309
Sta te P red iction of Chao tic System B a sed on ANN M odel
Table 1
Stip u lated
sym bo ls
R ep resen ting
m ean ing
X
(p )
Y (p )
Inp u t vecto r
of inp u t layer
T
R eal ou tp u t vect 2
o r of netw o rk
w hen inp u t vecto r
in inp u t layer is
X
nety(p )
(p )
Exp ected
ou tp u t vecto r
neth(p )
h (p )
Inp u t vecto r
of ou tp u t
layer
O u tp u t
vecto r of
h iden layer
Inp u t vecto r
of h iden
layer
(p )
w here p rep resen t s the p th sam p le, p = 1, 2, 3, …, k.
A ssum e sam p le size is k , from F ig. 1 and T ab le 1, w e can get fo rm u la ( 11) :
h1 ,
h2 ,
…,
hl
(2)
h2 ,
(2)
…,
hl
(1)
h1 ,
(1)
(1)
(2)
(k )
(k )
h2 ,
i1
w
i2
g
g
h1 ,
w
…,
(k )
w
hl
( nety(1) ) i
=
( nety(2) ) i
g
( 11)
( nety(k ) ) i
il
set cha racterist ic funct ion a s fo llow :
1
1 + l-
y = f (x ) =
( 12)
x
get fo llow ing fo rm u la
( nety(p ) ) i = ln
T
(p )
i
( 1 - T i(p ) )
, p = 1, 2, …, k
if a ssum e tha t
bi =
( nety(1) ) i , ( nety(2) ) i , …, ( nety(k ) ) i
w i = [w i1 , w i2 , …, w il ]
T
T
w e can get the equ iva len t fo rm u la of ( 11) a s fo llow :
[ h ] k 3 lw i = bi , i = 1, 2, …, m
( 13)
ba sed on the fundam en ta l theo rem of sy stem of linea r equa t ion, if fo rm u la ( 13 ) ha s so lu 2
t ion, the necessa ry and sufficien t cond it ion shou ld be r ( h ) = r (h , b). O n the o ther hand, if
every row of h is linea rly indep enden t, and w hen l < k , fo rm u la ( 13) ha s no so lu t ion, so it
is requ ired tha t l ≥k. Fo r the sim p licity of netw o rk st ructu re, w e choo se l = k. If on ly m a 2
t rix h is non singu la r, fo rm u la ( 13) ha s so lu t ion.
3. 2　D eterm ina tion of Network Input
W hen w e app ly ANN m odel to the fu tu re sta tes p red ict ion of chao t ic t im e series, a
p rob lem tha t how to choo se the netw o rk inp u t series is invo lved. T he t rad it iona l reso lving
m ethod is ca rried ou t by m ean s of qua lita t ive ana ly sis. It w ill p u t sub ject ive facto rs in to
the determ ina t ion of ANN m odel, and m akes it im po ssib le to exp ress the law h id ing in
t im e series exact ly. Fo r reso lving th is p rob lem , w e p u t fo rw a rd a com p letely quan t ita t ive
m ethod in the determ ina t ion of m odel inp u t series.
A ssum e {x ( 1) , x ( 2) , …, x (N ) } is a chao t ic t im e series, w e first get the op t im a l em 2
bedd ing d im en sion m and op t im a l t im e delay Σ acco rd ing to p ha se sp ace recon st ruct ion
techno logy, and then, ba sed on T aken s em bedd ing theo rem , there ex ist s a sm oo th m ap f
∶R 0 →R tha t con ten t s:
© 1995-2004 Tsinghua Tongfang Optical Disc Co., Ltd. All rights reserved.
310
V o l111 N o 13　Jou rna l of System s Science and System s Eng ineering
x ( k + 1) = f ( x ( k -
(m - 1) Σ) , x ( k -
(m - 2) Σ) , …, x ( k ) )
( 14)
w here k = (m - 1) Σ+ 1, (m - 1) Σ+ 2, …, N .
If m ap f can be determ ined, w e can do sta tes p red ict ion of t im e series above ba sed on
fo rm u la ( 14). H ere, w e shou ld po in t ou t tw o po in t s: ( i) T he g iven t im e series shou ld ex 2
ist op t im a l em bedd ing d im en sion and op t im a l t im e delay. T h is m ean s tha t the series
shou ld be chao t ic; ( ii) Even if cond it ion ( i) is sa t isfied, w e can ′
t do long 2term p red ict ion.
T h is is becau se chao t ic a t t racto r is very sen sit ive to it s in it ia l va lues. In the sho rt p eriod of
t im e, the d ivergence of p ha se p a th is rela t ively w eak, so sho rt 2term p red ict ion is m uch
fea sib le.
In th is p ap er, w e u se ANN m odel w ith th ree layer con st ruct ion to appox im a te m ap f .
W hen the ou tp u t va lue is x (k + 1) , the inp u t series is { x ( k - (m - 1) Σ) , x ( k - (m - 2) Σ) ,
…, x ( k ) }, and the num ber of nodes in the h iden layer is determ ined by the lea rn ing sam 2
p les. T he determ ina t ion of netw o rk w eigh t s fo llow s the a lgo rithm described in 3. 1.
4　Ca se Test
T he t im e series genera ted by L og ist ic m ap x n + 1 = Κx n ( 1- x n ) , w a s app lied to test the theo 2
ry above. Set Κ= 319, x 1 = 011, and genera te tw o thou sand s po in t s. T he first one thou 2
sand a re rem oved a s t ran sien t sta te po in t s, and the la st one thou sand a re kep t a s o rig ina l
da ta series {x ( i) }. W e ca rry ou t test a s fo llow ing step s.
4. 1　D eterm ina tion of Σ and m
B a sed on fo rm u la ( 5) , w e can get the da ta of Θ1～ Θ6 listed in T ab le 2.
Table 2
Θvalue
Θ1
Θ2
Θ3
Θ4
Θ5
Θ6
A u toco rrelation
coefficien t
- 0. 4984
- 0. 0193
0. 2636
- 0. 2487
0. 1694
- 0. 037
A cco rd ing to the cho ice standa rd of Σ described by fo rm u la ( 6 ) , w e can get tha t,
w hen k = 2, g Θ2 g = 010193< 011, and Σ= 2. Set in it ia l m = 1, then increa se m va lue g radu 2
a lly, and get a series of co rrela t ion d im en sion s listed in T ab le 3.
Table 3
m value
1
2
3
4
5
6
Co rrelation
d im en sion
0. 6954
0. 7359
0. 7445
0. 7476
0. 7485
0. 7487
O b serving T ab le 3, w e can get tha t, w hen m ≥3, co rrela t ion d im en sion s keep a lm o st
con stan t, so op t im a l em bedd ing d im en sion shou ld be 3.
4. 2　Chaotic Character istic Test of T im e Ser ies
1) F rom N = 1000, m = 3 and fo rm u la ( 7) , w e can get tha t N > 10m g2 , w h ich con ten t s
criterion ( 1) ;
2 ) B a sed on the com p u t ing in 411, there ex ist D = 0174 and Α= 3 tha t m ake fo rm u la
© 1995-2004 Tsinghua Tongfang Optical Disc Co., Ltd. All rights reserved.
YU E Y i2hong et a l.
311
Sta te P red iction of Chao tic System B a sed on ANN M odel
( 8) estab lished. It show s tha t criterion ( 2) is sa t isfied;
3) T here ex ist s K = 017021 tha t m akes fo rm u la ( 10) estab lished, and criterion ( 3) is
sa t isfied;
4 ) T he m ax im um L yap unov exponen t of the g iven t im e series, Κm ax = 014464 > 0,
w h ich m akes criterion ( 4) sa t isfied.
B a sed on the criteria above, w e have verified the chao t ic cha racterist ic of the g iven
t im e series genera ted by L og ist ic m ap.
4. 3　D eterm ina tion of ANN M odel
Fo r the g iven t im e series above, there ex ist s a certa in sm oo th m ap f tha t m akes
x ( k + 1) = f ( x ( k - 4) , x ( k - 2) , x ( k ) )
w e u se ANN m odel show n in F ig. 1 to appox im a te m ap f . If the netw o rk ou tp u t va lue is
x ( k + 1) , the inp u t va lues shou ld be x ( k - 4) , x ( k - 2) and x ( k ). Con sidering the p rop er
num ber of nodes in the h iden layer, w e choo se 50 sam p les to determ ine ANN m odel, and
it is done by app ly ing the a lgo rithm described in 311.
4. 4　Test of Appox im a ting Effect
U sing the ANN m odel described in 413 to appox im a te the po in t s betw een 2001 and
2020 th in the g iven t im e series, w e can get resu lt s listed in T ab le 4.
Table 4
SN
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
RV
0. 095
0. 336
0. 870
0. 442
0. 962
0. 143
0. 477
0. 973
0. 103
0. 359
AV
0. 097
0. 308
0. 845
0. 475
0. 992
0. 137
0. 446
0. 901
0. 105
0. 319
E rro r
2. 0%
- 8. 2%
- 2. 8%
7. 4%
3. 1%
- 4. 0%
- 6. 5%
- 7. 4%
2. 4%
- 8. 3%
SN
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
RV
0. 897
0. 359
0. 898
0. 358
0. 896
0. 364
0. 903
0. 342
0. 878
0. 418
AV
0. 862
0. 389
0. 819
0. 324
0. 941
0. 395
0. 866
0. 311
0. 824
0. 451
- 3. 9%
8. 2%
- 8. 8%
- 9. 4%
5. 0%
8. 5%
- 4. 1%
- 9. 1%
- 6. 1%
7. 9%
E rro r
w here SN rep resen t s Series N um ber; RV rep resen t s R ea l V a lue; AV rep resen t s A ppox i2
m a t ing V a lue.
O b serving the da ta in T ab le 4, w e can find tha t the erro rs betw een rea l va lues and ap 2
pox im a t ing va lues a re a ll loca ted in an idea l scop e, so the appox im a t ing effect is good.
5　Conclus ion
P rop er t im e delay and em bedd ing d im en sion determ ine the qua lity of recon st ructed p ha se
sp ace of o rig ina l t im e series. In th is p ap er, w e u se au toco rrela t ion funct ion to choo se t im e
delay, and u se ANN m odel w ith th ree layer con st ruct ion to appox im a te rea l chao t ic sy s2
tem. T he netw o rk inp u t s a re com p letely determ ined by op t im a l t im e delay and op t im a l em 2
bedd ing d im en sion, w h ich avo id s sub ject ive facto rs b rough t abou t by qua lita t ive ana ly sis.
T he determ ina t ion of netw o rk w eigh t s is ca rried ou t by app ly ing a com p letely new a lgo 2
© 1995-2004 Tsinghua Tongfang Optical Disc Co., Ltd. All rights reserved.
312
Jou rna l of System s Science and System s Eng ineering
rithm.
V o l111 N o 13　It overcom es som e defect s of B P a lgo rithm ba sed on the conven t iona l itera t ion
th ink ing. In the end, the fea sib ility of sta tes p red ict ion of chao t ic sy stem ba sed on the the2
o ry described in th is p ap er is show n th rough a ca se of the resea rch abou t L og ist ic t im e se2
ries.
References:
[ 1 ] Garcia C E, P rettt D M , M o rariM. M odel p red ictive con tro l. T heo ry and P ractice —A Su rvey, 1989, 25 (3) : 335
～ 348.
[ 2 ] Farm er J D , Sido row ich J J. P red icting chao s tim e series. Phys R ev L ett, 1987, 59.
[ 3 ] W ong F. Tm e series fo recasting u sing BP neu ral netw o rk s. N eu rocom p u ting, 1991, 2.
[ 4 ] Ep am inondas Panas, V assilia N inn i. A re o il m arkets chao tic? A non 2linear dynam ic analysis, Energy Econom ics
2000, 22: 549～ 568.
[ 5 ] Guo Gang, Sh i Zhongke, D ai Guanzhong. Select op tim al num ber of variab le to non linear m odeling w ith chao tic
theo ry. Con tro l and D ecision, 2000, 3: 233～ 238.
[ 6 ] Zhou X ifeng. A novel algo rithm fo r system iden tification and m odeling. Info rm ation and Con tro l, 2000, 4: 131～
138.
[ 7 ] Zhang D aiyuan. A com p letely new algo rithm of ANN. Com p u ting T echno logy and A u tom ation 2000, 6: 35～ 38.
[ 8 ] L i Y ing, L iu Bao, M a Shoufeng. T he iden tification of chao s in traffic flow u sing su rrogate2data techn ique. System
Eng ineering, 2000, 11: 54～ 58.
[ 9 ] J iang Chuanw en, Chen Sh i. Chao s techno logy and its app lication in the runoff fo recasting. System Eng ineering
1999, 11: 69～ 71, 80.
[ 10 ] X iang X iaodong, Guo Yaohuang. T he com p u ting of fractal d im en sion of chao s attracto r. System Eng ineering,
2000, 11: 75～ 78.
© 1995-2004 Tsinghua Tongfang Optical Disc Co., Ltd. All rights reserved.