Development of a new EDRNN procedure in Control of Human
Arm Trajectories
Shan Liu*，Yongji Wang* and Quanmin Zhu**
* Key Lab of Image Processing and Intelligent Control, Department of Control Science and Engineering,
Huazhong University of Science and Technology, Wuhan, Hubei, 430074, China
shan_liu_susan@hotmail.com
**Faculty of Computing, Engineering and Mathematical Sciences (CEMS), University of the West of England (UWE),
Frenchay Campus, Coldharbour Lane, Bristol, BS16 1QY, UK
Abstract: In this paper the trajectory tracking control of a human arm moving on the
sagittal plane is investigated by an interdisciplinary approach with the combination of
neural network mapping, evolutionary computation, and dynamic system control. The
arm in the study is described by a musculoskeletal model with two degrees of freedom
and six muscles, and the control signal is applied directly in the muscle space. A new
control system structure is proposed to manipulate the complicated nonlinear dynamical
arm motion. To design the intelligent controller, an evolutionary diagonal recurrent
neural network (EDRNN) is integrated with proper performance indices, in which genetic
algorithm (GA) and evolutionary program (EP) strategy are effectively integrated with
the diagonal recurrent neural network (DRNN). The hybrid GA with EP strategy is
applied to optimize the DRNN architecture and an adaptive dynamic back-propagation
algorithm (ADBP) with momentum for the multi-input multi-output (MIMO) systems is
used to obtain the network weights. The effectiveness of the control scheme is
demonstrated through a simulated case study.
1 Introduction
For whatever reasons, human beings have skillfully performed various movements to
achieve their targeted objects. Understanding the process and mechanisms of performing
such motion activities can provide a significant insight into the neural mechanisms of
motor learning and adaptation, development of interactions between robots and human
operators, and enormous applications in computer animation of the human motion and
neuromotor rehabilitation. It may be advantageous to use humanlike control strategies for
improving the performance of robot operation, in particular with regard to safety.
Although the use of control theories to provide an insight into neurophysiology has been
a long history, the complexity associated with the human movement arising from the
highly redundant and nonlinear dynamics has not been perfectly resolved, nevertheless
addressed.
There have been widespread research activities on modeling and controlling of the human
arm movements in the last five decades. Hierarchical model [15], neural network [12] [13] [20],
optimal control
[1] [10]
, adaptive control
[6] [11]
, iterative learning
[14]
, and combination of
these methods [17] have been applied in various motion controls of the human arm. So far
most of the studies have focused on the arm movement control on the horizontal plane,
with gravity compensation. However, human arm works in the gravity field. Therefore,
the human arm movement on the sagittal plane, that is, in the vertical plane, is more
natural and realistic. Accordingly proper research efforts should be contributed in this
field to provide a new angle to understand the operational mechanism of the normal
human arm movement.
Some experimental findings do suggest that the gravity force, at least for the vertical arm
movements in the gravity field, have an important role on the arm movement planning
process
[17] [18]
. The gravitational torque can either initiate or brake arm movements and
consequently should be represented in the motor command. And, the gravitational force
can be considered as an actuator which can be used to drive the movements. The
gravitational force acting to the arm even is dependent of the arm posture
[19]
. Thus
researches on the arm movements on the sagittal plane should establish some
relationships between the gravity, joint torques and movements.
It has been witnessed that neural networks have been increasingly adopted in the
biological motor control systems, due to their model-free approximation capability to the
complex decision making processes. Compared with the static neural network, the
diagonal recurrent neural network (DRNN) is more suitable for the dynamic
representations to efficiently capture the dynamic behavior of a complex system. Despite
the success of neural networks in solving complex problems, their design procedure is
based on still trial-and-error skills. A major problem in designing a neural network is the
proper choice of the network architecture, especially used as a controller for multi-input
multi-output (MIMO) nonlinear systems. The architecture of a neural network includes
its topological structure and the transfer function of each node in the network, which has
significant role in the way to process information. Recently, there have been many
attempts in this field, in which the combination of evolutionary algorithm and neural
networks has attracted a great deal of attention. Design of the optimal architecture for a
neural network can be formulated as a search problem in the architecture space where
each point represents the architecture. Previous studies have indicated that the
evolutionary approach is a better candidate for searching the space than the other
heuristic algorithms [3] [4].
In the main, the contributions of the study are intended
1) To understand the movement mechanism of the human arm on the sagittal plan;
2) To propose a new control strategy to manipulate the complicated nonlinear dynamical
arm motions;
3) To implement the controller by integrating the DRNN with genetic algorithm (GA)
and evolutionary program (EP) strategy;
4) To implement the adaptive dynamic back-propagation (ADBP) algorithm with
momentum to obtain the DRNN weight matrices in the control of MIMO systems.
5) To undertake simulation studies to demonstrate the new design procedure and compare
it with other representative approaches to show the enhancement.
This paper is organized as follows. In section 2, a realistic musculoskeletal model of the
human arm moving on the sagittal plane and the control system structure are presented to
set a foundation for the control system design. In section 3, the DRNN based controller is
constructed. Subsequently the hybrid GA and EP approach for optimizing the DRNN
architecture and the ADBP algorithm with momentum for determining the DRNN
weights are presented. In section 4, the feasibility of the proposed neural control system
is demonstrated by numerical simulation studies. Finally in section 5, the conclusions and
the future works are provided.
2 Problem Statement
2.1 Human Arm Model
The plant used in this study is a simulated, two degree-of-freedom, planar arm model
moving in the sagittal plane. A schematic picture of the arm model is shown in Fig. 1. In
this model there are two joints (i.e. shoulder and elbow) driven by three pairs of opposing
muscles; two pairs of muscles individually actuate the shoulder and the elbow joints,
while the third pair actuates both joints simultaneously. The movement range of the
shoulder is from -80° to 90° and that of the elbow is from 0° to 150°. For simplify, it is
assumed that the reference trajectory exists in the joint space, the mass effect of each
muscle is not considered in the model because the mass transfer of each muscle coming
from the movements of muscles is not so large, and the external force only involves the
gravity.
The inverse dynamics of the arm model is
M ( )  C ( ,  )  B  G ( )   m
(1)
Fig.1 Schematic picture of the arm model
In Eq.(1) ,   R 2 is the joint angle vector (   [1 , 2 ]T , 1 : shoulder,  2 : elbow),
M ( )  R 22 is the positive definite symmetric joint inertia matrix, C ( , )  R22 is the
vector of centrifugal and Coriolis torques, B  R22 is the joint friction matrix, G( )  R 2
is the vector of gravity torques, and  m  R 2 is the joint torque produced by six muscles.
The applied muscle model is a simplified version of the model proposed by [2]. It
represents the combined activation a and length l and velocity v dependent contractile
forces. The tension produced by a muscle is represented as
T (a, l , v)  a( FL (l ) FV (l , v)  FP (l ))
(2)
where
2
FL (l )  exp( l 2  1 )
 6  v
v0
 6  2lv ,
FV (l , v)  
2
1  (3  5l  3l )v , v  0
1 v

FP (l )  0.02 exp(13.8  18.7l )
and the muscle force is
(3)
F (a, l , v)  F0maxT (a, l , v)
where F0max is the maximum isometric force.
The moment arm is defined as the perpendicular distance from the muscle's line of
action to the joint's center of rotation, which is equal to the transpose of the Jacobian
matrix from the joint space to the muscle space. Here, the relation between muscle forces
and joint torques can be expressed by using the principle of virtual work:
 m  Mo( ) F (a, l , v)  0
(4)
where Mo( )  R 26 is the moment arm.
2.2 Control System Structure
The control system is illustrated in Fig. 2, where the system is composed of single DRNN
controller (In Fig. 2, k is the discrete time.) and the plant. The DRNN controller plays an
important role in the trajectory tracking control. The architecture of the DRNN is
optimized by GA and EP; and the weights are trained by the ADBP. The inputs of the
DRNN controller are the desired position (  r (k ) ) in the reference input
( yr (k )  [ r (k ),  r (k )] ), the previous tracking error between the desired and the actual
response of the plant ( e _ y(k  1)  yr (k  1)  y(k  1) , y(k )  [ (k ), (k )] ), and the
previous control signal to the plant, i.e. the previous muscular activation
( u (k  1)  a (k  1) ), and the integral terms of the position tracking errors. The reference
input is set in advance, so no delay exists in the controller for the reference input. The
DRNN makes advantage of the input and output information of the plant to generate the
proper control signal. Details of the controller design are described in Section 3.
Fig. 2 Block diagram of the human arm control system
3 Evolutionary Diagonal Recurrent Neural Network Controller
The DRNN is a simple recurrent neural network and has a similar dynamic mapping
ability to the fully connected recurrent neural network. A typical DRNN structure
comprises three layers: an input layer, a hidden layer, and an output layer
[5]
. The hidden
neurons are dynamic neurons in that the output of each hidden neuron is fed back to its
input through a delay unit. These local feedback paths introduce a dynamic behavior into
the network, so the DRNN has a more powerful capability to express the dynamical
systems than the feed-forward neural networks.
3.1 Structure of the DRNN
In the DRNN controller, the numbers of the neurons in the input layer and the output
layer are fourteen and six, respectively. The input I (k ) includes the input and the output
information
of
the
plant
(
I (k )  [e _ y(k 1) r (k ) a(k 1)

k 1
0
e( )d ]
,
e(k )  r (k )   (k ) ). The last one in I (k ) is the integral of position errors to eliminate
the steady-state position error [9]. The number of the hidden neurons is determined by the
hybrid GA and EP approach.
The mathematical model for the DRNN is shown below:
al (k )  W jlO X j (k ), X j (k )  f ( S j (k ))
j
S j (k )  WijI I i (k )  W jD X j (k  1)
(5)
i
where for each discrete time k, I i (k ) is the i th input to the DRNN, S j ( k ) is the sum of
inputs to the j th recurrent neuron, i.e. the j th hidden neuron, X j (k ) is the output of the
j th recurrent neuron, and al (k ) is the l th output of the DRNN. Here f () is the
nonlinear transfer function, and W I , W D and W O are input, recurrent and output weight
matrix, respectively in R n14 , Rn , and R 6n .
n
is the number of hidden neurons.
3.2 Optimization of the DRNN Structure
For practical purpose, the simultaneous evolution of the neural network connection
weights and the architecture is beneficial. However, in this paper, the DRNN controller is
a fourteen inputs six outputs neural network, the amount of the connection weights is
n  (14  1  6) . The evolution of the architecture with connection weights will be time
consuming. So, we just optimize the architecture of the network; the connection weights
are trained through the ADBP algorithm. Several researches on evolving the neural
network architectures for control problems have been carried out in recent years [8] [25] [26].
In [26], genetic algorithm with internal copy operator is used to organize the structure in
a network, but the effectiveness depends on the gene coding and how the gene blocks are
chosen and copied, which is still an open problem. Pasemann et al. [25] adopted an
ENS3-algorithm (evolution of neural systems by stochastic synthesis) to generate
recurrent neural networks with non-trivial internal dynamics, which is used for
development of the network structure, optimizing parameters as well. In terms of the
required computation time, the ENS3 can not compete with those learning algorithms like
the hybrid GA with EP strategy from [8].
In this paper, the optimum DRNN architecture is determined by implementing both GA
and EP methods, which evolves a pool of DRNN architectures until the control objective
is achieved, because GA can explore a wider area of search in a shorter number of
evolutions and EP can maintain the behavioral link of the selected offspring after the
competition stage
[8]
. The goal of the controller is to minimize the cost function J
( Eq.(7) , i.e. the sum-squared of the tracking error (SSE)), with T the number of
simulation epochs. Fig. 3 shows the process of the evolution procedure for the
optimization of the DRNN architecture. At each generation, all networks in the
population are trained to track the desired trajectories. The population is divided into
three different groups, namely the best, strong and weak group according to their cost
function.
T
2
J  12 [ jr (k )   j (k )]2
(6)
k 1 j 1
Binary string representation is used to represent the DRNN architecture. In the DRNN,
the sigmoid transfer functions are usually used in the hidden nodes
[22]
. The transfer
function has been shown to be an important part of the neural network architecture and
have a significant impact on the network's performance. The transfer function is often
assumed to be same for all the nodes in the same layer. White et al. [27] introduced a
simple approach to evolve both architectures and connection weights. The evolution was
used to decide the optimal mixture between two transfer functions (sigmoid transfer
function and Gaussian transfer function). In [28], EP is used to evolve general neural
networks with both sigmoidal and Gaussian nodes by adding or deleting a node (either
sigmoidal or Gaussian); and good performances were reported for some prediction
problems. However, it is still unclear about the performance of the network with different
transfer functions for control problems, and the training process of such a network
controller becomes complicated. We assume transfer functions are same for all nodes in
the same layer. The parameters evolved are the number of the neurons and the type of the
transfer functions in the hidden layer. Fig. 4 shows an example of the chromosome
representation in a binary string, which means that there are 28 neurons and a log sigmoid
transfer function in the hidden layer.
The explanation of the major steps in Fig. 3 is as follows:
1. Generate m number of networks in one population as depicted in Fig. 3. Initialize
weight matrix in each network with uniformly distributed random numbers.
2. Train each network with the ADBP algorithm and calculate the cost functions of the
networks (i.e. each network in the population is used to control the human arm model,
and then the control performance is monitored.).
Fig. 3 Flow chart of the evolution procedure
Fig. 4 Binary string representation of DRNN architecture
3. Divide these networks into three groups through the rank based selection method.
If one network in the best group can make the behavior converge satisfactorily to the
desired target, the procedure ends. Otherwise, continues.
4. Apply GA onto the individuals in the weak group to explore a wider area of search in a
shorter number of evolutions. The multipoint crossover and simple flip over mutation are
used in the computations. While these offspring networks are used to control the human
arm, their cost functions are compared with those of their parents. If the offspring is
better, it replaces its parent. Otherwise, the parent stays alive.
5. Apply EP onto each individual in the strong group to maintain the behavioral link of
the selected offspring. Three hidden neurons are deleted from each network, then the new
network is used for controlling the plant and calculating the offspring's cost function. If
the offspring is better than its parent, it replaces the parent. Otherwise, three hidden
neurons are added, the new network is used to control the plant again. If the offspring is
better than the parent, it replaces its parent. Otherwise, the parent stays alive.
6. Combine these networks obtained from steps 5 and 6 with networks in the best group
to get a new generation.
7. Repeat steps 3 to 6.
3.3 DRNN Weight Training
An ADBP algorithm with momentum for the MIMO systems is used for training the
DRNN weights. Back-propagation often gets trapped in a local minimum of the error
function and is incapable of finding a global minimum, and the evolutionary approach
can search all of solution space without the local minimum problem, but nevertheless the
evolutionary approach consumes an amount of time in network learning. The ADBP with
momentum for the MIMO systems can catch the dynamic information contained in the
plant, and the convergence of the DRNN based control system is guaranteed through
Lyapunov stability theorem [23].
The error function for a training cycle is defined as
2
E  12 [ jr (k )   j (k )]2
(7)
j 1
The gradient of error in (7) with respect to an arbitrary weight matrix W  R mr is
represented by
E
 (k )
a (k )
  e( k )
 e(k ) a (k )
(8)
W
W
W
where e(k )  r (k )   (k ) is the error between the desired and the output responses of the
plant, and the factor a (k )   (k ) / a(k ) represents the sensitivity of the plant with
respect to its input signal. From the mathematical model of the human arm (1) - (4),
 a (k )  m((kk)) am((kk))   M ( ) 1 h 2 F0max Mo( )( FL (l ) FV (l , v)  FP (l ))
(9)
where h is the sampling time.
Upon the DRNN model described in Section 3.1, the output gradients with respect to
the output, the recurrent, and the input weights, respectively, are given by
al (k )
 X j (k )
W jlO
al (k )
 W jlO Pj (k )
D
W j
(10)
al (k )
 W jlO Qij (k )
I
Wij
where Pj (k )  X j (k ) / WjD , Qij (k )  X j (k ) / WijI and satisfy
Pj (k )  f '( S j (k ))( X j (k  1)  W jD Pj (k  1)), Pj (0)  0
(11)
Qij (k )  f '( S j (k ))( I i (k )  W jDQij (k  1)), Qij (0)  0
We make use of the following gradient steepest descent method
[24]
to adapt the
weights of the DRNN:
E
)   (W (k )  W (k  1))
(12)
W
 I


 I D O
where  is the learning rate matrix (    D
 ,  ,  ,  represents the learning
O

 

W (k  1)  W (k )   (
rate matrix corresponding to W I , W D , W O ), the maximum learning rate  m in  is
chosen as
0  m 
2
 ei (k ) e j (k )
e (k ) e (k )
e (k ) e (k ) 
,
)  ( i D , j D )  ( i O , j O )
I
I
W
W
W
W
W
i , j 1 

2
2
(13)
  ( W
where (a, b) is the inner product between vector a and vector b , ei (k ) and e j (k ) is the
i th and j th tracking error of the controller respectively.  is a momentum factor
affecting the change W (k )  W (k  1) of the DRNN’s weights at the k th iteration. [23]
shows that (13) can be used as a guideline to find the optimal learning rate, and the
convergence can be guaranteed.
4 Simulation Results
In the simulation study, the EDRNN was programmed using MATLAB, the plant
parameters were obtained from [14], and the SSE function was monitored to demonstrate
the performance of the proposed control scheme. The reference trajectory equation was
assigned as
1r  60sin t
  60  60 cos t
 2r
(13)

1r  60 cos t
  60sin t
 2r
In the simulation, some limitations have been imposed on the DRNN. Though there is no
general solution for determining the number of hidden neurons in a neural network [20], it
was assumed that the range of the hidden neuron number was from one times to four
times than the number of the inputs. Therefore each network had a different hidden
neuron number from 14 to 56. For investigating the effectiveness of other differentiable
functions in the DRNN, tan-sigmoid, log-sigmoid, Gaussian, and the saturating linear
were chosen as the candidates of the transfer functions.
4.1 Evolution Process
In the evolution process, the number of the networks in one population m was set as m =
14, which is equal to the number of the network inputs; the crossover probability Pc and
the mutation probability Pm were set as Pc = 0.9 and Pm = 0.03 respectively; these two
parameters were determined from a trial and error procedure. The results of the evolution
over thirty generations are shown in Fig. 5 to show the best cost function value of each
population decreasing sharply in the first six generations, and converged after 20
generations. Table 1 indicates that the number of the hidden neurons is converged to from
twenty eight to thirty. Judging from the cost function and the running time shown in
Table 1, twenty eight is the optimum number of the hidden neurons. The transfer function
converges as a log sigmoid. For comparison, the GANNet algorithm presented in [27]
was implemented to evolve the architecture of the DRNN. Because the GA is sensitive to
the length of the chromosome string, the shorter this string can be made, the better, the
GANNet was modified to optimize the architecture of the DRNN; then the DBP was used
to obtain the weights. Comparisons of the results through the modified GANNet
algorithm and the GA and EP approach are shown in Table 2, which indicates the hybrid
GA and EP approach is more effective.
Fig. 5 Result of evolution
Table 1. Comparison of DRNN with different number of neuron
No. of neuron
26
27
28
29
30
Transfer Function
tans
logs
logs
logs
logs
Running time(sec) 17.92 18.13 18.86 23.64 30.02
Cost Function / T
0.04
0.034
0.02
0.02
0.02
Table 2. Comparison of the evolution results through the modified GANNet and GA+EP
Type of algorithm
Modified GANNet GA+EP
No. of hidden neurons
30
28
Running time(sec)
621.42
380.20
Cost function / T
0.024
0.020
Type of transfer function
Gauss
logs
Logs
Probability of transfer function
0.05
0.95
/
No. of generation
60
20
4.2 Control Result
In running the simulation studies, each DRNN controller was trained for 700 epochs. The
adaptive learning rates  O ,  D , and  I were used starting from the initial rate of 0.2, 0.2
and 0.2 respectively. After 100 training epochs, the learning rates were adjusted to 0.03,
0.05 and 0.005 respectively. The momentum factor  was assigned as 0.25. In this study,
randomly generated weight matrices were used as the initial weights. Therefore, the
whole program had been run for several times until satisfactory results were achieved.
The best simulation results are shown in Fig. 6 and Fig. 7. For comparison, the iterative
LQG (ILQG) presented in [16] was also applied to control the human arm on the sagittal
plane. It has been noticed that the ILQG was used to control the human arm movement on
the horizontal plane, and good simulation results were reported. Therefore ILQG was
selected as a critical bench test to compare the new procedure. Inspection of Fig. 6 and
Fig. 7, the EDRNN controller is capable of driving the human arm to follow the reference
joint trajectories with small tracking errors, and the performance of the EDRNN control
is better than that generated from the ILQG.
For investigating the generalization of the DRNN, the obtained DRNN controller from
the trajectory tracking control simulation was further used for the posture control in the
presence of external forces. In the same way, the ILQG was used as a comparable method
to compare with the EDRNN controller. The external force parallels the ordinate, and is
set as 9.8N . The simulation results are shown in Fig. 8 and Fig. 9. These figures indicate
that the obtained DRNN controller can maintain the arm at different location within the
allowable bound. Moreover, both methods can implement the posture control in the
presence of the external force.
Fig. 6 Trajectory of the joint angles
Fig. 7 Graph of the joint angle velocities
EDRNN control
ILQG control
reference
theta1(deg)
20
0
-20
0
100
200
300
400
Epochs
500
600
700
theta2(deg)
50
EDRNN control
ILQG control
reference
40
30
20
10
0
100
200
300
400
Epochs
500
600
700
Fig. 8 Angle response curve under the action of the external force
dtheta1(deg/s)
20
0
-20
-40
-60
-80
0
dtheta2(deg/s)
EDRNN control
ILQG control
reference
100
200
300
400
Epochs
500
600
700
EDRNN control
ILQG control
reference
100
0
-100
0
100
200
300
400
Epochs
500
600
700
Fig.9 Angle velocity curve under the action of the external force
Subsequently some pulse signals as disturbance were added into the inputs of the arm.
Each input had the same disturbance signal. The pulse amplitude was set as 1, the period
was set as 350 epochs, and the width as 15％ of the period. Fig. 10 and Fig. 11 show the
best simulation results. Compared with Fig. 6 and Fig. 7, it is observed that disturbance
affects the performance of these controllers. But, the arm still can follow the desired joint
inputs with the tracking error in tolerance. Further, the performance and robustness of the
EDRNN control is better than those delivered from the ILQG.
Then, the obtained DRNN from the above simulation testing is used for the posture
control with disturbance. The simulation results are shown in Fig. 12 and Fig. 13. These
figures illustrate the performance and robustness of the two methods for the posture
control in the presence of disturbance and external forces is all satisfactory.
Fig. 10 Trajectory of the joint angles with disturbance
Fig. 11 Graph of the joint angle velocities with disturbance
60
EDRNN control
ILQG control
reference
theta1(deg)
40
20
0
-20
0
100
200
300
400
Epochs
500
700
EDRNN control
ILQG control
reference
60
theta2(deg)
600
40
20
0
100
200
300
400
Epochs
500
600
700
dtheta1(deg/s)
Fig. 12 Angle response curve with disturbance and external force
0
-100
0
dtheta2(deg/s)
EDRNN control
ILQG control
reference
-50
100
200
300
400
Epochs
500
600
700
EDRNN control
ILQG control
reference
100
0
-100
0
100
200
300
400
Epochs
500
600
700
Fig. 13 Angle velocity curve with disturbance and external force
For better comparing the EDRNN and ILQG method, some more simulation results are
included. The running time of the two methods in different simulation testing are shown
in Table 3. In the trajectory tracking control simulation, the running time of the DRNN is
counted during the course of optimizing the architecture of DRNN and training the
DRNN. In the others simulations, the running time of the DRNN is the training time of
the obtained DRNN from the tracking control simulation. Table 3 indicates that the
computation burden of tuning the two controllers in the tracking simulation is almost
equal, the ILQG is somewhat better; however, the obtained DRNN controller can be used
for the different reference signals, and moreover, the ILQG need to repeat all the
optimizing procedure for different simulation testing.
Table 3 Comparison of computation burden of EDRNN and ILQG
Type of controller
EDRNN
ILQG
Trajectory tracking control
380.2
369.4
Trajectory tracking control with disturbance
25.14
409.3
Posture control with external force
16.02
261.2
Posture control with disturbance and external force
19.23
304.8
Running time (sec)
Type of simulation
5 Conclusions
In the first stage of concept development and system design, a number of cutting edge
techniques have been properly tailed into an interdisciplinary approach for the trajectory
tracking control of a human arm model on the sagittal plane. For the initial feasibility
study, simulation results are provided to show that the GA and EP approach can find out
the suitable architecture of the DRNN, the ADBP algorithm with momentum can
guarantee the convergence of the system, and the controlled arm is well behaved to
follow up those requested trajectories. It should be noticed that the proposed control
procedure for the human arm movement can be further expanded for motor control,
rehabilitation robot control, and various related applications.
With its favorable dynamical network structure, the DRNN has played a key role in the
tracking control in this study. The authors have tried to use a multilayer perceptron
network in the trajectory tracking, but unfortunately the outcomes are unacceptable,
which are not included in this paper. Furthermore, the performance comparisons between
the DRNN and the iterative LQG show that DRNN is a better method to control the
human arm motion in the sagittal plane. Compared with the modified GANNet algorithm,
the hybrid GA and EP strategy has a faster converging rate and is more effective.
In the second stage, further investigations will be expanded to the controller design for
the trajectory tracking control in the Cartesian coordinate rather than in the joint space.
The following development is also targeted for improving the computational efficiency. It
has already been mentioned that evaluating a structure involves training the
corresponding network and thus requires a large computational power. Obviously,
speeding up training would greatly reduce the required effort. Some efficient algorithms
should be developed while a fast real-time controller is the request.
6 Acknowledgements
This work is supported by grants from the National Nature Science Foundation of China,
No 60674105, the Ph.D. Programs Foundation of Ministry of Education of China, No
20050487013, and the Nature Science Foundation of Hubei Province of China, No
2007ABA027. The authors would like to thank Prof. Jiping He for the guidance regarding
the muscular control and providing the arm model in format of Matlab/Simulink. Further
the authors are grateful to the editor and the anonymous reviewers for their helpful
comments and constructive suggestions with regard to the revision of the paper.
References
1. Alfred C. Schouten, Erwin de Vlugt, Frans C. T. van der Helm, Guido G. Brouwn:
Optimal Posture Control of a Musculo-Skeletal Arm Model. Biol. Cybern. 84 (2001)
143-152
2. Brown, I. E., Cheng, E. J., and Leob, G.: Measured and Modeled Properties of
Mammalian Skeletal Muscle. II. the Effects of Stimulus Frequency on Force-Length
and Force-Velocity Relationships. J. of Muscle Research and Cell Motility 20 (1999)
627-643
3. Xin Yao: Evolving Artificial Neural Network. Proceedings of the IEEE. 87 (1999)
1423-1477
4. P.A. Castillo, J.J. Merelo, M.G. Arenas, and G. Romero, Comparing Evolutionary
Hybrid Systems for Design and Optimization of Multilayer Perceptron Structure
along Training Parameters. Information Science 177 (2007) 2884-2905
5. Chao-Chee Ku, Kwang Y. Lee: Diagonal Recurrent Neural Networks for Dynamic
System Control. IEEE Trans. on Neural Networks. 6 (1995) 144-156
6. David W. Franklin, Theodore E. Milner: Adaptive Control of Stiffness to Stabilize
Hand Position with Large Loads. Exp. Brain Res. 152 (2003) 211-220
7. Ernest J. Cheng, Ian E. Brown, Gerald E. Loeb: Virtual Muscle: a Computational
Approach to Understanding the Effects of Muscle Properties on Motor Control.
Journal of Neuroscience Methods 101 (2000) 117-130
8. S. B. Hussenin, H. Jamaluddin, M. Mailah, A. M. S. Zalzala. A Hybrid Intelligent
Active Force Controller for Robot Arms Using Evolutionary Neural Networks. In:
Peter J. Angeline (eds.) Proceedings of the 2000 Congress on Evolutionary
Computation, Vol. 1. IEEE Press, New Jersey (2000) 117-130
9. Kunihiko Nakazono, Masahiro Katagiri, Hiroshi Kinjo, Tetsuhiko Yamamoto: Force
and Position Control of Robot Manipulator Using Neurocontroller with GA based
Training. In: Yasushi Nakauchi (eds.): Proceedings of 2003 IEEE International
Symposium on Computational Intelligence in Robotics and Automation, Vol. 3. IEEE
Press, New Jersey (2003) 1354-1357
10. Ning Lan: Analysis of an Optimal Control Model of Multi-Joint Arm Movements.
Biol. Cybern. 76 (1997) 207-117
11. Robert M. Sanner, Makiko Kosha: A Mathematical Model of the Adaptive Control of
Human Arm Motions. Biol. Cybern. 80 (1999) 369-382
12. Shan liu, Yongji Wang, Jian Huang: Neural Network based Posture Control of a
Human Arm Model in the Sagittal Plane. In: J. Wang et al. (eds.): International
Symposium on Neural Networks 2006, Vol. 3973. Springer-Verlag, Berlin
Heidelberg New York (2006) 792-798
13. Stroeve SH: Impedance Characteristics of a Neuromusculoskeletal Model of the
Human Arm. I. Posture Control. Biol. Cybern. 81 (1999) 475-494
14. Weiwei Li, Emannuel Todorov: Iterative Quadratic Regulator Design for Nonliear
Biological Movement system. In: Joaquim Filipe (eds.): Proceedings of the 1st
International Conference on Informatica in Control, Automation and Robotics, Vol. 1.
Kluwer Academic Publisher, Netherlands(2004) 222-229
15. Weiwei Li, Emanuel Todorov and Xiuchuan Pan: Hierarchical Optimal Control of
Redundant Biomechanical Systems. In: D. Hudson, Z.P. Liang (eds.): Proceeding of
26th IEEE EMBS. IEEE Engineering in Medicine and Biology Society, Vol. 26.
IEEE Press, New Jersey (2004) 4618-4621
16. Emanuel Todorov, Weiwei Li: A Generalized Iterative LQG Method for
Locallyoptiml Feedback Control of Constrained Nonlinear Stochastic Systems. In:
Suhada J., S.N. Balakrishnan (eds.): Proceeding of 2005 ACC. American Control
Conference, Vol. 1. IEEE Press, New Jersey (2005) 300-306
17. Charslsmbos Papaxanthis, Thierry Pozzo, and Joe McIntyre: Arm End-point
Trajectories Under Normal and Micro-gravity Environments. Acta Astronautica.
43(1998) 153-161
18. Charslsmbos Papaxanthis, Thierry Pozzo, and Marco Schieppati: Trajectories of arm
pointing movements on the sagittal plane vary with both direction and speed. Exp.
Brain Res. 148 (2003) 498-503
19. Hiroyuki Kambara, Kyoungsik Kim, and Duk Shin, etc.: Motor Control-Learning
Model for Reaching Movements. In: 2006 International Joint Conference on Neural
Networks, (2006) 555-562
20. Andrew H. Fagg, Nathan Sitkoff, james C. Houk, and Andrew G. Barto: Cerebellar
Learning for Control of a Two-Link Arm in Muscle Space. In: 1997 IEEE
International Conference on Robotics and Automation, 3(1997) 2638-2644
21. Hagan, M. T., H. B. Demuth, and M. H. Beale: Neural Network Design. Boston, MA:
PWS Publishing, 1996
22. Yongji Wang, Jian Tu: Neural Network Control. Beijing: China Machine Press, 1998
23. Pan Wang, Youfeng Li, Shan Feng, Wei Wei: Convergence of Diagonal Recurrent
Neural Networks’ Learning. In: Proceedings of the 4th World Congress on Intelligent
Control and Automation. (2002) 2365-2369
24. H. Youlal, A. Kada, M. Ramzi and L. El-ismaili: Fast Diagonal Recurrent Neural
Networks for Identification and Control of Nonlinear System. In: International
Conference on Control 1994. 1(1994) 104-109
25. Pasemann, F., Steinmetz U., and Dieckman U. Evolving Structure and Function of
Neurocontrollers. in: Proceedings of the 1999 Congress on Evolutionary Computation.
3(1999) 1973-1978
26. Toru Kumagai, Mitsuo Wada, Sadayoshi Mikami, and Ryoichi Hashimoto. Structured
Learning in Recurrent Neural Network Using Genetic Algorithm with Internal Copy
Operator. (1997) 651-656
27. D. White, P. Ligomenides. GANNet: A Genetic Algorithm for Optimizing Topology
and Weights in Neural Network Design. In Proc. Int. Workshop Artificial Neural
Network (IWANN 93), Lecture Notes in Computer Science. 686(1993) 322-327
28 Y. Liu, X. Yao. Evolutionary Design of Artificial Neural Networks with Different
Nodes. In Proc. 1996 IEEE Int. Conf. Evolutionary Computation (ICEC 96). (1996)
670-675
Shan Liu is a doctoral student at Key Laboratory of Image Processing
and Intelligent Control, Department of Control Science and Engineering,
Huazhong University of Science and Technology, Wuhan, China. She
obtained her B.S. degree in Mechanical Automation, Hubei University of
Technology, Wuhan, China, in 2001 and M.S degree in system
engineering, Huazhong University of Science and Technology, in 2004.
Her main research interests involve application of advanced control to
nonlinear multivariate system, such as rehabilitation robots, the neuromusculoskeletal
model of human arm posture and movement, and industrial plant; neural network based
intelligent and optimal control for neural prostheses; development of neural implant
devices for epidural spinal cord stimulation for metabolic study in vivo of cat, and the
modeling, control and optimality of metabolic systems.
Yongji Wang was born in Ji’an, Jiangxi, P.R. China, in 1955. He received
the undergraduate degree in Electrical Engineering from Shanghai
Railway University, Shanghai, P.R. China, the M.S. degree and the Ph. D
degree in automation from Huazhong University of Science and
Technology, Wuhan, P.R. China, in 1982, 1984 and 1990, respectively.
Since 1984, he has been with Huazhong University of Science and
Technology, Wuhan, P.R. China, where he is currently a Professor of
Electrical Engineering. His main interest is in intelligent control, and he has done
research in neural network control, predictive control and adaptive control.
Dr. Wang is a member of IEEE, USA, President of Hubei Automation Association, China,
Standing member of council of Electric Automation Committee of Chinese Automation
Society and member of council of Intelligent Robot Committee of Chinese Artificial
Intelligence Society. He is an Area editor (Asia and Pacific) of Int. J. of Simulation
Identification and Control.
Quanmin Zhu is Professor in control systems at the Faculty of
Computing, Engineering and Mathematical Sciences (CEMS)
University of the West of England (UWE), Bristol, UK. He obtained his
MSc in Harbin Institute of Technology, China in 1983 and PhD in
Faculty of Engineering, University of Warwick, UK in 1989. His main
research interest is in the area of nonlinear system modelling,
identification, and control. Recently Professor Zhu started investigating electrodynamics
of acupuncture points and sensory stimulation effects in human body, modelling of
human meridian systems, and building up electro-acupuncture instruments. He has
published over one hundred papers on these topics. Currently Professor Zhu is acting as
Associate
Editor
of
International
Journal
of
Systems
Science
(http://www.tandf.co.uk/journals/titles/00207721.asp), Member of Editorial Committee
of Chinese Journal of Scientific Instrument (http://www.asiatest.org/), and Editor (and
Founder) of International Journal of Modelling, Identification and Control
(http://www.inderscience.com/ijmic). Prof Zhu’s brief CV can be found at
http://www.ias.uwe.ac.uk/people.htm