ARTICLE IN PRESS
Engineering Applications of Artificial Intelligence 23 (2010) 595–603
Contents lists available at ScienceDirect
Engineering Applications of Artificial Intelligence
journal homepage: www.elsevier.com/locate/engappai
Predictive human operator model to be utilized as a controller using linear,
neuro-fuzzy and fuzzy-ARX modeling techniques
Ozkan Celik ,1, Seniz Ertugrul
Department of Mechanical Engineering, Istanbul Technical University, Inonu Caddesi, No. 87 Gumussuyu, 34437 Istanbul, Turkey
a r t i c l e in fo
abstract
Article history:
Received 10 April 2009
Received in revised form
31 July 2009
Accepted 28 August 2009
Available online 12 November 2009
Modeling human operator’s behavior as a controller in a closed-loop control system recently finds
applications in areas such as training of inexperienced operators by expert operator’s model or
developing warning systems for drivers by observing the driver model parameter variations. In this
research, first, an experimental setup has been developed for collecting data from human operators as
they controlled a nonlinear system. Appropriate reference signals and scenarios were designed
according to the system identification and human operator modeling theory, to collect data from
subjects. Different modeling schemes, namely ARX models as linear approach, and adaptive-networkbased fuzzy inference system (ANFIS) as intelligent modeling approach have been evaluated. A hybrid
modeling method, fuzzy-ARX (F-ARX) model, has been developed and its performance was found to be
better in terms of predicting human operator’s control actions as well as replacing the operator as a
stand-alone controller. It has been concluded that F-ARX models can be a good alternative for modeling
the human operator.
& 2009 Elsevier Ltd. All rights reserved.
Keywords:
Human operator
Predictive modeling
Fuzzy-ARX model
Human controller model
Fuzzy identification
1. Introduction
A human carries out many different control tasks that range
from simple manipulation to relatively more complex tasks such
as operating a crane for carrying loads, driving a vehicle or
piloting an airplane.
Human operator (HO) models become an important tool when
one needs to design a system to be controlled by a human
(Sheridan and Ferrell, 1974). The model provides constraints for
the design and the system can be tested with the model at hand in
a closed-loop fashion if the model is accurate and reliable enough.
Earlier studies in this area were motivated by a need for pilot
models in order to utilize during the design phase of warplanes
(McRuer, 1980).
Some more recent studies on human operator modeling
concentrate on obtaining models for drivers’ control behavior
(Hess and Modjtahedzadeh, 1990; Prokop, 2001; Delice and
Ertugrul, 2007; Macadam, 2003; Pellecchia et al., 2005). The
ultimate and challenging goal of driver modeling is to obtain a
robust and reliable model that can eventually replace the human
driver. However, these models can more quickly find application
Corresponding author. Now with Department of Mechanical Engineering and
Materials Science, Rice University, MEMS MS-321, 6100 Main Street, Houston, TX
77005, USA. Tel.: +1713 348 2300; fax: +1713 348 5423.
E-mail addresses: celiko@rice.edu (O. Celik), seniz@itu.edu.tr (S. Ertugrul).
1
This manuscript is based on the M.Sc. Thesis of O. Celik prepared with
advisory of Dr. S. Ertugrul at Istanbul Technical University.
0952-1976/$ - see front matter & 2009 Elsevier Ltd. All rights reserved.
doi:10.1016/j.engappai.2009.08.007
areas as tools for implementing warning systems that would alert
the driver when it detects driver fatigue by keeping track of
several model parameters and comparing them with a nominal
model (Pilutti and Ulsoy, 1999; An and Harris, 1996; Ungoren and
Peng, 2005).
Another application for human operator models is utilizing
them for human skill transfer. If a successful model that
approximates and predicts human operator’s control behavior
well enough can be obtained, then a model of an expert operator
could be used as a teacher for novice operators by demonstrating
what the expert operator’s control inputs would be throughout
the control task. This approach has been applied by utilizing an
artificial neural network model in Nechyba and Xu (1995). More
examples of possible similar applications can be found in Nechyba
and Xu (1997). A different approach is presented in Vingerhoeds et
al. (1995) to the use of hybrid artificial intelligence.
In general, there are two main alternatives to obtain a dynamic
model; either based on knowledge or assumptions on the
dynamics of the system (dynamic modeling) or based on recorded
data and system identification techniques. It should be noted that,
although the term ‘‘modeling’’ usually refers only to the dynamic
modeling approach, throughout this paper it will be used in a
context that covers both dynamics-based and identification-based
methods and that all methods used in this study are identification-based.
Human behavior as a controller is generally quite nonlinear
(Sheridan and Ferrell, 1974). Human operators are able to gain
ARTICLE IN PRESS
596
O. Celik, S. Ertugrul / Engineering Applications of Artificial Intelligence 23 (2010) 595–603
experience and learn by repeating control tasks and improve their
behavior. Human operators can adjust themselves according to
the changes in the dynamics of the system that they control. These
aspects allow them to accomplish very different kinds of control
tasks while making the modeling more difficult compared to
other dynamic systems.
Although the control behavior of humans can be accepted as
quasi-linear under specific circumstances, these models require
the satisfaction of important and extensive limitations (Sheridan
and Ferrell, 1974; McRuer, 1980). In order to obtain a model that is
valid for a wide range of outputs of the plant under human control
or for plants that have nonlinear or nonstationary dynamics one
needs to use nonlinear modeling techniques.
Advancements in soft computing based identification and
modeling methods which includes artificial neural networks
(ANNs) and fuzzy logic (FL) triggered a more widespread use of
these methods in modeling of nonlinear dynamic systems
(Abonyi, 2003; Narendra and Parthasarathy, 1989). Hybrid
modeling schemes that take advantage of both ANNs and FL have
also been proposed in Jang (1993), Jang et al. (1997) and Nelles
(2001).
This study had two main purposes: (1) construction of an
experimental setup that allows implementation of previous
modeling attempts by Ertugrul that used computer based
experiments and simulations only (Ertugrul and Hizal, 2005;
Ertugrul, 2008), (2) investigating the possibility of utilizing local
ARX models by fuzzy transition modeling approach, called fuzzy
switching ARX (F-ARX) together with other well-known models
such as linear ARX and ANFIS using the constructed setup and
compare the performances in terms of prediction capability. Main
contributions of this study are the development and the
application of F-ARX modeling scheme to human operator
modeling and proposing an alternative for decomposition of the
operating region to use as the antecedents of the fuzzy inference
system in an F-ARX model.
In this paper, first, an experimental setup that has been
constructed for the purpose of conducting data acquisition
sessions with human operators is explained. The setup was used
to capture human operators’ control actions while they were in
charge of controlling a nonlinear system. The setup’s properties
are given in Section 2. The data acquisition protocol with five
subjects using this system is presented in Section 3.
Among three different methods, initially the linear system
identification techniques have been utilized to obtain AutoRegressive with eXogenous (ARX) inputs structured models. The
second modeling method made use of the developed ARX models
by means of a fuzzy inference system that allowed appropriate
transitions between linear ARX models (a fuzzy switching ARX or
F-ARX model). The last method was ANFIS, which produced a
fuzzy inference system model by training an initial model
structure using the data. These modeling techniques are covered
in Section 4. Results obtained by different models and observations on the setup after replacing the human operator by the
models are given in Section 6. Subsequently, findings and future
work are discussed.
2. Experimental setup
A flexible arm with dimensions of 2 mm 3 cm 50 cm is
attached to a 12 V brush DC motor with gear box forms the
mechanical part of the system to be controlled by a human
operator. The flexible arm moves in the horizontal plane only. The
bottom position of the arm is measured using a potentiometer.
The complete mechanical setup can be seen in Fig. 1. A similar
system was used by Sasaki et al. (1990) for similar purposes,
Fig. 1. Mechanical setup: the plant to be controlled by operators is a flexible arm
mounted to a DC gearmotor shaft. Weights are added on the tip to facilitate
oscillations.
however their system had a built-in closed-loop position
controller and operators only changed the reference input to the
system via a potentiometer. Our system differs in several aspects,
the most important one is that the human operator is the only
active controller in the closed loop in our setup.
Operator uses the x-axis of a PC joystick. Joystick command is
read into the computer and a proportional output is sent to DC
motor. Operators do not observe the plant directly while they are
controlling it; instead, they observe an online video of the system
provided by a color camera (Sony SSC-DC18P) assembled above
the plant. This video of the setup together with other components
of the operator interface is displayed on a 17" computer screen.
Operator uses only the x-axis of a standard PC joystick (Logitech
Extreme 3D Pro) to control the direction and the voltage to be
applied, hence approximately the speed of the motor. The joystick
is mounted on top of a stand so that operators can hold it while
their arms are on the armrest of the chair. The video is acquired at
720 576 pixels resolution and then cropped to a size of 493 300 pixels which corresponds to the area of interest. A white
semicircle that is slightly larger than the trace that the tip position
follows along the 03 21803 workspace is drawn on the video in
software together with a white ‘‘þ’’ sign that illustrates the
position of the rotation axis of the arm. The reference signal which
appears as a 53 interval in which the tip of the arm should be
positioned is also displayed on the video by two short yellow lines
at both ends of the intervals (see Fig. 2). The video is arranged
such that a left deflection on the joystick causes a counterclockwise rotation of the arm and a right deflection causes a
clockwise movement. A potentiometer for bottom position
measurement is also used but it is utilized just as a safety limit
switch on the two ends of the workspace. The acquired video is
processed to measure the tip point position using image
processing techniques. In order to improve the performance of
this algorithm, the background is covered with black fabric as well
as the base portion of the setup. The arm is marked with a 0.5 mm
wide orange strip and a 1 cm 1 cm blue marker is placed on the
tip. The weights on the tip are also covered with a cover that also
held the blue marker on it. In the tip position measurement
algorithm, first the blue layer is extracted and then the pixel with
maximum value (hence the bluest pixel) is determined. The
position is then converted to its degree equivalent using
trigonometric relations. The algorithm performed satisfactorily,
allowing all tasks to be run at 20 Hz loop rate.
A computer that had a P4 3.0 GHz processor and 1 GB RAM and
a PCI data acquisition card was used. All codes and interfaces were
prepared in LabVIEW 7 Express. IVision LabVIEW Toolkit was
utilized for the image gathering and processing portions of the
ARTICLE IN PRESS
O. Celik, S. Ertugrul / Engineering Applications of Artificial Intelligence 23 (2010) 595–603
597
Fig. 2. Operator interface. The camera display has black background, the arm is orange and its tip is marked with a blue square. The white semicircle and the ‘‘ þ ’’
representing the rotation axis of the arm are imposed on the gathered video by image processing. The reference interval that the operator should place the tip of the arm in
is demonstrated by two small yellow lines on the semicircle. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of
this article.)
program. More information on the setup and the interface can be
found in Celik (2006).
The task assigned to the operators is to control the tip position
of the arm so that the tip point follows the appearing references
within its working space of 1803 , and they should do this as fast as
possible and with minimum oscillations. Even tough flexible arm
control resembles a real world crane control system, compared to
a real crane, this system is faster and exhibits higher frequency
oscillations.
In order to emphasize the importance of the task goals, i.e.
positioning the tip of the arm in the reference interval quickly and
without too many oscillations, and reflecting these goals to
operators, a scoring scheme has been developed. According to this
scheme, a score counter is set to an integer value that is
proportional to the difference between the last reference position
and the new one each time a new reference position appears on
the screen. Then this score is decreased at a certain rate until the
tip of the arm is satisfactorily positioned in the interval. This
satisfaction criterion includes a threshold that the position error
between the midpoint of the interval and the tip of the arm should
stay below a specific value for a specific time. This threshold
ensures that the oscillations are almost completely damped.
Successful completion of a positioning task for a reference is
indicated to the operator by a virtual LED above the video window.
The score counter, the video window, total score indicator and the
‘‘target achieved LED’’ which constitute the operator interface are
shown in Fig. 2. All signals are recorded during data acquisition
sessions at 20 Hz sampling frequency.
There are several nonlinear effects in the plant. The proportional voltage created by the joystick deflection generates a dead
zone in the control loop, i.e. there is a range of control inputs on
both directions that does not make the motor move. Once this
threshold is exceeded, the speed of the motor is approximately
proportional to the joystick deflection. Another much less
pronounced nonlinearity stems from the backlash in the gearbox
of the motor. This backlash causes a play that significantly
changes the tip position of the arm when it oscillates. A friction
element is placed between the arm and the base to keep the
bottom position of the arm fixed during oscillations. But the
backlash is still in effect when a change in movement direction
occurs. These nonlinearities are left as is, since real world plants
controlled by HOs also contain various nonlinearities and they are
important from the perspective that they induce the nonlinear
control behavior.
3. Data acquisition sessions
Data for modeling purposes is collected from five operators, all
male graduate students. Their ages varied between 25 and 28 and
they were all computer users and had used a joystick before.
At each session, operators completed five different 3-min
length scenarios, namely 153, 303 , 453 , 603 , and random amplitude
(R) scenarios. In a 153 scenario, reference positions (the midpoints
of the reference intervals) appeared on 153 to the left or 153 to the
right of the center (zero) position which was the complete vertical
position of the arm on the screen. The other scenarios were
similar, except that in the R scenario, the position of the reference
signal was determined randomly between the 7 603 limits. The
duration between two successive reference positions was random
in all scenarios but the minimum was set to 2 sec since the task
becomes unrealistic or too challenging for reference inputs that
change more frequently than at every 2 sec. This lead to a
restriction on the obtained models: they are valid only for inputs
that have frequency content up to 0.5 Hz. In Fig. 3 a portion of
reference position and arm’s tip positions recorded in an R
scenario of Operator 3 is shown.
Each operator completed all scenarios in the mentioned order
above in each 15-min long session, and was required to complete
10 sessions in total. A specific schedule was not followed; all
operators completed the sessions in less than three weeks.
4. Modeling methods
Three different model structures have been used: linear system
identification techniques with an ARX model structure, a fuzzy
switching algorithm that provides switching among different
linear ARX models and ANFIS.
ARTICLE IN PRESS
598
O. Celik, S. Ertugrul / Engineering Applications of Artificial Intelligence 23 (2010) 595–603
1
60
0.9
0.8
0.7
20
Coherence
Position (degrees)
40
0
−20
0.6
0.5
0.4
0.3
−40
Reference position
Displayed reference interval
Arm’s tip position
−60
0.2
0.1
0
10
20
30
40
50
60
Time (sec)
Fig. 3. Example of a reference signal and the tip position of the arm while the
setup was being controlled by Operator 3 in an R scenario.
For all models, input and output signals are determined to be
the position error between the reference position and the current
position of the tip of the arm and the voltage applied to the motor
(proportional to the deflection of the joystick), respectively. The
outputs are scaled to the interval ½1; 1 while the inputs are
scaled by a factor of 0.1, producing a comparable range for the
input–output pair.
The reference signals for the first four scenarios are pseudorandom binary sequences (PRBS) with a lowest possible duration
limit of 2 sec (Soderstrom and Stoica, 1989). However, the
assumed input to the model is different and the order of
persistence excitation provided by the generated input signals
should be confirmed. For this purpose, the covariance matrices of
the inputs for a sample from each scenarios are calculated and it
was found that the input signals in all scenarios were persistently
exciting of order at least 10, allowing models of orders less than or
equal to 10 to be estimated (Soderstrom and Stoica, 1989).
For validating that a linear model could represent the HO
reasonably well for the ARX model case, the linearity in HOs
control actions was tested using coherence analysis (Bendat and
Piersol, 1986). Coherence analysis provides the frequency range
within which a linear model structure assumption would be valid.
For data belonging to different operators and scenarios, it was
found that the coherency between the input and the output
always remained above 0.6 along the 0–0.5 Hz range. This lowest
value varied between 0.6 and 0.8 for different data sets, implying
that an almost linear relationship existed between input and
output across this portion of the frequency spectrum. An example
coherence plot where the minimum coherency value within the
specified frequency range was 0.8 is given in Fig. 4.
4.1. ARX models
According to the definition of quasi-linear human operator
model given in Sheridan and Ferrell (1974), a model that is
composed of a linear differential equation and an error term in the
form of random noise should be sufficient to describe the
dynamics of a human operator. The structure of the ARX model,
as given in Eq. (1), is composed of a linear differential equation
and error term, besides it has fewer parameters to be determined
than ARMAX. Therefore, ARX model structure is chosen instead of
ARMAX, ARMA, OE, or other structures as it showed the best
prediction capability after estimating parameters for models of
0
10−1
100
Frequency (Hz)
101
Fig. 4. Example coherence plot for the input–output pair. For this specific data set,
coherency is equal to or greater than 0.8 for the frequency range of 0–0.5 Hz.
Testing with additional data sets revealed that coherency remained above 0.6–0.8
across the frequency range of 0–0.5 Hz for all scenarios and operators.
different orders and also due to its simpler structure. In the ARX
model structure, the output of the system at a specific time is
assumed to be linear combinations of the previous outputs and
inputs and the current input. A discrete-time designation of the
ARX model is
yðtÞ þ a1 yðt 1Þ þ a2 yðt 2Þ þ þ ana yðt naÞ
¼ b1 uðt nkÞ þ b2 uðt nk 1Þ þ þ bnb uðt nk nb þ 1Þ þ eðtÞ
ð1Þ
where t represents integer time steps, eðtÞ is the modeling error, y
is the output, u is the input, ai ’s and bj ’s are model parameters to
be estimated using the data, and na, nb and nk are the orders of
the output, input and input–output delay, respectively (Ljung,
1999). na is the model order. Least squares estimation (LSE)
algorithm is used for determining the model parameters (Soderstrom and Stoica, 1989). Using this structure, estimations are
carried out and a parameter set is obtained for each scenario in
each session for all operators. Then these values are averaged
across the sessions to obtain a set of average parameters for each
scenario of each operator.
Trials with different model order values revealed that models
with orders na ¼ 2 and nb ¼ 1 produced reasonably well fits to the
data and further increase in order did not improve the results
significantly. As for the order of the delay, generally an order of 6
or 7 gave the best results, which corresponded to 300 and 350 ms,
noting that the sampling time is 50 ms. Since it is known that the
reaction times for human motor actions varies between 200 and
300 ms, the order of the delay was found to be in good correlation
with the literature (Sheridan and Ferrell, 1974). The delay order nk
is selected to be 6 for all ARX models. This structure will be called
ARX216 in short hereafter.
4.2. Fuzzy-ARX models
The fuzzy-ARX (F-ARX) model represents a usual ARX structure
whose model parameters are updated based on a fuzzy inference
system, as illustrated in Fig. 5(a). This fuzzy transition scheme
provides an interpolation and smooth transition between an
infinite number of different local linear ARX models. The resulting
model is a nonlinear ARX (NARX) model that can be considered as
ARTICLE IN PRESS
O. Celik, S. Ertugrul / Engineering Applications of Artificial Intelligence 23 (2010) 595–603
599
a
|Δr(t)|
u(t)
Fuzzy
System
q-nk
y(t)
ARX
q-nk-1
q-1
q-nb-nk+1
q-2
q-na
b
Output membership
functions
Input membership
functions
|Δr|
Rule
Base
ARX model
parameters
Defuzzification
a1
Defuzzification
a2
Defuzzification
b1
Fig. 5. (a) F-ARX model structure. Fuzzy system updates the ARX model parameters based on the absolute reference value changes. (b) Schematic structure of the fuzzy
system. Input is the absolute change in the reference and outputs are the ARX model parameters.
a parameter varying model as well. The outputs of the fuzzy
inference system are model parameters, as stated, while for the
input(s) there are several possible choices: previous inputs/
outputs of the model, reference signal, or different combinations
of these or their variations. This model structure is in resemblance
to a fuzzy gain scheduling method (Zhao et al., 1993), however
here the fuzzy transition is between local linear models as
opposed to controller gains.
There are several applications of fuzzy gain scheduling to
mimic or attain human-like control behavior in the literature.
Fuzzy gain scheduling was used to achieve smooth automatic
lane-keeping of a vehicle in Wu et al. (2005) as well as the lateral
control of the vehicle (Chiang et al., 2006). It was also applied on a
car-like robot to obtain automated parallel parking (Chiu et al.,
2005). In all of these studies fuzzy gain scheduling was used as
part of tuning the parameters of the designed automatic
controllers. Here we propose to use fuzzy transitions as part of a
human operator model, which then can be used as a controller in
closed-loop fashion.
In Abonyi et al. (1999) and Abonyi and Babuska (2000), inputs
of a fuzzy system model were selected to be a subset of the
previous outputs and inputs of the system to be modeled, as
in regular Takagi–Sugeno type fuzzy modeling of dynamic
systems (Takagi and Sugeno, 1985). Thus the inputs are a subset
of fyðt 1Þ; yðt 2Þ þ þyðt naÞ; uðt nkÞ; uðt nk 1Þ þ þ
uðt nk nb þ1Þg, following the same notation in Section 4.1
(Abonyi et al., 1999). This selection leads to a direct fuzzy model of
the system that interpolates between local ARX models. However,
the identification scheme is not based on establishing individual
local ARX models by separate parameter estimations for predefined working regions. It is based on either a global least
squares estimation (LSE) or a local LSE that divides the problem
into weighted LSE problems.
A similar approach to the one in this study was given by
Johansen (1994). Johansen defined the problem on a multipleinput multiple-output (MIMO) NARX model, stated that the
modeling problem could be decomposed into local models in a
set of operating regimes. Remarks on how to choose operating
regions were made and it was mentioned that there were various
alternatives based on system’s characteristics.
In Hadjili and Wertz (2002), a fuzzy modeling method that is
used to identify the IF- and THEN-parts of a Takagi–Sugeno model
separately is proposed and explained. In the IF part identification,
again the previous input and output data were utilized, however a
systematic way of eliminating the components that are irrelevant
by a fuzzy clustering algorithm was developed.
Differing from the mentioned studies, in this study, the input
for the fuzzy system is selected to be the absolute value of the
change in the reference signal, as shown in Fig. 5(a). This selection
provides a constant model between two consecutive reference
positions, hence decreasing the computational load in implementation. Also, utilization of the reference signal which is always
ARTICLE IN PRESS
600
O. Celik, S. Ertugrul / Engineering Applications of Artificial Intelligence 23 (2010) 595–603
available increases the applicability of this modeling method to
different systems. One of the rules for F-ARX model can be given
as
3
IF the absolute change in Reference is 15
THEN a1 ¼ X1 ;
a2 ¼ X2 ;
a3 ¼ X3
ð2Þ
For the F-ARX model structure of concern in this study, a fuzzy
inference system is designed with four input membership
functions (two trapezoidal and two triangular) that correspond
to each reference input case, and four output membership
functions (fuzzy singletons) that are equal to the average model
parameters obtained in the previous ARX modeling step. A
schematic of this fuzzy system is shown in Fig. 5(b).
Hence, a two step modeling procedure was followed for the FARX modeling method in this study, similar to the decoupled IFand THEN-part identification method in Hadjili and Wertz (2002).
First the local linear ARX models were identified based on each
scenario, then a fuzzy inference system were designed to establish
the transitions between the local models.
4.3. ANFIS models
A problem that arises during pure fuzzy modeling is the
extensive number of free parameters and number of rules that
should be adjusted by trial and error (Zapata et al., 1999). An
approach that utilized ANNs for solving this problem has been
proposed by Jang (1993). In this hybrid modeling method called
ANFIS, an initial fuzzy inference system is manipulated and
adapted by training with the data collected from the system to be
modeled. Initial system is a first order Takagi–Sugeno fuzzy
system. The membership functions get adjusted through the
training by gradient descent in backward passes while the
coefficients of the first order output polynomial are tuned by
least squares method. When the training is complete, the final
model is obtained as a stand-alone fuzzy inference system.
Further information on ANFIS can be found in Jang (1993) and
Jang et al. (1997). ANFIS modeling scheme was applied for
modeling HO actions in Ertugrul and Hizal (2005), Ertugrul
(2008) and found to give successful results.
The ANFIS model structure used in this study was selected to
be equivalent to the ARX216 in order to allow fair comparison in
terms of prediction performance. Same input–output pair, namely,
the position error between the reference position and the tip
position of the arm and the control applied by the HO were used
in this modeling method. Thus the inputs of the ANFIS model were
^
fyðt 1Þ; yðt 2Þ; uðt 6Þg and its output was yðtÞ.
Gaussian
membership functions (MFs) given in the literature as
2
mðx; s; cÞ ¼ eðxcÞ
=2s2
ð3Þ
were used in this study. Number of MFs to be used for each input
is another parameter that should be chosen for an ANFIS model.
The number of rules that will be trained is given by mn, where m
denotes the number of MFs for each input and n denotes the
number of inputs. Consequently, although increasing the number
of MFs leads to a better approximation, training time will increase
significantly. In this study, initial models with two and three MFs
were used, but the model with three MFs was chosen due to its
slightly better performance. The training process was limited by
10 epochs.
There are many parameters that should be determined for an
ANFIS model and arriving at the optimal combination can be time
consuming. Here only the structure that is equivalent to the
ARX216 model was considered. A wide set of ANFIS models with
different combinations were tested with the purpose of HO
modeling in Ertugrul (2008).
5. Prediction performance of models
Models obtained by different methods are compared in terms
of their success in predicting HOs’ actions for varying prediction
horizons. If Y t ¼ fyðtÞ; yðt 1Þ; . . .g represents the information
available at time t, the equation
^ þ kjtÞ þ a1 yðt
^ þ k 1jtÞ þ þana yðt
^ þ k najtÞ ¼ b1 uðt þ k nkÞ
yðt
þ b2 uðt þ k nk 1Þ þ þ bnb uðt þ k nk nb þ1Þ
ð4Þ
can be used to simulate and predict the output of the model at
^ þ kjtÞ represents
time t þ k (Soderstrom and Stoica, 1989). Here yðt
the value of the simulated output at time t þ k, in which the
information in Y t is used. It should be noted that
(
^ þ ijtÞ if i 40
yðt
^yðt þ ijtÞ ¼
ð5Þ
yðt þ iÞ
if i r0
During simulations, all initial conditions are assigned as zero.
Prediction performance of the models is evaluated using the
formulation
Jy yr J
ð6Þ
Percent fit ¼ 100 1 m
Jyr y r J
where ym is the model’s output vector, yr is the real output vector
(recorded from the HO), and y r is the mean value of the real
output vector (Ljung, 1999). All models are evaluated for their one,
five and infinite step prediction performances. Infinite step
prediction is equivalent to pure simulation of the model with
zero initial conditions.
6. Results
Mean and standard deviation values for 10 sessions of model
parameters in the ARX216 model for all scenarios are given for
Operators 1 and 2 in Table 1. The table is limited to only two
operators’ data, since it has been observed that both mean and
standard deviation values of other operators did not show major
difference from the data set presented here. As it is apparent from
the table, parameters estimated using different operators’ data
were very similar. Also, relatively small values of the standard
deviations imply that the estimated parameters did not vary
considerably throughout 10 sessions. Furthermore, although b1
exhibits some variability for different scenarios, parameters did
not vary significantly with respect to the scenarios. All these
observations lead us to consider the possibility of obtaining a
single model with constant parameters suitable for all cases.
Table 1
Means and standard deviations (m 7 s) of ARX216 model parameters of Operators
1 and 2 for 10 sessions.
Operator 1
Scenario
3
15
303
453
603
R
Operator 2
Scenario
3
15
303
453
603
R
a1
a2
b1
1:583 7 0:053
1:573 7 0:059
1:571 7 0:045
1:559 7 0:047
1:564 7 0:061
0:679 7 0:048
0:660 7 0:048
0:640 7 0:039
0:618 7 0:045
0:640 7 0:056
0:01554 7 0:00176
0:01125 7 0:00299
0:00730 7 0:00182
0:00531 7 0:00085
0:00905 7 0:00127
a1
a2
b1
1:629 7 0:030
1:584 7 0:028
1:544 7 0:034
1:516 7 0:031
1:599 7 0:027
0:729 7 0:025
0:686 7 0:018
0:637 7 0:023
0:596 7 0:023
0:685 7 0:020
0:01792 7 0:00178
0:01457 7 0:00210
0:01046 7 0:00143
0:00769 7 0:00091
0:01101 7 0:00114
ARTICLE IN PRESS
O. Celik, S. Ertugrul / Engineering Applications of Artificial Intelligence 23 (2010) 595–603
Table 2
Prediction performance for ARX models obtained from different scenarios and
simulated on a randomly selected (sixth session) 153 scenario data of Operator 2.
Scenario from
which the
parameters
are estimated
153
303
453
603
R
Parameters
Prediction performance (%)
a1
a2
b1
One
step
Five
step
Infinite step
1.629
1.584
1.544
1.516
1.599
0.729
0.686
0.637
0.596
0.685
0.01792
0.01457
0.01046
0.00769
0.01101
89.4
89.1
88.8
88.5
89.0
60.8
57.0
52.5
49.9
55.5
55.5
49.7
40.5
33.0
45.3
Table 3
Prediction performance of the model derived from the R scenario data of Operator
2 (a1 ¼ 1:599; a2 ¼ 0:685; b1 ¼ 0:01101), simulated with the R scenario data of
the last session of each operator.
Operator data
1
2
3
4
5
Prediction performance (%)
One step
Five step
Infinite step
88.2
88.7
93.3
89.8
90.8
64.2
63.5
67.7
63.7
61.7
58.5
59.9
56.9
55.3
50.7
A single global model is highly desirable since it facilitates the
utilization and implementation of the obtained model for control
or prediction purposes, as in a model predictive controller.
The feasibility of a single constant model is further investigated by simulating different models (models based on different
scenarios) on a 153 scenario data recorded from Operator 2.
Operator’s recorded response and the predicted response by the
model for different horizons are used to calculate the prediction
performance values (see Section 5). Table 2 summarizes the
results of the described procedure for Operator 2. Estimations
based on R scenario data provide an average model, since these
estimations are based on the data calculated from random
amplitude reference scenario. All models perform equally well
based on their one step prediction performance. However for five
step and especially for infinite step, prediction performances
become considerably different and the average model is not
satisfactory.
In Table 2, it is apparent that small changes in model
parameters has a significant impact on the infinite step prediction
performance. Although the model obtained from the random
scenario or the average model that has average coefficients
calculated from the first four scenarios are successful to some
extent, approximately a 10% decrease in performance occurs for
infinite step predictions compared to the model derived from 153
scenarios only. These results motivated utilization of an F-ARX
model structure that could switch to the best set of parameters for
an approaching reference signal that should be followed.
In the case of generalization capability of a single model for
different operators, the average model derived from the R scenario
data of Operator 2 has reasonably well results for the R scenario
section of the last sessions of all operators, as summarized in
Table 3. The reason for selecting Operator 2 for a base model is
that he is assumed to be one of the two most expert operators by
outperforming others in terms of average total points, hence the
601
Table 4
Prediction performance of the F-ARX model based on Operator 2’s ARX216 models,
simulated with the random scenario data of the last session of each operator.
Operator data
1
2
3
4
5
Prediction performance (%)
One step
Five step
Infinite step
88.0
88.9
93.3
90.0
90.7
64.4
65.9
69.2
65.8
60.5
60.7
63.6
61.4
59.9
51.6
most successful controller for the defined control task. A onetailed paired t-test for mean total points across 10 sessions
revealed that Operators 1 and 2 performed significantly better
than all other operators [p o0:05] and the performance of
Operators 1 and 2 was not significantly different [tð9Þ ¼
0:025; p ¼ 0:490].
Since five step and infinite step predictions using a single
constant ARX model were not satisfactory, an F-ARX model was
obtained as described in Section 4.2, again using Operator 2’s ARX
models. The prediction capacity of this model is evaluated and
given in Table 4.
The prediction results for only the R scenarios of each
operator’s last session is considered descriptive enough, since
this scenario included random amplitude references throughout
the whole range. The prediction capacity of a single average model
given for the same set of data in Table 3 allows a comparison.
Comparing the results in Tables 3 and 4 reveals that although the
F-ARX model generally performs better in infinite step predictions, the difference between the performances of two models
remain around 1–4%. A one-tailed paired t-test on the mean
difference between the infinite step prediction performance, the
F-ARX model performed significantly better than the single
average ARX model [tð4Þ ¼ 4:455; p o0:05]. Results in Table 4
offer that the F-ARX model carries the ability to produce equally
well results for different scenarios and operators. To further
illustrate this, the same data set used to produce Table 2 was used
to evaluate the F-ARX model. F-ARX model demonstrated 89.4%,
59.8% and 53.7% fit values for one step, five step and infinite step
prediction horizons, respectively. Especially an increase in fit
around 8% in infinite step simulations can be accepted as a
considerable gain compared to the single constant average
ARX216 model’s performance.
The prediction performance of the ANFIS model for different
operators’ random scenario data in their last session is
given in Table 5. It is observed that the ANFIS model was not as
successful as other models. A single-tailed paired t-test indicated
that the infinite step prediction performance of the ANFIS
model was significantly lower than both the average
ARX model [tð4Þ ¼ 3:640; p o0:05] and the F-ARX model
[tð4Þ ¼ 4:525; p o 0:05].
The ANFIS model based on Operator 2’s random scenario data
in the last session demonstrated problems in the simulation. An
output that alternated between values well above or below the
upper and the lower limits (1 and 1) in a very fast manner for
short intervals of time was occasionally produced by the model.
When the same model structure is trained by using Operator 1’s
data (the other most expert operator), this problem occurred less
frequently but it did not completely vanish. Due to this aspect, the
ANFIS model evaluated in Table 5 is based on Operator 1’s data.
Also, the output of the model was saturated with the thresholds 1
and 1 in the simulations in order to limit the percent fit error
values for fair comparison and later to implement the model as
the active controller on the setup.
ARTICLE IN PRESS
602
O. Celik, S. Ertugrul / Engineering Applications of Artificial Intelligence 23 (2010) 595–603
ARX216 Model
Table 5
Prediction performance of the ANFIS model based on Operator 1’s last session
random scenario data, simulated with the random scenario data of the last session
of each operator.
HO and Model Outputs (normalized to [−1,1])
1
2
3
4
5
F−ARX Model
Reference position
Reference interval
Tip position
40
Prediction performance (%)
One step
Five step
Infinite step
85.7
86.4
91.4
84.5
87.2
66.1
60.3
65.4
58.1
60.1
55.7
52.3
53.7
51.4
49.3
Position (degrees)
Operator data
ANFIS Model
60
20
0
−20
−40
1
−60
0.5
0
10
20
30
40
50
60
Time (sec)
0
Fig. 7. Behavior of the setup in closed-loop control, with the developed HO models
integrated as the controller.
−0.5
Original HO output
Constant ARX216 model’s output
F−ARX model’s output
ANFIS model’s output
−1
0
5
10
15
Time (sec)
Fig. 6. HOs (Operator 3) output for the random scenario in the last session and the
simulation results of the three models. Constant ARX216 model uses the
parameters given in Table 3.
To better illustrate the imitating capabilities of the different
models, the simulations with zero initial conditions of all three
models are given together with the recorded output of a human
operator in Fig. 6. Data belong to the random scenario section of
the last session of Operator 3. Although the infinite step
prediction performances of the models remain around 50–60%,
it is seen in this figure that all models effectively reflect the basic
aspects of the actions taken by the HO. F-ARX outputs are
generally the closest ones to the HOs, followed by the constant
ARX model whereas ANFIS model’s outputs are relatively less
representative of the original signal. These observations are in
agreement with the results summarized in Tables 3–5.
ANFIS is a promising method for HO modeling as reported in
Ertugrul (2008). The relatively low performance of ANFIS model in
the application presented here may be due to the following: (1)
The data set used to train the model was not sufficiently rich. A
rich and representative training data set is an important step for
obtaining successful ANFIS models. (2) Imposing the model
structure ARX216 on the ANFIS model probably prevented the
best possible model to be obtained. The fast alternating outputs
problem was a major factor that affected the prediction performance of this type of model. This problem can potentially be
eliminated if the two limitations above were overcome and it is
decided to be addressed in a future study.
7. Implementation of the models to replace HO as controller
The obtained models were implemented as controllers in
closed loop to replace the HO. While ARX and F-ARX models were
successful in replacing the HO, ANFIS model displayed the
problem mentioned above. All models were occasionally able to
position the tip of the arm in the interval without inducing many
oscillations, while the expert operators could almost always do.
In Fig. 7, closed-loop behavior of the setup under the control of
the implemented models is given. The 20 sec portions randomly
selected from an R scenario test run are given for each model.
None of the models could perform precisely as well as a HO, in the
context of the control task defined in Section 2.
All models suffered from a common problem. For small
reference value changes, i.e. when a new reference position
appeared that was close to the last reference position, the models
did not respond. The position error at these instances was not
large enough to make the controller produce an output that could
move the motor. This property can be observed in Fig. 7, at the
ends of the 20 sec portions for the ARX216 and the F-ARX model.
ANFIS model seemed to suffer less from this issue, as it is
demonstrated in Fig. 7.
Apparently, this problem originated from the dead zone in the
DC motor, which was mentioned in the Section 2. This hard
nonlinearity in the setup found its reciprocal in the HO; HOs
learned the thresholds of this dead zone and responded by just
merely exceeding the limits of the zone when fine movements or
corrections were needed. A nonlinear element that resembles this
behavior of the HO can be explicitly introduced in the models to
overcome this problem as a future extension of this study,
following a similar approach that was used in Chen and Ulsoy
(2000).
8. Conclusion
An experimental setup that allowed the implementation of a
human operator control task has been used in this study. The
setup is used to collect data from five operators that performed a
defined task for 10 sessions each being 15 min long. Each session
also included five different scenarios which were later used to
implement different local and global models. Modeling methods
included well-known linear identification techniques with an ARX
model structure for local models, a fuzzy switching model that
provided smooth transition between appropriate local ARX
models (the F-ARX model), and the adaptive network-based fuzzy
ARTICLE IN PRESS
O. Celik, S. Ertugrul / Engineering Applications of Artificial Intelligence 23 (2010) 595–603
inference system (ANFIS) model. Models were evaluated and
compared based on their prediction performances as well as their
performance after being implemented on the system as a
replacement for the HO. Prediction performances of all models
were found to be relatively successful as reported in Tables 3–5.
Developed F-ARX models performed better than both ARX and
ANFIS models. The nonlinearity in the HOs behavior that was
capable of overcoming the dead zone in the plant was not
captured by any of the models, which is a direction for future
work. Another important direction for future work is the on-line
training of F-ARX model and taking over the control from HO at
any desired moment during real-time operation.
References
Abonyi, J., 2003. Fuzzy Model Identification for Control. Birkhauser, Basel.
Abonyi, J., Babuska, R., 2000. Local and global identification and interpretation of
parameters in Takagi–Sugeno fuzzy models. In: Proceedings of the IEEE
International Conference on Fuzzy Systems, San Antonio, TX, USA, May, pp.
835–840.
Abonyi, J., Babuska, R., Setnes, M., Verbruggen, H.B., Szeifert, F., 1999. Constrained
parameter estimation in fuzzy modeling. In: Proceedings of the IEEE
International Conference on Fuzzy Systems, Seoul, Korea, August, pp. 951–956.
An, P.E., Harris, C.J., 1996. An intelligent driver warning system for vehicle collision
avoidance. IEEE Transactions on Systems, Man and Cybernetics, Part A 26 (2),
254–261.
Bendat, J.S., Piersol, A.G., 1986. Random Data: Analysis and Measurement
Procedures. Wiley, New York.
Celik, O., 2006. Intelligent modeling of human operators. Master’s Thesis, Istanbul
Technical University, Istanbul (in Turkish).
Chen, L., Ulsoy, A.G., 2000. Identification of a nonlinear driver model via NARMAX
modeling. In: Proceedings of the American Control Conference, vol. 4, Chicago,
IL, June, pp. 2533–2537.
Chiang, H.H., Ma, L.S., Perng, J.W., Wu, B.F., Lee, T.T., 2006. Longitudinal and lateral
fuzzy control systems design for intelligent vehicles. In: Proceedings of the
2006 IEEE International Conference on Networking, Sensing and Control, 2006,
ICNSC’06, pp. 544–549.
Chiu, C.S., Lian, K.Y., Liu, P., 2005. Fuzzy gain scheduling for parallel parking a carlike robot. IEEE Transactions on Control Systems Technology 13 (6), 1084–1092.
Delice, I.I., Ertugrul, S., 2007. Intelligent modeling of human driver: a survey. In:
2007 IEEE Intelligent Vehicles Symposium, pp. 648–651.
Ertugrul, S., 2008. Predictive modeling of human operators using parametric and
neuro-fuzzy models by means of computer-based identification experiment.
Engineering Applications of Artificial Intelligence 21 (2), 259–268.
Ertugrul, S., Hizal, N.A., 2005. Neuro-fuzzy controller design via modeling human
operator actions. Journal of Intelligent and Fuzzy Systems 16 (2), 133–140.
Hadjili, M.L., Wertz, V., 2002. Takagi–Sugeno fuzzy modeling incorporating input
variables selection. IEEE Transactions on Fuzzy Systems 10 (6), 728–742.
Hess, R.A., Modjtahedzadeh, A., 1990. A control theoretic model of driver steering
behavior. IEEE Control Systems Magazine 10 (5), 3–8.
Jang, J.S.R., 1993. ANFIS: adaptive-network-based fuzzy inference system. IEEE
Transactions on Systems, Man and Cybernetics 23 (3), 665–685.
Jang, J.S.R., Sun, C.T., Mizutani, E., 1997. Neuro-Fuzzy and Soft Computing. PrenticeHall, Upper Saddle River, NJ, USA.
Johansen, T.A., 1994. Fuzzy model based control: stability, robustness, and
performance issues. IEEE Transactions on Fuzzy Systems 2 (3), 221–234.
Ljung, L., 1999. System Identification: Theory for the User. Prentice-Hall, Upper
Saddle River, NJ, USA.
Macadam, C.C., 2003. Understanding and modeling the human driver. Vehicle
System Dynamics 40 (1), 101–134.
603
McRuer, D., 1980. Human dynamics in man–machine systems. Automatica 16, 237–
253.
Narendra, K.S., Parthasarathy, K., 1989. Adaptive identification and control of
dynamical systems using neural networks. In: Proceedings of the IEEE
Conference on Decision and Control, pp. 1737–1738.
Nechyba, M., Xu, Y., 1995. Human skill transfer: neural networks as learners and
teachers. In: Proceedings of the IEEE International Conference on Intelligent
Robots and Systems, vol. 3, pp. 314–319.
Nechyba, M., Xu, Y., 1997. Human control strategy: abstraction, verification, and
replication. IEEE Control Systems Magazine 17 (5), 48–61.
Nelles, O., 2001. Nonlinear System Identification. Springer, Germany.
Pellecchia, A., Igel, C., Edelbrunner, J., Schoner, G., 2005. Making driver modeling
attractive. IEEE Intelligent Systems 20 (2), 8–12.
Pilutti, T., Ulsoy, A.G., 1999. Identification of driver state for lane-keeping tasks.
IEEE Transactions on Systems, Man and Cybernetics, Part A 29 (5), 486–
502.
Prokop, G., 2001. Modeling human vehicle driving by model predictive online
optimization. Vehicle System Dynamics 35 (1), 19–53.
Sasaki, M., Inooka, H., Ishikura, T., 1990. Manual control of a flexible arm and its
application to an automatic control system. IEEE Transactions on Systems, Man
and Cybernetics 20 (3), 718–722.
Sheridan, T.B., Ferrell, W.R., 1974. Man–Machine Systems. MIT Press, Cambridge,
MA.
Soderstrom, T., Stoica, P., 1989. System Identification. Prentice-Hall, Hertfordshire,
UK.
Takagi, T., Sugeno, M., 1985. Fuzzy identification of systems and its applications to
modeling and control. IEEE Transactions on Systems, Man, and Cybernetics 15,
116–132.
Ungoren, A.Y., Peng, H., 2005. An adaptive lateral preview driver model. Vehicle
System Dynamics 43 (4), 245–259.
Vingerhoeds, R.A., Janssens, P., Netten, B.D., Aznar Fernandez-Montesinos, M., 1995.
Enhancing off-line and on-line condition monitoring and fault diagnosis.
Control Engineering Practice 3 (11), 1515–1528.
Wu, S.J., Chiang, H.H., Perng, J.W., Lee, T.T., Chen, C.J., 2005. The automated lanekeeping design for an intelligent vehicle. In: IEEE Intelligent Vehicles
Symposium Proceedings, 2005, pp. 508–513.
Zapata, G.O.A., Kawakami, R., Galvao, H., Yoneyama, T., 1999. Extracting fuzzy
control rules from experimental human operator data. IEEE Transactions on
Systems, Man and Cybernetics, Part B 29 (3), 398–406.
Zhao, Z.Y., Tomizuka, M., Isaka, S., 1993. Fuzzy gain scheduling of PID controllers.
IEEE Transactions on Systems, Man and Cybernetics 23 (5), 1392–1398.
Ozkan Celik was born in Sivas, Turkey. He earned his B.Sc. and M.Sc degrees in
Mechanical Engineering in 2004 and 2006, respectively, both at Istanbul Technical
University. He was the recipient of TUBITAK (The Scientific and Technological
Research Council of Turkey) fellowship for M.Sc. students during his M.Sc. studies.
He is currently pursuing Ph.D. studies at Rice University and is a research assistant
at the Mechatronics and Haptic Interfaces (MAHI) Lab. His current research
interests are human motor control system, motor adaptation and learning, robotic
rehabilitation, mechatronics and haptics.
Seniz Ertugrul was born in Bursa, Turkey. She graduated from Istanbul Technical
University Mechanical Engineering Department with her B.Sc. and M.Sc. degrees in
1988 and 1992, respectively. She earned her Ph.D. degree at the Wichita State
University, KS, USA in 1996. She worked as a lecturer at WSU and as a visiting
researcher at the University of Michigan. She is currently an associate professor at
the Istanbul Technical University, Mechanical Engineering Department. Her areas
of interest are human operator modeling, mechatronics and intelligent systems.