Sprague & Geers. The Sprague & Geers metric (S&G) is a simple integral
comparison metric that includes both magnitude and phase errors, which are calculated
independently and then combined to give a comprehensive error (6). The error
calculation can be biased towards either the measured time history or the computed
(simulation) time history, with the former being most common. Given two time histories
of equal length, measured m(t) and computed c(t), the following time integrals are
defined:
t2
I mm  (t 2  t1 ) 1  m 2 (t )dt
[3]
t1
I cc  (t 2  t1 )
1
t2
c
2
[4]
(t )dt
t1
I mc  (t 2  t1 )
1
t2
 m(t )  c(t )dt
[5]
t1
The magnitude error is then defined as:
M SG  I cc / I mm  1
[6]
The phase error is defined as:
PSG 
1

cos 1 ( I mc / I mm I cc )
[7]
The comprehensive error is defined as:
2
2
CSG  M SG
 PSG
[8]
Due to the relative simplicity of the error metric, it can be implemented into a
spreadsheet program with little loss of accuracy (7). The integrals can be approximated
by summations using the trapezoidal method.
b
ba N
[9]
f
(
t
)
dt

[ f (ti )  f (ti1 )]
a
2 N i 1
ba
where N is the number of intervals such that t 
. Because equations [6] and [7]
N
use ratios of the integrals, the coefficients cancel leaving, for example;
N
I cc

I mm
 [c(t )  c(t
i 1
N
i
i 1
 [m(t )  m(t
i 1
i
)] 2
[10]
i 1
)]
2
An advantage of the Sprague & Geers metric is the independent treatment of
magnitude and phase errors, i.e. the magnitude component is insensitive to phase errors
and vice versa. This allows for an accurate representation of magnitude only and phase
only errors. Additionally, this independence can help in determining the cause of an
error.
Weighted Integrated Factor. The Weighted Integrated Factor method (WIFac)
has been proposed in the automotive literature, specifically for ATD model evaluation
(4). This metric is associative and calculates the agreement between two signals. This is
advantageous in situations where the experimental data is known to have significant
error. Shown in equation [11], WIFac is weighted by the absolute maximum of the
function values. This weighting has the effect of scaling local differences to increase the
effect of errors at high magnitude function values (4). A modified version of this metric
is examined in this paper, where the one minus portion has been removed to provide a
computation of error instead of agreement. This metric can also be implemented in a
spreadsheet program with the inclusion of a small delta function in the denominator in
order to avoid divide by zero errors. Below is new WIFac (1-max/max previously did not
have squared functions in the bottom max).
2
 max( 0, f (t )  g (t )) 
 max f (t ) , g (t )  1  max( f (t ) 2 , g (t ) 2 )  dt
Crit  1 
2
2
 max f (t ) , g (t ) dt

2
2



[11]
In addition to the philosophical reasons for the use of an associative metric, there
are also practical purposes. When utilizing an optimization routine to determine the ideal
model parameters needed to achieve agreement, such as in ATD model development,
associative metrics tend to produce computed results that do not under predict the test
results (4). This metric has been combined with scalar errors from the peak and time of
peak within the Quick Rating scheme from Madymo using a simple averaging routine.
Global Evaluation Method. The Global Evaluation Method (GEM) has also
been utilized in the automotive community to evaluate ATD models (8). This metric is
part of a larger software program, called ModEval, which is designed to evaluate an
entire ATD model, factoring in multiple data channels and multiple types of evaluation
methods. The GEM is intended to be used on primary interest channels, such as head x
and z acceleration in a typical forward crash without yaw. In general, to evaluate the test
and model, the test curve is parameterized to define important points and shapes, and the
model curve is analyzed to search for the defined parameters. A score is determined for
each parameter, factoring in the magnitude and phasing of the parameter, and then an
overall score is computed for the two data sets. These steps are completed by a standalone executable. Unlike the previous two metrics presented, the GEM can not readably
be coded into a spreadsheet program.
The parameters evaluated in the GEM are separated into three groups; points
(minima and maxima), shapes (rising/falling edges and local extrema), and global. For
the point group, one or two overall maxima and minima are chosen in the test data. Once
found in the model data, equations [12] and [13] are used to calculate a single point score,
which is the average of the time score (eqn. 12) and value score (eqn. 13). If a point is
not found in the simulation data, the time and value scores are both set to zero.

TimeTest _ Extremum  TimeModel _ Extremum 
  100%
Time _ ScoreOverall _ Extremum  1 


Re ference _ Period


 ValueTest _ Extremum  Value Model _ Extremum 
  100%
Value _ ScoreOverall _ Extremum  1 


Value
Test _ Extremum


[12]
[13]
The value score is essentially a relative error, with a one minus portion to calculate
agreement. The reference period is 40% of the total evaluation time, chosen based on
intuitive judgment for automotive crash events (8). The reference time allows the time
score to be independent of the time of occurrence within the data. For example, a 5 ms
difference in occurrence produces a different relative error if it occurs at 50 ms (10%
relative error) or 100 ms (5% relative error), while using a reference time of 100 ms,
which is 40% of the typical 250 ms test duration, produces the same error independent of
when the error occurs. Setting a reference time may allow for a more logical
understanding of an error, since the peak should occur in a region of interest and should
not depend on the specific time.
For the shape characteristics, the input curves are filtered with a low pass
Butterworth filter with a corner frequency of 40 Hz in order to extract the general shape
of the channels. One or two rising/falling edges are selected, along with one or two local
extrema if appropriate. Equations [14] and [15] are used to determine the shape score for
edges (taking the average of the time and value scores). Equations 12 and 13 are used for
local extrema. Once again, if a feature is not found in the simulation data, the time and
value scores are set to zero.

TimeTest _ Edge  TimeModel _ Edge 
  100%
[14]
Time _ Score Edge  1 


Re ference _ Period



min SlopeTest , Slope Model  
  100%
[15]
Value _ Score Edge  1 
 max SlopeTest , Slope Model  
The global group consists of two factors, the normalized cross-correlation
function (NCCF) and a shape corridor. The NCCF determines a general time shift
(phasing) of the two input curves, and the max of the NCCF provides the value. For the
shape corridor, the input curves are filtered as in the shape group, and a corridor is
defined based on the test data surrounded by a square rectangle with dimensions of +/5%. The time and value scores from the NCCF and shape corridor are combined to give
the global score.
The final GEM score is computed by summing the point score (twice) plus the
shape and global scores, divided by four. Like WIFac, the GEM produces a score
reflecting the agreement between test and simulation. For the purposes of this paper, the
error between the two curves was calculated by subtracting the agreement score from
one. By definition, the GEM score combines the error at the peak with the error
associated with the time to peak and curve shape, with additional weight given to the
peak value. The usefulness of this combination will be discussed in a later section.
Normalized Integral Square Error (NISE). Originally developed to evaluate
acceleration responses from side impact test dummies (9) and used extensively in
automotive biomechanical research, the NISE method evaluates the amplitude, phase,
and curve shape errors of time histories. At the heart of the method, values are calculated
with and without a phase shift to allow for an independent calculation of phase error and
amplitude error. Four basic values are defined (equations 16-19), similar to the S&G
metric.
1 N
R xx (0)   X i X i
[16]
N i 1
1 N
R yy (0)   Yi Yi
[17]
N i 1
1 N
R xy (0)   X i Yi
[18]
N i 1
1 N n
R xy ( ) 
[19]
 X iYi  n
N  n i 1
Where n = τ / ∆t, X = test data, Y = simulation data, and N = number of points in
the data set. Rxy (τ) is calculated for all time leads/lags, with the maximum value being
used in the subsequent equations. Next the phase, amplitude, and shape scores are
calculated.
2 R xy ( ) max  2 R xy (0)
NISE phase 
[20]
R xx (0)  R yy (0)
NISE amplitude  Pxy ( ) 
2 R xy ( ) max
R xx (0)  R yy (0)
NISE shape  1  Pxy ( )
[21]
[22]
where Pxy is the correlation coefficient, defined as:
Pxy ( ) 
R xy ( ) max)
R xx (0)  R yy (0)
[23]
A total NISE score can also be computed using equation 24, which is effectively the
summation of the three components.
NISE  1 
2 R xy (0)
R xx (0)  R yy (0)
[24]
Because the total score is a simple summation, if there are sign differences
between any of the three components, the total score can be lower than one of the
components (suggesting less error than one might expect). Like S&G and WIFac, this
metric can be implemented into a spreadsheet program with little difficulty.
4. Spit HH, Hovenga PE, Happee RM, Uijldert M, Dalenoort AM. Improved
Prediction of Hybrid-III Injury Values using Advanced Multibody Techniques and
Objective Rating. SAE 2005-01-1307. 2005.
5. Russell DM. Error Measures for Comparing Transient Data: Part II: Error
Measures Case Study. Proceedings of the 68th Shock and Vibration Symposium.
185-198. 1997.
6. Sprague MA and Geers TL. A Sprectral-Element Method for Modeling
Cavitation in Transient Fluid-Structure Interaction. International Journal for
Numerical Methods in Engineering. 60 (15), 2467-2499. 2004.
7. Schwer LE. Validation Metrics for Response Histories: A Review with Case
Studies. -UPDATE
8. Jacob C, Charras F, Trosseille X, et al. Mathematical Models Integral Rating.
International Journal of Crashworthiness. 5(4), 417-731. 2000.
9. Donnelly B, Morgan R, Eppinger R.
Durability, Repeatability and
Reproducibility of the NHTSA Side Impact Dummy. Proceedings of the 27th Stapp
Car Crash Conference, SAE Paper No. 831624. 299-310. 1983.
New references for WIFac
1. Twisk D., Spit H.H., Beebe M., Depinet P. "Effect of Dummy
Repeatability on Numerical Model Accuracy". SAE paper 2007-01-1173.
2007.
2. D.Twisk, P.A. Ritmeijer. "A Software Method for Demonstrating
Validation of Computer Dummy Models Used In the Evaluation of Aircraft
Seating Systems". SAE paper 2007-01-3925. 2007.