Psychometrics, Dynamics, and
Functional Data Analysis
“
T
h
e
v
i
e
w
s
Jim Ramsay
McGill University
Testing as Input/Output Analysis





A test score is actually a derivative with respect
to time.
Consequently a differential equation model for
testing data seems natural.
Dynamic testing data will be more and more
important.
We have some new tools for working with
dynamic data.
So let’s consider how to use time as a covariate.
Learning to Play Golf




We buy some clubs. We play a few games, each
being an 18 item test. It’s harder than it looks.
We take a lesson. We play a few games. Our
score improves to a better level.
We take another lesson, play some games, and
things improve again.
Key question: How quickly is a lesson reflected
in an improvement in score?
Brainergy


Energy is defined as
“the capacity to do
work.”
Kinetic energy E =
Mv2/2, and involves
mass, distance, and
time.
• We are interested in
“the capacity to solve
problems.”
• Problems involve
difficulties (=mass),
number of problems
(=distance) and time.
• Let’s call mental
energy brainergy.
Brain Power





What counts is problem solving per unit
time.
Power = energy expended per unit time.
Brain power = maximum difficulty of
problem solvable per unit time, or
number of lighter problems solved per unit
time.
That is, brain power = d brainergy/dt.
Brain Power and Time Scales
We need the concept of brain power when we
consider intelligence on two time scales:
1.
Long term: How much knowledge is available
over large time intervals, like a school year
2.
Short term: How much new knowledge can be
acquired over a short time interval, like a
single class.
Tests Measure Brain Power



Mental tests and psychological scales are
one of the greatest technological
achievements of the 20th century.
Tests work so well because they are timelimited.
Test scores reflect brain power rather than
brainergy.
Inputs to Brain Power





Information about the structure of the
problems.
A set of tools to solve them.
Training in the use of these tools.
All these require time.
Inputs to acquisition of brain power are
functions of time.
A Differential Equation in Time





Links one or more time-derivatives, dx/dt,
d2x/dt2,…, to the function x(t) itself.
Is a model for system dynamics: change
over time.
Can also include one or more input or
covariate functions.
x(t) is a long-term description.
dx/dt is a short-term description.
A Simple Example
dE
   E (t )   f (t )
dt



E(t) is brainergy, dE/dt is brain
power.
f(t) is an input function of time,
such as education.
α and β are constants, β > 0.


Most differential equations don’t have explicit
solutions, but this one does.
Let E0 be brainergy at time t = 0, and which
will often be 0.
E (t )  e
t
t
[ E0    e f (u)du]
t
0
see what happens when α=1, β varies,
and f(t) is a step function.
Let’s
E (t )  e
t
t
[ E0    e f (u)du]
0
t




The slope of E(t) when f(t) goes positive is
β.
β controls how fast the system responds
to the input f(t).
If the system is a problem solver, then β
indicates how quickly the person learns to
solve a problem.
After about 4/β time units, full capacity is
reached, and the system is ready for more
input.
Fitting Differential Equations



We have noisy discrete-time data, and
want to use them to estimate a differential
equation.
We want a solution E(t) to the equation to
fit the data as well as possible.
We need lots of flexibility in choosing a
differential equation, and we can’t assume
that there is an explicit solution to the
equation.
Functional Data Analysis




A collection of methods for analyzing
curves or functions as data
A common theme is using derivatives in
various ways
See Ramsay and Silverman (1997)
Functional Data Analysis. Springer.
And Ramsay and Silverman (2002) Applied
Functional Data Analysis. Springer.
Two Functional Data Analysis
Techniques


L-spline Smoothing: given noisy data and
a differential equation, find a function E(t)
that will smooth the data and at the same
time be nearly a solution to the differential
equation.
Principal Differential Analysis: given a
function E(t), estimate a linear differential
equation for which E(t) is a solution.
Estimating a DIFE from noisy data



We’ve recently combined these two methods
into a technique for estimating a differential
equation from noisy data.
In our simple example, this amounts to
estimating parameters α and β.
But much more complex DIFE’s can be
estimated as well, including linear or nonlinear,
and single or multiple variable systems.
An Oil Refinery



Here are some data from an oil refinery in
Corpus Christi.
The input f(t) (reflux flow) is negatively
coupled to the output E(t) (tray 47 level).
The smooth curve is a solution to the
differential equation that best represents
this relationship.
Perhaps this oil refinery is not too smart!



Many situations will call for multiple
outputs: Performance with a putter, a
driver, and an iron, for example. Or in
algebra and geometry.
And many situations will involve multiple
inputs: Regular classes, tutoring sessions,
labs and etc.
The technology used in these illustrations
can handle these situations, at least for
linear differential equations. Nonlinear
equations don’t pose any problem in
principle.
Some Simulated Data


Imagine that the data are golf scores over
successive games, and that the input is a
set of three equally-spaced lessons from a
golf pro.
The following slides show three golfers.
Which is a future Tiger Woods?
These lessons are nicely timed.
This person needs to find another sport!
This person should get lessons more often!
Is this Model Good Enough?



Specifying β to be constant is too simple.
Allowing for fatigue, boredom, and other
things requires a function β(t).
A first order equation can’t allow for
sudden transient effects like insight. We
may need a differential equation involving
higher derivatives.
We may need nonlinear equations as well.
A Nonlinear Differential Equation
2
dE1
A( BE2 (t ))
   [ E1 (t ) 
]   f (t )
2
dt
1  C ( BE2 (t ))
2
dE2
A( BE1 (t ))
   [ E2 (t ) 
]   f (t )
2
dt
1  C ( BE1 (t ))
•The summed output from these two equations
will exhibit both the rapid learning and long-term
retention required of human learners.
•See H. R. Wilson (1999) Spikes, Decisions and
Actions, Oxford, for many more examples of
differential equation models in neuroscience.
Control Theory


Engineers who work with input/output
systems have developed ways of
designing feedback loops to optimize
outputs.
We’re working with a team of chemical
engineers at Queen’s University.
Where Would the Data Come From?




Can we design customized learning
situations, like golf, and track how a
learner makes progress as a function of
time and inputs?
Perhaps video and computer games are
nearly what we need.
We already know that people will pay big
money to have these experiences.
Would corporations with deep pockets pay
for this kind of testing?
Conclusions



Dynamic testing would generate
performance data over time that depend
on one or more functional covariates.
New tools are available for these data that
fit them with a differential equation.
Dynamic psychometrics looks promising!