"Virtual" work: the D'Alembert principle and upper-body movement in mixed-reality systems.
By Bradly Alicea (freejumper@yahoo.com)
Department of Telecommunication, Information Studies, and Media and Cognitive Science Program, Michigan State University
Keywords: Ergonomics, Neurotechnology, Human-Machine Interfaces, Biomechanical Modeling
Introduction
In tracker-based mixed-reality computing systems (see Figure 1), it is common to manipulate computer-generated objects with devices such as pinch gloves. The unstable mismatch between proprioceptive feedback and object kinetics in such systems, especially during periods of training, requires looking to dynamic models for explanations of how to optimize the operation of these devices in real-time.
of momenta, and r equals displacement, represents this relationship.
“Virtual” objects or objects set in contexts determined by a tracking system are not associated with an inherent mass or inertia; for these objects, the equation often fails to balance. This creates a condition of
"virtual" work; applying this equation continuously to an arm movement trajectory yields a time series with varying degrees of stability.
In mixed-reality environments, the left-hand side of the D'Alembert equation can act as a tunable order parameter. Rewriting the expression as
1.2
Figure 1: Operational example of a mixed-reality system.
Method and Model
This poster uses D'Alembert's principle to explain why perceptualmotor learning of movements based on adaptation is an inherent feature of these tasks. In “real” contexts, forces equal in magnitude to that of a physical object and its acceleration are mechanically matched by the nervous system
1
. Opposing forces can simulate kinetic equilibrium. The equation where F
1.1
equals forces along a trajectory, p equals the time derivative where I t equals integration, or how much the forces encountered and work applied balance out at a specific time interval. results in an equation that adapts to local conditions much like a difference equation
2
.
Predictions
It is hypothesized that an operator's learning curve is influenced by adaptations to forces and delays along a trajectory. The D'Alembert equation and its derivatives act as both a parameter that characterizes stability in upper-body movements and the computational basis for an inverse model. If the left-hand term in either version of the D'
Alembert equation exceeds a critical value for extended time periods, operators must learn completely new “movements” for otherwise common tasks.
The inverse model itself is hypothesized to compare the production of forces at time t with afferent sensory feedback at time t+1 . This type of closed-loop adaptation has been empirically shown by applying Coriolis forces in various directions to the arm during ballistic movement
3
. Inverse models can selectively update neural representations of movement trajectories that are a product of training, but during times of rapid and irregular change have limited efficacy
4
.
Experimental Outcomes
The D'Alembert principle is violated mostly in cases where operators

initially learn to manipulate virtual objects in a novel task.
Experimental data has shown that movement trajectory and time to target decrease logarithmically, with performance improvements converging after the 30th trial (Figure 2).
Preliminary EMG recordings from the flexor carpi radialis and extensor carpi radialis suggest that either adaptive electromechanical delays or the spectral signature of each muscle may serve as temporal or magnitudinal markers of learning and performance improvements.
Discussion and Conclusions
The model presented here can give us two types of information about movements in mixed-reality: 1) that adaptation occurs when
I t does not equal 0, and 2) that the rate of adaptation may not be the same for all phases of the movement. When forces and displacements are heterogeneously matched across a given movement, the inverse model may retain all ‘solved’ portions of the trajectory and search for revisions among a subset of combinatorial options
4
. Specifically, combinations of visual and spinalmotor inputs act as selection mechanisms that balance
D'Alembert's equation and impose order upon the tuning parameter
I t
5.
The fusion of these two modalities at the internal model is used to estimate a force output
6
. This output indirectly results in the selective electrical activation of arm muscles (see Figure 3).
Figure 2: Movement Time data averaged over 45 trials.
While we cannot directly extract force information from the EMG signal, future research may involve using EMG to find when overcompensating of control in response to forementioned
Figure 3: left graph shows EMG time-series recordings for the flexor carpi radialis (blue) and the extensor carpi radialis (red) averaged over 30 trials; right graphs (A and B) show corresponding Fast Fourier spectrograms.
sensorimotor inputs may be occurring. This may indicate specific points in the movement when D’Alembert’s principle is being violated. This may help to predict the state of the tuning parameter I t in specific contexts, especially if different EMG profiles are sampled across the training period.
This information can also be used as a form of supervised learning to help individuals correct their motor activity relative to the environment.
Potential applications of this model include the evaluating performance gains and deficits in wearable computing systems and instructor stations used in laparoscopic surgery.
1
Sources
Kumar, S. (1999). Biomechanics in Ergonomics. Philadephia, PA:
Taylor and Francis.
2
Strogatz, S.H. (1994). Nonlinear dynamics and Chaos: with applications to physics, biology, chemistry, and engineering. Reading,
3
MA: Addison-Wesley.
Karniel, A. and Mussa-Ivaldi, F.A. (2002). Does the motor control system use multiple models and context switching to cope with a variable environment? Experimental Brain Research , 143, 4, 520-524.
4
Berthouze, L., and Lungarella, M. (2004). Motor skill acquisition under environmental perturbations: on the necessity of alternate freezing and
5 freeing of degrees of freedom. Adaptive Behavior , 12(1), 47-63.
Kawato, M. (1999). Internal models for motor control and trajectory planning. Current Opinion in Neurobiology , 9, 718-727.
6
Hillis, J.M., Ernst, M.O., Banks, M.S., and Landy, M.S. (2002). Combining sensory information: mandatory fusion within, but not between, senses.
Science , 298, 5598, 1627-1630.