Structure of Presentation
• Motion Tracking Technique
–
–
–
–
Motion Tracking Techniques
and Its Extension Application Projects
Definition of Motion Tracking Techniques
Classification of Motion Tracking Techniques
General Requirements for Motion Tracking Techniques
Evaluation Criteria for Motion Tracking Techniques
• Extension Application Projects
– Digital Work Evaluation System
– Project EADS
Liang MA 马靓
liang.ma@irccyn.ec-nantes.fr
IRCCYN Bureau 416
Motion Tracking Techniques and Its Extension Application Projects
Motion Tracking Techniques and Its Extension Application Projects
Page 2 of 75
Definition of Motion Tracking Techniques
Classification of Motion Tracking Techniques
• Definition of motion tracking
• Physical Principles
– To get the position and orientation of a moving object relative to a stationary
object
•
•
•
•
•
View Control
Navigation
Object selection or manipulation
Instrument tracking
Avatar animation
– Outside-looking-in
– Inside-looking-out
• Record the movement of the human body
• Motion analysis and synthesis
• Character animation
Page 3 of 75
General Requirements for Tracking Techniques
– Tiny
– Complete
– Accurate
–
–
–
–
–
–
Fast
Immune to occlusion
Robust
Tenacious
Wireless
Cheap
Motion Tracking Techniques and Its Extension Application Projects
Page 4 of 75
Mechanical Motion Tracking
Size as small as possible
Position and Orientation
Better than 1mm in Position and 0,1 degree
in Orientation
1 000 Hz with latency less than 1ms
No clear line of sight to anything else
Resisting degradation from other noises
No matter how far it goes
No influences on human motion
Cost factor
• Principle
– Measure joint angles and
lengths between joints
– Electromechanical transducers
• Classification
– Body-based
– Ground-based
l1
O ( x0 , y 0 )
Page 5 of 75
Phantom Desktop Device
P
θ1
Motion Tracking Techniques and Its Extension Application Projects
Acoustic motion tracking technique
Magnetic motion tracking technique
Mechanical motion tracking technique
Inertial motion tracking technique
Optical motion tracking technique
• Arrangement of Sources and Sensors
– Human motion tracking
Motion Tracking Techniques and Its Extension Application Projects
–
–
–
–
–
θ2
l2
l3
θ3
3
⎧
⎪ x p = ∑ li cos θ i
⎪
i =1
⎨
3
⎪ y = l sin θ
∑
p
i
i
i =1
⎩⎪
Motion Tracking Techniques and Its Extension Application Projects
FullBodyTracker
Page 6 of 75
1
Applications of Mechanical Tracking
Technical Specifications
• Technical Specifications
– Sensors: 37 potentiometers & 2 inertial gyroscopes
– Sensor Resolution: 0.125 degree
– Number of Joints / Rotations Sensed: 17
• Physical Specifications
– Setup time (from opening case):2 to 10 minutes
– Time to put on Sensor Suit: 2 to 5 minutes
– System Weight No Battery: 6 Kg
Gypsy5
http://www.inition.co.uk/inition/product.php?URL_=product_mocaptrack_animazoo_gypsy&SubCatID_=20
Immersion CyberGrasp
Motion Tracking Techniques and Its Extension Application Projects
Page 7 of 75
Evaluation for Mechanical Tracking
Page 8 of 75
Acoustic Motion Tracking
• Principles
Advantages
Disadvantages
+ high update rate
+ low latency
-Restricted movement from mounted devices
+accurate
+No blocking problem, no interference from
Motion Tracking Techniques and Its Extension Application Projects
– Ultrasonic above 20,000[Hz]
– Time of flight method
– Phase coherence method
-Bulky devices for body-based devices
-Time consuming for mounting the devices
(body based)
Speed =
γRT
M
d [m] = v[m / s ]t[ s ]
environment
+best choice for force feedback
Motion Tracking Techniques and Its Extension Application Projects
Page 9 of 75
Application of Acoustic Tracking
Motion Tracking Techniques and Its Extension Application Projects
Page 10 of 75
Technical Specifications
• Resolution
– Position (X/Y/Z): 1.5 mm
– Angular (P/R/Y): 0.05°
•
•
•
•
Intersense IS-600 Mark 2
The Logitech acoustic mouse tracker
Maximum update rate: 180 Hz
Latency: 2-5 ms
Sensor weight: 11.3g
Tracking Range:
One - 2.5 m x 2.5 m, Four - 25 m2
Intersense IS-600 Mark 2
http://www.inition.co.uk/inition/products.php?CatID_=11
Motion Tracking Techniques and Its Extension Application Projects
Page 11 of 75
Motion Tracking Techniques and Its Extension Application Projects
Page 12 of 75
2
Evaluation for Acoustic Tracking
Magnetic Motion Tracking
• Principles
– Generating Magnetic Fields
• Low frequency AC fields
• Pulsed DC fields
Advantages
Disadvantages
+ Very high freedom of movement
+ Cheap
− Line-of-sight problems
− Either high range or high accuracy
−Environmental interference (Temperature)
− Drift errors (phase-coherent)
− High latency, low update rates
Motion Tracking Techniques and Its Extension Application Projects
– Detecting Magnetic Fields
• Three orthogonal triaxial coils
• Four magnetic field
– X
– Y
– Z
– Earth Magnetic
Page 13 of 75
Applications of Magnetic Tracking
Motion Tracking Techniques and Its Extension Application Projects
Page 14 of 75
Technical Specifications
• Maximum Tracking Range
– Up to 5 feet (about 5/3 meter)
• Static Accuracy
– 0.1 inch RMS for the X, Y, or Z position (about 2.5 mm)
– 0.75º RMS for sensor orientation
Ascension 3D Navigator
Polhemeus Patriot
• Update rate: 60 Hz
• Latency: 17 ms
• Resolution
– 0.0015 inch per inch of source and sensor separation
(about 0.003 mm)
– 0.1º orientation
Inition 3DiStick
Polhemus Liberty
Ascension SpacePad
Motion Tracking Techniques and Its Extension Application Projects
Page 15 of 75
Motion Tracking Techniques and Its Extension Application Projects
Evaluation for Magnetic Tracking
Inertial Motion Tracking
Advantages
Disadvantages
+ High update rates
+ Very low latency
+ High robustness
+ No shadowing
+ Rather cheap
+ Acceptable accuracy in artificial
environment
− High sensitivity to background magnetic fields
− Inaccurate in practical environment due to interference
caused by ferrous metal devices
− Low range of the magnetic field
− Tracking scope is low due to cable
• Principles
– Accelerometer
– Gyroscope
Motion Tracking Techniques and Its Extension Application Projects
Page 17 of 75
Page 16 of 75
M = Iα
F = mr
M = Iω
Motion Tracking Techniques and Its Extension Application Projects
Page 18 of 75
3
Applications of Inertial Tracking
Technical Specifications
• Intersense additional (stand-alone) wireless IC3 sensors
–
–
–
–
–
–
–
Ascension 3D-BIRD
Wireless InertiaCube3
Update rate: 180 Hz
Latency: <6ms
Weight: 20g
Range: 30 m (wireless)
Dynamic accuracy: 0.25°
Resolution: Angular: 0.03°
Degree of freedom: 3
Xsens Xbus Master
Motion Tracking Techniques and Its Extension Application Projects
Page 19 of 75
Evaluation for Inertial Tracking
Advantages
Disadvantages
+ Very high range
+ High freedom of movement (if
wireless)
+ No shadowing
+ No environmental interference
− Long term drift of orientation
− Expensive
Motion Tracking Techniques and Its Extension Application Projects
Page 20 of 75
Optical motion tracking
•
Principles
Terminals
– Computer vision and Image
Processing
I = I0
• 1 2D position Æ 1 Ray
• 2 2D positionsÆ 3D position
•
( L − x)
L
Devices
– Detector
• Lateral Effect Photo Diodes
• Quad Cells
• Charge Coupled Devices CCD
s=
1 2
10
at = 0.5 ×
[ms − 2 ] ×10 2 [ s 2 ] = 5m
2
100
Motion Tracking Techniques and Its Extension Application Projects
Page 21 of 75
Applications of Optical Tracking
• Classification
•
•
•
•
• Passive marker
– Reflective material
– High contrast pattern
• Active marker
– Laser
– LED
– Simple light bulbs
(i1 + i2 ) − (i3 + i4 )
i1 + i2 + i3 + i4
y=
(i1 + i4 ) − (i2 + i3 )
i1 + i2 + i3 + i4
Motion Tracking Techniques and Its Extension Application Projects
Classification of Optical Motion Tracking
– Marker-based tracking
x=
Page 22 of 75
Gait Analysis
Avatar Animation
Character Animation
Work Load Evaluation
Ascension ReActor 2
– Marker-less tracking
Motion Tracking Techniques and Its Extension Application Projects
Page 23 of 75
Motion Tracking Techniques and Its Extension Application Projects
Page 24 of 75
4
Technical Specifications
Optical Motion Tracking
• Update rate
Advantages
Disadvantages
+ High accuracy
+ High freedom of movement
+ High update rate
+ Low latency
+ Multiple markers possible
+ High capturing volume
− High cost
− Shadowing of markers
− High sensibility to reflective objects
− Markers distinction if the distance between two
markers is too short
Passive Markers
– 2,000 frames per second
– Maximum: 10 000frames per second
• Resolution: 0.3 MegaPixel, VGA, 659 x 493
• Accuracy: 2D accuracy: 0.2 pixel
• Degree of freedom: 3
Vicon MX+
Active Markers
Motion Tracking Techniques and Its Extension Application Projects
Page 25 of 75
Advantages
Disadvantages
+ High accuracy
+ No correspondence problems
+ High update rate
+ Low latency
+ Multiple markers possible
+ High capturing volume
− Extremely high cost
− Shadowing of markers
− Low freedom of movement through cable for
markers
Motion Tracking Techniques and Its Extension Application Projects
Hybrid System
Structure of Presentation
• Definition
• Motion Tracking Technique
Several mediums exhibit complementary behavior, and these systems can be
combined to leverage the strengths of each medium as needed. Systems that
employ such mixed mediums are called hybrid systems.
• Applications
–
–
–
–
Page 26 of 75
Definition of Motion Tracking Techniques
Classification of Motion Tracking Techniques
General Requirements for Motion Tracking Techniques
Evaluation Criteria for Motion Tracking Techniques
• Extension Application Projects
Control Panel Evaluation Project
Data Glove + Optical Motion Tracking
– Digital Work Evaluation System
– Project with EADS
Ascension Hy-BIRD
Motion Tracking Techniques and Its Extension Application Projects
Page 27 of 75
Motion Tracking Techniques and Its Extension Application Projects
Page 28 of 75
Digital Work Evaluation
Objective of the Project
• Project Background
• Collaborative project between Tsinghua University and MITSUBISHI
Heavy Industry
• Objectives
– Objectives of the project
– Existing Problems in the traditional work design evaluation methods
– Solution
– To track, process, simulate, and examine the operation feasibility in
special environment in a real-time manner
– To automatically analyze, simulate, and optimize a typical work task in a
real MHI workshop
• Digital Work Evaluation System
–
–
–
–
System Structure
Motion Tracking Module
Motion Analysis Module
Motion Simulation Module
Motion Tracking Techniques and Its Extension Application Projects
Page 29 of 75
Motion Tracking Techniques and Its Extension Application Projects
Page 30 of 75
5
Existing problems and supporting techniques
Motion Time Analysis
• Existing problems in current work evaluation
• Definition
– Constraints of the field environment
– Subjective influences on work evaluation
– Work efficiency evaluation
– developing the preferred system and method – usually the one with the lowest
cost
– standardizing this system and method
– determining the time required by a qualified and properly trained person
working at a normal pace to do a specific task or operation
– assisting in training the worker in the preferred method
• Supporting techniques for this projects
– Motion tracking technique
Digitalize and Quantify the worker’s motion
– Virtual Reality and virtual environment technique
Interactive Simulation
– Work Measurement techniques
Work efficiency evaluation
Motion Tracking Techniques and Its Extension Application Projects
• Predetermined Time System
– as a field of work measurement to estimate the time needed by qualified
workers to perform a particular task at a specified level of performance.
– MTM, Method Time Measurement
– MOST, Maynard Operation Sequence Technique
Page 31 of 75
MOST time standards
•
•
•
General Move
Controlled Move
Tool Use
Motion Tracking Techniques and Its Extension Application Projects
Page 32 of 75
Hardware Structure
n
Ts = 10 ⋅ TMU ⋅ ∑ ( Ai + Bi + Gi + Pi )
i =1
TMU = 0.0036 sec
Motion Tracking Techniques and Its Extension Application Projects
Page 33 of 75
Software Structure
Motion Tracking Techniques and Its Extension Application Projects
Motion Tracking Techniques and Its Extension Application Projects
Page 34 of 75
Software system in Details
Page 35 of 75
Motion Tracking Techniques and Its Extension Application Projects
Page 36 of 75
6
Motion Tracking Structure
Motion Tracking Algorithms
• Typical problems in optical motion tracking
– 2D image processing
2D information
• Image capture card control
• Image processing
– Marker recognition
– Marker Center Determination
– Marker position prediction
– 3D human model construction
– Camera calibration
– Multiple camera collaboration
– Marker matching
Motion Tracking Techniques and Its Extension Application Projects
Page 37 of 75
Motion Tracking Techniques and Its Extension Application Projects
Different Aspect of Tracking
Marker Recognition
• One Marker with One Camera
• Objective
– Marker Recognition
– Marker Center Determination
– Marker Position Prediction
Page 38 of 75
– Distinguish marker from background
• Features of markers
– Shape
– Size
– Color
• One Marker with Several Cameras
– 2D Reverse Calculation (at least 2 Cameras)
– Camera collaboration
n
GT =
• Methods
– Threshold Methods
• Several Marker with One Camera
∑
i =1
Gi
n
Color feature
• Mean value threshold
• Histogram threshold
– Marker Matching (History based matching)
• Several Markers with Several Cameras
GT =
– Marker Matching based on human constraints
– All the above
Motion Tracking Techniques and Its Extension Application Projects
Page 39 of 75
Motion Tracking Techniques and Its Extension Application Projects
Marker Center Determination
Marker 2D Position Prediction
• Objective
• Objective
– Get the precise centre of the marker to ensure the 3D reconstruction
G1 + G2
2
Page 40 of 75
– To minimize the computation consumption for searching markers
• Methods
• Methods
– Gravity method
– Shape centre method
– Circle shape based center method
n
⎧
∑ xi
⎪
⎪C = i =1
⎪ x
n
⎨
n
⎪
yi
∑
⎪
i =1
⎪C y =
n
⎩
3D human motion info
n
⎧
∑ xi Gi
⎪
⎪Cx = i =1
⎪
n
⎨
n
⎪
y i Gi
∑
⎪
i =1
⎪C y =
n
⎩
Motion Tracking Techniques and Its Extension Application Projects
– Prediction models
• Linear models
– Acceleration and Velocity model
– Velocity model
• Statistical models
– HMM
xa + xb
⎧
⎪⎪C x = 2
⎨
⎪C = yc + y d
⎪⎩ y
2
Page 41 of 75
Motion Tracking Techniques and Its Extension Application Projects
Page 42 of 75
7
Marker Matching
Multiple Camera Collaboration
• Objective
• Objective
– To match the marker to the corresponding joint of human body
– To coordinate cameras and make best use of redundant data
• Methods
• Methods
– Historical information based marker distribution
– Priority algorithms
• Position
• Shape
• Size
• Priority for each marker in each camera
– Visible or invisible in Camera
– Distances of the current marker between the other markers
– Human constraints based marker distribution
• Limb lengths
• Limb orientation
• Other physical constraints
Motion Tracking Techniques and Its Extension Application Projects
Page 43 of 75
Motion Tracking Techniques and Its Extension Application Projects
Camera Calibration
Work Efficiency Evaluation Algorithms
• Principle
• Principle
– See textbooks about machine vision.
– Efficiency evaluation
• Productivity in unit time
• Consumed time for specified task
– Actual consumed time
» Measurement with frame number and frequency
Start frame of a motion
End frame of a motion
– Standard time
» Recognized motion in MOST standards
• Objective
– To get external and internal parameters of Cameras
– To mapping the 3D world coordinate and 2D image coordinate
•
•
•
•
•
World coordinate
Camera coordinate
Realistic imaging plane coordinate
Actual imaging plane coordinate
Pixel coordinate
Motion Tracking Techniques and Its Extension Application Projects
Page 44 of 75
Page 45 of 75
How to Get Standard Time
Motion Tracking Techniques and Its Extension Application Projects
Page 46 of 75
Static Posture
• Static posture template matching
Static Posture
– Static posture template definition
•Example
• Dynamic motion recognition
•Stand Posture
– Transition from one static posture to another static posture
•Sit Posture
• Dynamic motion mapping to MOST standards
– From dynamic motion to get MOST standards index
• Standard Time
Rules
B
Body
LeftLeg
RightLeg
/Rules
Motion Tracking Techniques and Its Extension Application Projects
Page 47 of 75
Sit
Z_Axis
Relative_Angle
Relative_Angle
0
90
90
Motion Tracking Techniques and Its Extension Application Projects
5
20
20
Page 48 of 75
8
Static Posture to Dynamic Motion
Flowchart of Motion Analysis
Dynamic motion is the movement between two different static
postures
End Frame
Start Frame
STATIC
Stand
Sit
Bend_50
Stand
No_Motion
Sit
Bend_50
Bend
Sit
Stand
No_Motion
Bend_50
Bend
Bend_50
Arise_50
Arise_50
No_Motion
Bend_50
Bend
Arise
Sit
Arise_50
No_Motion
Motion Tracking Techniques and Its Extension Application Projects
Bend
Page 49 of 75
Motion Analysis Result
Motion Tracking Techniques and Its Extension Application Projects
Page 50 of 75
Virtual Environment
• Virtual Working Environment
– Virtual working environment construction
• CAD software
• MULTIGEN Creator
– Virtual working environment visualization
• Vega
• WTK
• VTK
Motion Tracking Techniques and Its Extension Application Projects
Page 51 of 75
Virtual Human
Motion Tracking Techniques and Its Extension Application Projects
Page 52 of 75
Data Transferring via Network
• Virtual Human Modeling
• Server
– Virtual human modeling
– Socket initialization
– Maximum permissible connection
• Structure of virtual human
– Virtual human assembly
• Joint compensation
• Data smoothing
Motion Tracking Techniques and Its Extension Application Projects
• Client
– Server’s IP address
– If there is an available connection
– If data is receivable
Page 53 of 75
Motion Tracking Techniques and Its Extension Application Projects
Page 54 of 75
9
Simulation Result
Motion Tracking Techniques and Its Extension Application Projects
Démonstration- Motion Analysis
Page 55 of 75
Motion Tracking Techniques and Its Extension Application Projects
Page 56 of 75
Project EADS
Project Introduction
• Project introduction
• Research steps
• Research Methods
• To evaluate of the human fatigue during a task simulation
– Subjective Evaluation
– Objective Evaluation
– Subjective and Objective Evaluation
– Trajectory Optimization
• Supporting Techniques
• To analyse globally human tasks taking into account the
ergonomics and the fatigue (Stress, Workload, and Fatigue)
of the human in the context of aircraft industry.
– Human Modeling
– Muscle Fatigue Modeling
– Posture Analysis
Motion Tracking Techniques and Its Extension Application Projects
Page 57 of 75
Subjective and Objective Evaluation
Motion Tracking Techniques and Its Extension Application Projects
Research Steps
• Subjective Evaluation
1st stage
Technical support
Theory preparation
– To Carry out the task in virtual working environment
– To Evaluate the task by the participants
– To Record the human motion using Motion Tracking System
• Objective Evaluation
– To Generate theoretical path based on biomechanics and ergonomics study
– To Compare the human motion with the theoretical optimum trajectory
– To Evaluate the task
Motion Tracking Techniques and Its Extension Application Projects
Page 58 of 75
Page 59 of 75
Supporting technologies
•Human simulation
•Human modeling
•Biomechanics
•Ergonomics
•Task analysis
•Motion capture
2nd stage
Technique realization
Theory application
Motion Tracking Techniques and Its Extension Application Projects
Page 60 of 75
10
Research Steps
Initial
Trajectory
Supporting Techniques
Human Motion Analysis
•Real simulation
•Virtual simulation
Human
Motion
Evaluation result
•Ergonomics
•Fatigue
•Difficulty
• Human Modeling
– Kinematic modeling
• A sequences of rigid bodies connected by joints
• DH Methods (Denvavit and Hartenberg Notation)
– Biomechanical modeling
• Muscle Fatigue Model
Human Motion Improvement
•Trajectory comparison
•Fatigue analysis
•Ergonomics analysis
Motion Tracking Techniques and Its Extension Application Projects
Page 61 of 75
DH Parameters and Transformation Matrices
Motion Tracking Techniques and Its Extension Application Projects
Motion Tracking Techniques and Its Extension Application Projects
Page 62 of 75
Target point in global coordinate
Page 63 of 75
Overall Kinematic Modeling
Motion Tracking Techniques and Its Extension Application Projects
Page 64 of 75
Muscle Fatigue Models
• Muscle Models
– Hill’s Model
– Zajac’s modification to the Hill Model
– http://www.digital-humans.org/Report2004/
• Muscle Fatigue Analysis
– Objective methods
• EMG Electromyography
– Subjective methods
• Experiments design
• Questionnaires
Motion Tracking Techniques and Its Extension Application Projects
Page 65 of 75
Motion Tracking Techniques and Its Extension Application Projects
Page 66 of 75
11
Physical Strain Analysis Methods
Pen-paper-based Methods (1)
• Concentration
– Working postures/movement
– Additional factors: force and task duration
• Observational methods
– Pen-paper-based observational techniques
– Videotaping and computer-aided observational methods
• Direct or instrumental methods
• Self-report assessment
Motion Tracking Techniques and Its Extension Application Projects
Page 67 of 75
Pen-paper-based Methods (2)
Motion Tracking Techniques and Its Extension Application Projects
Motion Tracking Techniques and Its Extension Application Projects
Page 68 of 75
Videotaping and computer-aided methods(1)
Page 69 of 75
Videotaping and computer-aided methods(2)
Motion Tracking Techniques and Its Extension Application Projects
Page 70 of 75
Direct Methods
• Posture assessment
– Manual devices
– Electric instrumental methods
• Postural strain or local muscle fatigue assessment
– Electromyography
– Stadiometer
– Intradiscal pressure measurement
Motion Tracking Techniques and Its Extension Application Projects
Page 71 of 75
Motion Tracking Techniques and Its Extension Application Projects
Page 72 of 75
12
Self-report assessment
Bibliography
•
•
•
•
•
• Motion Tracking: No Silver Bullet, but a Respectable Arsenal.
• Motion Tracking Requirements and Technologies in Handbook of Virtual
Environment Technology
• Current techniques for assessing physical exposure to work related
musculoskeletal risks, with emphasis on posture-based methods
• http://www.digital-humans.org/Report2004/
• www.immersion.fr
• http://www.motionanalysis.com/
• http://www.vicon.com/applications/
Body map
Rating scales
Questionnaires
Interviews
Checklists
Motion Tracking Techniques and Its Extension Application Projects
Page 73 of 75
Motion Tracking Techniques and Its Extension Application Projects
Page 74 of 75
• Merci!
• Questions ?
Motion Tracking Techniques and Its Extension Application Projects
Page 75 of 75
13