Motion Capture History,
Pipeline, and Areas of Interest
Advanced Computing Center for the Arts
and Design
Ohio State University
©2003-2010, The Ohio State
University
Motion Capture
• Motion capture (mocap) is sampling and
recording motion of humans, animals and
inanimate objects as 3d data for analysis,
playback and remapping.
©2010, The Ohio State University
History of Motion Capture
• Eadweard Muybridge (1830-1904)
• Etienne-Jules Marey (1830-1904)
• Harold Edgerton (1903-1990)
©2010, The Ohio State University
Eadward Muybridge
• The Flying Horse
• Zoopraxiscope
• Animal and Human
Locomotion
© Kingston Museum
©2010, The Ohio State University
Etienne-Jules Marey
• First person to analyze
human and animal
motion with video
• Created
chronophotographic
gun and fixed plate
camera
©2010, The Ohio State University
Harold Edgerton
© Palm Press Inc.
©2010, The Ohio State University
Rotoscope
• Allowed animators to trace cartoon
characters over photographed frames of
live performances.
• Invented in 1915 by Max Fleischer
• Koko the Clown
• Snow White
©2010, The Ohio State University
1980’s
• Military and medical research purposes
• First computer graphics use in research labs
• First production use
o Brilliance by Robert Abel (1985 Superbowl ad)
o http://citinite.wordpress.com/2007/12/28/robert-abelassociates/
o Waldo C. Graphic (1988) PDI for Jim Henson tour
o Mike the Talking Head (Siggraph 88)
o http://mambo.ucsc.edu/psl/mike.html
o Don’t Touch Me (1989)
o http://www.poetv.com/video.php?vid=15102
©2010, The Ohio State University
Motion Capture Systems
Electromechanical
Optical fiber
Strobing LED
Acoustic
©2010, The Ohio State University
Optical
Acoustic
Electromechanical suits
• Linked structures
• Potentiometers determine rotation
of each link
• No occlusion
• No magnetic or electrical
interference
• Large capture range, portable
•
•
•
•
•
No global translation
Restricted movement
Fixed configuration of sensors
Low sampling rate
Inaccurate joints
http://www.metamotion.com/gypsy/gypsy-motion-capture-system-mocap.htm
©2010, The Ohio State University
Optical fiber system
• Fiber-optic sensor
• Bend and twist sensors measure
transmitted light
• No occlusion
• Multiple subjects
• Portable
• Adjustment to individual proportions
is limited
• Less accurate data
http://www.finger-motion-capture.com/
©2010, The Ohio State University
Electromagnetic systems
• Electromagnetic sensors placed
on joints of moving object
• Measures orientation and position
of sensor relative to
electromagnetic field generated
by transmitter
• No occlusion, multiple subjects
• Electromagnetic interference,
small volume
http://www.ascension-tech.com/
©2010, The Ohio State University
Acoustic system
• Set of microphones receive sound wave
• Other sounds in frequency range can
disrupt capture
• Accuracy not as high as other systems
©2010, The Ohio State University
Strobing LED marker system
• Fewer Occlusion
Problems
• Limited volume
http://www.inition.co.uk/inition/product.php?URL_=product_mocaptrack_ascension_reactor2&SubCatID_=59
©2010, The Ohio State University
Optical motion capture system
• Reflective markers
• Multiple cameras digitize
different views of
performance
• High speed capture
• High data accuracy
• Occlusion
• Position sensitive
cameras
• Limited volume
http://www.vicon.com
©2010, The Ohio State University
Video-based Motion Analysis
• Research problem for 20+ years
• Difficulties
o
o
o
Complex varying environment
Segmentation issues
Occlusion
©2010, The Ohio State University
Video-based Motion Analysis
Research Areas
Tracking
3D surface reconstruction (surface scanning)
3D movement reconstruction (video-based motion
capture)
o Recognition
o
o
o
Challenges:
o
o
o
Complex environment variability
Body segmentation
Occlusion
©2010, The Ohio State University
3D Reconstruction
• Single camera
o
Model assumptions required
• Multiple cameras
o
o
Require at least 2 cameras, unique with 3
Camera calibration
• Motion capture with markers
o
Use retroreflective markers to simplify video
information
©2010, The Ohio State University
Optical Mocap Process
• Calibration
o
Specifying how each camera is related to the others in
reference to origin
• Tracking
o
Tracking of points…acceleration and identification
problems
• Data
o
3D positional…needs to be converted to rotational
data
©2010, The Ohio State University
Problems Related to Marker Occlusion
©2010, The Ohio State University
Skeletal Fit
(remapping mocap data to a character model)
• How to make markers move a skeleton
o
o
o
o
o
Photo reference of performer
CG model
Motion Builder
Vicon Blade
Other methods
• Problems with detecting joint centers…
• Organization of joint hierarchies
©2010, The Ohio State University
Motion Capture Pipeline
•
•
•
•
•
•
Planning
Shooting (Vicon IQ)
Data processing (ViconIQ, MotionBuilder)
Skeleton creation (Maya)
Mapping to characters (MotionBuilder)
Binding skeleton to a model (Maya,
MotionBuilder)
©2009, The Ohio State University
Planning
• Character/Prop setup
o
o
Target skeleton/character topology
Ready stance considerations
• Marker setup
o
o
o
Heavy marker redundancy
Three markers per segment
Place markers close to bone
©2010, The Ohio State University
Planning
• Shot list
o
o
o
o
o
o
o
Number of characters involved in performance
Performance space dimensions
Interactions in shot
Shots to be blended or looped
Length of shots
Size and location of props
Gross proportional differences
for retargeting
©2009, The Ohio State University
Planning
• Technical Issues
o
o
o
o
o
Output format
File naming conventions
Frames per second
Target software platform
Database management
©2009, The Ohio State University
Other Areas of Interest
• Re-use of data
• Generating 3d
geometry
• Scene tracking with
motion capture
http://accad.osu.edu/~vberezin/bm/lp_anim.html
©2010, The Ohio State University
New Optical Technology
• 16 megapixel cameras ( T-series Vicon
and Motion Analysis), 2000 fps
• 10 bit greyscale – Vicon (better centroid
calculation)
©2010, The Ohio State University
Applications
• Medicine
Mixed Reality Rehabilitation http://ame2.asu.edu/projects/mrrehab/
NIH Clinical Movement Analysis Lab http://pdb.cc.nih.gov/
• Military http://www.motionrealityinc.com/
• Sports http://www.highwaygames.com/products/view.php?id=2807
• Video Games
http://www.spike.com/video/nba-live-08-mo-cap/2897719
http://vimeo.com/1096600
©2010, The Ohio State University
Other uses
Cognitive Psychology
http://www.mcgill.ca/spl/projects/motion/
Historical Preservation
http://accad.osu.edu/research/mocap_htmls/NAD.htm
Art
http://www.openendedgroup.com/index.php/artworks/
©2010, The Ohio State University
Feature Films
•
•
•
•
•
•
•
•
Avatar
Curious Case of Benjamin Button
Pirates of the Carribean
The Incredible Hulk
Lord of the Rings
Polar Express
Matrix
Titanic
©2009, The Ohio State University
Video Games
• Kinect
• Move
• Wii
©2009, The Ohio State University
Future
•
•
•
•
•
Univesity of South California (Paul Debevec)
NYU (Chris Briegler)
Stanford University (Vlad Koltun)
Carnegie Mellon (Jessica Hodgins)
Max Planck Center
•
•
•
•
Organic Motion
Mova
Image Metrics
Captive Motion
©2010, The Ohio State University
References
1. Menache, Alberto, “Understanding Motion Capture for
Computer Animation and Video Games”
2. http://www.kingston.ac.uk/Muybridge/
3. http://www.anotherscene.com/cinema/firsts/marey.html
4. http://cmp1.ucr.edu/exhibitions/edgerton/edgerton.html
5. Siggraph 2001 Course 51:
©2010, The Ohio State University