Particle Tracking
For Bulk Material Handling Systems
Using DEM Models
By: Jordan Pease
Introduction
•  Motivation for project
•  Particle Tracking
•  Application to DEM models
•  Experimental Results
•  Future Work
•  References and Acknowledgments
Motivation
•  Bulk Handling Head Station
Motivation
•  What is DEM?
Bulk Handling Particle
Tracking
•  Why Track Particles?
•
•
•
•
Material Flow
Chute Plugging
Material Distribution
Reduce Wear On:
•  Chute Walls
•  Liners
•  Belt
Motion Estimation Techniques
•  Feature-based methods
•  Extract visual features (corners, textured areas) and track
them over multiple frames
•  Sparse motion fields, but more robust tracking
•  Suitable when image motion is large (10s of pixels)
•  Direct methods
•  Directly recover image motion at each pixel from spatiotemporal image brightness variations
•  Dense motion fields, but sensitive to appearance variations
•  Suitable for video and when image motion is small
Particle Tracking
•  Methods/Background
•  Brownian Motion
•  Particle Filter
•  Optical Flow
•  Phase Correlation
•  Block-based Methods
•  Differential Methods
•  Horn-Schunck
•  Black-Jepsen
•  Buxton-Buxton
•  Lucas-Kanade
Optical flow
•  Definition: optical flow is the apparent motion of
brightness patterns in the image
•  Ideally, optical flow would be the same as the
motion field
•  Have to be careful: apparent motion can be
caused by lighting changes without any actual
motion
Optical Flow
•  How to estimate pixel motion from image H to image I?
–  Find pixel correspondences
•  Key assumptions
–  Color (Brightness) constancy: a point in H looks “like” a point in
image I
–  Small Motion: Objects move slowly (or access to high frame rate)
Optical Flow:
Lucas-Kanade Method
Assumes that the displacement of the image contents between two nearby instants
(frames) is small and approximately constant.
Gray-scale image Example
•  How to get more equations for a pixel?
•  Spatial coherence constraint: pretend the
pixel’s neighbors have the same (u,v)
•  If we use a 5x5 window, that gives us 25 equations per pixel
Uh oh!: We Have More
Equations Than Unknowns
Solution: solve least squares problem
•  minimum least squares solution given by solution (in d) of:
•  The summations are over all pixels in the K x K window
•  This technique is Lucas Kanade Method!
Conditions for Solvability
When is this solvable?
•  ATA should be invertible
•  Images must match! (size, pixel density etc.)
•  ATA should not be too small
–  eigenvalues λ1 and λ2 of ATA should not be too small
•  ATA should be well-conditioned
–  λ1/ λ2 should not be too large (λ1 = larger eigenvalue)
Optical Flow Issues
Iterative Refinement
•  Estimate velocity at each pixel using one iteration of
Lucas and Kanade estimation
•  Warp one image toward the other using the
estimated flow field
•  Refine estimate by repeating the process
CSE 576, Spring 2008
15
Motion estimation
Optical Flow: Iterative
Estimation
estimate
update
Initial guess:
Estimate:
x0
x
(using d for displacement here instead of u)
CSE 576, Spring 2008
16
Motion estimation
Optical Flow: Iterative
Estimation
estimate
update
Initial guess:
Estimate:
x0
CSE 576, Spring 2008
17
x
Motion estimation
Optical Flow: Iterative
Estimation
estimate
update
Initial guess:
Estimate:
x0
CSE 576, Spring 2008
18
x
Motion estimation
Optical Flow: Iterative
Estimation
x0
CSE 576, Spring 2008
19
x
Motion estimation
Optical Flow With Matlab
Calibration Images
Results
•  Input Image
Warp Image
Estimated Flow Field
Results
•  Input Image
Warp Image
Estimated Flow Field
Results: Failed Run
Future Work
•  Real-time Video
Results
Input Images
Warp Image
Estimated Flow Field
References & Acknowledgements
•  TAKRAF TENOVA
•  Bulk Flow Analyst
•  Overland Conveyor
•  MIT CSAIL Group
•  Richard Szeliski
•  Steve Seitz
•  Complete list of sources can be found on subsequent slides
Other Runs
Other Runs
Other Runs
Bibliography
•
J. R. Bergen, P. Anandan, K. J. Hanna, and R. Hingorani. Hierarchical modelbased motion estimation. In ECCV’92, pp. 237–252, Italy, May 1992.
•
M. J. Black and P. Anandan. The robust estimation of multiple motions:
Parametric and piecewise-smooth flow fields. Comp. Vis. Image Understanding,
63(1):75–104, 1996.
•
Shi, J. and Tomasi, C. (1994). Good features to track. In CVPR’94, pages 593–
600, IEEE Computer Society, Seattle.
•
Baker, S. and Matthews, I. (2004). Lucas-kanade 20 years on: A unifying
framework: Part 1: The quantity approximated, the warp update rule, and the
gradient descent approximation. IJCV, 56(3), 221–255.
CSE 576, Spring 2008
53
Motion estimation
Bibliography
•
T. Darrell and A. Pentland. Cooperative robust estimation using layers of support.
IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(5):474--487,
May 1995.
•
S. X. Ju, M. J. Black, and A. D. Jepson. Skin and bones: Multi-layer, locally affine,
optical flow and regularization with transparency. In IEEE Computer Society
Conference on Computer Vision and Pattern Recognition (CVPR'96), pages
307--314, San Francisco, California, June 1996.
•
M. Irani, B. Rousso, and S. Peleg. Computing occluding and transparent motions.
International Journal of Computer Vision, 12(1):5--16, January 1994.
•
H. S. Sawhney and S. Ayer. Compact representation of videos through dominant
multiple motion estimation. IEEE Transactions on Pattern Analysis and Machine
Intelligence, 18(8):814--830, August 1996.
•
M.-C. Lee et al. A layered video object coding system using sprite and affine motion
model. IEEE Transactions on Circuits and Systems for Video Technology, 7(1):
130--145, February 1997.
CSE 576, Spring 2008
54
Motion estimation
Bibliography
•
S. Baker, R. Szeliski, and P. Anandan. A layered approach to stereo reconstruction.
In IEEE CVPR'98, pages 434--441, Santa Barbara, June 1998.
•
R. Szeliski, S. Avidan, and P. Anandan. Layer extraction from multiple images
containing reflections and transparency. In IEEE CVPR'2000, volume 1, pages
246--253, Hilton Head Island, June 2000.
•
J. Shade, S. Gortler, L.-W. He, and R. Szeliski. Layered depth images. In Computer
Graphics (SIGGRAPH'98) Proceedings, pages 231--242, Orlando, July 1998. ACM
SIGGRAPH.
•
S. Laveau and O. D. Faugeras. 3-d scene representation as a collection of images. In
Twelfth International Conference on Pattern Recognition (ICPR'94), volume A,
pages 689--691, Jerusalem, Israel, October 1994. IEEE Computer Society Press.
•
P. H. S. Torr, R. Szeliski, and P. Anandan. An integrated Bayesian approach to layer
extraction from image sequences. In Seventh ICCV'98, pages 983--990, Kerkyra,
Greece, September 1999.
CSE 576, Spring 2008
55
Motion estimation