Teraflops for the Masses:
Killer Apps of Tomorrow
Pradeep K. Dubey
Senior Principal Engineer
Corporate Technology Group
EDGE
UNC, Raleigh, May 23, 2006
Evolution
Evolution continues
continues …
…
Media Evolution
Upcoming Transition
Next Transition
Modality-specific streaming
Modality-aware transformation
Graphics Evolution
Scene complexity: moderate
Local processing dominated
Multimodal recognition
Scene complexity: large
Global processing dominated
Mining Evolution
Scene complexity: real-world
Physical simulation dominated
Dataset: static/structured
Response: offline
Dataset: dynamic, multimodal
Response: interactive
Dataset: massive+streaming
Response: real-time
2
Workload
convergence:
multimodal
recognition
Workload
convergence:
multimodal
recognition and
and synthesis
synthesis over
over complex
complex datasets
datasets
May 23, 2006
Pradeep K. Dubey
pradeep.dubey@intel.com
1
Evolving
Evolving towards
towards model-based
model-based computing
computing
Large dataset mining
Semantic Web/Grid Mining
Streaming Data Mining
Mining
Distributed Data Mining
Content-based Retrieval
Multimodal event/object Recognition
IndexingCollaborative Filters
Statistical Computing
Multidimensional Indexing
Machine Learning
Streaming
Dimensionality Reduction
Dynamic Ontologies
Clustering / Classification
Efficient access to large, unstructured, sparse datasets
Model-based:
Recognition
Stream Processing
Bayesian network/Markov Model
Photo-real Synthesis
Neural network / Probability networks
Real-world animation
LP/IP/QP/Stochastic Optimization
Synthesis
3
May 23, 2006
Pradeep K. Dubey
Ray tracing
Global Illumination
Behavioral Synthesis
Graphics Physical simulation
Kinematics
Emotion synthesis
Audio synthesis
pradeep.dubey@intel.com
Video/Image synthesis
Document synthesis
Recognition
Mining
Synthesis
What is a tumor?
Is there a tumor here?
What if the tumor progresses?
ItIt is
is all
all about
about dealing
dealing efficiently
efficiently with
with complex
complex multimodal
multimodal datasets
datasets
Images courtesy: http://splweb.bwh.harvard.edu:8000/pages/images_movies.html
4
May 23, 2006
Pradeep K. Dubey
pradeep.dubey@intel.com
2
RMS:
RMS: Recognition
Recognition Mining
Mining Synthesis
Synthesis
Recognition
Mining
Synthesis
What is …?
Is it …?
What if …?
Model
Find a model
instance
Create a model
instance
Today
Model-less
Real-time streaming and
transactions on
static – structured datasets
Model-based
multimodal
recognition
Real-time analytics on
dynamic, unstructured,
multimodal datasets
Very limited realism
Tomorrow
5
May 23, 2006
Pradeep K. Dubey
Photo-realism and
physics-based
animation
pradeep.dubey@intel.com
Interactive
Interactive RMS
RMS (iRMS)
(iRMS)
Recognition
Mining
Synthesis
What is …?
Is it …?
What if …?
Model
Find an existing
model instance
Create a new
model instance
Graphics Rendering + Physical Simulation
Learning &
Modeling
Computer
Vision
Visual Input
Streams
Reality
Augmentation
Synthesized
Visuals
Most
Most RMS
RMS apps
apps are
are about
about enabling
enabling interactive
interactive (real-time)
(real-time) RMS
RMS Loop
Loop or
or iRMS
iRMS
6
May 23, 2006
Pradeep K. Dubey
pradeep.dubey@intel.com
3
Next-Generation
Next-Generation Entertainment
Entertainment
RMS Primitives:
Model the ball
Find the ball
Replace the ball
Shade/Bounce
the ball
Going
Going beyond
beyond media-stream
media-stream encode-decode-transcode!
encode-decode-transcode!
7
May 23, 2006
Pradeep K. Dubey
pradeep.dubey@intel.com
Real-time
Real-time DCC
DCC Loop
Loop
What if … what if …
Model the ball
Mine/Track/Replace
the ball
RMS Closed Loop
Rendering+Physics+Vision
Shade/Bounce
the ball
Going
Going beyond
beyond ‘red-eye
‘red-eye removal’
removal’
8
May 23, 2006
Pradeep K. Dubey
pradeep.dubey@intel.com
4
Where are we headed …
Machine learning
Neural networks
Probabilistic reasoning
Rendering
Simulation
Fuzzy logic
Belief networks
Evolutionary computing
Chaos theory
collision detection
force solver
global illumination
…
…
Physics
Soft
Computing
Dynamics
9
May 23, 2006
Constraint
Dynamics
Constraints
Pradeep K. Dubey
Soft
Physics?
pradeep.dubey@intel.com
RMS
RMS Computing
Computing Core
Core
Unstructured Information
Management
Analytics
Vision/Tracking
Gaming
Benefiting Applications
Real-time asset management, text mining
camera stream mining, 3D graphics …
Pool of Mathematical Techniques
Conic optimization, Subspace projection …
Pool of RMS Functions
Interior-point, Spectral Bundle, SVM …
10
May 23, 2006
Pradeep K. Dubey
pradeep.dubey@intel.com
5
Computer Vision
Rendering Physical Simulation
Face
Body
Tracking Detection
Global
Illumination
CFD
Face
(Financial) Analytics
Portfolio
Management
Cloth
Data Mining
Derivative
Pricing
Clustering
Indexing
Classification
SVM
Classification
PDE
SVM
Training
NLP
IPM
(LP, QP)
Level Set
Particle
Filtering
Springs
Fast Marching
Method
K-Means
Monte Carlo
Direct Solver
(Cholesky)
Krylov Iterative Solvers
(PCG)
FIMI
Basic Iterative Solver
(Jacobi, GS, SOR)
Text
Indexer
Integrator
Basic matrix primitives
(dense/sparse, structured/unstructured)
May 23, 2006
11
Workload Convergence Î RMS Primitives
Pradeep K. Dubey
pradeep.dubey@intel.com
Hollywood
Hollywood to
to Wall
Wall Street
Street
Common Core
Computing Kernel
Entertainment
Tomorrow
Stochastic Optimization
(Sim Annealing, Genetic Alg,
Bayes Learning)
Vision
Object Recognition
Object Tracking
Numerical Integration
Convex Optimization
(LP, QP, SOCP, SDP, Network, SVM)
Mesh Refinement
Graphics
Portfolio Selection
Asset Allocation
(Monte Carlo, Quasi-MC, Gaussian)
Computer Vision:
Depth from Stereo
Combinatorial Optimization
Rendering:
Path Tracing
Computational
Fluid Dynamics
Business/Finance
Today
Asset-Liability
Management
(Integer Prog, Dynamic Prog)
Risk Management
Differential Equations Solvers
Interest-Rate
Derivative Pricing
(Parabolic, Elliptic, Hyperbolic,
Finite Element Method, Stochastic)
AI for Games:
Path planning
Multi-Look
Option Pricing
Multi-Party Auctions
Iterative Solvers
(Conjugate Gradients, Gauss-Seidel,
Jacobi, GMRES)
12
May 23, 2006
Pradeep K. Dubey
pradeep.dubey@intel.com
6
RMS Computing Core:
Scaling to Next Generation Needs
Map-based shading
Global Illumination based
SIMPLEX based linear
optimization
IPM based LP/QP/NLP
optimization
Mass-Spring based deformation
FD/FE/FV based deformation
Marker-based explicit surface
tracking
Non-Linear
Level Set based
implicit surface
tracking And
Linear manifold based
recognition/modeling
Non-linear manifolds computer
vision
Linear Complementarity problem
Non-linear Complementarity
Low dimension classifiers
High dimension classifiers
13
May 23, 2006
Pradeep K. Dubey
Generative
pradeep.dubey@intel.com
Summary
There are mass applications that require significant increase in
compute density
• There is nothing as general-purpose as physics!
• Visual computing is a proxy of this much larger class (RMS)
These applications are not linear extensions of existing usage
• Optimal platform for such apps should not be linear extension
either
There is a significant performance difference between a bruteforce CMP Vs. a smart CMP targeted for this class
• There is significant opportunity for silicon differentiation
These apps will likely be the driver for most future technology
vectors
• Programming to processor to memory technology
14
May 23, 2006
Pradeep K. Dubey
pradeep.dubey@intel.com
7
15
May 23, 2006
Pradeep K. Dubey
pradeep.dubey@intel.com
8