Realtime 3D model
construction with Microsoft
Kinect and an NVIDIA Kepler
laptop GPU
Paul Caheny
MSc in HPC 2011/2012 Project
Preparation Presentation.
Today's Presentation

Project motivation.

Introduction to Microsoft Kinect and depth maps.


A taster on algorithms for constructing 3D models from
depth maps.
Particular goals and constraints for this dissertation.
Paul Caheny 2012
Project Motivation
Proposal from Holoxica Ltd.




Most holograms produced from 3D models which are
computer generated ab initio.
Some customers already posses high
quality 3D models of real
world objects.
Commercial scanning solutions
prohibitively expensive for
Holoxica's purposes.
A cheaper, good quality, portable 3D scanning & model
construction system could make holograms a more viable
and attractive proposition for customers.
Paul Caheny 2012
Microsoft Kinect & Depth Maps



Depth maps have been the subject of much research for
3D model construction over the past 20 years.
The Kinect provides unprecedented
quality to cost ratio as a depth sensor.
Video Camera, IR Projector, IR Camera
IR Projector/Camera constitute a depth
sensor providing a 640 x 480 depth map
at 30Hz frame rate.

Paul Caheny 2012
3D Modelling from Depth Maps
Put simply – fusing multiple viewpoints of the scene or
object into a single unified 3D model.


Early approaches involved highly calibrated
motion of the object plus simple surface
construction techniques (e.g. directly
meshing the depth data points).
Current state of the art replaces calibrated
motion with software tracking & high quality
3D model construction techniques.

Two techniques: Iterative Closest Point (ICP) algorithm for object
tracking and Volumetric Integration for high quality model
generation.
Paul Caheny 2012
Iterative Closest Point (ICP)
A method for the alignment of 3D surfaces.



Introduced independently by Besl & McKay at GM and Chen &
Medioni at Uni. Southern California in early '90s.
If corresponding points on surface in two views are known, trivial to
compute exact transform which aligns surfaces. But we don't know
corresponding points on surfaces in distinct depth maps.
Make a heuristic guess of corresponding points, compute transform
with selected points, achieving closer alignment. Reselect points
heuristically on transformed surfaces, iterate until close enough.
Paul Caheny 2012
Volumetric Integration
A technique for fusing data from multiple aligned depth maps.



ICP alignment & depth sensors have inherent margin of error /
measurement uncertainty. Results in noisy data points following ICP
alignment.
Reduce noise & improve result by combining multiple views using
heuristics which minimises noise.
Curless &
Levoy,
Stanford '96
Paul Caheny 2012
What's New?
ICP and Volumetric Integration introduced in the '90s.




Both techniques have have been subject of much refinement since.
In the 90s workflow looked like: Scan -> ICP Batch process ->
Volumetric Integration Batch process -> Finished 3D Model.
Mid 2000s saw systems with realtime ICP phase plus batch
Volumetric Integration phase to create finished model.
2011 saw publication of research by Augmented Reality Group at
Microsoft Research Cambridge demonstrating a realtime, synchronous
ICP and Volumetric Integration system called KinectFusion running at
Kinect full frame rate of 30Hz.
Paul Caheny 2012
My Dissertation
Focus on small object scanning and Laptop GPU.



Implement the algorithms from scratch using CUDA.
Plan to use NVIDIA's next generation Kepler architecture, recently
launched for consumer desktop & laptop market.
Laptop GPU TDP ~35W versus Tesla 2090 TDP of 225W – Fewer
Cores, Less Memory, Less Bandwidth, Lower Clock Speed.
Paul Caheny 2012
Wrap Up
References:




R. Newcombe et al. KinectFusion: Real-time dense surface mapping and tracking.
In Proc. 10th IEEE Int. Symp. on Mixed and Augmented Reality, 2011.
B. Curless and M. Levoy. A volumetric method for building complex models from
range images. ACM Trans. on Graphics, 1996.
P. Besl and N. McKay. A method for registration of 3D shapes. IEEE Trans. on
Pattern Analysis & Machine Intelligence, 14:239–256, 1992.
Y. Chen and G. Medioni. Object modeling by registration of multiple range images.
Image and Vision Computing (IVC), 10(3):145–155, 1992.
Paul Caheny 2012