My Research Experience
Cheng Qian
Outline
• 3D Reconstruction Based on Range Images
• Color Engineering
• Thermal Image Restoration
3D – Overview
To reconstruct the geometry and texture of a scene in a virtual environment.
--- 3D scanning
Method 1：2D color-image-based reconstruction
Create an arbitrary view by interpolation
3D – Overview
Method 2：Range-image-based reconstruction
CCD Image--RGB
Range image -----Depth
Intensity image --Reflectance
3D – Overview
Range Image
Intensity Image
CCD Image
Geometric Structure
Materials
Texture
A digital model
3D – Overview
System architecture
Knowledge
from Object Recognition
Visual Information
(Geometry, Texture)
Preprocessing
Registration
Mesh
Texture Mapping
Lighting
Raw Data
(3D coordinates,
Intensity, RGB)
Shading …..
Modeling
3D – Range Image Registration
Objective
Image of a left view
Image of a right view
Images registered
3D – Range Image Registration
Scheme
Range image I1
Range image I2
Noise filtering, outlier removing
……
Feature Extraction: surface, curve, corner point
……
Description based on geometric features
and their interrelationships
……
Construct feature correspondence M and measure the Similarity S between the two images
------ Find the Ｍ maximizing S
3D – Range Image Registration
Noise filtering, outlier removing
•
Polar window filtering,
•
Pseudo-median filtering
•
Isolated point filtering
Before
After
3D – Range Image Registration
Feature extraction
Surface: adaptive-shape window
Curve, corners: edge evolution
3D – Range Image Registration
Descriptions of the geometric features
Related geometric features are nested
Interrelationship contained in nested geometric features
3D – Range Image Registration
Correspondence and similarity measure
Virtual features
Virtual features
3D – Range Image Registration
Improvement of the registration results
Global optimization
Before
After
3D – Range Image Registration
What was left: Texture
Color Engineering
• Proposed a method for measuring luminance distribution of indoor scenes
using a digital camera rather than an expensive luminance meter.
1.4
1.35
1.3
1.25
1.2
1.15
1.1
90
1.05
1
90
60
30
60
0
30
0
-30
-30
-60
-90
-60
• Proposed a novel radiometric model for CCD sensors and a color self-calibration
algorithm based on this model. The objective of this project is to calibrate the color
performance of 100 CCDs in a lightfield-rendering system for 3D scene reconstruction.
Transformed to be
CCD 1
To approximate
Fake CCD 2
CCD2
Color Engineering
• Calibrated a line CCD sensor with poor color performance. The radiometric
correlation between r, g, b channels is considered.
 r    m11 m12
Min  g    m21 m22
b   m31 m32
m13
m23
m33
r 
m14   
g
m24   
b 
m34   
1
m12 , m13 , m21, m23 , m31, m32  0
Thermal Imaging
•Proposed a radiometric model for infrared cameras and developed relevant model
reconstruction methods, which resulted in obtaining a very precise forward
function for thermal image restoration. Regularization techniques, such as
Tikhonov, Total Variation, and Lasso, were applied to the restoration procedure and
their performances were compared.
Thermal camera model
Thermal Imaging
Thermal image restoration
Original image
Image restored by
Tikhonov regularization,
Edges are strongly
penalized
Image restored by Total
Variation regularization,
Edges are preserved
Image restored by DiscontinuityAdaptive model regularization, Edges
are adaptively penalized
Noise is suppressed
Convexity of energy function is well
controlled.
Thermal Imaging
• With the adjustment of the camera setting, the point spread function (PSF) of the
camera system can be changed. Therefore we try to develop a semi-blind image
restoration algorithm that can recover the original image and the PSF simultaneously.
Original image
Iteration 1
Iteration 2
Iteration 6
Iteration 7
(a)
Iteration 3
Iteration 4
Iteration 1
Iteration 5
Iteration 8
Iteration 9
Final restoration results
• Thanks