Image Mosaic Techniques
for the Restoration of Virtual Heritage
2003. 8. 28
Yong-Moo Kwon, Ig-Jae Kim,
Tae-Sung Lee, Se-Un Ryu, Jae-Kyung Seol
KIST
KOREA
Contents

Revisiting Image Mosaic Technique

Our Researches for Image Mosaic


IR Reflectography Image Mosaic

X-Ray Image Mosaic
Summary
Revisiting Image Mosaic Technique


Image Mosaicing

Panorama Image

Image Based Rendering (IBR)
Basic Algorithm

Registration using Features

Image Warping based on Homography Matrix

Blending Images
Target Dimension in view of Image Mosaicing

2D Target

Planar Paintings Image

Homography Technique


Feature-Based Image Mosaicing
3D Target

3D Real World Image

Limitation using Homography

Due to Depth Difference b/w Features in Target
Our Research for Image Mosaic

2D Target

IR Reflectography


Special 3D Target

X-Ray Imaging


Mural Underdrawings Mosaic
Old Sword X-Ray Image Mosaic
Research Topics

How to extract and use Features

Imaging Media (IR, X-Ray)

Feature’s characteristics are different from the previous ones
IR Image Mosaic

IR Reflectography System

IR Source

IR Filter

IR Camera
Murals
IR Reflectography Principle
Visible Light
Reflection
UnderDrawing
IR
Color Painting,
Reflection
Dust
Absorbed
Back Frame
IR Reflectography Camera

IR Camera : Super eye C2847 (~1.9㎛)

Hamamatsu

IR Source : ~1.9㎛

IR Filter : CVI Laser Corp.

NIR bandwidth filter


Bandpass Filter


800nm ~ 2000nm Pass
every 100 nm bandpass filter (800nm, 900, …, 2000nm)
IR Characteristics to Mural according to WL

IR Camera



HAMAMATSU Super eye
C2847
WL Range : 0.4㎛ ~ 1.9㎛
IR Source

HAMAMATSU C1385-02
 Filter

HAMAMATSU IR-D80A : 0.8㎛ ~ 1.9㎛

CVI Laser corporation


Near IR Interference BP filter
800nm, 900nm, … 2000nm
Sony PC-115


Digital Image Capture
Night Shot
IR Image Mosaic for Mural Underdrawing


Basic Method

Automatic Feature Extraction

Registration Using Features

Image Warping

Image Blending
Main Considerations

IR Wavelength Characteristics

Penetration Ratio into Paintings

Color (Red, Green, Blue etc)

Color Painting Depth
Our Approach
▶ Automatic Feature Extraction & Registration
- Cross Points in IR Underdrawing Image
- Grid Pattern for Blank Space
▶ Adaptive Overlapping Area For Image Blending
- Trade-Off between Registration and Blending
* Large Overlapping Area: Good for Registration
* Small Overlapping Area: Good for Blending
▶ Use feature of IR Spectrum
- Use Different IR Wavelength according to paining color
Automatic Feature Extraction
 Feature of Korea Murals
- Many Blank Space
- Not so much good features
1> Visible Light Pattern
2> Twice Captures
- w/o IR Filter
- w/i IR Filter
IR Image Mosaic
-
Homography Estimation using Grid Image & IR Image
- Apply Homography to IR Image
X-RAY Image Mosaic
Why we use X-ray Technique ?

Old Sword


Old Sword is inside Sword Cover

Weak for Touch & Manipulation

Can’t Open Sword Cover
Use X-Ray Technique for the restoration of
Old Sword inside Sword Cover

Sword for experiment
Schema of a x-ray imaging using a linear X-Ray Camera
1.
X-Ray Image
2.
X-Ray Tube
3.
X-Rays
4.
X-Ray Detector
5.
PC
6.
Object
Why X-Ray Image Mosaic ?

For High Resolution Imaging


Multiple X-Ray Imaging

Setting Object

X-Ray Image Capture

Move Object Upward or Downward Step-By-Step
Stitching X-Ray Images into High Resolution Image
X-Ray Imaging Principle

Basic Principle

X-Ray Particle Penetrates through Target

One Point Depth -> Grey Value Pixel

Dependency

Target Depth

Target Material
X-Ray Image Characteristics: 2D or 3D ?

Target Dimension in view of Image Mosaic

Well Controlled Penetration Angle

Image Pixel Depends on Penetration Angle

Usually Same Penetration Angle for Each Capture


Just Planar 2D Image Using CCD Camera


Orthogonal axis Movement according to X-Ray Beam
Object -> X-Ray Camera -> CCD Camera
2D Target: Homography Technique
X-RAY Image Equipment
X-TEK X-Ray System
X-Ray Source & Object (Sword)
X-RAY Image Capture


For High-Resolution Restoration
 Multiple X-Ray Imaging
 Image Stitching Technique
 Feature-based Registration
Problem ?
 Difficult to use features in X-ray Image
 Using Feature Pattern
Feature Extraction

Feature Extraction From Known Pattern

Circle Type & Rectangular Type
 Circle
Type -> Pattern Matching
 Rectangular -> Feature Points
Feature
Ex traction
Binary pattern
for feature ID
Feature Extraction

Method


Circle Type Pattern -> Apply Image Labeling
Rectangular Type Pattern -> Corner Detection
  D x2
C 
  D x D y
 

D x  D y 
2
- For every pixel of image, computes
first derivatives Dx and Dy.
- The eigenvalues are found by solving
det(C- λI )= 0
 DxDy 
2 
D
 y 
2
( D x   D y )  4( D x
2
2
2
2
2
If λ1, λ2 > t, where t is some threshold, then a
corner is found at that location

D y  ( D x D y ) )
2
2
Feature Point Matching

Semi-Auto(Present)



Automatic Matching
(On-going)


Automatic Feature
Extraction of Rectangle
Type pattern
Manual Matching
Classify the features using
pattern ID from Circle Type
Pattern
Homography Matrix

Apply LS-Method(Least
Square Method) using
Matched feature Points
Semi-auto Demo
Implemented S/W
X-ray
Image File
Handling
Feature
Extraction
&
Select
Points
Homograph
y Matrix
Estimation
& Stitching
Generated HighResolution X-ray
Image
More Experimentation
Summary


Application of Image Mosaicing Techniques

Infrared Image

X-Ray Image
Our Approach

Feature Pattern

Automatic Feature Extraction & Registration

Homography Technique

Imaging Media (IR, X-Ray) & Feature’s Characteristics
Thank You !
ymk@cherry.kist.re.kr