Image-based
Water Surface
Reconstruction with
Refractive Stereo
Nigel Morris
University of Toronto
Motivation
Computational Fluid Dynamics are
extremely complex and difficult to simulate
 Why not capture fluid effects from reality?
 We present the first step to capturing fluids
from reality – reconstructing water
surfaces
 May eventually be useful for determining
fluid flow

Previous Work

Shape from shading [Schultz94]
 Requires
large area light source or multiple
views

Shape from refractive distortion
[Murase90]
 Limited
wave amplitude, orthographic camera
model

Laser range finders [Wu90]
 Specialized
equipment
Previous work

Shape from refractive
irradiance [Jähne92],
[Zhang94] &
[Daida95]
 Requires
underwater
lens, orthographic
camera model
Goals of our system
Physically-consistent water surface
reconstruction
 Reconstruction of rapid sequences of
flowing, shallow water
 High reconstruction resolution
 Use of a minimal number of viewpoints
and props

Technical Contributions
We present a design for a stereo system
for capturing sequences of dynamic water
 System implementation and results
 Refractive stereo matching metrics and
analysis
 Effective localization of surface points of
shallow water

Refraction

Snell’s Law
 r1sin
Θi = r2sin Θr
For air → water:

sin Θi = rwsin Θr
Imaging water



Image point f at q
without water
Image f at q’ with
water
qq’ is the refractive
disparity
Deriving the surface normal



Suppose we know the
location of the surface
point p and its depth
from the camera z
We know the angle θδ
between the refracted
rays u and v
Can compute the
incident angle θi, then
the normal n:
Solution space

For given refractive
disparity, set of
solution pairs:
 nmzm

For every depth z,
there is at most one
normal n
Reconstruction with Stereo
Same setup as with one camera, but with
additional calibrated camera
 We search through the <nmzm> solution
space for a particular refractive disparity
 We use the second camera to determine
the error for each instance of nmzm
 Return best surface point p

Refractive stereo matching
Camera 2
Camera 1
n2 n1
n
Tank
Bottom
Matching metric

Normal collinearity metric
 Measure
the angle between the two normals
n1 and n2 to give an error.

Disparity difference metric
 Swap
n1 and n2 and reproject to tank plane,
measure disparity from the projection before
swapping.
 Seeks to minimize error due to inaccurate
normal measurements as water depth
approaches localization error range.
Disparity Difference Metric
Camera 2
Camera 1
Tank
Bottom
e1
e2
Metric Comparison


Disparity difference metric in red
Normal collinearity metric in blue
Implementation details

Pattern choice
 Checkered

pattern used
Tracking pattern and localization
 Lucas-Kanade

matching
Interpolation of the discrete pattern
System Inputs



Calibrated stereo
camera system
Images of pattern
without water from
both cameras to give
refractive disparities
Distorted pattern
image sequences
Corner tracking
In order to reconstruct a sequence of
frames, the corners must be localized at
every frame
 We employ a Lucas-Kanade matching
technique, matching templates of the
corners to the next frame

Corner Interpolation




We cannot assume that our
verification ray will land on
one of the corners
We thus find the four nearest
non-collinear corners
The surface may be distorted
so we cannot assume a grid
formation
We interpolate between these
corners to find the distortion of
the verification ray
Results
Ripple Drop
 Waves
 Pouring water

Future Work
Global surface minimization vs local
 Planar tank constraint removal
 More complex water scenario capturing

References

[Jähne92] B. Jähne, J. Klinke, P. Geissler, and F. Hering. Image sequence
analysis of ocean wind waves. In Proc. International Seminar on Imaging in
Transport Processes, 1992.

[Murase90] H. Murase. Shape reconstruction of an undulating transparent
object. In Proc. IEEE Intl. Conf. Computer Vision, pages 313–317, 1990.

[Schultz94] H. Schultz. Retrieving shape information from multiple images of
a specular surface. IEEE Transactions on Pattern Analysis and Machine
Intelligence, 16(2):195–201, 1994.

[Wu90] Z. Wu and G. A. Meadows. 2-D surface reconstruction of water
waves. In Engineering in the Ocean Environment. Conference Proceedings,
pages 416–421, 1990.

[Zhang94] X. Zhang and C. Cox. Measuring the two-dimensional structure
of a wavy water surface optically: A surface gradient detector. Experiments
in Fluids, Springer Verlag, 17:225–237, 1994.