Phase and color extraction for color fringe projection system
based on optimum frequency selection
Z H Zhang1, C E Towers2 and D P Towers2
1
School of Engineering and Physical Science, Heriot-Watt University, Edinburgh, UK
EH14 4AS
2
School of Mechanical Engineering, University of Leeds, Leeds. UK. LS2 9JT
z.zhang@hw.ac.uk, zhzhangtju@hotmail.com
Abstract. In this paper, we present a novel method to simultaneously extract the phase (and
hence XYZ) and color (RGB) information from composite color fringe patterns captured by a
color fringe projection system. Three sinusoidal fringe patterns with different frequencies
determined by the optimum frequency selection are encoded in the red, green and blue
channels of the composite fringe pattern. Due to the color mismatch of a 3-chip color CCD
camera and a Digital Light Processing (DLP) video projector, coupling effects between the
three channels are unavoidable and they have effects on separating the color channels. Though
coupling effects can be calibrated in advance by projecting pure red, green and blue fringe
patterns onto a white plate, software-based and hardware-based compensation methods can be
used to decrease coupling effects. Using the off-the-shelf 3-chip color CCD camera and DLP
projector without any modification, we explored a software-based method to calculate the
phase and color information by including the coupling effects into an optimization algorithm.
Each pixel is represented as the combination of the three channels by the obtained coupling
effects. Three phases and modulations encoded in the three color channels are simultaneously
calculated by an optimization method. Experiments based on simulated data and captured data
are conducted. Results show that even with high coupling effects between channels, the
obtained phases have high phase resolutions and the modulations have stable values in each
channel. Since optimum three-frequency selection was used to get the absolute phase, it has the
ability to measure objects with discontinuities.
1. Introduction
Non-contact optical 3D imaging methods have been widely studied in research fields [1] and applied
in commercial applications [2]. For measuring objects with discontinuities or large slope changes on
the surface, the mostly used technique is the temporal phase unwrapping algorithm developed by
Huntley et al [3]. A set of fringe patterns with different fringe pitches are consecutively projected onto
an object’s surface and are captured by a camera from a different viewpoint, which needs long time to
finish acquisition. Recently, Towers et al developed an optimization method for three-frequency
selection where a geometric series of synthetic wavelengths is defined to maximize the overall process
reliability [4,5]. The absolute fringe order for each pixel can be independently determined from the
three obtained phase-wrapped maps corresponding to the three frequencies respectively. Using the red,
green and blue channels of camera and projector, we explored a color fringe projection system to
modulate the three fringe patterns into the three color channels to reduce the capturing time further [6]:
one image for the Fourier transform and four images for the four-frame phase shifting algorithm. For
the most color CCD cameras and digital light processing (DLP) video projectors, the spectra of the
three channels are designed to overlap so that there is no color-blind area, which means coupling
effects between channels are unavoidable. Due to coupling effects, calculating the phase by directly
separating the three channels has large noise, as reported in [6].
Software-based and hardware-based compensation methods can be used to decrease coupling
effects [7]. Using the off-the-shelf 3-chip color CCD camera and DLP projector without any
modification, we will explore a software-based method to simultaneously extract the phase and color
information from the composite color fringe patterns. When the parameters of a camera and a
projector are unchangeable, the system’s coupling effects can be calibrated by projecting pure primary
color (red, green and blue) fringe pattern onto a white board [6]. Each pixel of the captured image is
represented as the combination of the three color channels by the obtained coupling effects. Three
phases and modulations encoded in the three color channels are simultaneously calculated by an
optimization method. Therefore, not only shape but also color of a captured object can be obtained
from the composite color fringe patterns.
2. Principle
When three fringe patterns with different pitches are simultaneously projected from the red, green and
blue channels of a DLP projector and the composite RGB fringe patterns are captured by a 3-chip
color CCD camera, one pixel I i in a chosen channel i containing coupling effect ki , j can be
represented as



I i x, y   I i, dc x, y   I i, m x, y cosi x, y   i    ki, j I j , dc x, y   I j , m x, y cos  j x, y    j  I i, D x, y , (1)
i, j  R, G, B, but j  i
where I i , dc and I i , m are the average intensity and intensity modulation, respectively; i is the phase
corresponding to the object height; i is the phase shift; ki , j is coupling effect calibrated beforehand
[6] and I j , dc , I j , m ,  j ,  j have the same meanings as the corresponding parts in the chosen channel i.
I i , D is the background intensity from the dark current of the camera for the chosen channel. x and y are
the indexes of one pixel and in the following we will omit them for brevity.   represents the
crosstalk coming from the other two channels.
For four-frame phase shifting algorithm, i , j  0, / 2, ,3 / 2 , the corresponding phase-shifted
fringe patterns in the red, green and blue channels can be represented as
I r ,n  I r ,dc  I r ,m cosr  n / 2  k g ,r I g ,dc  I g ,m cosg  n / 2 kb,r I b,dc  I b,m cosb  n / 2  I r ,D , (2)


I g ,n  I g ,dc  I g ,m cos g  n / 2  kr ,g I r ,dc  I r ,m cosr  n / 2 kb,g I b,dc  I b,m cosb  n / 2  I g ,D , (3)



I b,n  I b,dc  I b,m cosb  n / 2  kr ,b I r ,dc  I r ,m cosr  n / 2 k g ,b I g ,dc  I g ,m cos g  n / 2  I b,D , (4)
where n  0,1,2,3 . In fact, equations (2)-(4) contain twelve equations altogether. Combining the twelve
equations gets the following equations
 I r 0  I r180  2 I r ,m cosr   2k g ,r I g ,m cos g   kb,r I b,m cosb 
I
 r 270  I r 90  2 I r ,m sin r   2k g ,r I g ,m sin  g   kb,r I b,m sin b 
 I g 0  I g180  2 I g ,m cos g   2k r , g I r ,m cosr   kb, g I b,m cosb 
.
(5)

 I g 270  I g 90  2 I g ,m sin  g   2k r , g I r ,m sin r   kb, g I b,m sin b 
 I b 0  I b180  2 I b,m cosb   2k r ,b I r ,m cosr   k g ,b I g ,m cos g 

 I b 270  I b90  2 I b,m sin b   2k r ,b I r ,m sin r   k g ,b I g ,m sin  g 
There are six unknown variables, I r , m , I g , m , I b, m , r , g , b , in equation (5), so solving these equations
can simultaneously get the phase and intensity modulation in the three channels. A nonlinear
optimisation subroutine called lsqnonlin in Matlab 7 is used to calculate the unknown variables.
Applying this algorithm to all the pixels estimates phase and modulation information in the three color
channels together.
3. Experiments and results
3.1. Simulation
First we evaluated this algorithm using the simulated data. Because the optimized three-frequency
selection method is used in our color fringe projection system [6], the synthetic composite fringe
patterns have 90, 99 and 100 fringes in the red, green and blue channels, respectively. The average
intensity and intensity modulation for all channels are 100 and 60. In order to test the robustness to the
noise, we added 3.5% Gaussian noise to the generated fringe patterns. Twenty-five RGB fringe
patterns with coupling effects of 1, 2, …, 25 % were synthesized by software in a computer. To
simplify the evaluation without losing its generality, we just consider a 1-D case along the x axis
including all fringe patterns. One synthetic composite RGB fringe pattern with 20% coupling effect is
shown in Figure 1. Using the proposed algorithm, we calculated phase and modulation in each channel
and then evaluated the performance by using resolution for phase and standard deviation for
modulation, as shown in Figure 2(a) and (b). In comparison, we also calculated phase resolution and
standard deviation of modulation by using the standard four-frame phase shifting algorithm [8], as
shown in Figure 2(c) and (d). We can see coupling effect has no effect on calculating phase and
modulation by using our proposed algorithm. The standard phase shifting algorithm becomes much
worse for calculating phase and modulation when coupling effect increases.
(a)
(a)
150
200
250
300
350
400
450
500
(b)
250
200
150
100
50
150
Modulation std
100
Phase resolution
50
(b)
4
250
200
150
100
50
100
50
5
10
15
20
3.5
3
2.5
2
25
5
10
(c)
15
20
25
15
20
25
(d)
100
150
200
250
300
350
400
450
500
(c)
250
200
150
100
50
50
100
150
200
250
300
350
400
450
500
Figure 1 Synthetic composite RGB fringe pattern
with 20% coupling effect and 3.5% Gaussian
noise. The fringe number in R, G and B is 90, 99
and 100, respectively. (a) red channel; (b) green
channel and (c) blue channel.
150
Modulation std
50
Phase resolution
14
100
50
5
10
15
20
25
12
10
8
6
4
2
5
10
Figure 2 Phase resolution and modulation
standard deviation with different coupling effects.
The solid, dashed and dotted line corresponds to
the red, green and blue channels, respectively.
3.2. Experiment
We conducted the experiments on the actual composite RGB fringe pattern on a white plane board
captured by our color fringe projection system [6]. Such four composite fringe patterns with π/2 phase
shift consecutively in each channel were generated in a computer and sent out onto the white board by
a DLP projector. A 3-chip color CCD camera captured these deformed color fringe patterns from
another viewpoint. For different color cameras and DLP projectors, and even for the same camera and
projector with different parameters, coupling effects between channels are different. We calibrated our
system to obtain coupling effect. Applying the proposed algorithm to each pixel calculated the
wrapped phase and modulation. Figure 3 shows one composite RGB fringe pattern and the obtained
phase in the three channels. Table 1 gives the phase resolution by using our proposed algorithm and
the standard phase shifting algorithm. In order to quantitatively display the modulation, we averaged
all these rows to get a profile, as illustrated by the solid line in Figure 4. In the meanwhile, we
calculated the modulation by the standard method in [8], as illustrated by the dash line in Figure 4. In
actual situations, due to the influences of the number of fringes, the intensity values and the uneven
intensity in the captured image, coupling effect in each channel is not a constant. Therefore, the
acquired modulations have unstable values in the three channels. Table 1 shows the standard deviation
of modulation for the three channels by the two algorithms. Our proposed algorithm gives a much
better result in the red channel, a little better in the green channel, while the blue channel is a little
worse than the standard phase shifting and modulation calculation algorithm. We believe this comes
from the assumed unchangeable coupling effect used in our algorithm.
(a)
(b)
(c)
Figure 3 Phase calculated
by the proposed algorithm.
(a) is one composite RGB
fringe pattern captured by
our color projection system;
(b), (c) and (d) are phase in
the red, green and blue
channels, respectively.
(d)
(a)
100
80
60
100
200
300
400
500
600
700
800
900
1000
600
700
800
900
1000
600
700
800
900
1000
(b)
100
80
60
100
200
300
400
500
(c)
100
80
60
100
200
300
400
500
Figure 4 Average modulation in the red, green and blue channels. The solid and dotted lines
correspond to the proposed algorithm and the algorithm in [8]. The three rows correspond to the
red, green and blue channels, respectively.
Table 1 Performance in the three channels by the two algorithms.
Phase
resolution
Standard
deviation of
modulation
Channel
Algorithm in [8]
Proposed algorithm
Algorithm in [8]
Proposed algorithm
Red
44
151
10.00
3.03
Green
93
144
4.12
3.09
Blue
121
129
2.51
3.68
4. Conclusions
In this paper, we explored a novel method to simultaneously extract the phase (XYZ) and color (RGB)
information from the composite color fringe patterns including the coupling effects between channels.
The color fringe patterns are captured by the 3D imaging system, including an off-the-shelf 3-chip
color CCD camera and DLP projector without any modification. Simulated and experimental data are
tested on the proposed method. Compared to the standard four-phase shifting algorithm, the obtained
phase had a large resolution and the modulations have stable values. Even for a high value of coupling
effect between channels, this algorithm extracts phase and modulation reliably. Since optimum threefrequency selection method is used, the absolute fringe order for each pixel can be determined from
the three obtained wrapped phase. Therefore, the color fringe projection system can not only measure
the shape but also the color of objects with discontinuities and large slope changes on the surface.
Because each pixel needs an optimization method, the computational cost is a big problem for now.
On a personal computer with the following configurations: Pentium 4, 3.0 GHz processor and 1,024M
RAM (random access memory), the processing time for an image with size 1024x768 is about ten
hours. Another improvement is how to efficiently calibrate the coupling effects for a given DLP video
projector and a color CCD camera.
References
[1] Chen F, Brown G M and M. Song 2000 Opt. Eng. 39 10-22
[2] Blais F 2004 J. Electron Imaging 13 231-240
[3] Huntley J M and Saldner H O 1993 Appl. Opt. 32 3047-52
[4] Towers C E, Towers D P and Jones J D C 2004 Opt. Lett. 29 1348-50
[5] Towers C E, Towers D P and Jones J D C 2005 Opt. Lasers Eng. 43 788-800
[6] Zhang Z H, Towers C E and Towers D P 2006 Opt. Express 14 6444-55
[7] Pan J H, Huang P S and Chiang F P, 2006 Opt. Eng. 45
[8] Creath K, 1988 Phase measurement interferometry techniques Progress in Optics XXVI, Vol 26
ed E. Wolf (Amsterdam: North Holland Publ.) chapter 5 pp 349-393