STEREO IMAGING OF WELD POOLS
Andrew Neill
Introduction
‣ In-Situ weld pool 3D vision
‣ Robotic welding systems
– Deaf, dumb, and blind
– Limited in control and adaptability
‣ Holy Grail of Robotic Welding:
– Comprehensive sensing and feedback
– Weld pool characterization
– Adaptive real time control
Introduction
‣ Problem domain
– Camera Calibration
– OpenCV stereo implementation
– Refine matching by algorithms and adjusting matching criteria
‣ Assumptions
– Stereo camera rig fixed in position on welding torch
‧ All regions of interest will be the same
– Reading images from files rather than directly from process
‧ Better in terms of cost, time, and working outside of lab
‧ Allows analysis of speed
Previous Work
‣ Off-line stereo
– 3D over time
– Visualization of weld pool motion
– Effect of parameter variation on length,
width, height, etc.
[2]
‣ Other’s techniques: all postprocess
– Structured light
– Bi-prism stereo
[1]
[4]
Approach
‣ Calibrate cameras
‣ Acquire stereo image sets
‣ Implement stereo matching
– Stereo Block Matching
– Semi-Global Block Matching
‣ Refine matching parameters
Experiments
‣ Camera Calibration
‣ Choosing exposure for
collecting weld images
‣ Block Matching
parameter tuning
BMTune Screenshot
[3]
Results: Test Scene
Results: Stereo Matching Initial Results
Original Image:
Rectified Image:
Results: BM vs. SGBM
‣ BM vs SGBM
Results: Calibration Tuning
‣ Several
controls to
consider
1000
1000
1000
1000
1000
1000
1000
10000
100000
200
200
200
200
200
Inf
1.00E‐05
1.00E‐05 X
1.00E‐05 X
1.00E‐05
1.00E‐05
1.00E‐05 X
1.00E‐05
1.00E‐05
1.00E‐05
1.00E‐05
1.00E‐05
1.00E‐05
1.00E‐05
1.00E‐05
1.00E‐05
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
FixK5
FixK4
FixK3
FixK2
FixK1
ZeroTangentDist
SameFocalLength
FixFocalLength
FixAspectRatio
FixPrincipalPoint
UseIntrinsicGuess
Avg Error
1
208.1
2
172.58
3
172.59
4 1007.43
5
275.83
6
172.59
7 1007.43
8
231.77
9
282.39
10
230.86
11
234.07
12 307.354
13 2111.75
14
662.4
15
172
FixIntrinsic
Run
MinError
Calibration
more or less
automated
Iterations
‣ OpenCV
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
Discussion and Conclusions
‣ Calibration
– Poor calibration limits matching ability
– Matching requires epipolar geometry
‣ OpenCV Stereo
– Very fast, 0.5 frames/second with high resolution images
– Tunable for different scenes to improve speed
Works Cited
‣
‣
[1] Chen, S. B. (2005). Acquisition of Weld Seam Dimensional Position Information for Arc Welding Robot Based on Vision Computing. 43.1, 77-79.
Hirshmuller, H. (2008). Stereo processing by semiglobal matching and mutual information. IEEE Transactions on pattern analysis and machine
intelligence , 30 (2), 328-41.
‣
Hoff, W. (2012). Stereo Vision. EGGN 512, Computer Vision. Class Lecture . Colorado School of Mines.
‣
Liu, Y. K., Zhang, W. J., & Zhang, Y. M. (2013, November). Estimation of Weld Joining Penetration under Varying GTA Pools. Welding Journal , 313-321.
‣
[2] Mnich, C. A.-B. (2004 йил 19-21-July). In Situ Weld Pool Measurement using Stereovision. ASME Symposium on Flexible Automation .
‣
Mnich, C. M. (2004). Development of a Synchronized, High-Speed, Stereovision System for In Situ Weld Pool Measurement. Masters Thesis, Colorado
School of Mines, Department of Engineering, Golden, Colorado.
‣
Peris, Martin. "OPENCV: STEREO CAMERA CALIBRATION." Web log post. OPENCV: STEREO CAMERA CALIBRATION | Martin Peris' Blog. N.p., n.d. Web. 04
Oct. 2013.
‣
[3] Peris, Martin. "OPENCV: STEREO MATCHING." Weblog post. OPENCV: STEREO MATCHING | Martin Peris’ Blog. N.p., n.d. Web. 04 Oct. 2013.
‣
Seyffarth, P. a. (n.d.). Image Processing for Automated Robotic Welding. Ingenieurtechnik Und Maschinenbau GmbH Rostock .
‣
Szeliski, R. (2011). Computer Vision: Algorithms and Applications. London: Springer.
‣
Zhang, W., & Zhang, Y. (2013). Dynamic Control of the GTAW Process Using a Human Welder Response Model. Welding Journal , 92.
‣
[4] Zhang, Y. D., Liang, Z. M., Wang, D. L., & Wang, J. (2012). 3D Rconstruction of Weld Pool Surface by a Biprism Stereo System. Applied Mechanics and
MAterials , 469-473.
‣
Zhang, Y. M., Kovacevic, R., & Li, L. (1996). Characterization and Real-time Measurement of Geometrical Appearance of the Weld Pool. International
Journal of Machine Tools and Manufacture , 36.7, 799-816.