THERMOBOT: TOWARDS SEMI-AUTONOMOUS,
THERMOGRAPHIC DETECTION OF CRACKS
C. Eitzinger(1), S. Ghidoni(2), E.Menegatti (2)
(1)
Profactor GmBH, Im Stadtgut A2, 4407 Steyr-Gleink, Austria
Department of Information Engineering, University of Padova
Via Gradenigo 6/A I-35131 Padova, Italy
(2)
ABSTRACT. The detection of cracks in parts of complex geometry requires a cumbersome
process based on “magnetic particle inspection”. This includes the application and removal of
liquids and is difficult to automate. In this paper a semi-autonomous system for crack
detection is proposed that uses a robot to move a part in front of a thermographic image
acquisition system. At the heart of the inspection system is a laser combined with a thermocamera that will provide image information to enable the robust detection of cracks on or near
the surface. The analysis of the heat flow will reveal any inhomogeneities such as cracks in
the part. This is combined with automatic path planning for the robot to enable the inspection
of complex parts. The system concept is presented and details about the various system
components are explained.
INTRODUCTION
Despite considerable progress in the automation of production processes visual inspection of
products and parts is still done manually in a wide variety of inspection tasks. The current
procedure for crack detection is a process that dates back to the 1920s and is called “magnetic
particle inspection”. This method is infamous in industry, because it is a cumbersome, dirty
process that is often done manually even in otherwise fully automatic production lines. The
component to be tested is magnetized before applying a suspension of finely divided coloured
or fluorescent magnetic particles. Cracks or inclusions cause the magnetic flux to break the
surface forming free magnetic poles and the magnetic particles will collect at these locations
indicating a crack. Using UV-light the fluorescent particles in the suspension are activated to
increase the visibility of the cracks. Inspection of complex parts is usually done manually,
whereas for simple geometries machine vision systems can be used for automatic detection.
The project ThermoBot aims at replacing this old method for crack detection with a new
technology that is based on autonomous inspection robots using thermography to recognize
cracks on parts of complex geometry. The robot will scan the whole part with a thermocamera and analyse the heat-flow to find cracks and other defects hidden under the surface.
To achieve this, the project aims at making progress in the following areas:
 thermographic processes and process models for the automatic detection of cracks in
parts of complex geometry, varying surface structures and for different materials.

Automatic path and motion planning that uses a thermographic process model to
automatically generate a path for the inspection robot from 3D CAD data.
 Thermo-image analysis methods that not only aim at detecting cracks and making an
accept/reject decision for the whole part, but also have the capability of self-evaluating
their performance in the crack detection.
The following sections will describe the state of the art with respect to these areas and
explain the progress that has been made so far in solving the open technological and scientific
questions.
STATE OF THE ART
Models of thermographic imaging
Most frequently the models used in applications of thermography have their focus on
conduction of heat in solids in the context of the energy source, which is used to heat up the
specimen under inspection [1, 2, 3, 4]. Heat transport is the main mechanism that enables
thermography to detect defects such as cracks. In addition to heat flow inside the part two
additional models are required to fully cover the thermographic image acquisition process.
The first one deals with the radiation from the surface of a specimen, which includes the
actual (heat) signal as well as components [5] such as noise, reflections from environment or
the influence of the surface properties. The second one, which is much simpler, deals with the
properties of sensor and imaging considerations [6, 7]. Adding radiation from environment,
one creates a model that describes the “radiometric measurement problem” [8]. The model for
description of the surface temperature distribution and evolution is based on heat production
and transfer in a solid. For crack detection an in-plane flux must be induced, to obtain
measurable temperature differences along the crack. This can be achieved by heating up only
a small area of the surface. From this “hot spot”, heat will flow mainly in parallel to the
surface, and this flow is interrupted by a crack. To improve robustness of the detection, an
evaluation method [9] was developed, that identifies heat flux phenomena by watching the
evolution of temperature profiles. Description of temperature as consequence of heat
conduction is based on the differential equations of heat conduction, for which explicit
solutions exist only for simple models [10, 11]. In more complex situations simulations [12]
such as finite elements methods (FEM) are used to predict signals [13]. In case of real time
applications, current simulation methods would take too much time, so it is mainly used for
design purposes.
Robot path planning for inspection tasks
To calculate a path on a 3D object that can be used for inspection, several constraints need to
be adhered. For example, the camera has to be in a certain (range of) distances from the part
and it has to view the relevant surface under a certain angle. This problem is inherent in many
camera-based inspection problems, such as completeness inspection [14] or for automated 3D
object reconstruction [15]. It aims at the automatic generation of an (optimal) sequence of
viewpoints that satisfy certain constraints, such as full coverage of the part [16]. This is
known as “view planning problem”. In order to solve the associated optimization problem
algorithms that can deal with discrete search spaces, such as genetic algorithms [17] or
particle swarm optimization are regularly used.
Thermo-image analysis
Most of the systems proposed in literature for thermographic image processing are quite
basic: the processing work flows are composed of thresholding, edge detection, region
growing, and template matching [18]. However, more advanced image processing techniques
have been proposed. Besides traditional techniques coming from the field of computer vision,
several specific methods have been developed for thermal image processing [19]. These
unique techniques are sometimes based on the underlying heat-conduction physics. The
methods are used either at image preprocessing and/or processing stages [20]. For instance,
thermal contrasts have the advantages of being little sensitive to noise and to the surface
optical properties. Another image processing technique which proved to be useful for image
segmentation is pulsed phase thermography (PPT). This is a processing method in which the
thermal images are transformed from the time domain to the frequency domain to calculate
the phase and the amplitude of the signal in the images [21]. The phase is particularly
advantageous since it is less affected by environmental reflections, emissivity variations, nonuniform heating, surface geometry and orientation. It should be noted for the sake of
completeness that all the image segmentation algorithms developed in computer vision are also
applicable in thermo-image analysis. We just cite [22] and [23] as recent results that explain
adaptive image segmentation methods as well as optimization methods that can automatically tune
segmentation algorithms to a particular task.
OUR APPROACH
System overview
The hardware components of the automatic inspection system include a robot, a heating system (in
our case a laser unit), a thermocamera and a processing unit. While the robot itself can be any
general purpose robot, the robot’s control unit has to fulfill the requirement that accurate, timestamped position information (joint angles) can be acquired at high frequency. This is required to
synchronize image acquisition and robot motion. The laser unit’s main function is to locally heat
the part. The specific parameters of the laser substantially depend on the thermodynamic properties
of the parts to be inspected. For the thermocamera the key feature – aside from high resolution – is
a sufficiently high frame rate to allow the acquisition of image sequences at high frame rates. This
is particularly important for metallic parts, where the heat dissipates in less than a second. A central
processing unit links all the component and collects the robot’s position as well as the camera
image for each time instance. These data are then processed by image segmentation and
classification algorithms to distinguish between different types of defects and to find a final
good/bad classification for the whole part.
It should be noted that for practical reasons in many applications the robot will be holding the part
and not the camera/laser unit.
Model of thermographic imaging
The thermographic imaging model is at the heart of the path planning algorithm. The model
basically answers the question which area on the part can be inspected, if the part is placed in a
certain position and orientation under the camera.
Figure 1. Area that can be inspected (basic concept).
The area that can be inspected is determined by multiple factors. In the most simple case of a
flat surface placed at an ideal angle under the camera and laser the area will have a ringshaped form as shown in figure 1. At the centre of the ring there is the laser spot that results in
a bright, white region in the image, where the pixels of the thermocamera are fully saturated
and no analysis is possible. As the heat dissipates isotropically the signal becomes weaker
until no more contrast is achieved. This area defines the outer edge of the ring-shaped region.
During inspection the ring is wandering across the surface leaving a trail where the part has
been inspected. Cracks will become visible as a temperature gradient that is caused by the
interruption of the heat flow. It should be noted that the sensitivity of the detection depends
on the orientation of the crack relative to the laser spot. If the crack is propagating radially
from the laser spot, it will not be visible as there is no heat flow across the crack.
In the more realistic case of a non-flat part, the situation becomes significantly more complex.
The model has to consider that laser and camera are not placed in an ideal position relative to
the part’s surfaces and that the heat propagates in a non-flat area. An approximation of the
area that can be checked, may be obtained by projecting the ring-shaped region onto the part’s
3D surface. Additionally, self-occlusions of the part have to be considered as well as areas of
high curvature, where the above mentioned approximation is invalid. Those areas have to be
excluded from the region that can be checked. The calculations that are involved in such
estimations can only be done numerically.
Robot path planning
The task of path planning is to generate a sequence of viewpoints through which the robot has to
pass. This sequence has to fulfill a few strict boundary conditions:
 all relevant areas on the part’s surface have to be covered by acquiring images from
these viewpoints;
 when passing through the viewpoints the part must not collide with the robot, the
camera or other devices in the workspace of the robot;
 all the viewpoints must be within the reach of the robot.
These conditions apply to all robot-based inspection systems that include camera as the main
sensor system. Thermography, however, has one additional constraint that is quite difficult to
fulfil. In order to do a proper analysis of the heat flow, each position on the part has to be
recorded in several images at specific time intervals. Therefore an additional requirement is
that

the path of the robot has to follow a specific trajectory, that ensures that the
viewpoints are reached at a particular time.
It should be noted that the motion of the robot has to be continuous and not a start/stop motion
as is possible e.g. in standard machine vision applications.
This set of boundary conditions still allow a large (actually infinite) number of different
solutions for the sequence of viewpoints that is chosen for the particular inspection task. Even
if the restrictions of the robot’s kinematics are fully considered, there is still a wide range of
different paths. Therefore, an additional optimization criterion is required that reduces the set
of possible solution to (ideally) a single one. In many real-world applications this
optimization criterion corresponds to
 the time required for inspecting all areas on the part should be minimal.
While this optimization criterion may sound trivial it poses a number of challenges.
Estimating the time required for following a complex path in 6 degrees of freedom requires
detailed knowledge about the dynamical behaviour of the robot as well as detailed
consideration of the thermodynamic processes that are involved.
The starting point for path planning is a 3D CAD model of the part. This model is enhanced
by splitting the region into small elements that correspond to one main position (“process
points”) of the inspection process. Each surface element is augmented with an outward
pointing normal vector that points towards the ideal position of the laser/camera unit as shown
in figure 2.
Figure 2. Process points on small section of a crankshaft with a small outward pointing
normal vector.
Given the fact that there is an ideal operating distance for the image acquisition unit, a space
of ideal positions can be generated from which the surface can be inspected.
Tests on objects of different complexity included parts with an essentially flat surface that
have only low curvature as well as more complex parts such as the crankshaft shown in the
figure above. Path planning was able to generated useful paths for both of these test parts. The
current implementation of the path planning just tries to find a feasible path for the robot that
complies with the above mentioned boundary conditions. It does not yet use an optimization
criterion to e.g. reduce the time needed for the inspection task. For inspecting the relevant
regions on a whole crank shaft a total time of a few minutes is required, but there is still
substantial potential for optimization so that cycle times below 1 minute can be expected.
Thermo-image analysis
Automatic processing of thermo-image was implemented for two kinds of sample parts: a
metal crankshaft and a carbon fiber sideblade, shown in figure 3.
Figure 3. Sample parts tested using the thermo-image analysis: a carbon fiber sideblade (left)
and a metal crankshaft (right).
Such sample parts were cut into smaller portions that were analyzed, in order to ease the
acquisition process. The thermal excitation method is different for the two types of material,
because a strong power is needed for changing the thermal state of metal, while carbon fiber
would be damaged by such a strong heating. A high-power laser was used in the former case,
while small portions of carbon fiber parts were heated using a thermal chuck, even though in
the final system configuration this will not be feasible because of the complex geometry of
such parts.
Image analysis techniques employed for crack detection on metal parts focus on the laser spot.
The high amount of energy concentrated in a small spot causes a strong heat flow to the
surrounding areas. Cracks modify the heat transfer profile, and can therefore be detected by
measuring the regularity of the heat status around the laser spot. An example of detection
result can be seen in figure 4 (left): a large crack causes the heat transfer to be less effective
towards the left part of the image. The analysis of heat gradients depends on multiple
parameters, including the laser power and the speed at which it scans the part under
inspection. Patterns are in fact circular at lower speed, around 40 mm/s, while they become
oval-shaped at higher speed. Laser power also has an impact, since it should be strong enough
to cause heat transfers visible in the thermo-images. Common values are around 7-10 W.
Images of carbon fiber parts show a more uniform thermal status, given by the different
heating system: the thermal chuck has a surface size similar to that of the sample. The parts
are framed from one side, and heated on the opposite face. The alteration of the thermal status
of the part is capable of highlighting intrusion of unwanted material inside the fiber element.
Image processing is exploited to segment the single fiber elements and to analyze the content
of each one, that must be uniform without strong gradients. In figure 4 (right) the output of
the image analysis is shown: fiber elements are marked using elliptical contours, whose color
indicates whether the element has uniform color (green) or not (red). Some false positives are
visible towards the borders, because the contrast is too low in such locations. This will be
fixed in the final system configuration, when images acquired with the final heating system
will be available.
Figure 4. Automatic detection of a crack in a metal part (left) and carbon fiber element
segmentation and analysis (right).
CONCLUSIONS
In this paper we presented the progress that was made towards the implementation of a system
for the semi-autonomous detection of cracks in complex parts. The system is based on
thermography and uses a robotic system to position the part in front of the camera. Automatic
path planning allows a quick setup of the system for new geometries. First results indicated
that a robust detection of the crack is possible using image analysis methods and that the
automated path planning allows the inspection of a complex part in a few minutes. Future
work includes the optimization of all components in order to speed up the inspection process
and the setup of the whole inspection system. Additional work on defect classification and
extending the range of possible materials is also needed.
ACKNOWLEDGMENTS
The research leading to these results has received funding from the European Union Seventh
Framework Programme (FP7/2007-2013)under grant agreement No. 284607.
REFERENCES
[1] K.-J. Kremer, W. Kaiser, P. Möller (1985): Das Therm-O-Matic-Verfahren – ein
neuartiges Verfahren für die Inline-Prüfung von Stahlerzeugnissen auf Oberflächenfehler.
Stahl u. Eisen 105, 71-76
[2] J.Baumann, U.Netzelmann, R.Bilgram, T.Hierl, V.Carl, H.R. Subach, G. Zenzinger.
(2005) “Untersuchungen zu aktiver thermografischen Prüfmethoden zur Absicherung von
Hochleistungsfertigungsverfahren – Ergebnisse eines BMBF-Projekts”, DGZfP-Berichtsband
94, DGZfP-Jahrestagung, Rostock
[3] B. Oswald-Tranta. (2004): Thermoinductive Investigations of Magnetic Materials for
Surface Cracks.
Verformungskundliches Kolloquium 23, 229-237, Herausgeber: B.
Buchmayr, Lehrstuhl für Umformtechnik, Montan Universität
[4] Xavier P. V. Maldague. (2001). Theory and practice of infrared technology for nondestructive testing, ISBN 0-471-18190-0
[5] E.Grinzato, V.Vavilov, P.G.Bison, S.Marinetti, C.Bressan. (1995), Methodology of
processing experimental data in Transient Thermal NDT, SPIE Vol. 2473 / 167,
[6] A. Rogalski, K. Chrzanowski. (2002). Infrared devices and techniques, Opto-Electron.
Rev., 10, no. 2
[7] G.Eppeldauer, M.Racz. (2000). Spectral power and irradiance responsivity calibration of
InSb working-standard radiometers, APPLIED OPTICS, November 2000, Vol. 39, No. 31
[8] H.Kaplan.(2007). Practical Applications of Infrared Thermal Sensing and Imaging
Equipment, SPIE 2007, ISBN 978-0819467232
[9] G.Traxler, P.Thanner, W.Palfinger, A.Poetscher. (2010) Advanced evaluation for
thermographic crack detection with inductive exitation for steel billets, Budapest, 20th
manufacturing conference
[10] H.S.Carslaw, J.C.Jaeger. (1959) Conduction of Heat in Solids, 2nd Edition, Oxford
University Press, ISBN 0-19-853368-3
[11] Özisik, M. N. (1989). Boundary Value Problems of Heat Conduction. Dover Publications
Inc., New York, first edition
[12] U.Siemer. (2006 )Simulation and Evaluation of New Thermographic Techniques for the
Deployment in the Automotive Industry, ECNDT - European conference on non-destructive
testing.
[13] H.D.Baehr, K. Stephan: Wärme und Stoffübertragung, Springer 5.Auflage, ISBN-10 3540-32334-1
[14] T.H. Park, H.J. Kim, and N. Kim.( 2006). Path Planning of Automated Optical
Inspection Machines for PCB Assembly Systems, International Journal of Control,
Automation, and Systems, vol. 4, no. 1, pp. 96-104, February 2006
[15] W. R. Scoot and G. Roth. (2003). View Planning for Automated Three-Dimensional
Object Reconstruction and Inspection, ACM Computing Surveys, Vol. 35, No. 1, March
2003, pp. 64–96.
[16] Tarbox, G., Gottschlich , S. ( 1995). Planning for complete sensor coverage in
inspection. Comput. Vis. Image Understand. 61, 1 (Jan.), 84–111.
[17] C. G. Lu, D. Morton, M. H. Wu and P. Myler. (1999) Genetic Algorithm Modelling and
Solution of Inspection Path Planning on a Coordinate Measuring Machine (CMM),
International Journal of Advanced Manufacturing Technologies (1999) 15:409–416
[18] M. Pilla, M. Klein, X. Maldague, A. Salerno. (2002) New Absolute Contrast for Pulsed
Thermography, in: D. Balageas, G.M. Busse, G. Carlomagno (Eds.), QIRT 2002, pp. 53–58.
[19] C. Ibarra-Castanedo, D. Gonzalez, M. Klein, M. Pilla, S. Vallerand, X. Maldague.
(2004). Infrared image processing and data analysis, Infrared Physics & Technology, Volume
46, Issues 1-2, Workshop on Advanced Infrared Technology and Application, December
2004, Pages 75-83, ISSN 1350-449
[20] X. Maldague. (2001) Theory and Practice of Infrared Technology for NonDestructive
Testing, John Wiley-Interscience, 2001, p. 684
[21] Maldague X. and Marinetti S., “Pulse Phase Infrared Thermography”, J.Appl. Phys., Vol.
79, pp. 2694-2698, 1996.
[22] Peters S. (2008). Automated Design of a Hybrid Texture Analysis System for Defect
Detection., OAGM Workshop, Hagenberg, 2008
[23] Martin V., M. Thonnat. (2007). A Learning Approach for Adaptive Image Segmentation,
in Scene Reconstruction, Pose Estimation and Tracking, Book edited by: Rustam Stolkin,
ISBN 978-3-902613-06-6, pp.530, I-Tech, Vienna, Austria, June 2007