2010 1st International Conference on Applied Robotics for the Power Industry
Delta Centre-Ville
Montréal, Canada, October 5-7, 2010
1
Robotized Inspection of Power Lines with Infrared
Vision
Jonathan Henrique Efigênio de Oliveira, and Walter Fetter Lages, Member, IEEE
Abstract—This paper presents a method to automatically
detect faults in transmission lines by using thermographic images.
It involves the acquisition of infrared images over a network
and its processing. Since the method is intented to be used
embedded in a robot, low demands on processing power are
required, in order to enable inspection on large extensions of
transmission lines. The automatic inspection of power lines,
allows for inspections over a longer period of time and with
a decreased risk for the professionals involved. Experimental
results show that the proposed method is able to detect cables
with problems.
Index Terms—Filed robotics, Automation, Sensor, Vision, Implementation, Inspection, Maintenance.
I. I NTRODUCTION
T
HE methods for inspection of power transmission lines
are expensive, dangerous and inaccurate. This article
presents a method for making preventive inspections of transmission power lines. The preventive inspections are important
as they will be able to detect faults before they occur.
The proposed method is based on thermographic analysis
of the line by using a infrared camera mounted on a robot
that moves on the line. This way, there is a decrease in the
need for maintenance personnel to stay close to energized lines
and therefore an increase in safety, besides enabling a more
accurate process since operator fatigue is avoided.
Typically, problems on cables, connections and even insulators have a direct consequence on their electrical resistance
and therefore on the electrical current through the device,
increasing the temperature around the point with problem.
Note that although the usual behavior of conductors (cables
and connections) and insulators are very different, under
failure both present an increase in temperature. In conductors,
the increase in the temperature is due to an increase in their
electrical resistance, while in insulators, the increase in the
temperature is due to an increase in the leak current through
them. Thus an abnormal increasing in temperature can be
associated to a failing device.
Thermographic inspection has been growing in various
industry segments, since it is a nondestructive method [1],
is fast and does not require contact with the body under
inspection. For electrical plants, it is even more convenient,
since in general it is easier to perform the inspection without
the interruption of system operation.
The visual inspection of power lines, even if just searching
for hotspots in infrared images, is a very monotonous and
Authors are with the Department of Electrical Engineering, Federal University of Rio Grande do Sul, Porto Alegre, RS, 90035-190 BRAZIL e-mails:
jonathan@ece.ufrgs.br, fetter@ece.ufrgs.br.
978-1-4244-6634-4/10/$26.00 ©2010 IEEE
tedious task and therefore if done by human beings it is very
prone to fatigue of the operator. The method proposed in this
paper overcomes this problem by performing the analysis of
the infrared images automatically. Also, since the robot is
equipped with a differential GPS receiver, detected problems
are georeferenced so that they can be easily located by
maintenance teams. Furthermore, since the camera is mounted
on a robot moving on the line (aerial or underground), the
image is obtained very close to the line, thus avoiding the wellknown problem of averaging of thermal pixels. The averaging
of pixels can hide hotspots in images taken far from the object
under inspection.
This paper describes the procedures to acquire images from
the camera and the processing of the images to detect potential
problems on conductors and insulators. Details on the robot
can be found on [2]–[5].
The thermographic camera is an IP camera working as a
RTSP (Real-Time Stream Protocol) [6] server, thus the images
are provided as data streams. In order to have individual
images to be automatically processed, the images should be
captured from the received data stream.
The location of each object under risk of failure, obtained
by the robot GPS, is stored along with the image and details of
the analysis for further validation and schedule of maintenance
teams.
This article is organized as follows: Section II presents some
aspects that should be taken into account when processing
thermographic images. Section III addresses the main aspects
of the method developed in this work. Sections V and VI
present the experimental results and conclusions.
II. T HERMOGRAPHIC I MAGE
A thermographic image is a digital image formed by electromagnetic waves in the infrared portion of the spectrum,
rather than in the range corresponding to the visible light, like
images generated by usual cameras. Each pixel represents the
temperature of a point in the objects in image or the energy
emitted by its material.
In this paper a thermographic camera from FLIR, model
A320 is used (see Fig. 1). Its operation is based on microbolometers and it generates images with resolutions up to
320×240 pixels. It has two temperature ranges: from -20 o C
to 120 o C or 0 o C to 250 o C, providing images in different
formats like standard composite video (NTSC or PAL) or
RAW and MPEG4 over an Ethernet connection.
The camera provides several different streams of the same
thermal image:
2
Fig. 1.
Thermographic camera FLIR A320.
Fig. 2.
1) MPEG4 compressed video in three resolutions
(640×480, 320×240, 160×128): Useful only for
presenting an operator view, since the MPEG4
compression imposes some loss on the information
contained on the captured image, usually observed as
pixellization of the image.
2) FCAM FLIR: This stream is in a proprietary format of
the manufacturer. Since details of the format are not
available, it is not useful for the general user.
3) RAW IR signal: This stream provides the raw data from
the infrared sensors. Since the data has not been calibrated for the sensors characteristics, and a calibration
pattern for the sensor is not usually available for the
user, this stream is not very useful too.
4) linear IR temperature image in two resolutions
(320×240, 160×120): These streams are thermal images
calibrated in degrees Kelvin with resolutions of 0.1 K or
0.01 K. The stream with resolution of 320×240 pixels
and 0.01 K is the one selected to be automatically
processed, since it can be easily interpreted and has not
distorted the captured data.
In this paper the linear IR temperature image with a
resolution of 320×240 is used. In this format, each pixel is
a 16-bit value representing the temperature in Kelvin of that
point. The precision can be selected to be either 0.1 K or
0.01 K. However, since the camera accuracy is ±2 o C, the
precision can selected much more as a function of the desired
temperature range.
III. I MAGE C APTURE
The camera behaves as a RTSP server providing a stream
with thermographic images. Stream is a continuous flow of
images wrapped in the Real Time Protocol, RTP [7]. In order
to capture the stream, the processing software should talk to
the camera by using the stack of protocols shown in Fig. 2.
To receive streams, The client should in the first place
establish an RTSP session. This session configures some
parameters to be used for the streaming between the server
and the client. The RTSP protocol refers to the protocol used
to control the operation of the streaming server and provides
commands to select the stream, obtain a description the stream
properties, start, stop, pause and restart the data stream. Table I
Protocol stack.
TABLE I
RTSP C OMMANDS S UPPORTED BY THE C AMERA
Command
OPTIONS
DESCRIBE
SETUP
GETPARAMETER
PLAY
PAUSE
TEARDOWN
Description
list the supported optional commands
list the streams available by the camera
set one RTP session using a specific stream
get parameters like frame rate and file format
begin send the streams
pause the sending
close the RTP session
shows the RTSP commands supported by the camera used in
this work.
The connection between the server on the camera and
the client processing software on the robot to transfer
an image stream is called a multimedia session. Before
starting a multimedia session its parameters should be
configured. The Session Description Protocol, SDP [8] is
used to describe the desired session parameters such as
such as session name, media name, address and connection
information. The SDP protocol is used in the setup phase.
In this phase, the server describes the available streams
to the client, which then configures the parameters for
the desired stream. Figure 3 shows the reply sent by
the camera to a RTSP DESCRIBE command. The SDP
session description returned by the camera begins at the
line which starts with v=0. Notice the lines starting with
a=rtpmap: which starts the description of each stream
available from the camera. The stream used in this work
is the one described by a=rtpmap:103 raw/90000,
a=framesize:103 320-240
and
a=fmtp:103
sampling=mono; width=320; height=240;
depth=16.
The stream itself is sent though the RTP (Real Time
Protocol) [7]. This protocol defines how the data is packed
to be sent. This protocol enables the client to interpret the
information that it receives and decode the stream in images.
Nowadays, the Ethernet, IP, UDP and TCP protocols are
standard features of most operating systems, but RTSP, RTP
and SDP should be provided by extra software. This work uses
the implementations of the RTSP, SDP and RTP protocols
available through the Live 555 library [9]. That library has
3
RTSP/1.0 200 OK
CSeq: 2
Date: 24 Jun 2010 20:57:18 GMT
Content-Type: application/sdp
Content-Length: 1028
Content-Base: rtsp://10.1.32.1/
v=0
o=- 0 0 IN IP4 10.1.32.1
s=IR stream
i=Live infrared
t=nowc=IN IP4 10.1.32.1
m=video 13124 RTP/AVP 96 97 98 100 101 103 104
a=control:rtsp://10.1.32.1/sid=96
a=framerate:30
a=rtpmap:96 MP4V-ES/90000
a=framesize:96 640-480
a=fmtp:96
profile-level-id=5;config=000001B005000001B509000001010000012002045D4C28A021E0A4C7
a=rtpmap:97 MP4V-ES/90000
a=framesize:97 320-240
a=fmtp:97
profile-level-id=5;config=000001B005000001B509000001010000012002045D4C285020F0A4C7
a=rtpmap:98 MP4V-ES/90000
a=framesize:98 160-128
a=fmtp:98
profile-level-id=5;config=000001B005000001B509000001010000012002045D4C28282080A4C7
a=rtpmap:100 FCAM/90000
a=framesize:100 320-240
a=fmtp:100 sampling=mono; width=320; height=240; depth=16
a=rtpmap:101 FCAM/90000
a=framesize:101 160-120
a=fmtp:101 sampling=mono; width=160; height=120; depth=16
a=rtpmap:103 raw/90000
a=framesize:103 320-240
a=fmtp:103 sampling=mono; width=320; height=240; depth=16
a=rtpmap:104 raw/90000
a=framesize:104 160-120
a=fmtp:104 sampling=mono; width=160; height=120; depth=16
Fig. 3.
Fig. 4.
Original radiometric image.
Fig. 5.
Segmented image.
SDP session description returned by the camera.
C++ classes to start a session and send RTSP commands
to control the server (the camera). In order to receive an
thermographic image the client should send the commands
to choose the stream (in this case RAW stream with a
resolution of 320×240) and then send the command for the
server start sending the RTP packets. On reception the packets
are extracted from the stream. The payload of each RTP
packet is according to the RFC4715 (RTP Payload Format for
Uncompressed Video) [10]. Each of these arrays is a image
that will be analyzed to detect flaws.
The analysis of the infrared images is based on the detection
of meaningful hotspots and an assessment of the degree of risk
of failure that they represent. Depending on the temperature
of hotspots and its morphology it will be considered mediumrisk, serious-risk or imminent-risk of failure.
IV. FAILURE D ETECTION
The purpose of this algorithm is to process the images to detect hotspots and decide weather it represents a problem on the
transmission line or not. Each received image is processed for
inspection, searching for the existence and severity of failures.
The proposed method is based on the Infrared Thermography
Anomaly Detection Algorithm (ITADA) proposed in [11].
The original ITADA was proposed for inspection of electrical equipment in operation and is based on detection of
high temperatures and/or extreme variations in temperature.
Furthermore, it uses a visual 8 bit palette-based image, while
in this paper a 16 bit radiometric image is used (see Fig. 4).
As in many computer vision methods, the first step is the
segmentation of the image. To segment an image is to separate
the target object(s) from the background. In this case, the
background are the areas with low temperatures. Therefore
the result of the segmentation step is an image where only the
interesting high temperatures areas have a temperature value
different from the background. Furthermore, the background
itself is adjusted to have a single value, which can be easily
distinguished from the high temperatures areas, usually 0. The
segmentation is done by thresholding the image with a value
determined in a similar way to [12]. Therefore, all pixels
with temperature below the threshold value is considered
background and has its temperature set to 0 (see Fig. 5).
Let α be the original image(see, for example 9(b)) and β
the binarized version of α, T is the threshold value from the
Otsu method, W and H the width and height of he image, x
and y the coordinates of a pixel. Then, the segmented image
is:
(
α(x, y), if α(x, y) ≥ T
γ(x, y) =
0,
if α(x, y) < T
∀0 ≤ x ≤ W, 0 ≤ y ≤ H
The hotspots are then detected on the segmented image.
4
Fig. 6.
Hottest pixels.
Fig. 7.
First the pixels with highest temperature value are located (see
Fig. 6).
Thot =
max
0≤W, 0≤H
γ(x, y)
(1)
These pixels with highest temperature value are used to all
connected hot ares. Each Thot is used as a seed for a dilation,
denoted by Ω0 .
(
1 if α(x, y) = Thot
Ω0 (x, y) =
∀0 ≤ x ≤ W, 0 ≤ y ≤ H
0 otherwise
(2)
Then, a dilation operation is used to find connected hot areas
around the hotspots, resulting in a set of hot areas as shown
in Fig. 7:
Ωk = (Ωk − 1 ⊕ B) ∩ C k = 1, 2, 3, ..
(3)
where B is an 8-neighbors mask, C is a constraint representing
the experimental limit on the gradient between neighbors
in the foreground image γ (a value of 16 was adopted).
The algorithm converges when Ωk = Ωk−1 , resulting in the
converged image Ω∗ , which represents the all hotspots in the
thermographic image.
The very small areas with less than 5 pixels are regarded as
noise and are discarded for the purposes of subsequent analysis. The largest hotspot is used to characterize the situation
of the cable. The temperature of the largest hotspot is defined
by the mean value of its pixels:
Thot =
1
D
W
−1 H−1
X
X
x=0 y=0
!
γ(x, y), ∀ (x, y) ∈ A
(4)
where A represents all pixels of the largest hotspot and D the
number of pixels in the largest hotspot.
The criticality of each detected hotspot is determined by a
look-up table, with values based on standards such from IEC
Hotspots.
(International Electrical Commission) or ABNT (Brazilian
Association of Technical Standards).
The criticality of a hotspot can be defined by a “quantitative”
or a “qualitative” analysis of its temperature. The quantitative
analysis considers the exact measured temperature of the
hotspot. This method is generally not as important, as the
accuracy of these values are often affected by environmental
factors such as the current ambient temperature, humidity and
emissivity, etc..
The “qualitative” analysis considers the temperature values
for a hotspot in relation to other parts of the equipment with
similar conditions, based on the delta of temperature defined
as
∆T = Thot − Tref
(5)
where Tref is the temperature of the component if it were
operating normally. It is computed from the temperature of
pixels not present in any hotspot:
Tref =
M
N
(6)
where
M=
W
−1 H−1
X
X
γ(x, y), if Ω∗ (x, y) 6= 1
(7)
γ(x, y)
, ifΩ∗ (x, y) 6= 1
γ(x, y)
(8)
x=0 y=0
N=
W
−1 H−1
X
X
x=0 y=0
Table II presents absolute temperature values and Table III
presents relative temperature values used to decide on the criticality of each hotspot. The reference temperature, Tref , takes
into account the voltage levels temperature of the background.
It is generally a value around 70 o C [13]. This value can vary
due to cable manufacturer and climatic conditions. Hence, the
actual temperature of the coldest parts of the image is used as
reference, which represents just a portion of cable that is free
of flaws.
5
TABLE II
Q UANTITATIVE A NALYSIS
Condition
Normal
Not Serious
Medium
Serious
Emergency
Temperature Limits (o C)
Thot ≤ 68.3
68.3 < Thot ≤ 76.7
76.7 < Thot ≤ 85.0
85.0 < Thot ≤ 98.9
98.9 < Thot
TABLE III
Q UALITATIVE A NALYSIS
Condition
Normal
Not Serious
Medium
Serious
Emergency
Temperature Limits (o C)
∆T ≤ 10
10 < ∆T ≤ 20
20 < ∆T ≤ 30
30 < ∆T ≤ 40
40 < ∆T
(a) Visual image.
V. E XPERIMENTS
An aluminum cable with 25mm2 section (see Fig. 8(a))
is used in this experiment. It was submitted to a current of
500 A. The thermographic image is shown in Fig. 8(b). In
this image it is possible to see a warm region, detected as the
hotspot shown in Fig. 8(c), but this region have an acceptable
temperature. It is just about 5 o C warmer than the cold areas.
The qualitative and quantitative analysis results in a no fail
decision.
Then the very same cable, has been damaged, lowering its
conductivity, as shown in Fig. 9(a), and a new thermographic
image was obtained as shown in Fig. 9(b). Note that at the
failure point there is an increase in the temperature relative
to the previous image. The temperature on the cable in the
first image is around 27 o C, the temperature at the point
where the failure occurs is around 42 o C. The here proposed
method detected the region where the damage occurred as
a hotspot (see Fig. 9(c)). This temperature does not pass
the quantitative limits test. However, the qualitative analysis
results in a damage.
(b) Thermographic image.
VI. C ONCLUSION
In this paper a method for detection of failures from
thermographic images was presented. The next step is to
submit aluminum cables, like those used in transmission lines,
to currents on the order of hundreds of Amperes, simulate a
damage in it and automatically detect this fault.
ACKNOWLEDGMENT
The authors would like to thank to Companhia Estadual de
Energia Elétrica (CEEE), Coordenação de Aperfeiçoamento de
Pessoal de Nı́vel Superior (CAPES) and Conselho Nacional de
Pesquisa (CNPq) for the financial support.
R EFERENCES
[1] F. Chunli, S. Fengrui, and Y. Li, “Investigation on nondestructive
evaluation of pipes using infrared thermography,” in Proceedings of the
IEEE International Conference on Terahertz Electronics, vol. 2, Sept.
2005, pp. 339–340.
(c) Hotspot.
Fig. 8.
Aluminum cable without problems.
[2] V. M. de Oliveira and W. F. Lages, “Linear predictive control of a
brachiation robot,” in IEEE Canadian Conference on Electrical and
Computer Engineering. Ottawa, Canada: IEEE, May 2006, pp. 1517–
1520.
[3] ——, “Predictive control of an underactuated brachiation robot,” in
6
Elsevier, 2006.
[4] ——, “MPC applied to motion control of an underactuated brachiation
robot,” in Proceedings of the 11th IEEE International Conference on
Emerging Technologies and Factory Automation. Prague: IEEE Press,
2006.
[5] ——, “Comparison between two actuation schemes for underctuated
brachiation robots,” in Proceedings of the 2007 IEEE/ASME International Conference on Advanced Intelligent Mechatronics, Zürich, 2007.
[6] H. Schulzrinne, A. Rao, and R. Lanphier, “Real time streaming protocol
(RTSP),” Network Working Group, RFC 2326, April 1998, available at
hftp://ftp.ietf.org/rfc/rfc2326.txti.
[7] H. Schulzrinne, S. Casner, R. Frederick, and V. Jacobson, “RTP: A
transport protocol for real-time applications,” Network Working Group,
RFC 1889, January 1996, available at hftp://ftp.ietf.org/rfc/rfc1889.txti.
[8] M. Handley, V. Jacobson, and C. Perkins, “SDP: Session description
protocol,” Network Working Group, RFC 4566, July 2006, available at
hftp://ftp.ietf.org/rfc/rfc4566.txti.
[9] LIVE555 Streaming Media, Live Networks, Inc., Mointain View, CA,
available at hhttp://www.live555.com/LiveMediai.
[10] L. Gharai and C. Perkins, “RTP payload format for uncompressed
video,” Network Working Group, RFC 4566, September 2005, available
at hftp://ftp.ietf.org/rfc/rfc4175.txti.
[11] Y.-C. Chou and L. Yao, “Automatic diagnosis system of electrical
equipment using infrared thermography,” in Proceedings of the 2009
International Conference on Soft Computing and Pattern Recognition.
IEEE Computer Society, 2009, pp. 155–160.
[12] N. Otsu, “A threshold selection method from gray-level histograms,”
Systems, Man and Cybernetics, IEEE Transactions on, vol. 9, no. 1, pp.
62–66, jan. 1979.
[13] I. E. C. (IEC), “IEC 60826: Design criteria of overhead transmissionlines,” 2006.
(a) Visual image.
Jonathan Henrique Efignio de Oliveira received
the B.Sc. in Electrical Engineering from the Universidade Federal do Rio Grande do Sul (UFRGS),
Porto Alegre, Brazil, in 2007 and is working towards
his M.Sc. degree in Electrical Engineering at the
same University.
(b) Thermographic image.
Walter Fetter Lages (S’91, M’99) received the
B.Sc. in Electrical Engineering from Pontifı́cia Universidade Católica do Rio Grande do Sul (PUCRS),
Porto Alegre, Brazil in 1989 and the M.Sc. and D.Sc.
degrees in Electronic and Computer Engineering
from Instituto Tecnológico de Aeronáutica (ITA),
São José dos Campos, Brazil in 1993 and 1998,
respectively. From 1997 to 1999 he was an Adjoint
Professor in the Physics Department of the Fundação
Universidade Federal do Rio Grande (FURG), Rio
Grande Brazil. Currently he is an Associate Professor in the Electrical Engineering Department of the Federal University of Rio
Grande do Sul (UFRGS), Porto Alegre, Brazil. Dr. Lages is a member of
IEEE, ACM, Brazilian Automation Society and Brazilian Computer Society.
(c) Hotspot in the damaged aluminum cable.
Fig. 9.
Damaged aluminum cable.
Proceedings of the 8th IFAC Symposium on Robot Control.
Bologna: