Proceeding of the IEEE
International Conference on Information and Automation
Wuyi Mountain, China, August 2018
Real-Time Circuit Breaker State Detection and
Identification Method Based on Faster R-CNN*
Yuanjing Ma1,2, Ruifeng Guo2, Min Liu3, Shuai Wang1,2, Xiaolei Zhou2,
1
University of Chinese Academy of Sciences, Beijing, China
Shenyang Institute of Computing Technology, Chinese Academy of Sciences
Shenyang, Liaoning Province, China
3
Shenyang City Environmental Monitoring Center Station
Shenyang, Liaoning Province, China
mayjing@sict.ac.cn, grf@sict.ac.cn, zhouxl@sict.ac.cn shuaishuaiwh@163.com, lm_xm@163.com
2
knowledge and features of the sample. The traditional
algorithms are mainly used to solve specific types of problems.
They do not have universal applicability and scalability, and
can't adapt today's massive data processing. It is difficult to
meet the requirements of the data processing efficiency,
performance, speed and intelligence in the object detection.
With the rapid development of the theory and practice of
Deep Learning, object detection and classification based on
machine learning has entered a new stage. Differing from the
traditional feature extraction algorithms based on prior
knowledge, the Deep Convolutional Neural Networks are
characterized with local connectivity, weight sharing and
pooling operation, which decrease the complexity of the
network and the number of training parameters. As a result, the
models are invariant to geometric transformation, deformation
and illumination [3]. Aiming at the complex massive data
images in actual inspection, this paper introduces a method
that the condition of circuit breaker can be detected and
identified based on Deep Learning. Built a circuit breaker state
recognition model with Deep Convolutional Neural Networks,
and it can be constructed adaptively under the drive of training
data. The feature description provided the circuit breaker state
recognition process without preprocessing the raw data, which
overcame the shortcomings of the shallow model in
convergence and optimization methods and improved the
accuracy of circuit breaker state recognition [4-5]. The remote
monitoring system of ambient air automatic monitor station
and inspection data are used to build an experimental data
platform [6], which can deal with the difficulties caused by the
appearance diversity of the circuit breaker and made the
automatic identification and detection of circuit breaker state
have higher flexibility and generalization capability.
Abstract - Deep Learning is the latest research achievement
in the field of Artificial Intelligence. To detect and identify the
state of circuit breaker in ambient air automatic monitor station,
Deep Learning was proposed to realize the circuit breaker state
recognition in real-time. The Deep Convolutional Neural Network
was used to construct the state detection and identification model
of the circuit breaker and provided the detection process. The
remote monitoring system of ambient air automatic monitor
station and inspection data were used to build the experimental
data platform, with the circuit breaker state recognition data was
acquired from the multiple ambient air automatic monitor
stations. Through training the model with the Deep Learning, the
experimental results showed that the circuit breaker state
detection and identification method have a simple recognition
process and high accuracy. The real-time Detection and
Identification function of the circuit breaker state was realized,
meanwhile, the assistant decision-making function was provided
to the ambient air automatic monitor station.
Index Terms - object detection; deep learning; state
recognition; transfer learning； data augmentation.
I. INTRODUCTION
With the development of ambient air automatic monitor
station and the application of patrol robots, the patrol data
recording technology has been improved in the automatic way.
The need for information mining and object detection for
patrol data is growing. Using the remote monitoring system of
ambient air automatic monitor station to detect object
information of inspection robots, is the essential part of
realizing the unattended technology, and it is also important to
make safe operation and protection of the ambient air
automatic monitor station [1]. It can both reducing the labor
intensity of inspection personnel, and also avoids the
subjectivity.
The object detection includes location and identification,
that is to area location and category judgement from a specific
image. The object detection is concerned with the image
recognition problem, because it focuses on the partial area of
the image and the specific object class set [2]. The traditional
object detection method mainly depends on the designer's prior
II. TRADITIONAL CIRCUIT BREAKER STATE
DETECTION AND IDENTIFICATION METHOD
A. Circuit Breaker State Detection and Identification Based
on Image Processing
Based on digital image processing technology to detect
and identify the state of the circuit breaker, applying shadow
*
This work is supported by Technology Transformation of Environmental Quality Early Warning Platform Based on Internet of Things and Big Data # Z17-7029 and Key Technologies of Big Data Based on Environmental Internet of Things#17-141-2-00.
978-1-5386-8069-8/18//$31.00 ©2018 IEEE
1540
Authorized licensed use limited to: SHIV NADAR UNIVERSITY. Downloaded on June 20,2022 at 11:11:23 UTC from IEEE Xplore. Restrictions apply.
removal [7], binarization [8], and other operations to the
image. The non-target objects in the image are eliminated, and
the circuit breaker is extracted. According to the binary image
in the horizontal and vertical directions, the number of
projected connected areas determines the state of the circuit
breaker.
B. Circuit Breaker State Detection and Identification Based
on SIFT
SIFT (Scale Invariant Feature Transform) algorithm [9] is
a kind of image local feature description operator based on the
scale space, which is invariant to the image scaling, rotation
and even affine transformation. The SIFT algorithm maintains
a certain degree of stability with respect to the change of
viewing angle, affine transformation, and noise. And SIFT
algorithm is unique, multi-quantitative, high-speed, and
extensible. The SIFT algorithm is used to achieve the exact
positioning of the circuit breaker in the image. Based on this,
the circuit breaker is abstracted as a straight-line segment, and
the state of the circuit breaker is identified through the
straight-line detection using the Hough transform [10].
C. Circuit Breaker State Detection and Identification Based
on Template Matching
Template matching is an automatic recognition algorithm
based on image processing [11]. The algorithm automatically
recognizes template matching information by referring to the
target area difference. Template matching can be divided into
two types including feature-based template matching and
template-based template matching. It is a method that is most
commonly used in the detection process of power equipment.
The advantage is that the detection speed is fast, but when
there are multiple circuit breaker devices in the image, or the
size and direction of the circuit breaker device changed, the
method always cannot make an exact match. The template
matching result is biased.
R-CNN, which reduces the training and testing time of the
algorithm.
Structurally, Faster R-CNN has integrated feature
extraction, region proposal, bounding box regression (rect
refine), and classification in a network, resulting in a
significant improvement in overall performance, especially in
terms of detection speed obviously. Faster R-CNN uses the
VGG-16 network model to perform object detection tasks on
K40 GPUs at speeds up to 5 frames per second, with mAP
(mean Averaged Precision) reaching 73.2% on PASCAL VOC
2007, and also on VOC 2012 70.4% [12].
The Faster R-CNN structure is shown in Figure 1.
Fig. 1 Faster R-CNN Structure.
1) Conv layers: As a CNN network object detection
method, Faster R-CNN first uses a set of basic layers to extract
the feature maps of the image, such as convolutional layer,
rectified linear units layer, and pooling layer. The feature maps
are shared for the subsequent RPN layer and full-connection
layer.
2) Region Proposal Networks: The RPN network is used
to generate region proposals. This layer uses softmax to
determine that the anchor belongs to the foreground or the
background, and then uses the bounding box regression to
correct the anchors to get the exact proposals. This completes
the positioning function of the monitoring.
3) Roi Pooling: This layer collects the feature maps of the
input image and the proposal boxes generated by the RPN
network. After synthesizing this information, feature maps are
extracted and sent to the subsequent full-connection layer to
determine the object category.
4) Classification: Using the proposed feature maps,
calculate the category of each proposal through the full
connect layer and softmax and output the probability vector of
category or classification probability. At the same time, use the
bounding box regression to obtain the final exact position of
each frame.
B. Transfer Learning
Transfer learning is to transfer the learned and trained
model parameters to a new model to help the new model train.
Considering that most of the data or tasks are relevant, we can
transfer the learning model parameters (which can also be
III. METHOD OF CIRCUIT BREAKER IDENTIFICATION
BASED ON FASTER R-CNN
The object detection and identification algorithm based on
region selection is the most mature and most widely used
object detection and identification framework at the present
stage. It simplifies the entire detection and identification
process into a classification task, and utilizes the superior
performance of deep learning methods in large-scale complex
data classification to improve detection and identification
accuracy. The object detection system, we used in this paper is
Faster R-CNN.
A. Faster R-CNN
Faster R-CNN is composed of two modules [12]. One of
these modules is a deep fully convolutional network that
proposes regions, called RPN (Region Proposal Network).
And the other one is the Fast R-CNN detector that uses the
proposed regions. The entire system uses RPN networks
instead of Selective Search, EdgeBoxes and other methods,
which greatly saves the time for generating candidate
windows. The RPN network is trained to share the features of
the convolutional layer with the object detection network Fast
1541
Authorized licensed use limited to: SHIV NADAR UNIVERSITY. Downloaded on June 20,2022 at 11:11:23 UTC from IEEE Xplore. Restrictions apply.
understood as model knowledge) in a way that we can share to
the new model to accelerate and optimize. The learning
efficiency of the model does not have to learn from zero like
most networks. Especially when training classifiers for small
data sets, they usually take advantage of weights trained in
another larger data set.
If a Deep Convolutional Neural Network is trained using a
data set with a small sample size, the overfitting phenomenon
can easily occur due to the large number of network
parameters. Therefore, referring to the idea of transfer
learning, after the Deep Convolutional Neural Network is
trained by large-scale image data sets, the model is considered
as a feature extractor for transfer learning after supervised pretraining.
Usually, based on the idea of transfer learning, we use the
ImageNet-pre-trained model to fine tune the Faster R-CNN
network.
C. Circuit Breaker State Detection and Identification
Algorithm for Based on Faster R-CNN
In this paper, the Faster R-CNN model is applied to the
circuit breaker state detection and identification model. Since
the circuit breaker state is divided into two categories: “on”
and “off”. The network model parameters are modified and
made the model output two categories. The result made the
model to be more accurately applied to circuit breaker state
detection and identification [6]. The algorithm steps are as
follows:
Step1：Input the circuit breaker state image.
Step2：Training the RPN network，which is transferred
to the target dataset using ImageNet-pre-trained model. The
model is initialized, and the end-to-end micro-invocation is
performed on the area proposal task and the candidate area is
output.
Step3 ：We use the proposals generated by the second
step RPN to train a separate detection network by Fast RCNN. This detection network is also initialized by ImageNetpre-trained model. At this time, the two networks have not yet
shared the convolutional layers.
Step4 ：We use the detector network to initialize RPN
training, but we fix the shared convolutional layers and only
fine-tune the layers unique to RPN. Now the two networks
share the convolutional layers.
Step5：Keep the shared convolutional layers fixed and
fine-tune the fully connected layer of Fast R-CNN. In this way,
both networks share the same convolutional layers and form a
unified network.
Step6：Output the detected image, indicating the circuit
breaker position and the detection state result.
At this point, the end of the network training test, the
detection flow of the circuit breaker state detection and
identification model based on Faster R-CNN is as follows in
Figure 2:
Fig. 2 Circuit Breaker State Detection and Identification Model Processing.
Through the Faster R-CNN deep learning algorithm,
useful feature information is extracted from the input image.
Further, the image features are extracted hierarchically, and
through a large amount of data training, an effective circuit
breaker state recognition model of ambient air automatic
monitor station is obtained. Realize automatic identification
and detection of circuit breaker state to achieve real-time
detection.
IV. EXPERIMENTAL RESULTS AND ANALYSIS
A. Experimental Environment
Experimental Hardware Configuration: CPU Intel(R)
Xeon(R) CPU E3-1231 v3 @ 3.40GHz, quad-core processor
with 4GB of memory, graphics card type NVIDIA GeForce
GTX 960, and memory size of 2GB.
Experimental Software Environment: Ubuntu 16.04
operating system, Caffe deep learning framework, CUDA7.5,
opencv3.1.
B. Data Set
Since the recognition accuracy and processing speed of
the Faster R-CNN model on PASCAL VOC 2007 have
obvious advantages, the collected raw data is made into the
PASCAL VOC 2007 data set format.
The dataset is composed of two parts: training validation
set and test set, and the data ratio is 1:1. The training
validation set is also divided into training set and validation
set, and each set accounts for fifty percent of the training
validation set.
Monitoring video information through multiple ambient
air automatic monitor stations, we acquired a lot of related
circuit breaker image information. A total of 3,312 collected
dataset A were selected according to the ratio of the width and
height of the VOC2007 data set image, in which the circuit
breaker image marked the state of the circuit breaker image
“on” and “off”.
C. Experimental Results and Analysis
This paper applies Faster R-CNN model under Caffe
framework. Using transfer learning method, the pre-training
model adopts VGG-16 classification model, and the pretraining model parameters come from the training results of
VGG-16 under ImageNet data set.
Accelerate the operation of Faster R-CNN model under
GPU and CUDA, and train the dataset A. The basic learning
rate is 0.001, the momentum is 0.9, the weight attenuation is
0.005, and the non-maximum suppression nms is the last value
of 300. The IoU threshold for nms is 0.7 and training is
performed using an alternating optimization algorithm.
After 29 hours of training, a circuit breaker state detection
and identification model was obtained, with a mAP value of
0.9000 (AP for on = 0.8985, AP for off = 0.9014) and an
1542
Authorized licensed use limited to: SHIV NADAR UNIVERSITY. Downloaded on June 20,2022 at 11:11:23 UTC from IEEE Xplore. Restrictions apply.
average detection time of 0.219 s/sheet, which achieved the
effect of real-time detection. The results are shown in Figure 3.
And the features extracted from each convolutional layer are
shown in Figure 4. The process of gradually extracting from
the low-level features to the high-level features can be
observed through the convolutional visualization.
elasticity, and intercepting part of the original image to the part
of the original data set, and generate a large number of
horizontal flips. The images were trained to find that the
dataset B contained a total of 10k pieces.
The circuit breaker state detection and identification
model based on Faster R-CNN was trained again, and the mAP
value was 0.9053 (AP for on = 0.9043 and AP for off =
0.9063). The purpose of improving the detection effect was
achieved. The test results are shown in Figure 6, and the loss
curves is shown in Figure 7.
Fig. 3 The Result of Circuit Breaker State Detection and Identification
Faster R-CNN loss
Fig. 6 Multi-Target Detection Results Based on Faster R-CNN.
Fig. 4 The Result of Circuit Breaker State Detection and Identification
But there are some state recognition occasions where
missed or false positives are detected for multi-target
situations, as shown in Figure 5.
iterations
Fig. 7 Faster R-CNN loss
curves.
In view of the above situation, randomly sample spatial
data from the dataset B for deep learning training respectively
such as 400, 1000, 2000, 5000 and 8000. The result of
statistical operation is that the mAP value increases with the
increase of dateset, and finally reaches 90%, as shown in Table
I. It can be seen that when the mAP value rises to a stable
level, increasing the sample spatial data can improve the
detection effect.
Date size
400
1000
2000
3312(dataset A)
5000
8000
10k(dataset B )
Fig. 5 Missed Situation of Multi-Target Detection Results
Based on Faster R-CNN.
For multi-object detection problems, use data
augmentation technology [1] to perform operations such as
rotating small angles, adding random noise, deformation with
TABLE I
MEAN AVERAGE PRECISION
Mean average precision
AP for on
AP for off
0.713
0.766
0.809
0.805
0.903
0.899
0.899
0.901
0.906
0.901
0.904
0.908
0.904
0.906
mAP
0.740
0.807
0.901
0.900
0.904
0.906
0.905
1543
Authorized licensed use limited to: SHIV NADAR UNIVERSITY. Downloaded on June 20,2022 at 11:11:23 UTC from IEEE Xplore. Restrictions apply.
V. CONCLUSION
This paper proposed the application of Faster R-CNN
deep learning method for ambient air automatic monitor
station inspection problems. By modifying the network model
and data augmentation methods, the circuit breaker state can
be detected and identified. Not only the different types of
circuit breaker can be recognized, but also it can be detected
simultaneously about multiple circuit breaker devices. The
detection accuracy is high, the real-time effect is good, it is
easy to expand, and it has strong practicability. It provides
auxiliary decision-making functions for the inspection of
ambient air automatic monitor station.
REFERENCES
[1]
Liangliang Yan, “Discussion on main points of daily inspection of
environmental air automatic monitoring station,” Environmental
Research and Monitoring, vol. 29, pp. 76-77, December 2016.
[2] Shun Zhang, Yihong Gong, and Jinjun Wang, “The development of deep
convolution neural network and applications on computer vision,”
Chinese Journal of Computer, vol. 40, no. 144, pp. 1-30, September
2017.
[3] Huangsheng Song, Xiangqing Zhang, Baofeng Zheng and Teng Yan,
“Vehicle detection based on deep learning in complexscene,” Application
Research of Computers, vol. 35, no. 4, pp. 1270-1273, April 2018.
[4] Szegedy C., Toshev A., and Erhan D., “Deep neural networks for object
detection,” Proceedings of the Advances in Neural Information
Processing Systems, Lake Tahoe, USA, 2013: 2553-2561.
[5] Erhan D., Szegedy C., Toshev A., et al.. “Scalable object detection using
deep neural networks,” Proceedings of the IEEE Conference on Computer
Vision and Pattern Recognition, Columbus, USA, 2014: 2147-2154.
[6] Cunji Zhang, Xifan Yao, Jiangming Zhang, and Erhui Liu, “Tool wear
monitoring based on deep learning,” Computer Integrated
Manufacturing Systems, vol. 23, no. 10, pp. 2146-2155, October 2017.
[7] Sihai Ding, Yuxue Liu, and Linji Lu, “Application of digital image
processing in the switch identification,” Microcomputer Applications,
vol. 30, no. 5, pp. 39-40, 2013.
[8] Shen Fang, Xiaohua Shu, and Dewu Li, “A method based on machine
vision for opening-closing status recognition of substation disconnection
switches,” Journal of Hunan University of Technology, vol. 31, no. 6, pp.
32-36, November 2017.
[9] Lowe D G. “Distinctive image features from scale-invariant keypoints,”
International Journal of Computer Vision, vol. 60, no. 2, pp. 91-110,
2004.
[10] Anwei Chen, Quanming Yue, Zongyin Zhang, and Yong Sun, “An
image recognition method of substation breakers state based on robot,”
Automation of Electric Power Systems, vol. 36, no. 6, pp. 101-105,
March 2012.
[11] Li Li, Binghai Wang, Wanguo Wang, and Jian Li, “An automatic status
recognition approach for outdoor high voltage circuit breaker based on
power station equipment monitoring robot,” Bulletin of Science and
Technology, vol. 27, no. 5, pp. 732-736, September 2011.
[12] Shaoqing Ren, Kaiming He, Ross Girshick, Jian Sun, “Faster R-CNN:
Towards Real-Time Object Detection with Region Proposal Networks,”
Proceedings of the Advances in Neural Information Processing Systems,
Montreal, Canada, 2015: 91-99.
1544
Authorized licensed use limited to: SHIV NADAR UNIVERSITY. Downloaded on June 20,2022 at 11:11:23 UTC from IEEE Xplore. Restrictions apply.