Engineering Applications of Artificial Intelligence 131 (2024) 107866
Contents lists available at ScienceDirect
Engineering Applications of Artificial Intelligence
journal homepage: www.elsevier.com/locate/engappai
Improved YOLOv8-GD deep learning model for defect detection in
electroluminescence images of solar photovoltaic modules
Yukang Cao a ,1 , Dandan Pang a ,1 , Qianchuan Zhao b , Yi Yan a , Yongqing Jiang c , Chongyi Tian a ,∗,
Fan Wang a , Julin Li a
a
Shandong Key Laboratory of Intelligent Buildings Technology, School of Information and Electrical Engineering, Shandong Jianzhu
University, Jinan 250101, Shandong province, China
b Department of Automation, Tsinghua University, Beijing 100018, China
c MOE Key Laboratory of Deep Earth Science and Engineering and Department of Civil Engineering, College of Architecture and Environment, Sichuan
University, Chengdu 610065, Sichuan province, China
ARTICLE
INFO
ABSTRACT
Keywords:
Photovoltaic systems
Health monitoring
Defect detection
Deep learning
YOLOv8
Photovoltaic defect detection is an essential aspect of research on building-distributed photovoltaic systems.
Existing photovoltaic defect detection models based on deep learning, such as YOLOv5 and YOLOv8, have
significantly improved the accuracy of photovoltaic defect detection. However, these models are too large,
and their feature extraction ability is insufficient, leading to low detection efficiency and inability to cope
with the continuous evolution of defects. Therefore, this study proposes an accurate and lightweight YOLOv8
(You Only Look Once v8) GD algorithm. The algorithm is an improved version of YOLOv8, wherein DW-Conv
(DepthWise-Conv) is applied to the YOLOv8 backbone network. Moreover, convolution is replaced with the
GSConv (Group-shuffle Conv) and the BiFPN (bidirectional feature pyramid network) structure is added to the
architecture. Several electroluminescent photovoltaic defect datasets are used to verify the effectiveness of the
proposed method. The final experimental results show that the map@0.5 and map@0.5∼0.95 of YOLOv8-GD
are 92.8% and 63.1%, respectively, which are 4.2% and 5.7% higher than those of the original algorithm,
respectively, and the model volume is reduced by 16.7%. Thus, the proposed algorithm shows considerable
potential in the field of photovoltaic defect detection.
1. Introduction
can easily lead to fires and cause significant losses (Addabbo et al.,
2017). Therefore, PV defect detection should be an essential part of
safety inspection and maintenance.
Currently,PV defect detection methods can be classified into traditional object detection methods (Juan and Kim, 2020) and object
detection algorithms based on deep learning (Tang et al., 2022a, 2020).
The traditional object detection algorithms utilize a sliding window
approach to traverse the entire image and generate a certain number
of candidate frames. The features of the candidate frames are then
extracted and classified using a classification method such as a support
vector machine (SVM) to obtain the results. The primary detection
methods include the Viola–Jones detector (Viola and Jones, 2001), the
HOG detector (Dalal and Triggs, 2005), and the part-based deformable
model (DPM) (Felzenszwalb et al., 2008). Owing to the lack of an
effective image representation, traditional object detection methods can
only use limited computational resources to design complex feature
representations for detection.
Building distributed photovoltaic power generation systems (Zhu
et al., 2015; Brown and Green, 2004) are the primary tools for resolving building energy resource shortage, energy conservation, and
environmental protection issues; however, they are directly related to
the effectiveness and safety of building electricity consumption (Gorge
et al., 2013). The compact configuration of distributed photovoltaic
power generation systems also places higher demands on the quality
of power generation, and fault detection (Mansouri et al., 2012). An
accurate assessment of the health of solar panels is of great important
to ensure the smooth operation of the power system (Kun et al., 2019).
However, in the actual operation of building distributed photovoltaic
power generation systems, solar cells are prone to defects such as hot
spots, hidden cracks, ash accumulation, and shading, and many other
types of defects (Wang et al., 2019). This is particularly significant in
hot spot areas where the temperature is relatively high; such defects
∗ Corresponding author.
1
E-mail address: tianchongyi86@163.com (C. Tian).
Yukang Cao and Dandan Pang are co-first authors of the article.
https://doi.org/10.1016/j.engappai.2024.107866
Received 7 June 2023; Received in revised form 19 October 2023; Accepted 6 January 2024
Available online 13 January 2024
0952-1976/© 2024 Elsevier Ltd. All rights reserved.
Engineering Applications of Artificial Intelligence 131 (2024) 107866
Y. Cao et al.
Deep learning-based defect detection methods primarily involve
defect imaging and feature extraction. They rely on infrared and
electroluminescent technology (Demirci et al., 2021) to realize defect
imaging and use a K-means color quantization and clustering algorithm to segment and extract defect regions (Sun et al., 2013). In
2014, the R-CNN algorithm (Girshick et al., 2014) was introduced
as a deep learning method for object detection. However, with the
extensive development of computer vision technology, the mainstream
object detection algorithms are now divided into one-stage and twostage object detection methods. The one-stage object detection algorithms usually produce candidate frames and simultaneously performs classification and boundary frame regression which is characterized by high speed but poor accuracy. Examples include the
YOLO (Redmon et al., 2016) series and the SSD algorithms (Berg
et al., 2015), such as the classic YOLOv3 (Redmon and Farhadi,
2018), and YOLOv4 (Bochkovskiy et al., 2020) algorithms, and then
the subsequent emergence of YOLOv5, YOLOv6 (Li et al., 2022a),
YOLOv7 (Wang et al., 2022) and YOLOv8 algorithms. These methods characterized by high speed but poor accuracy. The two-stage
object detection algorithms first generate regional candidate frames
and extract the features of each candidate frame; they finally generate
position frames and predict the corresponding categories. Examples
include Fast R-CNN (Girshick, 2015), SPPNet (He et al., 2014), Faster
R-CNN (Ren et al., 2017), and FPN (Lin et al., 2017) algorithms. They
are characterized by high accuracy but low speed. Therefore, improving
the accuracy and speed of the algorithms and their deployment ability
for the sparse and regional characteristics of building distributed PV
defects is necessary.
To identify abnormal regions, Li et al. (2020) proposed using the Kirsch
operator that could reduce computational complexity and error rate.
They then used a deep convolution neural network to extract defect
features and a multi-classification support vector machine to assist
defect detection decisions. Su et al. (2021) proposed an attention-based
top-down and bottom-up architectures to accomplish multiscale feature
fusion and solve the feature disappearance problem by deepening the
network. This architecture known as the bidirectional attention feature
pyramid network (BAFPN), enables all the pyramid layers to share
similar semantic features. Zhang et al. (2013) indicated that defect
images were reconstructed based on independent component analysis
(ICA) to enhance local anomalies to detect defects. Fioresi et al. (2022)
proposed a segmented Deeplabv3 model using the ResNet-50 backbone
to identify and segment defects in silicon photovoltaic cells’ electroluminescence (EL) images. Doll et al. (2021) used photoluminescence
(PL) as an outdoor characterization method for full-size module imaging to provide economical, fast, and non-destructive detection of PV
defects. Akram et al. (2020) proposed an automated detection method
for PV module defects using deep learning methods based on infrared
thermal imaging technology. Ding et al. (2018) used migration learning
to make a PV defect detection algorithm more robust. Wang et al.
(2021) proposed using unsupervised learning to automatically detect
defects in EL images and to statistically classify them based on electrical
sources. Pratt et al. (2021) identified and segmented PV defects in EL
images based on the U-Net architecture. Demirci et al. (2021) used
a lightweight convolutional neural network (L-CNN) architecture to
reduce the number of parameters in the model and thus enhancing its
deployment capability.
In general, algorithm innovation for the detection of building distributed PV defects needs to be based on accuracy, speed, and deployment ability, not only to achieve a balance between accuracy and speed
but also to ensure that it can meet the requirements for deployment,
particularly on unmanned aircraft and embedded devices for large-scale
building-distributed PV detection and maintenance. This study makes
the following contributions in the areas of accuracy, speed balance, and
deployment ability.
The main contributions of this study are as follows:
2. Related methods
Research on solar panel defect detection can be broadly divided
into two categories. One approach is to use devices such as unmanned
aerial vehicles (UAVs), to acquire large-scale aerial images acquisition
for defect detection. For example, Li et al. (2019) combined deep
learning, data argumentation, and migration learning to propose a nondestructive vision inspection system based on UAV and edge computing
to solve the problem of a lack of training samples on edge devices. Grimaccia et al. (2017) used a light UAV to collect the image of the surface
conditions of the photovoltaic panels. Image post-processing technology was then utilized for rapid fault detection to minimize the impact
of faults on the photovoltaic power generation performance. Tang et al.
(2022b) considered the balance between defect detection performance
and computational complexity and proposed the distribution of computational tasks among edge devices, edge servers, and cloud servers to
perform defect detection using deep learning methods. Li et al. (2017)
indicated that an image-processing algorithm based on the first-order
derivative of the Gaussian function and feature matching can detect
defects in aerial parts. Salimzadeh and Hammad (2017) used building
information modeling (BIM) and ranging (LiDAR) techniques to build
distributed PV for potential estimation. Quan et al. (2017) compressed
the defect mapping image by eliminating the information redundancy
in the sensing stage of defect location; that is, in the process of defect
location, the photocurrent of test points was measured using an LCD
to illuminate at the same time instead of point-by-point; hence, a small
number of photocurrent measurements were used to solve the convex
problem of positioning of defects. Xiaoxia et al. (2019) proposed using
images captured by drones for defect pattern recognition. Convolution
neural networks are used during machine learning to classify various
forms of modular defects.
The second category of methods include the use of deep learning
algorithms for defect detection. For example, Zhang (2023) proposed
a virtual convolution method based on BottleneckCSP and YOLOv5
(GBH-YOLOv5) with tiny object prediction heads for PV panel defect
detection, which increases the inference speed and reduces the number
of parameters by adding prediction heads and replacing convolutions.
• This paper proposes an improved method based on YOLOv8 for
building distributed PV defects, named YOLOv8-GD.
• YOLOv8-GD includes improved methods such as GSConv, BiFPN
and DW-Conv, which shows improved accuracy and speed.
• YOLOv8-GD can be used for defect detection and health monitoring in building distributed PV, which can provide a reference for
future research in this field.
3. YOLOv8-GD defect detection algorithm for building distributed
PV
In this study, we proposed an improved defect detection algorithm
based on YOLOv8 for the sparse characteristics of building distributed
PV defects. After verifying the performance of YOLOv8 using the PV
defect datasets, the best-performing YOLOv8s were selected for improvement and named YOLOv8-GD. The main improvement strategies
include the following:
(1) Application of DW-Conv to the YOLO v8 backbone network. This
is used to reduce the number of parameters by 14% and improve the
inference speed.
(2) Utilization of the GSConv to replace the standard convolution in
the YOLOv8 neck network. This better balances the accuracy and speed
of the model.
(3) Utilization of the BiFPN structure in the neck network. This
enriches the contextual information (information around the object
pixel).
The specific network structure is illustrated in Fig. 1.
2
Engineering Applications of Artificial Intelligence 131 (2024) 107866
Y. Cao et al.
Fig. 1. The network architecture of YOLOv8-GD.
Fig. 2. The DW-Conv structure diagram.
3.1. DW-Conv (DepthWise-Conv)
and computational effort involved in extracting defect information from
the model backbone network.
In the DW-Conv, the number of DW-Conv (Chollet, 2017) channels
is equal to that of the channels (Fig. 2). Each channel is convolved
separately as a group, and the results of the convolution of each group
are spliced for the output. Here, output channels are input to DWConv, divided into groups with an equal number of input channels,
and there is no information flow between channels. Specifically, for
several convolution kernels, each convolution kernel only completes
the convolution of the input single channel to acquire the feature map
and then stacks a plurality of feature maps to get the output with the
same input depth. However, the difference is that the spatial dimension
is compressed. Briefly, the number of channels in the convolution
kernel of a conventional convolution operation is the same as the
number of channels in the input image. Conversely, deep convolution
can be regarded as a special case of grouping convolution. By grouping
the number of channels in the input picture, a convolution operation is
performed, thus reducing the parameter quantity of the model and the
convolution calculation cost and improving the inference speed. This
module is primarily designed to solve the problems of the parameters
3.2. GSConv (Group-shuffle Conv)
In the CNN structure, image transmission must undergo a conversion process in which spatial feature information is gradually transferred to channel information. The feature map must be compressed
in height and width during the conversion process; however, the compression process tends to lose the semantic information of defective
features. If a dense convolution is used, the connections between channels must be preserved to the maximum extent possible; however,
this will significantly increase the computational cost. With sparse
convolution, these connections are completely severed, and critical
information about defective features is easily lost.
Conversely, the GSConv approach (Li et al., 2022b) (Fig. 3) preserves as many of these connections as possible. However, if used in
both the backbone and neck stages, it would increase the number
of network layers while exacerbating the data flow resistance and
inference time. Therefore, because no transformation of the feature
map is required in the neck stage, the channel dimensions reach their
3
Engineering Applications of Artificial Intelligence 131 (2024) 107866
Y. Cao et al.
Fig. 3. The GSConv structure diagram.
maximum. GSConv is used to process the spliced feature map obtained
from the backbone and reduce redundant repetitive information. It
increases the inference speed while ensuring feature extraction accuracy; moreover, it reduces the number of parameters and computational
costs.
3.3. BiFPN (bidirectional feature pyramid network)
The BiFPN proposed by EfficientDet (Tan et al., 2020) is a weighted
bidirectional feature pyramid network. In contrast to previous feature
fusion networks, the BiFPN does not treat input features equally during
the fusion process. It allocates additional weights to each input feature,
enabling the network to learn the importance of each input feature.
The BiFPN has branches from top to bottom and from bottom to top.
Moreover, it increases the direct jump connection between the input
and output nodes; thus, the feature information (context information)
around the target pixel can be fused through various branches and
jump connections to extract features. The BiFPN structure used in
this study was provided by YOLOv5 officials, with learnable weight
parameters set to handle the input features. Unlike previous feature
fusion networks, the BiFPN no longer treats input features equally
during the fusion process, but allocates additional weights to each input
feature, allowing the network to learn the importance of each input
feature.
Fig. 4. The defect data set after data enhancement.
Defects types such as black core sheet, black spot sheet, and hot
spot can similarly affect the cell’s performance; hence, these types were
uniformly labeled as spots for easy detection (Cao et al., 2023). The
total defect types were divided into cracks and spots (Fig. 5). The samples were labeled using labelimg1.8.3 dataset labeling software. Finally,
there is no fixed basis for the division between training and validation
or test sets. Generally, it is divided according to the number of features
in the dataset; however, if the conditions permit, more images can be
used for training. In this study, the division ratio of 9:1 was selected
for the experiment. And the experimental results in Section 4.3 also
showed that the training results were not significantly different from
the test results, which fully meets the relevant requirements of object
detection algorithms.
4. Experiments
4.1. Data set and preprocessing
The data set (Buerhop et al., 2018) used in this study was presented at the 35th European Photovoltaic Solar Energy Conference and
Exhibition. Images of solar cells extracted from high-resolution electroluminescence images of photovoltaic modules were thus provided at
the conference. The data set comprised 44 PV modules, including 18
single crystals and 26 polymorphic modules. The images could be split
into 2624 EL images of 300 × 300-pixel cells. These images contained
common internal and external defects such as black core sheets, black
spot sheets, short circuit black sheets, over-soldered sheets, broken
grids, bright and dark sheets, hidden cracks, etc. A total of 1550
defect images were selected from this dataset (Fig. 4). The dataset
was expanded using data enhancement methods, such as random flip,
rotation, translation transformation, and scale transformation, resulting
in a final dataset of 4530 defect images.
4.2. Experimental parameter setting and model training
To make the YOLOv8-GD algorithm more effective, all of the training and testing tasks described in this paper were performed on a server
with a single GPU (CPU: Intel(R) Core(TM) i7-9700F CPU: @3.00 GHz,
RAM: 16 GB GPU: NVIDIA GeForce GTX 3060) and Python 3.8. etc.
Fig. 6 shows that 16 samples were used to train the model. Each
sample consisted of four images augmented by stitched data, effectively
solving the small-object problem during model training. The algorithms
used for comparison were YOLOv3, YOLOv5s, YOLOv6s, YOLOXS,
YOLOv8n, YOLOv8s, and YOLOv8s-GS-BiFPN.
The specific parameter settings are listed in Table 1. The batch size
was set to 32, the input size of image was 416 × 416, and an SGD
optimizer was used, after which training was performed. At the end of
4
Engineering Applications of Artificial Intelligence 131 (2024) 107866
Y. Cao et al.
Fig. 5. Labeling process of data set.
Fig. 6. Training visualization.
Table 1
Hyperparameter setting.
Parameter
Settings
lr0(initial learning rate)
lrf(final learning rate)
momentum
weight_decay
warmup_epochs
warmup_momentum
warmup_bias_lr
epochs
optimizer
Mosaic
hsv_h(HSV-Hue augmentation)
hsv_s(HSV-Saturation augmentation)
hsv_v(HSV-Value augmentation)
0.01
0.01
0.937
0.0005
3.0
0.8
0.1
300
SGD
1.0
0.015
0.7
0.4
score, and PR curve. The area under the PR curve (Davis, 2006) was
used to evaluate the model’s accuracy, where P represents precision and
R represents recall. The area enclosed by the PR curve is the AP, and
the average AP of all categories is the map, as shown in Eq. (1).
𝑇𝑃
𝑇𝑃 + 𝐹𝑃
𝑇𝑃
𝑅𝑒𝑐𝑎𝑙𝑙 =
𝑇𝑃 + 𝐹𝑁
𝑃 𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 =
(1)
where TP represents the number of correctly detected ground-truth
boxes, including the number of defects in the ground-truth boxes;
FN represents the number of undetected ground-truth boxes and FP
represents the number of incorrect bounding boxes.
This experiment uses the YOLOv8s algorithm as a baseline for
improvement. The comparison algorithms used include the classic
YOLOv3-tinyu (the simplified version of YOLOv3), YOLOv5s (the smallest version of YOLOv5 in size), YOLOv6s (the lightest version of
YOLOv6), YOLOv8n (the smallest version of YOLOv8), YOLOv8s-DCNv2
(which replaces the Conv module with DCNv2), YOLOv8s-ConvNext
(which uses ConvNext as a backbone network), YOLOv8s-GS-SPD
(wherein SPDConv is inserted into the backbone and GSConv replaces
the Conv module), YOLOv8s-GS-SimAM (wherein GSConv replaces
convolution and the SimAM attention mechanism is added). Other
algorithms are obtained after training. The PR curves of training
process are shown in Fig. 7. The parameters floating point operations
(FLOPs), mean of average precision (MAP), and frames per second
(FPS) were taken as the model performance metrics of this experiment,
in which the parameters refer to the total number of parameters to
model training, metrics such as model layers, number of parameters,
computation complexity (FLOPs), map@0.5, and map@0.5∼0.95 were
estimated to demonstrate the model’s performance during the training
process.
4.3. Model validation and results
At the end of model training, the weights of the last epoch of the
training process and the best training weights were obtained. Generally,
the best training weights are used as the output weights of the model
for validation. The validation process generated a confusion matrix, F1
5
Engineering Applications of Artificial Intelligence 131 (2024) 107866
Y. Cao et al.
Fig. 7. Comparison of PR curves for different models.
Table 2
Test results comparison of different models.
Method
FLOPs (G)
Params (M)
Layers
Map@0.5
Map@0.5 ∼ 0.95
YOLOv3-tinyu
YOLOv5s
YOLOv6s
YOLOv8n
YOLOv8s
YOLOv8s-DCNv2
YOLOv8s-ConvNext
YOLOv8s-GS-SPD
YOLOv8s-GS-SimAM
YOLOv8s-GS-ConvNext
YOLOv8s-GS-BiFPN
YOLOv8s-GS-Repvgg-SPD
YOLOv8s-GS-DW
YOLOv8s-GS-slim
YOLOv8s-GS-DW-slim
YOLOv8s-GS-DW-SPD
YOLOv8-GD
19.9
15.8
57.3
8.1
28.4
25.4
23.7
7.3
26.8
21.6
26.4
8.4
24.3
23.0
19.3
6.5
24.5
12.13
7.02
18.75
3.00
11.13
11.24
9.06
10.13
10.19
7.99
10.06
11.08
9.20
8.88
7.32
9.35
9.27
63
213
446
168
168
168
198
190
191
219
189
210
174
266
266
175
174
0.834
0.865
0.906
0.896
0.886
0.875
0.821
0.862
0.901
0.886
0.928
0.796
0.916
0.896
0.894
0.8
0.928
0.486
0.487
0.558
0.558
0.574
0.55
0.467
0.509
0.589
0.563
0.64
0.437
0.613
0.575
0.578
0.437
0.631
be trained in model training, which is used to describe the size of the
model. FLOPs refer to the floating-point operands and the calculation
amount of the model; this is used to measure the complexity of the
algorithm or model. MAP refers to averaging the AP values of all classes
to describe the accuracy of the model prediction. FPS refers to the
number of frames transmitted per second and is used to measure the
model’s speed (Zhang et al., 2015).
Fig. 7(a) shows a comparison of the algorithms in the YOLO series,
where the YOLOv8s algorithm has the best training performance on
this data set, followed by YOLOv6s, YOLOv3-tinyu, and YOLOv5s. This
indicates that the YOLOv8s algorithm performs well in distinguishing
between positive and negative training samples while also aggregating positive samples to a high confidence level to increase coverage,
that is, the area enclosed by the PR curve and coordinate axis. In
Fig. 7(b), the curve coverage of YOLOv8-GD is slightly higher than that
of YOLOv8s, whereas the curve coverage of YOLOv8s is higher than
that of YOLOv8n, which indicates that the training performance of the
proposed YOLOv8-GD algorithm is better than that of the comparison
algorithm mentioned in Table 2.
The experiment validated the classical YOLOv3-tinyu and YOLOv5s
algorithms (Table 2), both of which effectively reduced the computational complexity and number of parameters while achieving the
accuracy of 83.4% and 86.5% on the data set, respectively. This confirmed the generality of the simplified classical algorithms in the field
of object detection. YOLOv6s also shows good potential owing to its improved network structure, achieving an accuracy of 90.6% on this data
set; however, its computational complexity and number of parameters
are considerable, which is not conducive to later quantitative deployment. By contrast, YOLOv8n and YOLOv8s achieved high accuracy
and lightweight. Therefore, this study mainly focused on the improvement of YOLOv8s. Comprehensive experimental validation of various
improvement methods was conducted by replacing the modules, replacing backbone, and Neck, including the more popular ConvNext,
DCNv2, GSConv, SPD-Conv, and other modules. Although these improvements showed promising results in terms of accuracy, the number
of parameters, and complexity, they could still not balance model
accuracy and speed. The best results were achieved by embedding
GSConv and DWConv into the network and using the BiFPN structure
on the experimental data set;this network was named YOLOv8-GD.
The map@0.5 and map@0.5∼0.95 of YOLOv8-GD achieved 92.8% and
63.1%, respectively, a 4.2% and 5.7% improvement compared to the
original algorithm. The FLOPs and parameters were reduced by 13.7%
and 16.7% respectively, thus achieving the ideal results. The final
detection results are shown in Fig. 8. The prediction box represents
the actual application results of the model, the red and the pink boxes
represent the positions of the two types of defects detected by the
model, and the numbers represent the probability of such defects. The
higher the probability, the more accurate is the defect detection.
The inference speed is also an integral aspect of the algorithm performance quantification, This experiment used the FPS to characterize
the inference speed of the model. The formula for calculating FPS is as
shown in Eq. (2):
𝐹𝑃𝑆 =
1000
𝑝𝑟𝑒 + 𝑖𝑛𝑓 𝑒𝑟𝑒𝑛𝑐𝑒 + 𝑝𝑜𝑠𝑡
(2)
where pre is the preprocessing time of the model, including the time
consumed by image scaling, padding, and channel transformation.
Inference is the inference time of the model, which refers to the time
from the preprocessed image input to the model output result. Post is
the post-processing time of the model, which refers to the time spent
on the line conversion of the model output results. FPS is the inference
speed of the model.
6
Engineering Applications of Artificial Intelligence 131 (2024) 107866
Y. Cao et al.
Fig. 8. Visualization of the detection process.
Fig. 9. PVEL-AD data set.
Table 3
Comparison of inference speed of different models.
Model
Pre-process (ms)
Inference (ms)
Post-process (ms)
FPS
YOLOv3-tinyu
YOLOv5s
YOLOv6s
YOLOv8n
YOLOv8s
YOLOv8s-DCNv2
YOLOv8s-ConvNext
YOLOv8s-GS-SPD
YOLOv8s-GS-SimAM
YOLOv8s-GS-ConvNext
YOLOv8s-GS-BiFPN
YOLOv8s-GS-Repvgg-SPD
YOLOv8s-GS-DW
YOLOv8s-GS-slim
YOLOv8s-GS-DW-slim
YOLOv8s-GS-DW-SPD
YOLOv8-GD
0.5
0.5
0.4
0.5
0.7
0.4
0.4
0.5
0.5
0.5
0.5
0.4
0.3
0.4
0.5
0.6
0.5
8.3
11
18.4
9.7
10.3
10.7
10.6
10.3
10.6
11.1
10.3
11.4
10.5
10.3
10.4
10.1
10.1
1.8
1.4
1.0
1.7
1.8
1.5
1.6
1.7
1.6
1.7
1.6
1.7
1.6
1.4
1.8
1.7
1.6
94.34
77.52
50.51
84.03
78.13
79.37
79.37
80
78.74
75.19
80.65
74.07
80.65
84.64
78.74
80.65
81.97
Table 3 shows a comparison of the inference speeds of different
models. With approximately the same model size, YOLOv8-GD improved the inference speed to 81.97 FPS, surpassing the benchmark
YOLOv5s, YOLOv6s, YOLOv8s, and various improved methods. This
also shows that YOLOv8-GD ensures a balanced speed with improved
model accuracy, and provides feasibility for subsequent deployment.
4.4. Performance of the proposed model on other data sets
In this study, the PVEL-AD data set (Su et al., 2023) was used to
demonstrate the generalizability of YOLOv8-GD further. The dataset includes crack, finger, black_core, thick_line, star_crack, corner, fragment,
scratch, horizontal_dislocation, vertical_dislocation, printing_error,
7
Engineering Applications of Artificial Intelligence 131 (2024) 107866
Y. Cao et al.
Table 4
Comparison of model test results.
Method
FLOPs (G)
Params (M)
Map@0.5
Map@0.5 ∼ 0.95
FPS
YOLOv5lu
YOLOv5mu
YOLOv5nu
YOLOv5su
YOLOv8l
YOLOv8m
YOLOv8n
YOLOv8s
YOLOv8-GD
134.7
64
7.1
23.9
164.9
78.7
8.1
28.5
24.1
50.68
23.89
2.39
8.69
41.60
24.65
2.87
10.61
9.27
0.925
0.908
0.901
0.907
0.917
0.923
0.916
0.93
0.926
0.557
0.587
0.559
0.578
0.581
0.589
0.573
0.58
0.585
54.64
68.03
90.09
80
53.48
68.97
80.65
72.99
75.19
GSConv
BiFPN
Map@0.5
Map@0.5 ∼ 0.95
FPS
✓
0.886
0.927
0.923
0.903
0.916
0.928
0.906
0.574
0.628
0.625
0.599
0.613
0.64
0.601
78.13
78.13
80
81.3
80.65
80.65
76.34
Table 5
Ablation experiment.
DW-Conv
✓
✓
✓
✓
✓
✓
✓
✓
short_circuit, and 12 other defects (Fig. 9). In this experiment, 4500
defect images with a resolution of 1024 × 1024 were selected from the
original data as datasets. The training and verification sets were divided
according to the ratio of 9:1, in which the training set included 4051
defect images and the verification set included 449 defect images.
The algorithm’s performance cannot be demonstrated using a single
dataset, and the defect category and quantity of the dataset used in this
study are insufficient. Therefore, this study evaluated the algorithm’s
performance on more larger datasets. As shown in Table 4, the accuracy, model size, FPS, and complexity of the YOLOv8-GD algorithm still
show promising results in the same experimental environment, fully
demonstrating the proposed algorithm’s generalization ability.
5. Discussion
The improved YOLOv8-GD algorithm comprises three main modules: DW-Conv, GSConv, and BiFPN. In this study, the mechanisms
of action of these three modules were experimentally investigated to
illustrate the advantages of YOLOv8-GD. The results are summarized
in Table 5.
After adding DW-Conv, the model’s accuracy improved by 4.1%
and 5.4%, respectively, but the inference speed was the same as that
of the original model. This indicates that DW-Conv improves the feature extraction capability of the backbone network while ensuring
convolution performance, resulting in a more defect-informed feature
representation.
The experimental results show that embedding GSConv into the
neck network with less redundant information maximizes its effectiveness, with a more significant increase in accuracy and speed. In
particular, the connections between channels retained by GSConv ensure the model’s accuracy while reducing the number of parameters,
enabling the network to obtain richer information on defect features.
The addition of the BiFPN structure improved the model accuracy
by 1.7% and 2.5%, respectively, and the inference speed with improved
feature fusion capability, demonstrating that BiFPN enables neck networks to focus on the contextual information of defective features and
ensure a balance between model accuracy and inference speed.
Fig. 10 shows a plot of the PR curve of YOLOv8s after inserting the
three modules. The accuracy of the improved method is higher than
that of the original YOLOv8s algorithm after the insertion of the three
modules separately.
Fig. 11 shows a heat map visualization after adding the three
modules. The insertion of DW-Conv and GSConv improve the model’s
accuracy more because the convolution operation is used more in the
Fig. 10. PR comparison graph for the three modules.
algorithm, which has a more far-reaching influence. The BiFPN structure, conversely, focuses on improving the feature fusion capability.
Overall, DW-Conv, when used to replace the standard convolution,
improves the extraction capability of the backbone network, thus increasing the focus on defective feature information. In contrast, the
neck network uses the GSConv and BiFPN structures to fuse further the
multiscale feature information extracted from the backbone network
and balance the accuracy and speed of the model while preserving
the connectivity between channels. The advantages of YOLOv8-GD in
model size, accuracy, and speed are illustrated.
Additionally, the interpretability (Duval, 2019) of a model is an
important aspect that requires attention. Establishing an interpretable
photovoltaic defect detection model (Nauck, 2019) while maintaining
a high level of learning performance (such as prediction accuracy)
is the next direction of study. In this process, balancing the model
interpretability and performance according to the principles of artificial
intelligence, fairness, privacy, and data fusion is necessary.
6. Conclusions
This study proposes a defect detection algorithm based on the improved YOLOv8 for detecting sparse and hard to distinguish distributed
PV defects; the algorithm mainly included DW-Conv, GSConv, and
BiFPN structures. The improved strategy proposed in this study were
8
Engineering Applications of Artificial Intelligence 131 (2024) 107866
Y. Cao et al.
Fig. 11. The Grad-CAM heat map of three modules.
Yongqing Jiang: Supervision. Chongyi Tian: Writing – review &
editing. Fan Wang: Software. Julin Li: Software.
validated by creating a dataset containing 4530 defect images. The
effectiveness of the proposed method was confirmed through model
training and validation.
PVEL-AD datasets with more types and abundant defects were used
to verify the robustness of the YOLOv8-GD algorithm to reflect the
generalization ability of the model. The accuracy and speed of YOLOv8GD were still improved, demonstrating the potential and popularization
value of YOLOv8-GD in photovoltaic defect detection.
The final experimental results show an improvement in detection
accuracy and speed. To our knowledge, most current practical applications in this field sacrifice accuracy to improve detection speed.
This algorithm’s improved accuracy and detection speed can meet the
actual deployment requirements. In the next step, we will deploy the
algorithm to unmanned aerial vehicles and other carriers to monitor the
distributed photovoltaics of buildings. This will facilitate the reduction
of energy consumption of buildings and improve the quality of power
generation.
Declaration of competing interest
We wish to confirm that there are no known conflicts of interest
associated with this publication and there has been no significant
financial support for this work that could have influenced its outcome.
We confirmed that we had reviewed the final manuscript and
approved its publication. To the best of our knowledge and belief, this
manuscript has not been published in whole or in part, nor has it been
considered for publication elsewhere.
We confirm that the manuscript has been read and approved by all
named authors and that there are no other persons who satisfied the
criteria for authorship but are not listed. We further confirm that the
order of authors listed in the manuscript has been approved by all of
us.
We confirm that we have given due consideration to the protection
of intellectual property associated with this work and that there are
no impediments to publication, including the timing of publication,
with respect to intellectual property. In so doing we confirm that we
have followed the regulations of our institutions concerning intellectual
property.
CRediT authorship contribution statement
Yukang Cao: Conceptualization, Methodology, Writing – original
draft. Dandan Pang: Data curation, Software, Validation. Qianchuan
Zhao: Supervision, Validation. Yi Yan: Visualization, Investigation.
9
Engineering Applications of Artificial Intelligence 131 (2024) 107866
Y. Cao et al.
We understand that the Corresponding Author is the sole contact
for the Editorial process (including Editorial Manager and direct communications with the office). He/she is responsible for communicating
with the other authors about progress, submissions of revisions and
final approval of proofs.
Grimaccia, F., Leva, S., Niccolai, A., 2017. PV plant digital mapping for modules’
defects detection by unmanned aerial vehicles. Iet Renew. Power Gener. 11 (10),
1221–1228. http://dx.doi.org/10.1049/iet-rpg.2016.1041.
He, K., Zhang, X., Ren, S., Sun, J., 2014. Spatial pyramid pooling in deep convolutional
networks for visual recognition. In: IEEE Transactions on Pattern Analysis & Machine Intelligence. pp. 1904–1916. http://dx.doi.org/10.1007/978-3-319-105789_23.
Juan, R., Kim, J., 2020. Photovoltaic cell defect detection model based-on extracted electroluminescence images using SVM classifier. http://dx.doi.org/10.
1109/ICAIIC48513.2020.9065065.
Kun, D., Fudong, C., Yuanliang, L., 2019. Research on the health status of photovoltaic
system based on Hilbert-Huang transform and grey relational degree. Renew.
Energy Resour.
Li, C., Li, L., Jiang, H., Weng, K., Geng, Y., Li, L., Ke, Z., Li, Q., Cheng, M., Nie, W.,
Li, Y., Zhang, B., Liang, Y., Zhou, L., Xu, X., Chu, X., Wei, X., Wei, X., 2022a.
YOLOv6: a single-stage object detection framework for industrial applications.
arXiv:2209.02976.
Li, H., Li, J., Wei, H., Liu, Z., Zhan, Z., Ren, Q., 2022b. Slim-neck by GSConv: A better
design paradigm of detector architectures for autonomous vehicles. arXiv preprint
arXiv:2206.02424.
Li, X., Li, W., Yang, Q., Yan, W., Zomaya, A.Y., 2019. Building an online defect
detection system for large-scale photovoltaic plants. In: The 6th ACM International
Conference.
Li, X., Li, W., Yang, Q., Yan, W., Zomaya, A.Y., 2020. An unmanned inspection system
for multiple defects detection in photovoltaic plants. IEEE J. Photovolt. 10 (2),
568–576. http://dx.doi.org/10.1109/JPHOTOV.2019.2955183.
Li, X., Qiang, Y., Chen, Z., Luo, X., Yan, W., 2017. Visible defects detection based
on UAV-based inspection in large-scale photovoltaic systems. IET Renew. Power
Gener. 11 (10), 1234–1244. http://dx.doi.org/10.1049/iet-rpg.2017.0001.
Lin, T.Y., Dollar, P., Girshick, R., He, K., Hariharan, B., Belongie, S., 2017. Feature
pyramid networks for object detection. In: 2017 IEEE Conference on Computer
Vision and Pattern Recognition. CVPR.
Mansouri, A., Zettl, M., Mayer, O., Lynass, M., Burhenne, R., 2012. Defect detection in
photovoltaic modules using electroluminescence imaging.
Nauck, D., 2019. Responsible AI. J. Inst. Telecommun. Prof. (Apr./Jun. Pt.2), 13.
Pratt, L., Govender, D., Klein, R., 2021. Defect detection and quantification in electroluminescence images of solar PV modules using U-net semantic segmentation.
Renew. Energy 178, http://dx.doi.org/10.1016/j.renene.2021.06.086.
Quan, L., Xie, K., Xi, R., Liu, Y., 2017. Compressive light beam induced current sensing
for fast defect detection in photovoltaic cells. Sol. Energy 150 (jul.), 345–352.
http://dx.doi.org/10.1016/j.solener.2017.04.059.
Redmon, J., Divvala, S., Girshick, R., Farhadi, A., 2016. You only look once: unified,
real-time object detection. Comput. Vis. Pattern Recognit.
Redmon, J., Farhadi, A., 2018. YOLOv3: an incremental improvement. http://dx.doi.
org/10.48550/arXiv.1804.02767, arXiv e-prints.
Ren, S., He, K., Girshick, R., Sun, J., 2017. Faster R-CNN: towards real-time object
detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell.
39 (6), 1137–1149. http://dx.doi.org/10.1109/TPAMI.2016.2577031.
Salimzadeh, N., Hammad, A., 2017. High-level framework for GIS-based optimization of
building photovoltaic potential at urban scale using BIM and LiDAR. pp. 123–134.
http://dx.doi.org/10.1061/9780784481196.012.
Su, B., Chen, H., Zhou, Z., 2021. BAF-detector: an efficient CNN-based detector for
photovoltaic cell defect detection. IEEE Trans. Ind. Electron. PP (99), 1. http:
//dx.doi.org/10.1109/TIE.2021.3070507.
Su, B., Zhou, Z., Chen, H., 2023. PVEL-AD: a large-scale open-world dataset for
photovoltaic cell anomaly detection. IEEE Trans. Ind. Inform. 19 (1), 404–413.
http://dx.doi.org/10.1109/TII.2022.3162846.
Sun, Z.J., Xue, L., Xu, Y.M., 2013. Marginal Fisher feature extraction algorithm based
on deep learning. Dianzi Yu Xinxi Xuebao/J. Electron. Inform. Technol. 35 (4),
805–811. http://dx.doi.org/10.3724/SP.J.1146.2012.00949.
Tan, M., Pang, R., Le, Q.V., 2020. EfficientDet: scalable and efficient object detection.
In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. CVPR.
Tang, W., Yang, Q., Hu, X., Yan, W., 2022a. Convolution neural network based polycrystalline silicon photovoltaic cell linear defect diagnosis using electroluminescence
images. Expert Syst. Appl. (Sep.), 202.
Tang, W., Yang, Q., Hu, X., Yan, W., 2022b. Deep learning-based linear defects
detection system for large-scale photovoltaic plants based on an edge-cloud
computing infrastructure. Sol. Energy 231, 527–535. http://dx.doi.org/10.3390/
electronics12030561.
Tang, W., Yang, Q., Xiong, K., Yan, W., 2020. Deep learning based automatic defect
identification of photovoltaic module using electroluminescence images. Sol. Energy
201, 453–460. http://dx.doi.org/10.1016/j.solener.2020.03.049.
Viola, P.A., Jones, M.J., 2001. Rapid object detection using a boosted cascade of
simple features. In: Computer Vision and Pattern Recognition, 2001. CVPR 2001.
Proceedings of the 2001 IEEE Computer Society Conference on.
Wang, C.Y., Bochkovskiy, A., Liao, H., 2022. YOLOv7: Trainable bag-of-freebies sets
new state-of-the-art for real-time object detectors. http://dx.doi.org/10.48550/
arXiv.2207.02696, arXiv e-prints.
Wang, Y., Li, J., Chen, Y., Zhou, J., Chu, J., 2019. Effects of working pressure and power
onphotovoltaic and defect properties of MagnetronSputtered Sb2Se3 thin-film solar
cells. Appl. Opt. 59 (4), http://dx.doi.org/10.1364/AO.382805.
Data availability
The authors do not have permission to share data.
Acknowledgments
The authors are grateful for the National Natural Science Foundation of China (No. 52007109, No. 62076150), the Natural Science
Foundation of Shandong Province, China (No. ZR2022MF302) and all
the reviewers for their constructive comments and valuable suggestions
to improve the quality of the article.
References
Addabbo, P., Angrisano, A., Bernardi, M.L., Gagliarde, G., Ullo, S., 2017. A UAV infrared
measurement approach for defect detection in photovoltaic plants. In: 2017 IEEE
International Workshop on Metrology for AeroSpace (MetroAeroSpace).
Akram, M.W., Li, G., Jin, Y., Chen, X., Ahmad, A., 2020. Automatic detection of
photovoltaic module defects in infrared images with isolated and develop-model
transfer deep learning. Sol. Energy 198, 175–186. http://dx.doi.org/10.1016/j.
solener.2020.01.055.
Berg, A.C., Fu, C.Y., Szegedy, C., Anguelov, D., Erhan, D., Reed, S., Liu, W., 2015. SSD:
single shot MultiBox detector.
Bochkovskiy, A., Wang, C.Y., Liao, H., 2020. YOLOv4: optimal speed and accuracy of
object detection. http://dx.doi.org/10.48550/arXiv.2004.10934.
Brown, A.S., Green, M.A., 2004. Impurity photovoltaic effect with defect relaxation:
Implications for low band gap semiconductors such as silicon. J. Appl. Phys. 96
(5), 2603–2609. http://dx.doi.org/10.1063/1.1777394.
Buerhop, C., Deitsch, S., Maier, A., Gallwitz, F., Brabec, C.J., 2018. A benchmark for
visual identification of defective solar cells in electroluminescence imagery. In: 35th
European Photovoltaic Solar Energy Conference and Exhibition.
Cao, Y., Pang, D., Yan, Y., Jiang, Y., Tian, C., 2023. A photovoltaic surface defect
detection method for building based on deep learning. J. Build. Eng. 70, 106375.
http://dx.doi.org/10.1016/j.jobe.2023.106375.
Chollet, F., 2017. Xception: Deep learning with depthwise separable convolutions. In:
2017 IEEE Conference on Computer Vision and Pattern Recognition. CVPR.
Dalal, N., Triggs, B., 2005. Histograms of oriented gradients for human detection. In:
IEEE Computer Society Conference on Computer Vision & Pattern Recognition.
Davis, J., 2006. The relationship between precision-recall and ROC curves. In:
Proceedings of the 23th International Conference on Machine Learning, 2006.
Demirci, M.Y., Beli, N., Gümüü, A., 2021. Efficient deep feature extraction and classification for identifying Defective Photovoltaic Module Cells in Electroluminescence
Images. Expert Syst. Appl. 175 (2), 114810. http://dx.doi.org/10.1016/j.eswa.2021.
114810.
Ding, S., Yang, Q., Li, X., Yan, W., Wei, R., 2018. Transfer learning based photovoltaic
module defect diagnosis using aerial images. In: 2018 International Conference on
Power System Technology (POWERCON).
Doll, B., Hepp, J., Hoffmann, M., Schuler, R., Buerhop-Lutz, C., Peters, I.M., Hauch, J.A.,
Maier, A., Brabec, C.J., 2021. Photoluminescence for defect detection on fullsized photovoltaic modules. IEEE J. Photovolt. (11–6), http://dx.doi.org/10.1109/
JPHOTOV.2021.3099739.
Duval, A., 2019. Explainable artificial intelligence (XAI). http://dx.doi.org/10.13140/
RG.2.2.24722.09929.
Felzenszwalb, P.F., Mcallester, D.A., Ramanan, D., 2008. A discriminatively trained,
multiscale, deformable part model. In: 2008 IEEE Conference on Computer Vision
and Pattern Recognition.
Fioresi, J., Colvin, D.J., Frota, R., Gupta, R., Li, M., Seigneur, H.P., Vyas, S., Oliveira, S.,
Shah, M., Davis, K.O., 2022. Automated defect detection and localization in
photovoltaic cells using semantic segmentation of electroluminescence images. IEEE
J. Photovolt. (12–1), http://dx.doi.org/10.1109/JPHOTOV.2021.3131059.
Girshick, R., 2015. Fast R-CNN. Comput. Sci. http://dx.doi.org/10.1109/ICCV.2015.
169.
Girshick, R., Donahue, J., Darrell, T., Malik, J., 2014. Rich feature hierarchies for
accurate object detection and semantic segmentation. IEEE Comput. Soc. http:
//dx.doi.org/10.1109/CVPR.2014.81.
Gorge, V., Migan-Dubois, A., Djebbour, Z., Pantzas, K., Gautier, S., Moudakir, T.,
Suresh, S., Ougazzaden, A., 2013. Theoretical analysis of the influence of defect
parameters on photovoltaic performances of composition graded InGaN solar cells.
Mater. Sci. Eng. B: Solid-State Mater. Adv. Technol. 178 (2), 142–148. http:
//dx.doi.org/10.1016/j.mseb.2012.10.033.
10
Engineering Applications of Artificial Intelligence 131 (2024) 107866
Y. Cao et al.
Zhang, J., Sclaroff, S., Lin, Z., Shen, X., Mech, R., 2015. Minimum barrier salient object
detection at 80 FPS. In: IEEE International Conference on Computer Vision.
Zhang, X., Sun, H., Zhou, Y., Xi, J., Li, M., 2013. A novel method for surface defect
detection of photovoltaic module based on independent component analysis. Math.
Probl. Eng. 2013 (pt.6), 147–160. http://dx.doi.org/10.1155/2013/520568.
Zhu, W., Wang, Z., Yuan, X., Fan, R., 2015. Security assessment of distribution
system with distributed photovoltaic. J. Power Energy Eng. 03 (04), 152–162.
http://dx.doi.org/10.4236/jpee.2015.34034.
Wang, Y., Li, L., Sun, Y., Xu, J., Akiyama, H., 2021. Adaptive automatic solar cell
defect detection and classification based on absolute electroluminescence imaging.
Energy 229, 120606. http://dx.doi.org/10.1016/j.energy.2021.120606.
Xiaoxia, L., Yang, Q., Lou, Z., Yan, W., 2019. Deep learning based module defect
analysis for large-scale photovoltaic farms. IEEE Trans. Energy Convers.
Zhang, T., 2023. GBH-YOLOv5: ghost convolution with BottleneckCSP and tiny target
prediction head incorporating YOLOv5 for PV panel defect detection. Electronics
12, http://dx.doi.org/10.3390/electronics12030561.
11