Study on Evaluation of Safety and Quality in Military Aircraft Maintenance
Ji Ding 1, Qiong-jian Fan 2, De-xiang Sun 1,Ji-jun Zhang1, Bo Ren 3
1
Training Department of Air Force Aviation University, Changchun, China
Aeronautical Control Engineering Department of Air Force Aviation University , Changchun, China
3
Aeronautical and Mechanical Engineering Department of Air Force Aviation University , Changchun, China
(sdx58964@gmail.com, iott1230@163.com)
2
Abstract - Evaluating safety and quality in military aircraft
maintenance is one of the most important aspects in
ensuring flight safety. Constructing index system of
maintenance safety and quality of grass-root level，Using
the method of set pair analysis (SPA) to evaluate
maintenance levels of several aeronautical maintenance
army and proving accuracy of the model compared with
TOPSIS and gray correlation analysis. Based on component
and network technologies, the assessment system is set up.
Provide basis for controlling maintenance safety and quality
and ensure aircraft support under the condition of
informatization.
Keywords - Military aircraft maintenance, safety,
quality, comprehensive evaluation
I. INTRODUCTION
When referring civilian or military aviation, safety is
always the eternal theme. At present, foreign countries
have formed a variety of aviation safety evaluation
systems which is more mature, such as the International
Operational Safety Audit (IOSA)[1] in International Air
Transport Association (IATA), Air Transport Monitoring
System (the ATOS)[2] in the U.S. Federal Aviation
Administration (FAA); As to domestic aviation safety
evaluation, the use of fuzzy comprehensive evaluation
method[3], multi-attribute decision making method[4],
bayesian network evaluation method[5] have been widely
used. Although aviation safety assessment has made a lot
of results, but it can be seen from the type of the results,
the decrease of risk caused by military aircraft
maintenance activities is not given enough attention as the
reduce of risk posed by the flight operations. At the same
time, the research has not formed as a unified safety and
quality comprehensive evaluation system on the safety
and quality of grass-roots-oriented military aircraft
maintenance, the achievement is limited and the related
safety and quality indicators remains mostly in isolated
and gap analysis. Therefore, it is necessary to establish a
quantified
grassroots-oriented
military
aircraft
maintenance safety and quality evaluation system.
Drawing on the useful experience of the foreign military,
using the Set Pair Analysis methods (SPA), this paper aim
to establishing and improving the maintenance of safety
and quality evaluation system, the significance to improve
the level of maintenance management and combat
effectiveness is obvious.
II. THE BASIC CONCEPT OF SET PAIR ANALYSIS
Set pair analysis[6](SPA) is a system analysis method
which is proposed by Keqing ZHAO, Chinese famous
scholar .SPA believe that uncertainly is the essential
nature of things, the certainly and uncertainly as a
comprehensive inspection system, the system of certain is
divided into the “same” and “opposition”, the uncertain is
defined as “different”. The “three” are interrelated
constraints, and under certain condition into each other.
The core concept of Set pair analysis is contract degree.
Set pair is associated with a certain set consisting of two
pairs, connection degree is a characteristic set of links in
the quantitative characterization, the expression is
  a  bi  cj . Where i is the different coefficient value
in the range of [-1,1]; j denotes the opposing coefficient
who’s value is set as -1; a, b and c denote the same
degree, different degree and opposing degree respectively.
A and c is certain relatively while b is not. We can know
from the definition that a  b  c  1.
III. THE MULTIPLE INDEX EVALUATE MODEL
A. Establish the evaluation index system and index
calculation criteria
Aircraft Maintenance Management of the U.S. air
force ordinance define eight index to judge the quality
of maintenance work, they are major failure rate of the
aircraft equipment, ground accident rate, the rate of
qualified personnel, aircraft status report accuracy,
plan efficiency, flight emergency rate, aircraft mission
support rate, staff compliance rate. Aeronautical
Engineering Protection Ordinance proposed by the
Russian air force in 1991, provides a unified index
system for the assessment of aviation equipment and
maintenance of equipment status and technical quality,
contains 279 indicators totally[7]. At present, the air
corps quality control room uses ten tips of index to
make judgment of the quality and safety of the aircraft
maintenance work. In order to meet the needs of
aviation equipment informatization development and
reform of the aviation maintenance, on the basis of
foreign military useful experience, we can consider the
use of maintenance safety index (SI) and service
quality index (QI). The gass-roots military aircraft
maintenance safety quality evaluation index
architecture is shown in table 1. The index calculation
type can be divided into three categories. The index of
I can be calculated by the data of the statistics from
the quality control room of air force units. The intact
rate of aircraft, success rate of the support task and the
time of direct maintenance belong to this class. What
should be noted is that all of the index quantity value
needed to be divided by the designed value or the
mean of same type of aircraft. The index of II is the
data that our quality control system doesn’t record,
and we should use special method to collect the
information. III index is a quality assessment score
index, such as the personnel pass rate. The value of the
index is the test result, and quality evaluation is a
combination of theory written and practical test.
B .Determine the links between the comparison space and
index [8-10]
Now the military aircraft maintenance safety and
quality assessment is denoted by G=(S,E,W,X), S={s1,s2,
… ,sm} denotes a set of the maintenance forces to be
assessed, Sr denotes the rth maintenance force; and
E={e1,e2,…,en} denotes the evaluation index set, er is the
rth index.
Type of Index
index weights
Maintenance Maintenance Maintenance Maintenance
force A
force B
force C
force D
Aircraft in good order
Ⅰ
0.025
0.90
0.75
0.80
0.85
Success rate of the support task
Ⅰ
0.022
0.94
1.15
1.20
0.90
Hourly direct maintenance time
Ⅰ
0.014
0.85
1.05
1.10
0.92
Ⅰ
0.013
1.16
0.87
0.94
1.00
Mean preventive maintenance
time
Frequency of replace engine
Ⅰ
0.010
0.86
1.04
1.06
0.96
Ⅱ
0.021
7
7
8
9
Mean time to repair
Check time of per-machine
Ⅱ
0.020
15
16
17
18
Re-dispatch time
4 of 4 aircrafts
in the same condition
Ⅱ
0.008
28
28
29
27
Maintenance flight rest rate
Ⅰ
Maintenance error perthousand rate
Flight error per-thousand rate
Flight accident portent per-ten
thousand rate
Serious flight accident per-ten
thousand rate
Three-grade accident per-ten
thousand rate
Serious ground accident per-ten
thousand rate
Common ground accident perten thousand rate
Mean flight time between
accident
Flight frequency to maintenance
day rate
0.005
0.60
0.45
0.55
Ⅰ
0.015
0.70
0.68
0.65
0.55
Ⅰ
0.120
0.65
0.65
0.55
0.50
Ⅰ
0.130
0.35
0.30
0.20
0.10
Ⅰ
0.150
0
0
0
0
Ⅰ
0.140
0
0
0
0
Ⅰ
0.160
0
0
0
0
Ⅰ
0.110
0.25
0.23
0.22
0.20
Ⅰ
0.002
0.72
0.65
0.63
0.63
Ⅰ
0.012
1.40
1.45
1.50
1.50
Ⅲ
0.008
95
95
96
98
Maintenance plan efficiency rate
Ⅱ
0.004
40
42
45
44
Accident spare exchange rate
Ⅰ
0.011
0.90
0.70
0.60
0.80
W={w1,w2,…,wn} means the index weight set, wr is
the
weight
of
index
r,
while
n
r
xkr
, 1
 [0,1] , they shows
1
u r  vr u r  vr
the approach degree of data xkr and ur, xkr and vr. The data
larger, the approach degree bigger, therefore the similarity
level, contrary level and the difference level are defined
as following:
complete rate meets
akr 
xkr
(1)
ur  vr
ckr 
1
xkr
 1, wr  0 ; x=(xkr)m×n indicates the
i
decision matrix, xkr shows the await maintenance troop
sk’s quantification value about the index er. The
maintenance security quality optimal evaluation set
U={u1,u2,…,un} and the worst evaluation set
V={v1,v2,…,vn} are determined by each index weight
1
1
u r  vr
(2)
(ur  xkr )( xkr  vr )
(3)
(ur  vr ) xkr
So, to the benefit index, set pair connection level is:
{xkr , ur } 
xkr
u r  vr

1
(ur  xkr )( xkr  vr )
x
i  1 kr 1 j （4）
(ur  vr ) xkr
u r  vr
Note: Index weight is calculated via AHP and expert survey.
As the same reason, to the cost index such as
maintenance responsibility aircraft error thousand rate, set
pair connection level is:
{xkr , ur } 
0.56
Personnel pass rate
w
1
xkr
bkr  1  akr  ckr 
TABLE I
INDEX WEIGHT AND VALUE
Index of the maintenance
quality evaluation
value, among them, ur, vr mean the maximum and
minimum value separately. The comparison space [V，U]
consists of the optimal and worst evaluation set.
While the benefit index such as the aircraft
1
xkr
1
1
ur  vr

(ur  xkr )( xkr  vr )
x
i  kr j （5）
(ur  vr ) xkr
ur  vr
The similarity and contrary rate of equation (4) and
(5) show the await troop’s evaluation index tendency of
good or bad. Also, the similarity and contrary rate of two
equations are symmetrical with the same difference level.
In conclusion, the equation is integrated as:
{sk , u}  ak  bk i  ck j （6）
Where ak 
n
n
n
r 1
r 1
r 1
 wr akr , bk   wr bkr , ck   wr ckr .
Since ak、ck are confirmed relatively, they indicate
the await maintenance troop sk’s optimal and worst level
approaching to maintenance security. Under relative
certain conditions, the relative approach level of await
maintenance troop sk (k=1,2,…,m) and optimal evaluation
set U is defined as:
rk 
ck
（7）
ak  ck
When the approach level ck is bigger, the await troop
sk get closer to the optimal evaluation set U, so according
to rk, await troop aircraft maintenance security quality can
be evaluated.
C. Example analysis
Using above evaluation method, the paper evaluate
four await troops. Each evaluation index value is showed
in table 1. The weights are confirmed in table 1, and the
integrated connection level and relative approach level are
calculated via equation (4)-(7), shown in table 2.
Meanwhile, in order to improve the reliability and
veracity of proposed method, gray relation evaluation
method and the TOPSIS evaluation method are used to
examine the results, shown in table 3. From table 2 and 3,
it is known that the results three methods are no
difference, they are: C>B>A>D. In summary, the set pair
analysis method’s application to aircraft maintenance
security quality evaluation is feasible and accurate.
TABLE II
THE SPA EVALUATION RESULTS OF AWAIT TROOPS.
Await maintenance troops
Integration
level ak
Diversity bk
Contradiction
level ck
Relative close
level rk
Rank
B
C
D
0.227
0.244
0.273
0.216
0.445
0.408
0.467
0.456
0.328
0.348
0.260
0.328
0.409
0.412
0.512
0.397
3
2
1
4
TABLE III THE GRAY RELATION AND TOPSIS
EVALUATION RESULTS OF AWAIT REOOPS.
Await troops Gray relation
Rank
TOPSIS
Webpage
request
Client browser
Webpage
retrieval
Web
browser
Client browser
Date
request
ODBC
Retrieval
data
Data
base
server
Fig. 1: Browser-Web server-data base server three-layer
In the evaluationframework.
process, the interactive of the
personnel and process are achieved by ASP dynamic
webpage and ActiveX control component. When other
systems are integrated, they send requests to the
evaluation server, and invoke the ports of server
evaluation components to put the evaluation into effect.
V. CONCLUSION
Parameters
A
Client browser
Rank
A
0.864
3
0.941
3
B
0.906
2
0.947
2
C
0.942
1
0.958
1
D
0.746
4
0.932
4
IV. ACHIEVMENT OF DYNAMIC DISTRIBUTED
APPLICATION SYSTEM
Given that the aircrafts of different troops are located
in diverse areas, when the military plane maintenance
security evaluation system is exploited, integrated,
layered and distributed system is required to ensure that
the system is compact and the components could be
transplanted, extended and operated. Exploited method
based
on
components
makes
it
possible.
CSCW(Computer Support Cooperation Workgroup) is a
set of software based on distributed computer system, its
goal is to design application system supporting all kinds
of synergy works, and improve people’s ability of solving
problems. Under B/S framework, client screen is realized
via WWW browser, the proposed browser—Web
server—data base server framework is shown as figure 1.
IE browser is installed by clients; and Microsoft IIS is
installed by Web browser, the ASP is used as the browser
script language, the interactive marking component and
ActiveX control component are installed to provided
download; Windows NT is installed in evaluation server,
the COM components are deployed and managed by NT’s
MTS; the system data base and maintenance module
procedure are installed in data base.
The paper begins with military aircraft maintenance
security evaluation index system, the set pair analysis
method is used, based on original data connection level,
the troop maintenance security quality is evaluated, and
the maintenance security quality evaluation system based
on component and network technology is discussed, it is
beneficial for the equipment department to master the
maintenance quality condition, and provide support for
information-based air battle.
REFERENCES
[1] IATA.IOSA Standards Manual[EB/OL]
.http://www.iata.org/NR/rdonlyres/E3C8A673-8C50-4C32899B-3EE54BFC4568/0/ISM_2nd_Edition.zip
[2] Guo-yan Chen. Aviation transportation monitor system
research. Civil Aviation University of China, 2008, pp. 512.
[3] Peng-yu Zhang, Yong-gang Wang. Multilevel fuzzy
integrate assessment of aviation safety centered on human
factors. Journal of Safety and Environment, 2006, Vol. 6,
pp. 146-147.
[4] Liang Li, Jian-bo Hu, Xiao-kui Sui, Peng Gao. Multiattribute Aviation Safety Risk Assessment Based on Fuzzy
Theory. Control Engineering of China, 2007,Vol.14, pp.
33-34.
[5] Song-bin Ding, Yi Ru. Airlines Flight Safety System
Assessment Method Based on Bayesian Networks. Journal
of Civil Aviation University of China, 2009, Vol. 27, No.3,
pp. 30-36.
[6] Ke-qin Zhao. Set Pair Analysis and its Preliminary
Application. Science and Technology Press, 2000.
[7] Feng-ming Zhang, Dong-liang Zheng, Zhen-zhong Lv.
Aviation Equipment Science Maintenance Introduction.
National Industry Press, 2006.
[8] Xiao Chen, Yi Yang, Yi-ming Bi. Study of Effectiveness
Evaluation of Maintenance Support System based on Set
Pair Analysis. Fire Control and Command Control, 2009,
Vol. 34, No. 1, pp. 75-78.
[9] Bao-lei Chen, Zhi-wei Shan, Shou-hua Chen, Yao-hui
Peng. Optimization Model of Equipment Support Scheme
in Wartime Based on Integration and Contrast Analysis.
Ordnance Industry Automation, 2009, Vol. 28, No.1, pp.
41-44.
[10] Hui-zhi Cao, Ya-peng Wang, Yuan-fei Xue, Yao-jun
Wang. Support Benefits Evaluation about Military
Vehicles’ Repair Socialization Based on Set Pair Analysis.
Journal of Ordnance Engineering College, 2008, Vol. 20,
No.2, pp. 9-12.