sensors Article An Adaptive Multi-Sensor Data Fusion Method Based on Deep Convolutional Neural Networks for Fault Diagnosis of Planetary Gearbox Luyang Jing 1 , Taiyong Wang 1, *, Ming Zhao 2 and Peng Wang 1 1 2 * School of Mechanical Engineering, Tianjin University, Tianjin 300354, China; jingl[email protected] (L.J.); [email protected] (P.W.) School of Mechanical Engineering, Xi’an Jiaotong University, Xi’an 710049, China; [email protected] Correspondence: [email protected]; Tel.: +86-22-8740-2033 Academic Editor: Vittorio M. N. Passaro Received: 30 December 2016; Accepted: 16 February 2017; Published: 21 February 2017 Abstract: A fault diagnosis approach based on multi-sensor data fusion is a promising tool to deal with complicated damage detection problems of mechanical systems. Nevertheless, this approach suffers from two challenges, which are (1) the feature extraction from various types of sensory data and (2) the selection of a suitable fusion level. It is usually difficult to choose an optimal feature or fusion level for a specific fault diagnosis task, and extensive domain expertise and human labor are also highly required during these selections. To address these two challenges, we propose an adaptive multi-sensor data fusion method based on deep convolutional neural networks (DCNN) for fault diagnosis. The proposed method can learn features from raw data and optimize a combination of different fusion levels adaptively to satisfy the requirements of any fault diagnosis task. The proposed method is tested through a planetary gearbox test rig. Handcraft features, manual-selected fusion levels, single sensory data, and two traditional intelligent models, back-propagation neural networks (BPNN) and a support vector machine (SVM), are used as comparisons in the experiment. The results demonstrate that the proposed method is able to detect the conditions of the planetary gearbox effectively with the best diagnosis accuracy among all comparative methods in the experiment. Keywords: fault diagnosis; multi-sensor data fusion; deep convolutional neural networks; feature learning; planetary gearbox 1. Introduction The planetary gearbox is a key component in mechanical transmission systems and has been widely used in wind turbines, helicopters and other heavy machineries . The wide range of gear ratios, small room in power transmission line, and high transmission efficiency are the most significant advantages of planetary gearboxes . Planetary gearboxes generally operate in tough working environments, which makes them suffer from a variety of failures and damages [1,3] causing unwanted downtime, economic losses, and even human casualties . Therefore, the fault diagnosis of planetary gearboxes is necessary to guarantee a safe and efficient operation of mechanical transmission systems. Health conditions of planetary gearboxes can be reflected by various kinds of measurements, including vibration signal, acoustic signal, driven motor current, shaft speed, oil debris, etc. Different measurements have different drawbacks and are sensitive to different types of damage modes or operation conditions. Thus, combining and analyzing these measurements together should be an appropriate approach to detect various types of faults of complex systems. This approach, Sensors 2017, 17, 414; doi:10.3390/s17020414 www.mdpi.com/journal/sensors Sensors 2017, 17, 414 2 of 15 named multi-sensor data fusion, can achieve more accurate and reliable result than approaches with a single sensor [5–7]. Nevertheless, multi-sensor data fusion for fault diagnosis suffers from two challenging problems . (1) One is the feature extraction of multi-sensory data. Generally, conventional fault diagnosis includes three steps : signal acquisition, feature extraction, and fault classification. In the feature extraction step, fault sensitive features are extracted and selected from raw data through signal processing technologies and data analysis strategies, such as Fourier spectral analysis, wavelet transformation (WT), and principal component analysis (PCA). However, the multiple types of sensory data of multi-sensor data fusion may cause a number of issues that make the feature extraction with multi-sensor much more difficult than with a single sensor. These issues [6,7] include more imprecision and uncertainties in measurements, various noises, more conflicting or correlating data, higher data dimensions, etc. Extracting features from these kinds of data will be a very challenging and cruel task. At the same time, multiple types of sensory data also increases the difficulties and consumes much time to choose an optimal handcraft feature or manual feature extraction method, even though to date, the optimal handcraft feature or feature extraction method for a specific type of sensory data still remains unanswered [8,9]; (2) The other challenge of the multi-sensor data fusion is the selection of different fusion levels. Similarly, different fusion levels have their own advantages and disadvantages, and the suitable one for different fault diagnosis tasks is usually different . Selecting an optimal fusion level for a specific fault diagnosis task always requires domain expertise, prior knowledge, and human labor. Deep neural networks (DNN), also known as deep learning, have been attracting increasing attention from researchers from various fields in recent years [10–12]. The key advantage of DNN is the feature learning ability , which can automatically discover an intricate structure and learn useful features from raw data layer by layer. A number of studies [11–13] have shown that DNN can fuse input raw data and extract basic information from it in its lower layers, fuse the basic information into higher representation information and decisions in its middle layers, and further fuse these decisions and information in its higher layers to form the final classification result. It can be seen that DNN itself is a fusion structure [11,13], which fuses the feature extraction, feature selection, data-level fusion, feature-level fusion, decision-level fusion, and classification into one single learning body. For this reason, a DNN-based low level fusion, e.g., data-level fusion, can not only learn features from fused raw data automatically, but also fuse these data, features and decisions adaptively through its deep-layered structure. DNN might be an ideal model to fuse multi-sensory data and detect faults of a mechanical system. However, although some applications [14–18] of DNN in feature learning and fault diagnosis with a single sensor have been found in recent years, no study, to the best of our knowledge, has investigated the effectiveness of DNN-based feature learning and the adaptive level fusion method for fault diagnosis. It is attractive and meaningful to investigate this adaptive fusion method, which can learn features from raw data automatically, and select and combine fusion levels adaptively. Deep convolutional neural networks (DCNNs), as one of the main types of DNN models, have been successfully used in mechanical fault diagnosis with automatic feature learning from single sensory data [9,14,19,20]. Benefitting from several unique structures, DCNN can achieve better results with less training time than standard neural networks. Firstly, DCNN has a large set of filter kernels in convolutional layers, which can capture representative information and patterns from raw data. Stacking these convolutional layers can further fuse information and build complex patterns; Secondly, DCNN is an unfully-connected network, where each filter shares the same weights. This structure can reduce both the training time and complication of the model. In addition, the pooling layer of DCNN further reduces the revolution of the input data as well as the training time, and improves the robustness of the extracted patterns (a detailed introduction to the DCNN model is presented in Section 2). Thus, DCNN should have great potential in processing the multi-sensory data of Sensors 2017, 17, 414 3 of 15 a mechanical system, which usually contains rich information in the raw data and is sensitive to training time as well as model size. Aiming to address the two problems of multi-sensor data fusion mentioned above, this paper proposes Sensors 2017, 17, 414 3 of 15 an adaptive data fusion method based on DCNN and applies it to detect the health conditions of a planetary Aiming from to address the two problems of the multi-sensor data fusion is mentioned above, thisfeatures paper from gearbox. Different conventional methods, proposed method able to (1) extract proposes an adaptive data fusion method based on DCNN and applies it to detect the health raw data automatically and (2) optimize a combination of different fusion levels adaptively for any specific conditions of a planetary gearbox. Different from conventional methods, the proposed method is fault diagnosis task with less dependence on expert knowledge or human labor. able to (1) extract features from raw data automatically and (2) optimize a combination of different Thefusion rest of the paper is organized as follows. In Section 2, the typical architecture of DCNN and levels adaptively for any specific fault diagnosis task with less dependence on expert an adaptive training method are briefly described. Section 3 illustrates the procedures of the proposed knowledge or human labor. method, theThe design of the DCNN model, and several comparative introduced to further rest of the paper is organized as follows. In Section 2, the typicalmethods architecture of DCNN and an adaptive training method are briefly described. Section 3 illustrates the procedures of the analyze the performance of the proposed method. In Section 4, an experimental system of a planetary method, thevalidate design ofthe the effectiveness DCNN model, and several comparative methods introduced to gearboxproposed test rig is used to of the proposed method. Finally, the conclusions further analyze the performance of the proposed method. In Section 4, an experimental system of a are drawn in Section 5. planetary gearbox test rig is used to validate the effectiveness of the proposed method. Finally, the conclusions are drawn in Section 5. 2. Deep Convolutional Neural Networks 2. Deep Convolutional Neural Networks 2.1. Architecture of Deep Convolutional Neural Networks 2.1. Architecture of Deep Convolutional Neural Networks DCNN is a type of DNN model inspired by visual system structure [11,21], and it has is a type of DNNfor model inspired by visual system [11,21], andinitimage has become become theDCNN dominant approach almost all recognition andstructure detection tasks and speech the dominant approach for almost all recognition and detection tasks in image and speech analysis analysis [22–24]. DCNN contains three kinds of layers , which are the convolutional layer, [22–24]. DCNN contains three kinds of layers , which are the convolutional layer, pooling layer, pooling layer, and fully-connected layer. As shown in Figure 1, the first several layers of a typical and fully-connected layer. As shown in Figure 1, the first several layers of a typical DCNN usually DCNN consist usuallyofconsist of a combination of two types of layers—convolutional followed by a combination of two types of layers—convolutional layers, followedlayers, by pooling poolinglayers—and layers—and is a fully-connected In the following part, we will thethe last last layerlayer is a fully-connected layer. Inlayer. the following part, we will describe thesedescribe threekinds kinds of layers detail. these three layersininmore more detail. Convolutional and pooling layer Input data Convolutional and pooling layer subsampling Convolutional filters Fully connected layer subsampling Feature maps Feature maps Softmax regression subsampling subsampling Convolution Convolution Feature maps Fully connect Feature maps subsampling subsampling Feature maps Feature maps Figure 1. A typical architecture of deep convolutional neural network (DCNN). Figure 1. A typical architecture of deep convolutional neural network (DCNN). The convolutional layer is composed of a number of two-dimensional (2D) filters with parameters. filters convolute with input data and obtain an output, as Theweighted convolutional layerThese is composed of a number of two-dimensional (2D) filtersnamed with weighted feature maps. Each filter shares the same weighted parameters for all the patches of the input data to maps. parameters. These filters convolute with input data and obtain an output, named as feature reduce the training time and complication of the model, which is different from a traditional neural Each filter shares the same weighted parameters for all the patches of the input data to reduce the network with different weighted parameters for different patches of the input data. Suppose the traininginput timeofand complication of the model, which is different from a traditional neural network the convolutional layer is , which belongs to × , and A and B are the dimensions of the with different weighted different patches the inputasdata. Suppose the input of the input data. Then theparameters output of thefor convolutional layer can of be calculated follows : A × B convolutional layer is X, which belongs to R , and A and B are the dimensions of the input data. −1 follows : Then the output of the convolutional layer be calculated as (1) = can (∑ ∗ + ) =1 CC Ccn = f ∑ cc=1 ! l −1 Xcc l ∗ Wcn + l Bcn (1) Sensors 2017, 17, 414 4 of 15 where Ccn is the cn-th output of the convolutional layer, and the output number is CN, which is also equal to the filter number; ∗ is an operator of convolution; Xcc represents the input data of cc-th channel l is the weight of cn-th filter of the current of previous layer l − 1, and the channel number is CC; Wcn layer l; the width and height of the filter are CW and CH, respectively; the cn-th bias is denoted with l ; f is an activation function, typically hyperbolic tangent or sigmoid function. bcn The pooling layer is a sub-sampling layer, which reduces the revolution of the input data and improves the robustness of learned features. A pooling layer generally follows a convolutional layer with a max pooling method and it outputs only the maximum of each sub-sampling patch of the feature maps to subsample the feature maps from the previous convolutional layer. The output can be described as follows : Pcn = max Ccn (2) Ccn ∈S where Pcn is the cn-th output of the pooling layer, and the output number is CN; S is the pooling block size. This function will sum over each distinct S pooling block in the input data so that the output will become S times smaller along both spatial dimensions. The fully-connected layer is the last layer of the DCNN model. It follows several combinations of the convolutional layers and the pooling layers, and classifies the higher-level information from the previous layers. A fully-connected layer is similar to a traditional multilayer neural network with a hidden layer and a classification layer, typically using a softmax regression. Assuming that the task is a K-label problem, the output of the softmax regression can be calculated as follows: Oj = P(y = 1| x; θ ) P(y = 2| x; θ ) ... P(y = k | x; θ ) 1 = K ∑ j=1 exp θ ( j) x exp θ (1) x exp θ (2) x . . . exp θ (K ) x (3) where θ (1) , θ (2) , . . . θ (K ) are the parameters of the model, and O j is the final result of the DCNN. 2.2. Training Method l , and θ ( j) are the learnable parameters It can be seen from the previous description that wil , bcn and will be optimized through model training with gradient decent algorithms. Since a gradient decent algorithm is easily trapped into local optima, we introduce several enhancement methods, including stochastic gradient decent (SGD), cross-validation, and adaptive learning rate, to solve this problem. SGD updates gradient  based on a few training data instead of the entire training set. This approach not only increases the training speed, but also improves the training reliability. Cross-validation selects a validation set from training data to test the performance of the parameters of the model to avoid overfitting. Since a global constant learning rate easily causes either a slow convergence with a lower learning rate or a serious fluctuation of the convergence with a higher learning rate, an adaptive learning rate is employed. The adaptive learning rate has a high rate at first and decreases with the increase of the training epochs adaptively to obtain a fast and reliable convergence result. 3. Adaptive Multi-Sensor Data Fusion Method Based on DCNN for Fault Diagnosis 3.1. Procedure of the Proposed Method An adaptive multi-sensor data fusion method based on DCNN is presented to learn features from raw data automatically and combine fusion levels adaptively to detect faults of a planetary gearbox. Through its deep-layered structure, DCNN can fuse input data and extract basic features in the lower layers, fuse basic features into high level features and decisions in the middle layers, and further fuse these features and decisions in the higher layers to obtain the final diagnosis result. Sensors 2017, 17, 414 5 of 15 Figure 2 displays the flowchart of the proposed method: (1) four types of signals, including5vibration Sensors 2017, 17, 414 of 15 signal [3,28,29], acoustic signal [4,30], current signal [31–33], and instantaneous angular speed (IAS) (IAS)are signal [34,35],according are selected to published studies and acquired a planetary signalspeed [34,35], selected toaccording published studies and acquired from afrom planetary gearbox; gearbox; (2) data preprocessing applied to standardize each signal andthem divide them into segments; (2) data preprocessing is applied toisstandardize each signal and divide into segments; (3) each of (3) each of theof four of the fourare signal types are combined together simply one data the four segments thesegments four signal types combined together simply as one data as sample to form sample to form the data-level fused input data of the DCNN model; (4) DCNN is trained and tested the data-level fused input data of the DCNN model; (4) DCNN is trained and tested with these fused with these fused input data, and its output will be the diagnosis result of the planetary gearbox. The input data, and its output will be the diagnosis result of the planetary gearbox. The testing accuracy of testing accuracy of the output result is used to evaluate the effectiveness of the proposed method. It the output result is used to evaluate the effectiveness of the proposed method. It should be noted that should be noted that although we use a data-level fusion in the third step, data is fused again in the although we layers use a of data-level fusion in to thefurther third step, datathe is fused again inThe the DCNN startingimplicitly layers of the starting the DCNN model optimize data structure. DCNN model to further optimize the data structure. The DCNN implicitly contains data-level fusion, contains data-level fusion, feature-level fusion, and decision-level fusion through the deep-layered feature-level decision-level fusionofthrough the deep-layered structure and ittooptimizes structure fusion, and it and optimizes a combination these fusion levels adaptively according the characteristic of the fusion data itself. a combination of these levels adaptively according to the characteristic of the data itself. Multi-sensor Accelerometer Data preprocessing Data level fusion Deep convolutional neural networks based feature learning and data fusion Divide signals into segments Combine four segments into one data sample Input data samples Convolutional and pooling layer Diagnosis result Fully connected layer Microphone Current sensor Optical encoder Figure 2. Flowchart of the proposed method. Figure 2. Flowchart of the proposed method. 3.2. Model Design of DCNN 3.2. Model Design of DCNN The model of DCNN is adjusted to satisfy the characteristics of mechanical fault diagnosis. The model DCNN isof DCNN adjusted to satisfy the characteristics of mechanical Although mostofapplications in image recognition chose a 2D convolutional structure fault [11,22,36], and some researchers [20,37] of also used the way to diagnosechose mechanical faults, we diagnosis. Although most applications DCNN in same image recognition a 2D convolutional select[11,22,36], a one-dimensional (1D) convolutional structure with the a 1Dsame filter way bank to as diagnose the kernelmechanical of the structure and some researchers [20,37] also used DCNN model. In our opinion, the main reason behind the applications of the 2D convolutional faults, we select a one-dimensional (1D) convolutional structure with a 1D filter bank as the kernel of structure of DCNN in image analysis lies in the natural 2D space correlation in images. However, the DCNN model. In our opinion, the main reason behind the applications of the 2D convolutional most of the measurement data for mechanical fault diagnosis only correlate with time, which is a 1D structure of DCNN in image analysis lies in the natural 2D space correlation in images. However, parameter. Thus, 1D convolutional structure should be an appropriate choice for a DCNN-based most fault of the measurement data for mechanical fault diagnosis only correlate with time, which is a 1D diagnosis problem. In addition, we choose a larger filter size than conventional ones used in parameter. Thus, 1D convolutional be be anmore appropriate choice for of a DCNN-based image recognition. While a largerstructure size of theshould filter may expensive in terms computation, a fault diagnosis In addition, we choosebetween a largerthe filter thanaway conventional ones, used in image largerproblem. filter can capture more information datasize farther from each other which recognition. While a larger of the filter may be more expensive in terms of computation, a larger may be the features in thesize frequency domain. In spite of the information many benefitsbetween of the deep-layered structure DCNN, a “deep” structure filter can capture more the data farther awayoffrom each other , whichalso may be means complicated hyper-parameters as well as various choices of architectures, which increases the the features in the frequency domain. difficulty build an benefits appropriate anddeep-layered efficient model. Although there areaseveral In spite oftothe many of the structure of DCNN, “deep”researches structure [39,40] also means about the automatic optimization of parameters of DCNN, the optimized process is usually complicated hyper-parameters as well as various choices of architectures, which increases the difficulty time-consuming and easily converges into a local optimum due to the large number of parameters of to build an appropriate and efficient model. Although there are several researches [39,40] about the DCNN. Thus, we build the DCNN model initially based on a few general design principles [38,41]. automatic optimization of parameters of DCNN, the optimized process is usually time-consuming and Then several configurations of the network are tested using the experimental data, and the one with easilybest converges into aislocal optimum due toofthe performance selected as the setting thelarge finalnumber model. of parameters of DCNN. Thus, we build the DCNN model initially based on a few general design principles [38,41]. Then several configurations Sensors 2017, 17, 414 6 of 15 Sensors 17, 414are tested using the experimental data, and the one with best performance is selected 6 of 15 of the 2017, network as the setting of the final model. 3.3. Comparative Methods 3.3. Comparative Methods methods are employed to further test and confirm the performance of the Several comparative proposed method. The flowcharts comparative methods in Figure Several comparative methodsofare employed to furtherare testshown and confirm the3.performance of the To evaluate ability of learning features from the raw of the proposed proposed method.the The flowcharts of comparative methods aredata shown in Figure 3. method, manual feature extraction is used as a replacement and comparison of the feature learning method, in each To evaluate the ability of learning features from the raw data of the proposed comparative method. Eight time-domain featuresand andcomparison several frequency-domain features are manual feature extraction is used as a replacement of the feature learning in each extracted. Root mean square (RMS), kurtosis, crest factor, skewness, mean, minimum, maximum, comparative method. Eight time-domain features and several frequency-domain features are extracted. and variance are chosen the handcraft features in the time [34,37,42]. The characteristic Root mean square (RMS),as kurtosis, crest factor, skewness, mean,domain minimum, maximum, and variance frequencies ofthe the handcraft planetary features gearbox , including the rotating frequencies of the sun gear, planetary are chosen as in the time domain [34,37,42]. The characteristic frequencies of gear and the carrier, the pass frequency of the planetary gear and the meshing frequency of the the planetary gearbox , including the rotating frequencies of the sun gear, planetary gear and planetary are selected the handcraft features in the frequency domain well as their ten carrier, thegearbox, pass frequency of theasplanetary gear and the meshing frequency of the as planetary gearbox, sidebands of thefeatures sensor signals [34,43,44] except signal. For the current are selectedfor asall thetypes handcraft in the frequency domainfor asthe wellcurrent as their ten sidebands for all signal,ofthe frequency its sidebands generated the modulation of thesignal, characteristic types theline sensor signalsand [34,43,44] except  for the current by signal. For the current the line frequenciesand of its thesidebands planetary gearbox are chosen its frequency-domain features.frequencies In addition, frequency generated by theas modulation of the characteristic of the fast Fourier-transform (FFT) energy of each sample, which is obtained by splitting the frequency planetary gearbox are chosen as its frequency-domain features. In addition, the fast Fourier-transform spectrum of each sample into 32 average bands and calculating the RMS of eachofband is also (FFT) energy of each sample, which is obtained by splitting the frequency spectrum each , sample into added intobands the handcraft features in the domain. features 32 average and calculating the RMS of frequency each band , is alsoWhile addedall intothe thedomain handcraft featuresare in processed together as the “handcraft features”, the “time-domain features” and “frequency-domain the frequency domain. While all the domain features are processed together as the “handcraft features”, features” are also tested respectively to reflect more features” information the sensitivity of the data. the “time-domain features” and “frequency-domain are about also tested respectively to reflect The comparison learningof features from data andbetween the handcraft features is marked more informationbetween about thethe sensitivity the data. Theraw comparison the learning features from in orange in Figure 3. raw data and the handcraft features is marked in orange in Figure 3. (a) Multi-sensor Data preprocessing (b) Feature learning Data preprocessing (c) Raw data Handcraft features DCNN, BPNN or SVM Results with manual feature extraction and data level fusion DCNN, BPNN or SVM Results with feature learning and feature level fusion Data preprocessing (d) Feature level fusion Feature level fusion DCNN, BPNN or SVM Results with manual feature extraction and feature level fusion Raw data DCNN, BPNN or SVM Decision level fusion Results with feature learning and decision level fusion Handcraft features DCNN, BPNN or SVM Decision level fusion Results with manual feature extraction and decision level fusion Raw data DCNN, BPNN or SVM Results with feature learning and single sensor data Handcraft features DCNN, BPNN or SVM Results with manual feature extraction and single sensor data Feature learning or Feature extraction Feature learning Data preprocessing DCNN or BPNN Handcraft features Feature learning Feature extraction Single sensor Results with feature learning and data level fusion Feature learning or Feature extraction Feature extraction Multi-sensor DCNN, BPNN or SVM Feature learning or Feature extraction Data level fusion Feature extraction Multi-sensor Raw data Feature learning Feature learning or Feature extraction Feature extraction Figure 3. Flowcharts of comparative methods: (a) Data-level fusion methods; (b) Feature-level fusion with single single sensory sensory data. data. methods; (c) Decision-level fusion methods; and (d) Methods with As a comparison of the DCNN model of the proposed method, two intelligent models, back-propagation neural networks (BPNN) and support vector machine (SVM), are introduced as replacements of DCNN in each comparative method. BPNN is built into a three-layer model with Sensors 2017, 17, 414 7 of 15 Sensorsactivation 2017, 17, 414 functions. SVM uses Gaussian radial basis function (RBF) as the kernel7 function of 15 sigmoid and the grid search method to optimize the parameters of the kernel. The three comparative models, DCNN, BPNN, and SVM, are marked in green 3. As a comparison of the DCNN model in ofFigure the proposed method, two intelligent models, For testing the performance of the different fusion levels, feature-level fusion back-propagation neural networks (BPNN) and support vector manual-selected machine (SVM), are introduced as and replacements decision-level explored and compared with is the data-level fusion ofmodel the proposed of fusion DCNN are in each comparative method. BPNN built into a three-layer with sigmoid functions. uses Gaussian radialfrom basis raw function (RBF) the kernel function method. Foractivation feature-level fusionSVM with feature learning data, onlyasDCNN and BPNN are and the method to optimize parameters of the kernel. The three comparative models, applied to grid learnsearch features from the rawthedata of each sensor, respectively. The outputs of the DCNN, BPNN, and SVM, are marked in green in Figure 3. second-last layers of DCNN are extracted as the learned features of each sensory data. Then, all the For testing the performance of the different fusion levels, manual-selected feature-level fusion and learned features from the four types of sensors are combined together as the feature-level fused decision-level fusion are explored and compared with the data-level fusion of the proposed method. features and used as the input of another DCNN for classification. In the same way, the outputs of For feature-level fusion with feature learning from raw data, only DCNN and BPNN are applied to the second-last layers of BPNN are extracted and fused. Then, the fused features are used as the learn features from the raw data of each sensor, respectively. The outputs of the second-last layers of inputDCNN of both andasSVM for classification. Thesensory comparison of different fusion features levels isfrom marked areBPNN extracted the learned features of each data. Then, all the learned in red in Figure 3. the four types of sensors are combined together as the feature-level fused features and used as the As a comparison of theformulti-sensory thethe proposed method, fault diagnosis input of another DCNN classification. input In the data same of way, outputs of the second-last layers ofwith single sensory data is also applied to the evaluate the effectiveness of the proposed method, which is BPNN are extracted and fused. Then, fused features are used as the input of both BPNN and SVM for classification. The comparison of different fusion levels is marked in red in Figure 3. marked in purple and shown in Figure 3d. As a comparison of the multi-sensory input data of the proposed method, fault diagnosis with single sensory is also applied to evaluate the effectiveness of the proposed method, which is 4. Experiment anddata Discussion marked in purple and shown in Figure 3d. 4.1. Experiment Setup 4. Experiment and Discussion An experimental system of a planetary gearbox test rig is established to evaluate the effectiveness of the proposed method. As shown in Figure 4, it consists of a one-stage planetary experimental of a planetary gearbox test rig is established to evaluate the20-tooth effectiveness gearbox, An a driven motor system and a magnetic brake. The planetary gearbox contains one sun gear of the proposed method. As shown in Figure 4, it consists of a one-stage planetary gearbox, a driven and three 31-tooth planetary gears, and transmits torque from the sun gear to the planetary carrier of motor andgears a magnetic The planetary gearbox contains one 20-tooth sun gear and three 31-tooth the planetary with brake. a standstill ring gear. planetary gears, and transmits torque from the sun gear to the planetary carrier of the planetary gears Figure 5 presents the four types of sensors employed in the experiment, including an with a standstill ring gear. accelerometer, microphone, current sensor, and optical encoder. Vibration signal, acoustic signal, Figure 5 presents the four types of sensors employed in the experiment, including an accelerometer, and motor current signal are measured by their corresponding sensors and acquired through a data microphone, current sensor, and optical encoder. Vibration signal, acoustic signal, and motor current acquisition box with a sampling frequency of 20 kHz and data length points. The of the signal are measured by their corresponding sensors and acquired throughofa 320 datakacquisition boxIAS with output shaft is calculated based on counting the number of high resolution pulses of the encoder a sampling frequency of 20 kHz and data length of 320 k points. The IAS of the output shaft is calculated ; based and down-sampling, using of the data acquisition box with the same frequencyusing and data on counting the number high resolution pulses of the encoder ;sampling and down-sampling, theas data box with the same sampling frequency and data length as the other three signals. length theacquisition other three signals. 4.1. Experiment Setup Magnetic brake Motor Planetary gearbox Figure gearboxtest testrig. rig. Figure4.4.Planetary Planetary gearbox Sensors 2017, 17, 414 Sensors 2017, 17, 414 8 of 15 8 of 15 Sensors 2017, 17, 414 Accelerometer Microphone Accelerometer 8 of 15 Microphone Optical encoder Current sensor Optical encoder Current sensor Figure 5. 5. Four and their their installations. installations. Figure Four types types of of sensors sensors and Figure 5. Four types of sensors and their installations. Seven health conditions of the planetary gearbox are tested, including normal, pitting tooth, Seven health conditions of the planetary gearbox are tested, including normal, pitting tooth, Seven health conditions the crack planetary gearbox tested, including normal, pitting tooth, in chaffing tooth, chipped tooth,ofroot tooth, slight are worn tooth, and worn tooth. As shown chaffing tooth, chipped tooth, root crack tooth, slight worn tooth, and worn tooth. As shown in chaffing chipped tooth, in root tooth, gears. slight In worn tooth, and worn tooth. shown in Figure 6, alltooth, the faults occurred thecrack planetary each experiment, only oneAs planetary gear Figure 6, allall the faults in planetary gears. Ineach eachexperiment, experiment,only only one planetary gear Figure the faultsoccurred occurred inthe planetaryand gears. one planetary gear with one 6,kind of health condition isthe examined, allIn the other gears are normal. Experiments are with one kind of health condition is examined, and all the other gears are normal. Experiments are with one kind of health condition is examined, and all the other gears are normal. Experiments are conducted under three motor speeds (600 rpm, 1200 rpm and 1800 rpm) and a load-free condition. conducted under three speeds (600 rpm, 1200 1200rpm rpmand and1800 1800rpm) rpm)and and a load-free condition. conducted threemotor motor (600 rpm, aisload-free condition. The detailed under description for thespeeds datasets and pattern labels of the experiment shown in Table 1. The detailed description labels of ofthe theexperiment experimentisisshown showninin Table The detailed descriptionfor forthe thedatasets datasets and and pattern pattern labels Table 1. 1. (a) (a) (d) (d) (b) (b) (e) (e) (c) (c) (f) (f) Figure tooth; (b) (b) Chaffing Chaffingtooth; tooth;(c)(c)Chipped Chipped tooth; Root Figure6. 6.Faulty Faultyplanetary planetarygears: gears: (a) (a) Pitting Pitting tooth; tooth; (d)(d) Root Figure 6. Faulty planetary gears: (a) Pitting tooth; (b) Chaffing tooth; (c) Chipped tooth; (d) Root crack crack tooth;(e)(e)Slight Slightworn worntooth; tooth;and and (f) (f) Worn tooth. crack tooth; tooth. tooth; (e) Slight worn tooth; and (f) Worn tooth. Table1.1.Description Description of of the pattern pattern labels data. Table labelsof ofplanetary planetarygearbox gearbox data. Table 1. Description of the pattern labels of planetary gearbox data. PatternLabel Label Gearbox Gearbox Condition Condition Input Pattern InputSpeed Speed(rpm) (rpm) Load Load Pattern Label Gearbox Condition Input1200 Speed (rpm) Load 1 Normal 600, and 1800 Zero 1 Normal 600, 1200 and 1800 Zero Normal 600, 1200 and 1800 Pitting tooth 600, and 1800 Zero 22 1 Pitting tooth 600,1200 1200 and 1800 Zero Zero 2 Pitting tooth 600, 1200 and 1800 Zero 3 Chaffing tooth 600, 1200 and 1800 Zero 3 3 Chaffing tooth 600,1200 1200 and Zero Chaffing tooth 600, and 18001800 Zero 4 Chipped tooth 600, 1200and and 1800 Zero Zero Chipped tooth 600, 1800 4 4 Chipped tooth 600,1200 1200 and 1800 Zero 55 Root tooth 600, 1200and and 1800 Zero Zero Rootcracked cracked tooth 600, 1200 1800 5 6 Root cracked tooth 600, 600, 1200 and 1800 Zero Slightworn worn tooth 1800 6 Slight tooth 600, 1200 1200and and 1800 Zero Zero 6 7 Slight worn tooth 600,1200 1200 and Zero Worn tooth 600, and 18001800 Zero 7 Worn tooth 600, 1200 and 1800 Zero 7 Worn tooth 600, 1200 and 1800 Zero 220.127.116.11. Data Processing Data Processing 4.2. Data Processing The acquired signal, current currentsignal, signal,and andIAS IASsignal signal are preprocessed The acquiredvibration vibrationsignal, signal, acoustic acoustic signal, are preprocessed The acquired vibration signal, acoustic signal,method, current signal, and IAS signal are preprocessed following thethe steps in in Section 3.1. For the proposed the collected signals are standardized and following steps Section 3.1. For the proposed method, the collected signals are standardized following the steps in Section 3.1. For the proposed method, the collected signals are standardized divided into segments at first. A total of 1024 points are selected as a segment, so each type of signal and divided into segments at first. A total of 1024 points are selected as a segment, so each type of and divided into segments ateach first.health Afor total of 1024 points are motor selected as aand segment, so each of will contain segments condition under one speed 6552 segments in total signal will312 contain 312 for segments each health condition under one motor speed and type 6552 signal willhealth contain 312seven segments for each health under onefour motor speed 6552 segments in total for healththree conditions undercondition three motor speeds. Next, each of the four for seven conditions under motor speeds. Next, each of the segments ofand the four segments inof total for health under three speeds. Next, each of the four segments the fourseven signal typesasconditions are together asmotor one input data sample to the form the input signal types are combined together onecombined data sample to form the vectors of DCNN model. vectors of the DCNN model. In this way, each data sample will be a 4096-dimensional vector (four segments of the four signal types are combined together as one data sample to form the input In this way, each data sample will be a 4096-dimensional vector (four times the size of segments), times of thethe sizeDCNN of segments), there willeach be 6552 samples A total of 40% ofvector the 6552 vectors model. and In this way, datadata sample will in betotal. a 4096-dimensional (four times the size of segments), and there will be 6552 data samples in total. A total of 40% of the 6552 Sensors 2017, 17, 414 9 of 15 and there will be 6552 data samples in total. A total of 40% of the 6552 data samples are selected randomly as the training data set, 10% are used as the validation set, and the remaining 50% are selected as the testing data set. Eight trails are carried out to avoid particularity and contingency of the diagnosis result. The average testing accuracy of the eight trails is calculated as the final result. 4.3. Model Design The model of the DCNN is developed using the principles described in Section 3.2. For different input data, different model settings are tested, and the one with the best performance among all the tested settings is selected to process this input data. The model setting of the proposed method is displayed in Table 2, which consists of three convolutional layers, two pooling layers, and a fully-connected layer with softmax regression. The convolutional layer corresponds to Equation (1), the pooling layer to Equation (2) and the fully-connected layer to Equation (3). The DCNN model is developed based on C++. Table 2. Variables of parameters and structures of the deep convolutional neural networks. Layer Type Variables and Dimensions Training Parameters 1 2 Convolution Pooling CW = 65; CH = 1; CC = 1; CN = 10; B = 10 S=2 SGD minibatch size = 20 Initial learning rate = 0.05 3 Convolution CW = 65; CH = 1; CC = 10; CN = 15; B = 15 4 Pooling S=2 Decrease of learning rate after each ten epochs = 20% Momentum = 0.5 5 6 7 Convolution Hidden layer Softmax CW = 976; CH = 1; CC = 15; CN = 30; B = 30 Relu activation function 7 outputs Weight decay = 0.04 Max epochs = 200 Testing sample rate = 50% SDG = stochastic gradient decent; CW = filter width; CH = filter height; CC = filter channel; CN = number of filter in the bank; B = bias; S = sub-sampling rate; No overlapping of convolutional window and no padding. 4.4. Experimental Results 4.4.1. Results of Single Sensory Data Following the flowchart shown in Figure 3d in Section 3.3, methods with three intelligent models, feature learning and manual feature extraction methods, and four types of single sensor data are tested in the experiment. The results are displayed in Table 3. Table 3. Average testing accuracy of comparative methods with single sensory data. Sensory Data Model Feature Learning from Raw Data Vibration signal DCNN BPNN SVM Acoustic signal Manual Feature Extraction Time-Domain Features Frequency-Domain Features Handcraft Features 81.45% 42.56% 45.11% 55.84% 55.62% 56.35% 70.74% 69.03% 72.23% 73.64% 72.36% 73.86% DCNN BPNN SVM 66.23% 19.80% 26.54% 31.42% 35.89% 33.62% 76.45% 76.04% 77.36% 76.02% 75.79% 76.32% Current signal DCNN BPNN SVM 85.68% 52.36% 51.64% 60.73% 60.47% 63.74% 61.45% 61.21% 63.53% 76.85% 76.43% 78.76% Instantaneous angular speed (IAS) signal DCNN BPNN SVM 90.23% 51.37% 48.22% 75.34% 75.36% 75.68% 84.42% 85.22% 85.65% 88.34% 89.82% 89.85% DCNN = deep convolutional neural network; BPNN = back-propagation neural networks; SVM = support vector machine. Sensors 2017, 17, 414 Sensors 2017, 17, 414 10 of 15 10 of 15 Sensors 2017, 17, 414 10 of 15 4.4.2.Results ResultsofofMulti-Sensory Multi-SensoryData Data 4.4.2. 4.4.2. Results of Multi-Sensory Data Followingthe theflowcharts flowchartsshown shown in in Figure Figure 3a–c 3a–c in in Section Section 3.3, Following 3.3, methods methods with withthree threefusion fusionlevels, levels, Following the flowcharts shown in Figure 3a–c in Section 3.3, methods with three fusion levels, three intelligent models, two feature extraction methods, and multi-sensory data are tested three intelligent models, two feature extraction methods, and multi-sensory data are testedin inthe the three intelligent models,are twodisplayed feature extraction methods, andthe multi-sensory data are tested in the is experiment. The results in Table 4, in which result of the proposed method experiment. The results are displayed in Table 4, in which the result of the proposed method is marked experiment. TheFigure results are displayed in Tableresults 4, in which the result of thethe proposed method is in bold. presents the testing of the eight trails top three methods, inmarked bold. Figure 7 presents7 the testing results of the eight trails of the topofthree methods, which are marked in bold. Figure 7 presents the testing results of the eight trails of the top three methods, which are the proposed method, the DCNN model with feature learning and feature-level fusion, the proposed method, the DCNN model with feature learning and feature-level fusion, and the SVM which the proposed DCNNand model with feature learning and feature-level fusion, and the are SVM model with method, handcraftthe features feature-level fusion. model with handcraft features and feature-level fusion. andFinally, the SVMthe model with handcraft features and feature-level fusion. average testing accuracies of all the comparative methods in the experiment are Finally, thethe average testing accuracies of all methods in the are shown accuracies of the all comparative the comparative methods inexperiment the experiment are shownFinally, together inaverage Figure 8testing for a clearer comparison between each other. together in Figure 8 for a clearer comparison between each other. shown together in Figure 8 for a clearer comparison between each other. Table 4. Average testing accuracy of comparative methods with multi-sensory data. Table 4. 4.Average methodswith withmulti-sensory multi-sensorydata. data. Table Averagetesting testingaccuracy accuracyof of comparative comparative methods Manual Feature Extraction Feature Learning Manual Feature Manual Feature Extraction Extraction Handcraft Fusion Level Model Feature Learning Time-Domain Frequency-Domain Feature Learning from Raw Data Time-Domain Frequency-Domain Handcraft Fusion Model Fusion Level Level Model Time-Domain Frequency-Domain Handcraft Features Features Features from fromRaw RawData Data Features Features Features Features Features Features DCNN 99.28% 66.08% 87.63% 90.23% DCNN 99.28% 66.08% 87.63% 90.23% DCNN 99.28% 66.08% 87.63% 90.23% Data-level fusion BPNN 53.28% 65.95% 87.89% 91.22% Data-level BPNN 53.28% 65.95% 87.89% 91.22% BPNN 53.28% 65.95% 87.89% 91.22% Data-level fusion fusion SVM 51.62% 67.32% 87.28% 90.67% SVM 51.62% 67.32% 87.28% 90.67% SVM 51.62% 67.32% 87.28% 90.67% DCNN 98.75% 86.35% 92.34% 94.08% DCNN 98.75% 86.35% 92.34% 94.08% DCNN 98.75% 86.35% 92.34% 94.08% Feature-level fusion BPNN 64.74% 86.81% 92.15% 94.04% Feature-level BPNN 64.74% 86.81% 92.15% 94.04% BPNN 64.74% 86.81% 92.15% 94.04% Feature-level fusion fusion SVM 56.27% 86.74% 94.62% 95.80% SVM 56.27% 86.74% 94.62% 95.80% SVM 56.27% 86.74% 94.62% 95.80% DCNN 93.65% 84.65% 90.23% 92.19% DCNN 93.65% 84.65% 90.23% 92.19% DCNN 93.65% 84.65% 90.23% 92.19% Decision-level fusion BPNN 77.62% 84.47% 91.19% 93.42% Decision-level BPNN 77.62% 84.47% 91.19% 93.42% BPNN 77.62% 84.47% 91.19% 93.42% Decision-level fusion fusion SVM 76.17% 86.32% 90.98% 93.44% SVM 76.17% 86.32% 90.98% 93.44% SVM 76.17% 86.32% 90.98% 93.44% Figure Testing accuracyof ofeight eighttrials trialsof the top three the DCNN Figure Testing accuracy of eight trials of the the top top three three methods methods(the method, the DCNN Figure 7.7.7. Testing accuracy methods (theproposed proposedmethod, method, the DCNN model with feature learning and feature-level fusion, and the SVM model with handcraft features modelwith withfeature featurelearning learning and feature-level fusion, and SVM model with handcraft features model and feature-level fusion, and thethe SVM model with handcraft features and andfeature-level feature-level fusion). FL == feature feature learning; FE feature extraction; and fusion). FL learning; FE == manual manual feature extraction; DF= =data-level data-level feature-level fusion). FL = feature learning; FE = manual feature extraction; DF = DF data-level fusion; fusion; FF==feature-level feature-level fusion. fusion; FF FF = feature-level fusion. fusion. The proposed method The proposed method Figure 8. Testing accuracy of all the comparative methods. Figure8.8. Testing Testingaccuracy accuracy of of all all the the comparative comparative methods. Figure methods. Sensors 2017, 17, 414 Sensors 2017, 17, 414 11 of 15 11 of 15 4.5. Data and and Learned LearnedFeatures Features 4.5.Principal PrincipalComponent ComponentAnalysis Analysisof of the the Experimental Experimental Data PCA toanalyze analyzeand and visualize learned features the proposed As PCAisis employed employed to visualize the the learned features of the of proposed method.method. As shown shown in Figure 9, the labels 1 to 7 correspond to the seven conditions of the planetary gearbox in Figure 9, the labels 1 to 7 correspond to the seven conditions of the planetary gearbox described described Table andtwo theprincipal first two components principal components (PCs) arebyobtained by PCA tothe represent in Table 1,inand the1,first (PCs) are obtained PCA to represent useful the useful information hidden in the data. Figure 9a shows the result of the input data of the testing information hidden in the data. Figure 9a shows the result of the input data of the testing dataset of dataset of the proposed method the PCs. first two PCs. 9b illustrates result of the learned the proposed method along thealong first two Figure 9bFigure illustrates the result the of the learned features features with adaptive fusion levels of themethod proposed method testing the are dataset, which are with adaptive fusion levels of the proposed for testing thefor dataset, which extracted from extracted from thesecond-last output of layer the second-last layer the DCNN.the For comparison, the results the output of the of the DCNN. Forofcomparison, results of feature-level fusedof feature-level fused features learned DCNN andhandcraft feature-level fused handcraft features learned through DCNN andthrough feature-level fused features along the firstfeatures two PCsalong are the first two PCs are shown in Figure respectively. It should bedisplay noted that we only display the shown in Figure 9c,d, respectively. It 9c,d, should be noted that we only the first two PCs of the first two of thevisualization, data for a clearer which that there may some be overlaps between data forPCs a clearer whichvisualization, means that there maymeans be overlaps between categories and some categories andbemany of them can bePCs. divided into higher PCs. many of them can divided into higher (a) (b) (c) (d) Figure component analysis (PCA) of of thethe experimental data and learned features: (a) Figure9.9.Principal Principal component analysis (PCA) experimental data and learned features: Data-level fusedfused inputinput datadata of the proposed method; (b) (b) Features of the proposed method; (c) (a) Data-level of the proposed method; Features of the proposed method; Feature-level fused features learned through DCNN; features. (c) Feature-level fused features learned through DCNN;and and(d) (d)Feature-level Feature-levelfused fused handcraft handcraft features. 4.6. 4.6.Discussion Discussion 1.1. The is able able to to diagnose diagnosethe thefaults faultsofofthe the Theexperimental experimentalresults resultsshow showthat thatthe the proposed proposed method method is planetary best testing testingaccuracy accuracyininthe theexperiment. experiment.ItItcan can planetarygearbox gearboxtest testrig rigeffectively, effectively, yielding the best bebeseen from Table 4 and Figure 8 that the proposed method achieves the best testing accuracy seen from Table 4 and Figure 8 that method achieves the best testing accuracy 99.28% this result resultisissignificantly significantlycorrelated correlated 99.28%among amongall allthe thecomparative comparativemethods. methods. We We think that this with proposedmethod. method.DCNN DCNNcan canfuse fuseinput input withthe thedeep deeparchitecture architectureof ofthe the DCNN DCNN model model of the proposed dataand andlearn learnbasic basicfeatures featuresfrom from itit in in its its lower layers, fuse data fuse basic basic features featuresinto intohigher higherlevel level featuresorordecisions decisionsininits itsmiddle middlelayers, layers, and and further further fuse these features these features featuresand anddecisions decisionstotoobtain obtain thefinal finalresult resultininits itshigher higher layers. layers. Although Although there there is a data-level the data-level fusion fusionbefore beforeDCNN DCNNininthe the proposedmethod, method,DCNN DCNNstill stillactually actually fuses fuses the the data again in proposed in its its starting starting layers layerstotofurther further optimize the data structure. Optimized features and combinations of different level fusions are formed through this deep-layered model, which provides a better result than with manually selected features or fusion levels. Sensors 2017, 17, 414 2. 3. 4. 5. 6. 12 of 15 optimize the data structure. Optimized features and combinations of different level fusions are formed through this deep-layered model, which provides a better result than with manually selected features or fusion levels. The ability of automatic feature learning of the DCNN model with multi-sensory data is proven through the experiment. It can obviously be seen from Figure 8 that both the proposed method and the feature-level fusion method with feature learning through DCNN obtain a better result, 99.28% and 98.75%, than any other comparative methods with handcraft features or feature learning through BPNN. This result proves that the feature learning through DCNN with multi-sensory data can improve the performance of the multi-sensor data fusion method for fault diagnosis. In addition, the result also implies that the proposed method with adaptive fusion-level selection can achieve a better result 99.28% than the result 98.75% of the method with manual-selected feature-level fusion, which is the only difference between these two methods. However, the method with automatic feature learning of DCNN from the raw signal of a single sensor cannot achieve a better result than methods with handcraft features. Table 3 displays the diagnosis results using signals from a single sensor. Only with a vibration signal and current signal, can the DCNN-based feature learning method achieve better results than conventional methods with handcraft features. By contrast, the results of the DCNN-based feature learning method with an acoustic signal and IAS signal are worse than that of conventional methods. This implies that the DCNN-based method with learned features from single sensory data cannot provide stable improvements for all kinds of sensory data. We think that the performance of the DCNN-based feature learning is influenced by the characteristics of the input data. As can be seen from the results shown in Table 3, the performance of feature learning has a stronger positive correlation with the performance of time-domain features than frequency-domain features, which infers that the DCNN-based feature learning from a raw signal may be more sensitive to time-correlated features than frequency-correlated features. The effectiveness of the automatic feature learning and adaptive fusion-level selection of the proposed method is further confirmed through PCA. As can be seen from Figure 9a, most of the categories of the input raw data overlap each other, which makes it difficult to distinguish them. After the processing of the proposed method, the learned features with adaptive fusion levels along the first two PCs become distinguishable in Figure 9b. Meanwhile, Figure 9c,d presents the results of PCA with feature-level fused learned features and handcraft features as comparisons, respectively. The feature-level fused features learned through DCNN have just a slightly worse distinction between each category than the features of the proposed method, which not only verifies the feature learning ability of DCNN used in both methods, but also proves the better performance of the adaptive-level fusion of the proposed method than that of the manual-selected feature-level fusion. On the contrary, the fused handcraft features show a much worse distinction between different categories than the learned features of the proposed method. These analyses further demonstrate the effective performance of the automatic feature learning and adaptive fusion-level selection of the proposed method. While DCNN has a much better feature learning ability than BPNN, the three comparative models, DCNN, BPNN and SVM, obtain similar results with handcraft features. Figure 8 shows clearly that feature learning through DCNN achieves much better testing accuracies than through BPNN. Nevertheless, with handcrafts features, these three intelligent models provide similar accuracies, which suggests that DCNN cannot achieve much more improvements than conventional methods without using its ability of feature learning. Methods with multi-sensory data provide better results than those with single sensory data. It can be seen from Figure 8 that methods with multi-sensory data achieve higher testing accuracies than with single sensory data, no matter which fusion level or intelligent model is selected. This phenomenon indicates that multi-sensory data can improve the reliability and accuracy for fault diagnosis. Sensors 2017, 17, 414 13 of 15 5. Conclusions and Future Work This paper presents an adaptive data fusion method based on DCNN to detect the health conditions of planetary gearboxes. The processes of data-level fusion, feature-level fusion, decision-level fusion, feature learning, and fault diagnosis are all fused into one DCNN model adaptively. The proposed method can learn features from raw data, and fuse data, features, and decisions adaptively through the deep-layered structure of DCNN with fewer requirements of expert knowledge or human labor for feature extraction and fusion-level selection. The performance of the proposed method is evaluated through the experiment of the planetary gearbox fault test rig. As comparisons, feature-level fusion, decision-level fusion, handcraft features, single sensory data, and two traditional intelligent models, BPNN and SVM, are also tested in the experiment. The comparative results of the experiment verify the effectiveness of the proposed method, which achieves the best testing accuracy among all the comparative methods in the experiment. Our future work will focus on testing the DCNN model-based feature learning and data fusion approaches on more mechanical objects, fault modes, operation conditions, and sensor types, which can further confirm the effectiveness of approaches and help us to find out other useful application guidance. Moreover, due to the large number of parameters of deep learning models, manual parameter optimization often takes many trials-and-errors to find the best one, and conventional automatic searching methods are usually very time-consuming and easily converge into a local optimum. It is meaningful to investigate more effective and faster approaches to optimize the parameters automatically. Finally, combinations of different deep learning architectures should improve the effect of fault diagnosis. Adding recurrent architecture may make the model suitable to predict future fault conditions, and combining with auto-encoder architecture may improve the feature learning ability to capture more complex features. Acknowledgments: The authors would like to thank the National Natural Science Foundation of China (Grant No. 51475324), and the National Natural Science Foundation of China and Civil Aviation Administration of China jointly funded project (Grant No. U1533103) for their support. The authors would also like to thank the editors and anonymous reviewers for their constructive comments and suggestions. Author Contributions: Luyang Jing and Taiyong Wang conceived and designed the research; Ming Zhao designed and performed the experiment; Peng Wang contributed to the theory studies; Luyang Jing wrote the manuscript; and all the authors reviewed and revised the manuscript. Conflicts of Interest: The authors declare no conflict of interest. References 1. 2. 3. 4. 5. 6. 7. 8. Lei, Y.; Lin, J.; Zuo, M.J.; He, Z. Condition monitoring and fault diagnosis of planetary gearboxes: A review. Measurement 2014, 48, 292–305. [CrossRef] Khazaee, M.; Ahmadi, H.; Omid, M.; Banakar, A.; Moosavian, A. Feature-level fusion based on wavelet transform and artificial neural network for fault diagnosis of planetary gearbox using acoustic and vibration signals. Insight Non-Destr. Test. Cond. Monit. 2013, 55, 323–330. [CrossRef] Lei, Y.; Lin, J.; He, Z.; Kong, D. A Method Based on Multi-Sensor Data Fusion for Fault Detection of Planetary Gearboxes. Sensors 2012, 12, 2005–2017. [CrossRef] [PubMed] Li, C.; Sanchez, R.V.; Zurita, G.; Cerrada, M.; Cabrera, D.; Vásquez, R.E. Gearbox fault diagnosis based on deep random forest fusion of acoustic and vibratory signals. Mech. Syst. Signal Process. 2016, 76, 283–293. [CrossRef] Hall, D.L.; Llinas, J. An introduction to multisensor data fusion. Proc. IEEE 1997, 85, 6–23. [CrossRef] Khaleghi, B.; Khamis, A.; Karray, F.O.; Razavi, S.N. Multisensor data fusion: A review of the state-of-the-art. Inf. Fusion 2013, 14, 28–44. [CrossRef] Serdio, F.; Lughofer, E.; Pichler, K.; Buchegger, T.; Pichler, M.; Efendic, H. Fault detection in multi-sensor networks based on multivariate time-series models and orthogonal transformations. Inf. Fusion 2014, 20, 272–291. [CrossRef] Lei, Y.; Jia, F.; Lin, J.; Xing, S. An Intelligent Fault Diagnosis Method Using Unsupervised Feature Learning Towards Mechanical Big Data. IEEE Trans. Ind. Electron. 2016, 63, 3137–3147. [CrossRef] Sensors 2017, 17, 414 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 32. 14 of 15 Ince, T.; Kiranyaz, S.; Eren, L.; Askar, M. Real-Time Motor Fault Detection by 1D Convolutional Neural Networks. IEEE Trans. Ind. Electron. 2016, 63, 7067–7075. [CrossRef] Bengio, Y.; Courville, A.; Vincent, P. Representation Learning: A Review and New Perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 2013, 35, 1798–1828. [CrossRef] [PubMed] Lecun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature 2015, 521, 436–444. [CrossRef] [PubMed] Schmidhuber, J. Deep learning in neural networks: An overview. Neural Netw. 2014, 61, 85–117. [CrossRef] [PubMed] Bengio, Y. Learning Deep Architectures for AI. Found. Trends® Mach. Learn. 2009, 2, 1–127. [CrossRef] Janssens, O.; Slavkovikj, V.; Vervisch, B.; Stockman, K.; Loccufier, M.; Verstockt, S.; Walle, R.V.D.; Hoecke, S.V. Convolutional Neural Network Based Fault Detection for Rotating Machinery. J. Sound Vib. 2016, 377, 331–345. [CrossRef] Jia, F.; Lei, Y.; Lin, J.; Zhou, X.; Lu, N. Deep neural networks: A promising tool for fault characteristic mining and intelligent diagnosis of rotating machinery with massive data. Mech. Syst. Signal Process. 2016, 72–73, 303–315. [CrossRef] Lu, C.; Wang, Z.Y.; Qin, W.L.; Ma, J. Fault diagnosis of rotary machinery components using a stacked denoising autoencoder-based health state identification. Signal Process. 2016, 130, 377–388. [CrossRef] Sun, W.; Shao, S.; Zhao, R.; Yan, R.; Zhang, X.; Chen, X. A sparse auto-encoder-based deep neural network approach for induction motor faults classification. Measurement 2016, 89, 171–178. [CrossRef] Zhao, R.; Yan, R.; Wang, J.; Mao, K. Learning to Monitor Machine Health with Convolutional Bi-Directional LSTM Networks. Sensors 2017, 17, 273. [CrossRef] [PubMed] Abdeljaber, O.; Avci, O.; Kiranyaz, S.; Gabbouj, M.; Inman, D.J. Real-Time Vibration-Based Structural Damage Detection Using One-Dimensional Convolutional Neural Networks. J. Sound Vib. 2017, 388, 154–170. [CrossRef] Guo, X.; Chen, L.; Shen, C. Hierarchical adaptive deep convolution neural network and its application to bearing fault diagnosis. Measurement 2016, 93, 490–502. [CrossRef] Ji, S.; Xu, W.; Yang, M.; Yu, K. 3D Convolutional Neural Networks for Human Action Recognition. IEEE Trans. Pattern Anal. Mach. Intell. 2013, 35, 221–231. [CrossRef] [PubMed] Girshick, R.; Donahue, J.; Darrell, T.; Malik, J. Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE Conference On Computer Vision and Patten Recognition, Columbus, OH, USA, 23–28 June 2014; pp. 580–587. Sermanet, P.; Eigen, D.; Zhang, X.; Mathieu, M.; Fergus, R.; Lecun, Y. Overfeat: Integrated recognition, localization and detection using convolutional networks. arXiv 2014, arXiv:1312.6229. Simonyan, K.; Zisserman, A. Very deep convolutional networks for large-scale image recognition. arXiv 2014, arXiv:1409.1556. Krizhevsky, A.; Sutskever, I.; Hinton, G.E. ImageNet Classification with Deep Convolutional Neural Networks. Adv. Neural Inf. Process. Syst. 2012, 25, 1097–1105. Nebauer, C. Evaluation of convolutional neural networks for visual recognition. IEEE Trans. Neural Netw. 1998, 9, 685–696. [CrossRef] [PubMed] Mnih, V.; Kavukcuoglu, K.; Silver, D.; Rusu, A.A.; Veness, J.; Bellemare, M.G.; Graves, A.; Riedmiller, M.; Fidjeland, A.K.; Ostrovski, G. Human-level control through deep reinforcement learning. Nature 2015, 518, 529–533. [CrossRef] [PubMed] Lei, Y.; Han, D.; Lin, J.; He, Z. Planetary gearbox fault diagnosis using an adaptive stochastic resonance method. Mech. Syst. Signal Process. 2013, 38, 113–124. [CrossRef] Sun, H.; Zi, Y.; He, Z.; Yuan, J.; Wang, X.; Chen, L. Customized multiwavelets for planetary gearbox fault detection based on vibration sensor signals. Sensors 2013, 13, 1183–1209. [CrossRef] [PubMed] Amarnath, M.; Krishna, I.R.P. Local fault detection in helical gears via vibration and acoustic signals using EMD based statistical parameter analysis. Measurement 2014, 58, 154–164. [CrossRef] Immovilli, F.; Cocconcelli, M.; Bellini, A.; Rubini, R. Detection of Generalized-Roughness Bearing Fault by Spectral-Kurtosis Energy of Vibration or Current Signals. IEEE Trans. Ind. Electron. 2009, 56, 4710–4717. [CrossRef] Kar, C.; Mohanty, A.R. Monitoring gear vibrations through motor current signature analysis and wavelet transform. Mech. Syst. Signal Process. 2006, 20, 158–187. [CrossRef] Sensors 2017, 17, 414 33. 34. 35. 36. 37. 38. 39. 40. 41. 42. 43. 44. 45. 46. 47. 15 of 15 Lu, D.; Gong, X.; Qiao, W. Current-Based Diagnosis for Gear Tooth Breaks in Wind Turbine Gearboxes. In Proceedings of the Energy Conversion Congress and Exposition, Raleigh, NC, USA, 15–20 September 2012; pp. 3780–3786. Fedala, S.; Rémond, D.; Zegadi, R.; Felkaoui, A. Contribution of angular measurements to intelligent gear faults diagnosis. J. Intell. Manuf. 2015. [CrossRef] Moustafa, W.; Cousinard, O.; Bolaers, F.; Sghir, K.; Dron, J.P. Low speed bearings fault detection and size estimation using instantaneous angular speed. J. Vib. Control 2014, 22, 3413–3425. [CrossRef] Turaga, S.C.; Murray, J.F.; Jain, V.; Roth, F.; Helmstaedter, M.; Briggman, K.; Denk, W.; Seung, H.S. Convolutional networks can learn to generate affinity graphs for image segmentation. Neural Comput. 2010, 22, 511–538. [CrossRef] [PubMed] Chen, Z.Q.; Li, C.; Sanchez, R.V. Gearbox Fault Identification and Classification with Convolutional Neural Networks. Shock Vib. 2015, 2015, 390134. [CrossRef] Szegedy, C.; Vanhoucke, V.; Ioffe, S.; Shlens, J.; Wojna, Z. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Patten Recognition, Seattle, WA, USA, 27–30 June 2016. Bergstra, J.; Bardenet, R.; Kégl, B.; Bengio, Y. Algorithms for Hyper-Parameter Optimization. In Proceedings of the Advances in Neural Information Processing Systems, Granada, Spain, 12–15 December 2011; pp. 2546–2554. Zhang, L.; Suganthan, P.N. A Survey of Randomized Algorithms for Training Neural Networks. Inf. Sci. 2016, 364–365, 146–155. [CrossRef] Abdel-Hamid, O.; Li, D.; Dong, Y. Exploring Convolutional Neural Network Structures and Optimization Techniques for Speech Recognition. In Proceedings of the INTERSPEECH, Lyon, France, 25–29 August 2013; pp. 1173–1175. Kane, P.V.; Andhare, A.B. Application of psychoacoustics for gear fault diagnosis using artificial neural network. J. Low Freq. Noise Vib. Act.Control 2016, 35, 207–220. [CrossRef] Fedala, S.; Rémond, D.; Zegadi, R.; Felkaoui, A. Gear Fault Diagnosis Based on Angular Measurements and Support Vector Machines in Normal and Nonstationary Conditions. In Condition Monitoring of Machinery in Non-Stationary Operations; Springer: Berlin/Heidelberg, Germany, 2016; pp. 291–308. Li, C.; Sánchez, R.V.; Zurita, G.; Cerrada, M.; Cabrera, D. Fault Diagnosis for Rotating Machinery Using Vibration Measurement Deep Statistical Feature Learning. Sensors 2016, 16, 895. [CrossRef] [PubMed] Lu, D.; Qiao, W. Adaptive Feature Extraction and SVM Classification for Real-Time Fault Diagnosis of Drivetrain Gearboxes. In Proceedings of the Energy Conversion Congress and Exposition, Denver, CO, USA, 15–19 September 2013; pp. 3934–3940. Abumahfouz, I.A. A comparative study of three artificial neural networks for the detection and classification of gear faults. Int. J. Gen. Syst. 2005, 34, 261–277. [CrossRef] Li, Y.; Gu, F.; Harris, G.; Ball, A.; Bennett, N.; Travis, K. The measurement of instantaneous angular speed. Mech. Syst. Signal Process. 2005, 19, 786–805. [CrossRef] © 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).