minerals Article An Enhanced Rock Mineral Recognition Method Integrating a Deep Learning Model and Clustering Algorithm Chengzhao Liu 1 , Mingchao Li 1, * , Ye Zhang 1 , Shuai Han 1 and Yueqin Zhu 2 1 2 * State Key Laboratory of Hydraulic Engineering Simulation and Safety, Tianjin University, Tianjin 300354, China Development Research Center of China Geological Survey, Beijing 100037, China Correspondence: lmc@tju.edu.cn Received: 26 May 2019; Accepted: 23 August 2019; Published: 26 August 2019 Abstract: Rock mineral recognition is a costly and time-consuming task when using traditional methods, during which physical and chemical properties are tested at micro- and macro-scale in the laboratory. As a solution, a comprehensive recognition model of 12 kinds of rock minerals can be utilized, based upon the deep learning and transfer learning algorithms. In the process, the texture features of images are extracted and a color model for rock mineral identification can also be established by the K-means algorithm. Finally, a comprehensive identification model is made by combining the deep learning model and color model. The test results of the comprehensive model reveal that color and texture are important features in rock mineral identification, and that deep learning methods can effectively improve identification accuracy. To prove that the comprehensive model could extract effective features of mineral images, we also established a support vector machine (SVM) model and a random forest (RF) model based on Histogram of Oriented Gradient (HOG) features. The comparison indicates that the comprehensive model has the best performance of all. Keywords: rock mineral; classification recognition; deep learning model; transfer learning model; clustering algorithm 1. Introduction The classification and identification of rock minerals are indispensable in geological works. The traditional recognition methods are based on the physical and chemical properties of rock minerals, which are used to identify rock minerals at macro- and micro-scales [1,2]. With the development of computer science and artificial intelligence, the recognition model can be established using machine learning methods [3–7]. Traditional recognition methods have definite physical meaning; while machine learning methods are driven by data [8,9]. The two methods have their own strengths and weaknesses. The significant strength of traditional identification model is that the model can be explained. Vassilev and Vassileva [10,11] analyzed the optical properties of low-temperature ash (LTA) and high-temperature ash (HTA) of the coal samples to get the weight of each mineral in the crystalline matter basis. Then they classified the inorganic matter in coal using the composition from the HTA and got a good result. But the experiment environment and operating procedures of this method are difficult to implement. Zaini [12] used wavelength position, linear spectral un-mixing and a spectral angle mapper to explore carbonate mineral compositions and distribution on the rock mineral surface and got a good result. Moreover, Zaini’s method needs SisuCHEMA SWIR sensor, which is integrated with a computer workstation to collect the data. Adep [13] developed an expert system for hyperspectral data classification with neural network technology. Their system was also used to Minerals 2019, 9, 516; doi:10.3390/min9090516 www.mdpi.com/journal/minerals Minerals 2019, 9, 516 2 of 17 calculate the depth and distribution of iron ore in a specific engineering project. The scholars chose the key factors to build the model. However, the professional knowledge and equipment are necessary for the process. It is not an efficient and economical way to conduct traditional model sometimes. In recent years, machine learning methods have been applied in image recognition, including sensing images [14,15] and mineral images [16]. Multi-source and multi-type mineral datasets are suitable for machine learning methods [17,18]. Moreover, the application of machine learning in the identification of rock images has been proved useful and efficient [19–23]. Aligholi et al. [24] classified 45 slices of 15 kinds of rock minerals automatically based on the color of the rock minerals and achieved high accuracy. Li et al. [25] applied feature selection and transfer learning to classify microscopic images of sandstone. The feature engineering is significant in machine learning model establishment. However, it is easy to be influenced by subjective factors. As a result, it is necessary to develop an end-to-end mineral identification model. At present, deep learning algorithms have been developed and studied widely. Google introduced the design and application of a deep learning framework, which was flexible to build deep learning models [26]. Moreover, deep learning methods have achieved a remarkable prominence in recognition of rock mineral images. Zhang et al. [27] retrained Inception-v3 to identify granite, phyllite and breccia, and thereby proved that the Inception-v3 model was suitable for rock identification. The neuro-adaptive learning algorithm was also used to classify the iron ore real-timely [28]. Lots of researchers studied the application of deep learning in geochemical mapping [29], mineral prospectivity mapping [30] and micro-scale images of quartz and resin [31]. It proves that deep learning models are outstanding in complex image data processing and analysis. While, it is hard to improve deep learning model, because the deep learning architecture is fixed. The model ensemble can be adopted to solve the problem. In this paper, a comprehensive model for rock mineral identification was developed by combining the Inception-v3 model and a color model. First, as shown in Section 3.1, the texture features of rock minerals in the image were extracted based on the color and brightness of rock minerals. Then, the Inception-v3 model was retrained with these preprocessed images. Moreover, a color model was proposed (as shown in Section 3.2) to recognize the mineral images based on the result of Inception-v3 model. Finally, the comprehensive identification model based on the deep learning model and color model can be established (as shown in Section 4). We also constructed two models using SVM and RF-based on Histogram of Oriented Gradient (HOG) features. The comparison of the three models indicates that the comprehensive model can achieve significant performance. In this research, the elaborate mineral images are adopted, whose features are obvious. We want to study whether the comprehensive model can extract the obvious features, such as cleavage and luster. If effective features can be extracted, the model can be used in a larger domain. 2. Methodology This research is based on the deep learning method, transfer learning and clustering algorithm. The overall of the method is shown in Figure 1. The process can be divided into two parallel ones. One of them cut the raw images (images without any pre-process) into pieces. Then T1 is calculated, and the color model is established using these image slices. T1 is broadcasted to the other process at the same time. The other process will calculate and outline the binary points in the raw images once received T1 . Then the processed images are going to be trained using the transfer learning method based on deep learning method. Finally, a comprehensive model is established, combining the Inception-v3 model and color model. T and T1 are thresholds defined in Section 3.1. Minerals 2019, 9, 516 3 of 17 Minerals 2019, 9, x FOR PEER REVIEW 3 of 16 Raw image data Mineral image slices Calcula te the color value of each point Calcula te the gray value of each point K-mea ns calculation T1 If the value changes more than T1, set the point as binary point If the value changes more than T, set the point as binary point Preprocesse d image data Transfe r learning Comprehensive model Inception-v3 model Image data Identific ation by Inception-v3 model Color model Results from Inception-v3 model Comprehensive result Identific ation based on color model and results from Inception-V3 model Figure 1. 1. The The training training process process and and identification identification process process of a comprehensive model. Figure 2.1. Deep Learning Algorithm 2.1. Deep Learning Algorithm Deep learning was proposed by Hinton and Salakhutdinov [32] and has been widely applied Deep learning was proposed by Hinton and Salakhutdinov [32] and has been widely applied in in semantic recognition [33], image recognition [34], object detection [35], and the medical field [36]. semantic recognition [33], image recognition [34], object detection [35], and the medical field [36]. In In image recognition, the end-to-end training mode is adopted in the training process. In each iteration image recognition, the end-to-end training mode is adopted in the training process. In each iteration of a training process, each layer can adjust its parameters according to the feedback from training data of a training process, each layer can adjust its parameters according to the feedback from training to improve the accuracy of the model. Moreover, the nonlinear function mapping ability is added to data to improve the accuracy of the model. Moreover, the nonlinear function mapping ability is the deep learning model. As a consequence, it reduces the computational time for extracting features added to the deep learning model. As a consequence, it reduces the computational time for from images and improves the accuracy of the deep learning model. extracting features from images and improves the accuracy of the deep learning model. In this paper, the Inception-v3 model [37] was set as a pre-trained model. In the Inception-v3 In this paper, the Inception-v3 model [37] was set as a pre-trained model. In the Inception-v3 model, there are five convolutional layers and two max-pooling layers in front of the net structure. model, there are five convolutional layers and two max-pooling layers in front of the net structure. To reduce the computation cost, the convolution kernel is converted from one 5 × 5 matrix to two To reduce the computation cost, the convolution kernel is converted from one 5 × 5 matrix to two 3 × 3 × 3 matrixes. Then, 11 mixed layers follow. These mixed layers could be divided into three categories, 3 matrixes. Then, 11 mixed layers follow. These mixed layers could be divided into three categories, named block module 1, block module 2 and block module 3. Block module 1 has three repetitive mixed named block module 1, block module 2 and block module 3. Block module 1 has three repetitive layers, and each layer is connected by a contact layer. The input size of first mixed layer is (35,35,192), mixed layers, and each layer is connected by a contact layer. The input size of first mixed layer and output size is (35,35,256). The rest mixed layers’ input, and output size are (35,35,288) in the block is(35,35,192), and output size is (35,35,256). The rest mixed layers’ input, and output size are module 1. Batch size is the number of training samples in each step. In the front of block module 2, (35,35,288) in the block module 1. Batch size is the number of training samples in each step. In the there is a small mixed layer, whose input size is (35,35,288) and the output size is (17,17,768). The rest front of block module 2, there is a small mixed layer, whose input size is (35,35,288) and the output of the mixed layers’ input and output size are (17,17,768) in the block module 2. In block module 3, size is (17,17,768). The rest of the mixed layers’ input and output size are (17,17,768) in the block the input size is (17,17,768) and output size is (8,8,2048). After the mixed layer, a convolutional layer, module 2. In block module 3, the input size is (17,17,768) and output size is (8,8,2048). After the mean pooling layer and dropout layer is followed to extracted image features. Finally, the softmax mixed layer, a convolutional layer, mean pooling layer and dropout layer is followed to extracted classifier is trained at the end of the net. image features. Finally, the softmax classifier is trained at the end of the net. In the process of convolution, a color image is converted into a 3D matrix including all RGB values; then, each pixel is evaluated using the kernel and pooling during each iteration. The average pooling layer and max-pooling layer could get the average value and max value of a matrix, Minerals 2019, 9, 516 4 of 17 Minerals 2019, 9, x FOR PEER REVIEW 4 of 16 In the process of convolution, a color image is converted into a 3D matrix including all RGB values; then, each pixel is evaluated using the kernelcan andbe pooling during each iteration. Theof average pooling respectively. Finally, the common features extracted from a large number images. In the layer and max-pooling layer could get the average value andwhich max value a matrix, Finally, mixed layer, convolution and pooling are synchronized, is ableofto achieverespectively. better performance the common features can be extracted from a large number of images. In the mixed layer, convolution in feature extraction. and pooling are synchronized, which is able to achieve better performance in feature extraction. 2.2. Transfer Leaning Method 2.2. Transfer Leaning Method According to transfer learning, the parameters in trained models from clustering domains can According to transfer learning, parameters trained fromrecognition, clustering domains can be be utilized to establish a new modelthe efficiently. In in a new taskmodels of images the parameters utilized to establish a new a new task Transfer of imageslearning recognition, the parameters of a of a pre-trained model aremodel set asefficiently. the initialInparameters. can help scholars avoid pre-trained model are set as the initial parameters. Transfer learning can help scholars avoid building a building a model from scratch and make full use of the data, which reduces computational cost and model from scratch and make full use of the data, which reduces computational cost and dependence dependence on big datasets. on bigIndatasets. this research, the Inception-v3 model was selected as the pre-trained model for rock mineral In recognition. this research,The the pre-trained Inception-v3deep model was selected the pre-trained model for rockand mineral image learning model as is able to reduce computational time image recognition. The pre-trained deep learning model is able to reduce computational and time cost when a new model is trained. Therefore, transfer learning based on a deep learning modelcost has when a new model is trained. transfer based onmodel a deepcontains learningabout model1.2 has been been widely used [38,39]. TheTherefore, original data set oflearning the Inception-v3 million widely The 1000 original data set The of the Inception-v3 about 1.2 million images imagesused from[38,39]. more than categories. model involvesmodel about contains 25 million parameters. However, from than 1000 categories. Thethe model involves about 25 million However, the with more the transfer learning method, training process based on thisparameters. model can be quickly with finished transfer learning method, the training process based on this model can be quickly finished even though even though the computer has no GPU in general. Figure 2 shows the process of transfer learning the computer has no GPU in general. Figure 2 shows the process of transfer learning method. method. Images of ImageNet Convolution layers and pooling layers Labels of ImageNet Softmax Transfer learning Rock mineral images Feature extraction Features of rock mineral images Softmax Result of rock mineral image classification Figure2. 2. Transfer Transferlearning learningwith withInception-v3. Inception-v3. Figure 2.3. 2.3. Clustering Clustering Algorithm Algorithm The The clustering clustering algorithm algorithm has has aa mass mass application application in in pattern pattern recognition recognition (PR) (PR) and and is is based based on on the the Euclidian Distance. Absolute distance is the main point of Euclidian Distance, which measures Euclidian Distance. Absolute distance is the main point of Euclidian Distance, which measures the the clustering of of datasets by absolute distance. Reasonably, a rankawill be will proposed by the distance clusteringofoflots lots datasets by absolute distance. Reasonably, rank be proposed by the size of thesize dataset. distance of the dataset. In thethe color model is established based on the K-means algorithm.algorithm. In the algorithm, Inthis thisresearch, research, color model is established based on the K-means In the the absolute distances of objects are calculated, and the objects that share the closest absolute distances algorithm, the absolute distances of objects are calculated, and the objects that share the closest are in the distances same category. a silhouette used to evaluate theisclustering effect: the absolute are inFinally, the same category.coefficient Finally, a is silhouette coefficient used to evaluate clustering effect: bπi −−aπi π π . Si = (1) ππ = max(ai , bi ) . (1) πππ₯( ππ , ππ ) In Equation Equation (1), (1), aai is is the the average average distance distance from from the the ith ith point point to to other other points points in in the the same same kind, kind, In i which is called intra-cluster dissimilarity; b i is the average distance from the ith point to the points in which is called intra-cluster dissimilarity; bi is the average distance from the ith point to the points any kind, which is called inter-cluster dissimilarity. When the the number of kinds is more thanthan 3, the in any kind, which is called inter-cluster dissimilarity. When number of kinds is more 3, average value is set as a silhouette coefficient. The silhouette coefficient is in [−1, 1]. In the training the average value is set as a silhouette coefficient. The silhouette coefficient is in [−1, 1]. In the training colormodel, model,the themodel modelreceiving receivingthe themaximum maximumsilhouette silhouettecoefficient coefficientisisgoing goingtotobe beestablished. established. color Minerals 2019, 9, 516 5 of 17 2.4. Support Vector Machine Support Vector Machine (SVM) is a supervised learning method. When given a binary classification N D dataset xn , yn N n=1 with xn ∈ R and yn ∈ [−1, 1], for each (xn , yn ) belongs to xn , yn n=1 . An SVM model can be established by the equation: f (x) = ωT x + b. (2) The optimal weight vector wT and bias b are obtained by the Hinge Loss Function [40]. In this research, the Histogram of Oriented Gradient (HOG) algorithm was used to extract the feature of rock mineral images and then an SVM model was trained with these features. 2.5. Random Forest Random Forest (RF) is proposed by Leo Breiman. The steps of establishing an RF model are as following: Step 1: Input T as training dataset size and M as feature numbers. Step 2: Input the feature number of each node (m, m << M) to get the decision result at a node in the tree. Step 3: Select K samples with playback from a dataset N randomly. Then, K taxonomic trees are constructed by these selected samples. The rest of the dataset is used to test the model. Step 4: Randomly select m features at each node and find the best splitting mode by the selected features. Step 5: Each tree grows completely without pruning and may be adopted after a normal tree classifier is built. 3. Algorithm Implementation Rock minerals have unique physical properties, such as color, streak, transparency, aggregate shape and luster [41]. By analyzing their properties, rock minerals can be identified more accurately. Table 1 shows the properties of 12 kinds of rock minerals. Table 1. Twelve kinds of rock mineral’s physical properties. Minerals Color Streak Transparency Aggregate Shape Luster Cinnabar Red Red Translucent Granule, massive form Hematite Red Cherry red Opaque Adamantine luster From metallic luster to soil state luster Calcite Colorless or white White Translucent Malachite Green Pale green From translucent to opaque Azurite Navy blue Pale blue Opaque Aquamarine Determine by its component White Transparent Augite Black Magnetite Black Pale green, black Black Molybdenite Gray Light gray Opaque Stibnite Gray Dark gray Opaque Cassiterite Crineus, yellow, black White, pale brown Gyp White, colorless White From opaque to transparent Transparent, translucent Opaque Opaque Multi-form Granule, massive form, threadiness, stalactitic form, soil state Emulsions, massive, incrusting, concretion forms or threadiness Granule, stalactitic, incrusting, soil state Cluster form Granule, radial pattern, massive Granule, massive form Clintheriform, scaly form or lobate form Massive, granule or radial pattern Irregular granule Clintheriform, massive, threadiness Glassy luster Waxy luster, glassy luster, soil state luster Glassy luster Glassy luster Glassy luster Metallic luster Metallic luster Strong metallic luster Adamantine luster, sub adamantine luster Glassy luster, nacreous luster Minerals Minerals 2019, 2019, 9, 9, 516 x FOR PEER REVIEW of 17 66 of 16 In Table 1, color is a key feature of rock minerals in rock mineral image recognition, because In Table 1, color is a key feature of rock minerals in rock mineral image recognition, because different compositions of rock minerals lead to different colors. Also, the composition of different different compositions of rock minerals lead to different colors. Also, the composition of different rock rock minerals usually changes, and the colors vary widely. Some rock minerals with a single minerals usually changes, and the colors vary widely. Some rock minerals with a single composition composition have a pure color. For example, cinnabar is usually red, malachite is green, and calcite is have a pure color. For example, cinnabar is usually red, malachite is green, and calcite is white. white. A rock mineral often shows a mixed color on the surface because of mixed composition, A rock mineral often shows a mixed color on the surface because of mixed composition, which is often which is often indicated by binomial nomenclature, such as lime-green. However, whether a indicated by binomial nomenclature, such as lime-green. However, whether a particular type of rock particular type of rock mineral has a pure or mixed composition, those in that classification always mineral has a pure or mixed composition, those in that classification always share some certain kinds share some certain kinds of colors. Therefore, color is an important classification criterion. of colors. Therefore, color is an important classification criterion. The texture is also an important feature for rock mineral image recognition. The main The texture is also an important feature for rock mineral image recognition. The main components components of rock mineral texture are aggregate shape and cleavage. Different rock minerals have of rock mineral texture are aggregate shape and cleavage. Different rock minerals have different different distinctive textures. For example, cinnabar’s aggregate is granular, with perfect cleavage distinctive textures. For example, cinnabar’s aggregate is granular, with perfect cleavage and uneven and uneven to sub-conchoidal fracture; aquamarine’s aggregate is cluster form, with imperfect to sub-conchoidal fracture; aquamarine’s aggregate is cluster form, with imperfect cleavage and cleavage and conchoidal to irregular fracture; malachite’s aggregate is multi-form, as shown in conchoidal to irregular fracture; malachite’s aggregate is multi-form, as shown in Figure 3. The texture Figure 3. The texture is also helpful in recognizing rock minerals. Therefore, the combination of color is also helpful in recognizing rock minerals. Therefore, the combination of color and texture can and texture can improve the accuracy of the rock mineral recognition model. improve the accuracy of the rock mineral recognition model. (a) (b) (c) Figure ofof rock mineral characteristics [42]. (a)(a) Aquamarine; (b)(b) cinnabar; (c)(c) hematite. Figure3.3.Samples Samples rock mineral characteristics [42]. Aquamarine; cinnabar; hematite. 3.1. 3.1. Mineral Mineral Texture Texture Feature Feature Extraction Extraction The The visual visual features features of of rock rock minerals minerals also also include include the the texture texture of of rock rock mineral mineral aggregates aggregates and and cleavage. In light conditions, the reflection of different cleavages is different. Meanwhile, the reflection cleavage. In light conditions, the reflection of different cleavages is different. Meanwhile, the of a given of rock mineral follows a pattern ofbecause its own of characteristics. Therefore, Therefore, the texturethe of reflection a given rock mineral followsbecause a pattern its own characteristics. rock minerals can be extracted based on the brightness and color variation of images. texture of rock minerals can be extracted based on the brightness and color variation of images. In In an an area area where where the the brightness brightness changes changes significantly significantly in in the the image, image, the the rock rock mineral mineral texture texture can can be outlined. According to the RGB color system, color images can be converted to grayscale be outlined. According to the RGB color system, color images can be converted to grayscale to to show show brightness brightness variations variations more more effectively. effectively.The Theconversion conversionrule rule[43] [43]isisas asfollows: follows: (3) ππππ¦ = 0.299 × π + 0.587 × πΊ + 0.114 × π΅, gray = 0.299 × R + 0.587 × G + 0.114 × B, (3) where, gray is the grayscale value of a pixel, and R, G and B are the three primary color values of the pixel. The value of thevalue image be calculated by Equation In primary the lightcolor condition, where, graygray is the grayscale ofcan a pixel, and R, G and B are the(3). three valuesthe of brightness in the area a rock image by is distinctly different of the the pixel. The grayopaque value of theofimage canmineral be calculated Equation (3). In thefrom lightthat condition, surrounding pixels. the rockarea mineral’s luster is brilliant in the image, some pixels from achieve a higher the brightness in theIf opaque of a rock mineral image is distinctly different that of the brightness value than thatrock of their surrounding while of the pixels do achieve not. If the luster surrounding pixels. If the mineral’s luster isarea, brilliant inthe the rest image, some pixels a higher of a rock mineral is dull in the image, which indicates that it has a low reflection, pixels achieve brightness value than that of their surrounding area, while the rest of the pixels do not. If the luster ofa value their surrounding pixels, in Figure 4. pixels achieve a lower alower rock brightness mineral is dull inthan the image, which indicates thatasitshown has a low reflection, brightness value than their surrounding pixels, as shown in Figure 4. Minerals 2019, 9, 516 Minerals 2019, 9, x FOR PEER REVIEW 7 of 17 7 of 16 (a) (b) (c) (d) Figure images. (a) (a) Calcite; Calcite;(b) (b)brightness brightnessdistribution distributionof Figure4.4.The Thebrightness brightnessdistribution distribution of of rock rock mineral images. ofcalcite calciteimage; image;(c) (c)cinnabar; cinnabar;(d) (d)brightness brightnessdistribution distributionofofcinnabar cinnabarimage. image. the same same light brightness value fluctuates around a fixed and theand changed InInthe lightcondition, condition,the the brightness value fluctuates around a value, fixed value, the value of the fluctuation is set as The brightness value dramatically changes if it is beside |βZ |. i as |Δππ |. The brightness value dramatically changes if it the changed value of the fluctuation is set is texture and the value is far more than variation a pixel is calculated: |βZmore |βZi |. The i |, namely beside the texture and the value is far than |βZ| |Δππ>> |, namely |Δπ| >> |Δππ |.forThe variation for a pixel is calculated: βZi = grayi − gray, Δππ = ππππ¦π − ππππ¦, (4) (4) where, π iisisthe where,π₯π βZ thevariation variationofofbrightness brightnessfor foreach eachpixel pixeland andthe thegrayscale grayscalevalue valuefor forpixel pixelpoints pointsisis determined by Equation (3). Moreover, gray is the mean value of the grayscale of the central pixel determined by Equation (3). Moreover, gray is the mean value of the grayscale of the central pixel and and 8 pixels around the central pixel. In order to eliminate edge influence, the maximum and 8 pixels around the central pixel. In order to eliminate edge influence, the maximum and minimum are minimum removed, as shown (5) in Equations removed,are as shown in Equations and (6). (5) and (6). n ο½9 gray ο½ (ο₯ grayi ο graymax ο graymin ) / 7 , (5) n =9 X i ο½1 gray = ( grayi − graymax − gray7min )/7, (5) 1 i=1| |Δππ |′ = ||Δπ (6) π − |Δπ|| = ||Δππ | − ∑|Δππ ||, 7 π=1 7 1 X ππππ¦ where, the grayi is the value|βZ ofi |0the ith i |pixel’s and ππππ¦πππ are the = |βZi | − value,|βZ (6) = |βZ − |βZ| grayscale k | , πππ₯ 7 calculated, maximum and minimum grayscale values of the pixels being |π₯π |′ is the variation of k =1 π the ith grayscale is selected as pixel’s the threshold value. After experimentation, we found that a where, the grayi pixel. is the T value of the ith grayscale value, gray max and graymin are the maximum ′ 0 value of T = 15 was suitable. If the point satisfies |Δπ | > T, it will be set as a feature point. π and minimum grayscale values of the pixels being calculated, |βZi | is the variation of the ith grayscale In Taddition, theastexture featuresvalue. couldAfter be extracted from color at the interface pixel. is selected the threshold experimentation, wechanges found that a value of T =of15rock was 0 minerals. Therefore, the color change at the interface of a rock mineral could be measured by the suitable. If the point satisfies |βZi | > T, it will be set as a feature point. variation of RGB proportion. Three linearly independent indexes, namely C1, C2 and C3 were selected, which were calculated in Equation (7). Moreover, n was set as 0.01 to avoid a denominator of zero. πΆ1 = π /(πΊ + π), πΆ2 = πΊ/(π΅ + π), πΆ3 = π΅/(π + π), (7) Minerals 2019, 9, 516 8 of 17 In addition, the texture features could be extracted from color changes at the interface of rock minerals. Therefore, the color change at the interface of a rock mineral could be measured by the variation of RGB proportion. Three linearly independent indexes, namely C1 , C2 and C3 were selected, which were calculated in Equation (7). Moreover, n was set as 0.01 to avoid a denominator of zero. C1 = R/(G + n), C2 = G/(B + n), C3 = B/(R + n), (7) where, C was calculated to evaluate the variation of C1 , C2 and C3 comprehensively, as shown in Equation (7). p C = K C1 2 + C2 2 + C3 2 , (8) where, K is the expansion coefficient, and in this research, K was set to 1. C will change sharply when any one of C1 , C2 and C3 changes. βC was set as the maximum difference in coefficients: βC = max{βCi }, (9) where, βCi wasMinerals the 2019, difference the pixel and the surrounding pixels, and 1 was used as a 9, x FOR PEERbetween REVIEW 8 ofT16 threshold value to determine whether the point was a texture boundary point; and T’ was a matrix, where, C was calculated to evaluate the variation of C1, C2 and C3 comprehensively, as shown in which was used to store Equation (7). the maximum distances within each class. T 1 indicated the max distance between the 12 rock minerals, which was contains the (8) πΆ = calculated πΎ√πΆ1 2 + πΆ2 2 + by πΆ3 2 , K-means. First, a matrix which RGB values of every of rock mineral image calculated algorithm. where, Ktype is the expansion coefficient, and in thiswas research, K was set toby 1. CK-means will change sharply when Then, a result any one ofthe C1, Cmax 2 and C3 changes. ΔπΆ was set as the maximum difference in coefficients: matrix that contained distance for these 12 kinds of rock mineral was assigned to T’: ΔπΆ = πππ₯{ π₯ πΆπ }, (9) where, ΔπΆπ was the difference between the pixel and the surrounding pixels, and T1 was used as a T0 = {D1 ,point D2 , was . . . ,aD , Dm }, point; and T’ was a matrix, (10) i , . . . boundary threshold value to determine whether the texture which was used to store the maximum distances within each class. T1 indicated the max distance the 12 rockin minerals, which was calculated by K-means. a matrix contains the where, Di is thebetween max distance each category from any point’s First, C value to which the mean colors’ RGB values; RGB values of every type of rock mineral image was calculated by K-means algorithm. Then, a result and m is the number of mineral classes. Finally, the maximum value in T’ was assigned to T 1 . When T 1 matrix that contained the max distance for these 12 kinds of rock mineral was assigned to T’: is smaller than βC, the pixel point will beπ ′ marked boundary point; and = {π·1 ,D2 , . . . as , π·π , a . . . texture , π·π }, (10) in the contrary case, it is not going marked. Theintexture and from cleavage canCbe outlined in the image where,toDibe is the max distance each category any point’s value to the mean colors’ RGB by combining values;color and m is the number of mineral classes. Finally, the maximum valuemineral in T’ was assigned to Tcan 1. the brightness and change. The recognition accuracy of rock images be increased When T1 is smaller than π₯πΆ, the pixel point will be marked as a texture boundary point; and in the with the extraction of texture and the cleavage of rock minerals. contrary case, it is not going to be marked. The texture and cleavage can be outlined in the image by In a word,combining the feature of rock images can be extracted by calculating the brightness and mineral color change. The recognition accuracy of rock mineral images can be the changes in increased the values. extraction of the cleavage of rock features minerals. of rock minerals were extracted brightness values and with color Intexture this and research, texture In a word, the feature of rock mineral images can be extracted by calculating the changes in to strengthen the texture features the Inoriginal rock mineral by marking and outlining the brightness values and colorof values. this research, texture featuresimages of rock minerals were extracted to strengthen features of the original rock mineral images by Inception-v3 marking and outlining the and the color texture boundary points,theastexture shown in Figure 5. After extraction, model texture boundary points, as shown in Figure 5. After extraction, Inception-v3 model and the color model will be trained. model will be trained. (a) (b) (c) (d) Figure 5. Cont. Minerals 2019, 9, 516 9 of 17 Minerals 2019, 9, x FOR PEER REVIEW 9 of 16 (e) (f) Figure 5. Samples of texture extraction for rock mineral images. (a) Cinnabar; (b) texture extraction of Figure 5. Samples of texture extraction for rock mineral images. (a) Cinnabar; (b) texture extraction of cinnabar; (c) calcite; (d) texture extraction of calcite; (e) malachite; (f) texture extraction of malachite. cinnabar; (c) calcite; (d) texture extraction of calcite; (e) malachite; (f) texture extraction of malachite. 3.2. Color Model of Rock Mineral 3.2. Color Model of Rock Mineral The colors of the 12 types of rock minerals considered here are shown in Table 1. A computer generated the color through combining red, green, and blue to build RGB color space, in which red, The colorsgreen, of the of rock considered here (226 are ×shown Table 1. A computer and12 bluetypes have values in [0, minerals 255]. As a result, there are 1,677,216 256 × 256 in = 1,677,216) colors in RGB color space. Any change of the value in R, G or B will lead to color variation. It is in which red, generated the color through combining red, green, and blue to build RGB color space, possible to implement recognition for rock mineral images based on the feature of color. The rock green, and bluemineral havelabel values [0, 255]. a result, there are 1,677,216 (226 × 256 × 256 = 1,677,216) can bein determined by As the following equation: 2 2 2 (11) variation. It is πji = πji 2 =of (π the R ) + (πΊ Gππ )R, + (π΅ , will lead to color colors in RGB color space. Any change value Gπ -B orππ )B π ji π - in where, ρji are the Euclid Distance. Ri, Gi and Bi are the mean values of RGB value in rock mineral possible to implement recognition for rock mineral images based on the feature of color. The rock images. Rji, Gji and Bji are the mean values of the RGB value of the ith color in the jth rock mineral. mineral label can be determined by the equation: According to the color distance Sji,following the rock mineral ranked top 6 are recognized, and shown in the result. See Equations (11–12). 2πππ{πππ } 2 2 (π =G0,1, . . .+ , π), Bi − B ji , S ji = ρ ji 2 =πcorπ Ri ′π−= 1R−ji∑π +πππ{π Gi }− ji π=1 (12) ππ (11) πcorπ ′π πcorππ = π . (13) ∑ πcorπ where, ρji are the Euclid Distance. Ri , Gi and B are the ′mean values of RGB value in rock mineral π π=1 i Here a color model for rock mineral will be established based on the colors of rock mineral. The images. Rji , Gji and Bji are the mean values of the RGB value of the ith color in the jth rock mineral. color model will be used to assist inception-v3 model in identifying rock minerals. During According to the color distance rockthe mineral ranked 6 are and identification of the typeSji of, athe mineral, inception-v3 model top identifies therecognized, mineral first, then a shown in the six-size set of identification result. See Equations (11) and (12).results is passed to the color model. The final result is given by the color model after matching the color of the mineral with the results from inception-v3 model. n o min S ji n o (i = 0, 1, . . . , k), Score0j = 1 − P n 4.1. Image Processing j=1 min S ji 4. Experiment and Results The dataset [42] consists of 4178 images of 12 kinds of elaborate rock minerals. The information about rock mineral images in each category is shown in0 Table 2. Twenty images were selected Score j randomly in each category as a test dataset, and the remaining images were used to retrain the Score j = Pn 0. Inception-v3 model. j=1 Score j (12) (13) Table 2. The types and numbers of rock mineral images. Here a color model for rock mineral will be established based on the colors of rock mineral. Rock Minerals Number Rock Minerals Number The color model will be used to assist inception-v3 model rock minerals. During Cinnabar 327 Hematite in identifying 324 Aquamarine 380 Calcite 451 identification of the type of Molybdenite a mineral, the inception-v3 model identifies the mineral first, then a six-size 316 Stibnite 423 Malachite 335 Azurite 323 set of identification results is passed to the color model. Augite The final result171 is given by the color model Cassiterite 378 Magnetite Gyp Inception-v3 415model. after matching the color of the mineral with 335 the results from 4. Experiment and Results 4.1. Image Processing The dataset [42] consists of 4178 images of 12 kinds of elaborate rock minerals. The information about rock mineral images in each category is shown in Table 2. Twenty images were selected randomly in each category as a test dataset, and the remaining images were used to retrain the Inception-v3 model. In establishing the model, numerous factors including the number and clarity of rock mineral images, background noise and differences between features of minerals all influence its accuracy. Generally, accuracy increases as the number of rock mineral images increases. However, the imbalance among different types in quantity may lead to low accuracy. Therefore, we collected the images and ensured that the total number of each category was at least 150. The proportion of rock minerals in an image was then adjusted to at least 80%. Finally, the noise of the image, such as complex backgrounds and labels, was removed to give the images the same size. In the process of training and the color Minerals 2019, 9, 516 10 of 17 Minerals 2019, x FOR PEER REVIEW 10 of 16 model, the 9, texture features of all images were extracted and strengthened. All images were also translated into the RGB matrix. Then, the brightness values and a matrix containing the βC values of In establishing the model, numerous factors including the number and clarity of rock mineral the points of each image were calculated by Equation (3) and Equations (7)–(9), respectively. If they images, background noise and differences between features of minerals all influence its accuracy. were determined to be boundary points, the points were marked and outlined. Some pre-processed Generally, accuracy increases as the number of rock mineral images increases. However, the results are shown in Figure 5. Then, the Inception-v3 model was trained. imbalance among different types in quantity may lead to low accuracy. Therefore, we collected the images and ensured thatTable the total number of each category at least 150. The proportion of rock 2. The types and numbers of rockwas mineral images. minerals in an image was then adjusted to at least 80%. Finally, the noise of the image, such as Rock complex backgrounds and labels, was removed same size. In the process of Rock Minerals Number to give the images theNumber Minerals training and the color model, the texture features of all images were extracted and strengthened. All Cinnabarinto the RGB 327 matrix. Hematite 324 values and a matrix images were also translated Then, the brightness Aquamarine 380 Calcite 451 containing the β³C values of the points of each image were calculated by Equation (3) and Equations Molybdenite 316 Stibnite 423 (7)–(9), respectively. IfMalachite they were determined points were marked and 335 to be boundary Azurite points, the 323 outlined. Some pre-processed shown in Figure Cassiteriteresults are 378 Augite5. Then, the 171inception-v3 model was Magnetite 335 Gyp 415 trained. 4.2. Model 4.2. Model Establishment Establishment 4.2.1. Model Training The processed images images were were used used as as the the raw raw data. data. The The input input image image size size was was set setto to299 299×× 299. 299. All the input images were pre-processed for training. There were three input channels for length, width, width, and color. The training steps were set to 20,000, and the learning rate was set to to 0.01. 0.01. The prediction results true label in in each step, so that the training and validation accuracy were results were werecompared comparedwith withthe the true label each step, so that the training and validation accuracy both to update the weights in the model. The true label the class name thefrom dataset. were calculated both calculated to update the weights in the model. The trueis label is the classfrom name the The dataset. primary The measures primary measures used to evaluate used to training evaluateeffectiveness training effectiveness are the training are the accuracy, trainingvalidation accuracy, validationand accuracy and train cross-entropy, shown accuracy train cross-entropy, as shown inasFigure 6.in Figure 6. Figure 6. Training process with Inception-v3. The have an an influence To get The lighting lighting conditions conditions have influence on on the the RGB RGB value value of of the the images. images. To get aa better better color color model, which is optimized by the coefficient, was used model, the theself-adaption self-adaptionK-means K-meansalgorithm, algorithm, which is optimized by silhouette the silhouette coefficient, was to train model based on 10 image forslices each category differentinlighting condition a size used tothe train the model based on 10slices image for each in category different lighting(with condition of about 300of× about 300). There wereThere aboutwere 900,000 points in each category. points were then were used (with a size 300 × 300). about 900,000 points in eachThese category. These points to train the color model K-means. the model was trained, the silhouette coefficient and the then used to train the by color modelWhen by K-means. When the model was trained, the silhouette maximum distance, which is the D in Equation (10), of each category from point to the mean i coefficient and the maximum distance, which is the Di in Equation (10), of each category from value point to the mean value were calculated. The training process ended until the silhouette coefficient was maximum. The training results of the color model is shown in Table 3. Additionally, Figure 7 shows samples of rock mineral images which were trained in color model. Minerals 2019, 9, 516 11 of 17 were calculated. The training process ended until the silhouette coefficient was maximum. The training results of the color model is shown in Table 3. Additionally, Figure 7 shows samples of rock mineral images which were trained in color model. Table 3. Training results of color model. Colors (R, G, B) Rock Minerals Cinnabar Hematite Molybdenite Calcite Cassiterite Magnetite Malachite Azurite Aquamarine Augite Stibnite Gyp Minerals 2019, 9, x FOR PEER REVIEW (a) Color1 Color2 Color3 (142,79,69) (168,96,80) (121,125,126) (193,199,193) (44,47,53) (36,33,31) (88,168,129) (0,19,151) (0,95,56) (82,80,82) (92,104,115) (161,161,163) (105,36,31) (127,157,138) (93,96,98) (175,177,167) (67,144,104) (44,77,201) (35,159,112) (122,120,124) (230,220,192) (184,107,102) (50,108,75) - (b) 11 of 16 (c) Figure7.7.Training Trainingsamples samplesofofcolor colormodel. model.(a) (a)Cinnabar; Cinnabar;(b) (b)calcite; calcite;(c)(c)aquamarine. aquamarine. Figure To eliminate the influence of different lightingresults condition, wemodel. trained numerous images in different Table 3. Training of color lighting conditions. Due to the difference in the reflection of rock minerals, some mineral’s RGB values Colors (R, G, B) minerals while some do not, therefore, the number of RGB value kinds of these minerals changedRock significantly, Color1 Color2 Color3 in the colorCinnabar model is different. (142,79,69) (105,36,31) (184,107,102) (168,96,80) (127,157,138) WhenHematite training the SVM model and RF model, the features of raw images were extracted by the Molybdenite (121,125,126) (93,96,98) HOG method. Then all features were resized to 256 × 256. The parameters of SVM and RF were Calcite (193,199,193) (175,177,167) determined by the grid search algorithm. Finally, we found that the penalty coefficient and -gamma of Cassiterite (44,47,53) SVM are 0.9 and 0.09. The number of(36,33,31) trees of RF is 180. The parameters of RF are the default. Magnetite Malachite (88,168,129) (67,144,104) (50,108,75) (0,19,151) (44,77,201) Aquamarine (0,95,56) (35,159,112) First, two Inception-v3 models were retrained from both raw images and texture feature-extraction Augite (82,80,82) (122,120,124) Stibnite (92,104,115) - with the images. Then, the retrained model from the texture feature extraction images was combined Gyp (161,161,163) (230,220,192) color model to create a comprehensive model. The comparison was made among the three -models to Azurite 4.2.2. Model Test and Results find one with the highest accuracy. To ensure the accuracy was more convincing, 440 images were eliminate the influence of different condition, we from trained images used toTotest these models, and the testing images lighting were randomly selected thenumerous image dataset of thein different lighting Due are to the difference in the reflection of rock some mineral’s rock mineral. Theseconditions. testing images going to be used as the input data for minerals, a comprehensive model RGB values changed significantly, while some do not, therefore, the number of RGB value kinds and Inception-v3 model trained with or without feature extraction images. Meanwhile, the images areof these minerals in the color model is different. going to be automatically processed when testing the comprehensive model and Inception-v3 model the SVM model RF model, theThe features of raw were extracted by the trainedWhen with training feature extraction imagesand by the program. test result is images shown in Table 4. HOG method. Then all features were resized to 256 × 256. The parameters of SVM and RF were determined by the grid search algorithm. Finally, we found that the penalty coefficient and gamma of SVM are 0.9 and 0.09. The number of trees of RF is 180. The parameters of RF are the default. 4.2.2. Model Test and Results Minerals 2019, 9, 516 Minerals 2019, 9, x FOR PEER REVIEW 12 of 17 12 of 16 Table five models. models. Table4. 4. Results Results of of five Validation TestTest Accuracy Accuracy Validation Accuracy Accuracy Top-1 Top-1 Top-3 Top-3 Models Models Model onimages raw images Model basedbased on raw 73.1% Model based on texture extraction imagesimages 77.4% Model based on texture extraction Comprehensive modelmodel 77.4% Comprehensive SVM-HOG SVM-HOG RF-HOG 73.1% 64.1% 77.4% 67.5% 77.4% 74.2% 64.1% 96.0% 96.0% 98.3% 67.5% 98.3% 99.0% 74.2% 99.0% 33.6% 33.6% 30.4% 30.4% RF-HOG ToToensure the precision precisionrate, rate,recall recallrate rateand and ensurethe themodels modelsininTable Table 44 are are statistically statistically feasible, feasible, the F1-measure value used to validate models. In Table the retrained inception-v3 model F1-measure value areare used to validate the the models. In Table 4, the4,retrained Inception-v3 model based on rawwas images was able to reach aaccuracy validation accuracy ofthe 73.1%, top-1 and top-3 onbased raw images able to reach a validation of 73.1%, and top-1and andthe top-3 accuracies were accuracies were respectively. 64.1% and 96.0%, respectively. test accuracies rate) SVM-HOG 64.1% and 96.0%, The test accuraciesThe (precision rate) of(precision SVM-HOG andofRF-HOG are and 32.8% RF-HOG are 32.8% andThe 31.2%, respectively. The accuracy is toothat low, which thatnot theeffective. HOG and 31.2%, respectively. accuracy is too low, which indicates the HOGindicates features are features are not effective. On the other side, the comprehensive model has a significant performance, On the other side, the comprehensive model has a significant performance, which proves it can extract which proves it can The extract texture features. The test accuracies of each category (recall rate) ofin the texture features. testthe accuracies of each category (recall rate) of these models are shown these8, models areF1-measure shown in Figure F1-measure value of the5.models shown 5. Asof Figure and the value8,ofand thethe models is shown in Table As weiscan see, in theTable accuracy we can see, the accuracy of each category of the comprehensive model is much higher than that of each category of the comprehensive model is much higher than that of the rest models, which indicates the rest models, which indicates that the comprehensive model is suitable for rock mineral that the comprehensive model is suitable for rock mineral identification. The F1-measure value of the identification. The F1-measure value of the comprehensive model indicates that the model is comprehensive model indicates that the model is statistically feasible. statistically feasible. Accuracy of each category of test Gyp Calcite 70.8% 31.2% 23.0% Comprehensive model HOG-SVM HOG-RF 66.7% 5.0% 7.2% Cassiterite 87.5% 26.6% 23.8% Stibnite 55.8% 9.1% 10.7% Molybdenite 64.2% 46.6% 43.4% Magnetite 21.7% Augite 18.6% 21.7% Aquamarine 59.7% 38.9% 66.7% 23.9% Azurite 75.0% 32.8% 32.3% Malachite 91.7% 38.4% 91.7% 39.1% Hematite 45.8% 43.4% Cinnabar 76.7% 57.3% 87.5% 43.9% 44.1% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% Figure 8. 8. Test Test accuracies accuracies of Figure of each eachcategory. category. Table 5. Validation parameters of models. Models Precision Rate Recall Rate F1-measure Value Comprehensive model SVM-HOG RF-HOG 74.2% 33.6% 30.4% 77.5% 28.5% 31.2% 0.758 0.308 0.308 100% Table 5. Validation parameters of models. Models Comprehensive model Minerals 2019,SVM-HOG 9, 516 RF-HOG Precision rate 74.2% 33.6% 30.4% Recall rate 77.5% 28.5% 31.2% F1-measure value 0.758 0.308 13 of 17 0.308 ItIt revealed accuracy of the model, with the texture feature feature extraction images, revealedthat thatthethe accuracy of Inception-v3 the inception-v3 model, with the texture extraction is higher than that with the raw images in Table 4. The Inception-v3 model with the texture images, is higher than that with the raw images in Table 4. The inception-v3 model with the feature texture extraction images reached a higher accuracy, which means the texture is a significant feature for rock feature extraction images reached a higher accuracy, which means the texture is a significant feature mineral identification. The texture extraction makes it easier to distinguish different rock minerals, for rock mineral identification. The texture extraction makes it easier to distinguish different rock which improves accuracy the retrained minerals, which the improves theofaccuracy of themodel. retrained model. The The comprehensive comprehensive model model achieved achieved the the highest highest accuracy accuracy in in the the three threemodels. models. Color Color properties properties could increase identification accuracy greatly. From the test accuracy of the comprehensive could increase identification accuracy greatly. From the test accuracy of the comprehensive model, model, we can see that the top-1 accuracy is 74.2%, and the top-3 accuracy is 99.0%. By combining the retrained we can see that the top-1 accuracy is 74.2%, and the top-3 accuracy is 99.0%. By combining the Inception-v3 model andmodel the color a comprehensive identification could be made different retrained Inception-v3 andmodel, the color model, a comprehensive identification couldfor be made for rock minerals. The texture and color features were used the most during identification. different rock minerals. The texture and color features were used the most during identification. Following Following are are some some recognition recognition samples samples in in Figure Figure 9. 9. As As you you can can see, see, these these models models sometimes sometimes achieved different results. When the texture features are similar, and the colors are different. For example, achieved different results. When the texture features are similar, and the colors are different. For magnetite’s cleavage is similar to the azurite’s, but their colors are not similar, the color model could example, magnetite’s cleavage is similar to the azurite’s, but their colors are not similar, the color identify them successfully. the minerals’ is similar, color but has cleavage such as model could identify themWhen successfully. Whencolor the minerals’ is different similar, but has different calcite and gym, the Inception-v3 model trained with feature extraction images could identify them cleavage such as calcite and gym, the Inception-v3 model trained with feature extraction images successfully. Therefore, the comprehensive model the advantages of both model and the could identify them successfully. Therefore, the has comprehensive model has the the color advantages of both Inception-v3 model trained with feature extraction images. Occasionally, however, the comprehensive the color model and the Inception-v3 model trained with feature extraction images. Occasionally, model is going to be wrong when theisminerals’ feature is vague, or the color is abnormal. however, the comprehensive model going totexture be wrong when the minerals’ texture feature is This condition will confuse the comprehensive model, as shown in Figure 10. It is magnetite vague, or the color is abnormal. This condition will confuse the comprehensive model, as shown in in Figure of the blue lighting vague feature, these all received an Figure 10, 10. but It isbecause magnetite in Figure 10, butand because oftexture the blue lighting andmodels vague texture feature, incorrect result. these models all received an incorrect result. (a) True (b) True (c) True (d) True (e) False (f) True Figure 9. Cont. Minerals 2019, 9, 516 14 of 17 Minerals Minerals2019, 2019,9,9,x xFOR FORPEER PEERREVIEW REVIEW 1414ofof1616 (g) (g)True True (h) (h)False False (j)(j)False False Figure are from the comprehensive model, (d–f) are from the Figure9.9.Samples Samplesof ofrecognition recognitionresults. (a–c) are from the comprehensive model, (d–f) are from the Figure Samples of recognition results.(a–c) (a–c) are from the comprehensive model, (d–f) are from Inception-v3 model trained with feature extraction images, Inception-v3 model trained with feature extraction images, arefrom fromthe theInception-v3 Inception-v3model model the Inception-v3 model trained with feature extraction images,(g–j) (g–j)are are from the without feature extraction. without feature extraction. without feature extraction. (a) (a) (b) (b) (c)(c) Figure True; (b) true; (c)(c) true. Figure10. 10.Identification Identificationresults resultsfrom fromthese thesethree threemodels. models.(a) (a)True; True; (b) true; true. (b) true; (c) true. 5. Conclusions 5.5.Conclusions Conclusions In established. InInthis this research, three models based on Inception-v3 model were established. The deep learning thisresearch, research,three threemodels modelsbased basedon onInception-v3 Inception-v3model modelwere were established.The Thedeep deeplearning learning model, coupled with color model reached top-1 and top-3 accuracies of 74.2% and 99.0%, respectively. model, model, coupled coupled with with color color model model reached reached top-1 top-1 and and top-3 top-3 accuracies accuracies ofof 74.2% 74.2% and and 99.0%, 99.0%, The retrainedThe model usingmodel raw images reached top-1 and top-1 top-3 accuracies of 64.1%ofof and 96.0%, respectively. retrained using images reached 64.1% and respectively. The retrained model usingraw raw images reached top-1and andtop-3 top-3accuracies accuracies 64.1% and respectively. The retrained model using texture extraction images reached top-1 and top-3 accuracies 96.0%, 96.0%,respectively. respectively.The Theretrained retrainedmodel modelusing usingtexture textureextraction extractionimages imagesreached reachedtop-1 top-1and andtop-3 top-3 of 67.5% and 98.3%, respectively. The comparison ofcomparison the three models indicates that the comprehensive accuracies ofof 67.5% and The ofofthe three indicates that accuracies 67.5% and98.3%, 98.3%,respectively. respectively. The comparison the threemodels models indicates thatthe the model is the best of all. The SVM-HOG model achieved a validation accuracy of 32.8%. The RF-HOG comprehensive model is the best of all. The SVM-HOG model achieved a validation accuracy of comprehensive model is the best of all. The SVM-HOG model achieved a validation accuracy of model a validation of The accuracy results indicate that The the learning model 32.8%. The model achieved a a31.2%. validation ofof31.2%. results indicate that the 32.8%.achieved TheRF-HOG RF-HOG modelaccuracy achieved validation accuracy 31.2%. Thedeep results indicate thatcan the extract effective image features. The comparison between the traditional models and the deep learning deep deeplearning learningmodel modelcan canextract extracteffective effectiveimage imagefeatures. features.The Thecomparison comparisonbetween betweenthe thetraditional traditional model shows that the deep learning models are much more effective than theare traditional ones. models and learning model shows that the much effective models andthe thedeep deep learning model shows that thedeep deeplearning learningmodels models are muchmore more effective The deep learning algorithm provides a new idea for rock mineral identification. The clustering than thanthe thetraditional traditionalones. ones. algorithm could also improve the performance of the deep learning model. Therefore, the combination The algorithm provides idea for rock identification. The clustering Thedeep deeplearning learning algorithm providesa anew new idea for rockmineral mineral identification. The clustering of a deep learning algorithm and clustering algorithm is an effective way for rock mineral identification. algorithm algorithm could could also also improve improve the the performance performance ofof the the deep deep learning learning model. model. Therefore, Therefore, the the Meanwhile, the features and texture features of rock algorithm minerals significant for combination ofofacolor learning algorithm and isisan way combination adeep deep learning algorithm andclustering clustering algorithmare aneffective effectiveproperties wayfor forrock rock identification. The identification accuracy ofcolor Inception-v3 model could features befeatures improved greatly by texture mineral Meanwhile, the features ofofrock are mineralidentification. identification. Meanwhile, thecolor featuresand andtexture texture rockminerals minerals are and color extraction. significant significantproperties propertiesfor foridentification. identification.The Theidentification identificationaccuracy accuracyofofInception-v3 Inception-v3model modelcould couldbebe Furthermore, the elaborate mineral images, with clear mineral features, are adopted. If the improved greatly and improved greatlyby bytexture texture andcolor colorextraction. extraction. features, such as cleavage and luster, can beimages, extracted, theclear comprehensive model can be further tested Furthermore, the mineral are adopted. IfIfthe Furthermore, theelaborate elaborate mineral images,with with clearmineral mineralfeatures, features, are adopted. the for mineral specimen identification, even for field survey. features, features,such suchasascleavage cleavageand andluster, luster,can canbebeextracted, extracted,the thecomprehensive comprehensivemodel modelcan canbebefurther further tested testedfor formineral mineralspecimen specimenidentification, identification,even evenfor forfield fieldsurvey. survey. Author AuthorContributions: Contributions:Modules Modulesdesign designand anddevelopment, development,C.L.; C.L.;Overall Overallframework frameworkdesign, design,M.L.; M.L.;Data Data collection collectionand andapplication applicationtest, test,Y.Z. Y.Z.and andY.Z.; Y.Z.;Algorithm Algorithmand andmodel modelimprovement, improvement,S.H. S.H. Minerals 2019, 9, 516 15 of 17 Author Contributions: Modules design and development, C.L.; Overall framework design, M.L.; Data collection and application test, Y.Z. and Y.Z.; Algorithm and model improvement, S.H. Funding: This research was funded by the National Natural Science Foundation for Excellent Young Scientists of China (Grant no. 51622904), the Tianjin Science Foundation for Distinguished Young Scientists of China (Grant no. 17JCJQJC44000) and the National Natural Science Foundation for Innovative Research Groups of China (Grant no. 51621092). Conflicts of Interest: We declare that we have no financial and personal relationships with other people or organizations that can inappropriately influence our work, there is no professional or other personal interest of any nature or kind in any product, service or company that could be construed as influencing the position presented in, or the review of, the manuscript entitled. References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. Yeshi, K.; Wangdi, T.; Qusar, N.; Nettles, J.; Craig, S.R.; Schrempf, M.; Wangchuk, P. Geopharmaceuticals of Himalayan Sowa Rigpa medicine: Ethnopharmacological uses, mineral diversity, chemical identification and current utilization in Bhutan. J. Ethnopharmacol. 2018, 223, 99–112. [CrossRef] [PubMed] Rustom, L.E.; Poellmann, M.J.; Johnson, A.J.W. Mineralization in micropores of calcium phosphate scaffolds. Acta Biomater. 2019, 83, 435–455. [CrossRef] [PubMed] Laura, B.; Nina, E.; Mona, W.M.; Udo, Z.; Andò Sergio Merete, M.; Reidar, I.K. Quick, easy, and economic mineralogical studies of flooded chalk for eor experiments using raman spectroscopy. Minerals 2018, 8, 221. Ramil, A.; López, A.J.; Pozo-Antonio, J.S.; Rivas, T. A computer vision system for identification of granite-forming minerals based on RGB data and artificial neural networks. Measurement 2017, 117, 90–95. [CrossRef] Sadeghi, B.; Madani, N.; Carranza, E.J.M. Combination of geostatistical simulation and fractal modeling for mineral resource classification. J. Geochem. Explor. 2015, 149, 59–73. [CrossRef] Li, R.; Albert, N.N.; Yun, M.; Meng, Y.S.; Du, H. Geological and Geochemical Characteristics of the Archean Basement-Hosted Gold Deposit in Pinglidian, Jiaodong Peninsula, Eastern China: Constraints on Auriferous Quartz-Vein Exploration. Minerals 2019, 9, 62. [CrossRef] Rajendran, S.; Nasir, S. ASTER capability in mapping of mineral resources of arid region: A review on mapping of mineral resources of the Sultanate of Oman. Ore Geol. Rev. 2019, 108, 33–53. [CrossRef] Shi, B.; Liu, J. Nonlinear metric learning for kNN and SVMs through geometric transformations. Neurocomputing 2018, 318, 18–29. [CrossRef] Sachindra, D.; Ahmed, K.; Rashid, M.M.; Shahid, S.; Perera, B. Statistical downscaling of precipitation using machine learning techniques. Atmos. Res. 2018, 212, 240–258. [CrossRef] Vassilev, S.V.; Vassileva, C.G. A new approach for the combined chemical and mineral classification of the inorganic matter in coal. 1. Chemical and mineral classification systems. Fuel 2009, 88, 235–245. [CrossRef] Vassilev, S.V.; Vassileva, C.G. A new approach for the classification of coal fly ashes based on their origin, composition, properties, and behaviour. Fuel 2007, 86, 1490–1512. [CrossRef] Zaini, N.; Van Der Meer, F.; Van Der Werff, H. Determination of Carbonate Rock Chemistry Using Laboratory-Based Hyperspectral Imagery. Remote. Sens. 2014, 6, 4149–4172. [CrossRef] Adep, R.N.; Shetty, A.; Ramesh, H. EXhype: A tool for mineral classification using hyperspectral data. ISPRS J. Photogramm. Remote. Sens. 2017, 124, 106–118. [CrossRef] Othman, A.A.; Gloaguen, R. Integration of spectral spatial and morphometric data into lithological. J. Asian Earth Sci. 2017, 146, 90–102. [CrossRef] Li, Y.; Chen, C.; Fang, R.; Yi, L. Accuracy enhancement of high-rate GNSS positions using a complete ensemble empirical mode decomposition-based multiscale multiway PCA. J. Asian Earth Sci. 2018, 169, 67–78. [CrossRef] Wang, W.; Chen, L. Flotation Bubble Delineation Based on Harris Corner Detection and Local Gray Value Minima. Minerals 2015, 5, 142–163. [CrossRef] Chen, J.; Chen, Y.; Wang, Q. Synthetic Informational Mineral Resource Prediction: Case Study in Chifeng Region, Inner Mongolia, China. Earth Sci. Front. 2008, 15, 18–26. [CrossRef] Shardt, Y.A.; Brooks, K. Automated System Identification in Mineral Processing Industries: A Case Study using the Zinc Flotation Cell. IFAC-Papers OnLine 2018, 51, 132–137. [CrossRef] Minerals 2019, 9, 516 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 32. 33. 34. 35. 36. 37. 38. 39. 40. 41. 16 of 17 SΜlipek, B.; MΕynarczuk, M. Application of pattern recognition methods to automatic identification of microscopic images of rocks registered under different polarization and lighting conditions. Geol. Geophys. Environ. 2013, 39, 373. [CrossRef] MΕynarczuk, M.; Górszczyk, A.; SΜlipek, B. The application of pattern recognition in the automatic classification of microscopic rock images. Comput. Geosci. 2013, 60, 126–133. [CrossRef] Shu, L.; McIsaac, K.; Osinski, G.R.; Francis, R. Unsupervised feature learning for autonomous rock image classification. Comput. Geosci. 2017, 106, 10–17. [CrossRef] Coates, A.; Ng, A.Y.; Lee, H. An analysis of single-layer networks in unsupervised feature learning. In Proceedings of the 14th International Conference on Artificial Intelligence and Statistics, Fort Lauderdale, FL, USA, 11–13 April 2011; pp. 215–223. Raina, R.; Battle, A.; Lee, H.; Packer, B.; Ng, A.Y. Self-taught learning: Transfer learning from unlabeled data. In Proceedings of the 24th International Conference on Machine Learning, Corvallis, OR, USA, 20–24 June 2007; pp. 759–766. Aligholi, S.; Lashkaripour, G.R.; Khajavi, R.; Razmara, M. Automatic mineral identification using color tracking. Pattern Recognit. 2016, 65, 164–174. [CrossRef] Li, N.; Hao, H.; Gu, Q.; Wang, D.; Hu, X. A transfer learning method for automatic identification of sandstone microscopic images. Comput. Geosci. 2017, 103, 111–121. [CrossRef] Google Deep Learning. Available online: http://deeplearning.net/tag/google/ (accessed on 21 March 2019). Zhang, Y.; Li, M.C.; Han, S. Automatic identification and classification in lithology based on deep learning in rock images. Acta Petrol. Sin. 2018, 34, 333–342. Kitzig, M.C.; Kepic, A.; Grant, A. Near Real-Time Classification of Iron Ore Lithology by Applying Fuzzy Inference Systems to Petrophysical Downhole Data. Minerals 2018, 8, 276. [CrossRef] Xiong, Y.; Zuo, R.; Carranza, E.J.M. Mapping mineral prospectivity through big data analytics and a deep learning algorithm. Ore Geol. Rev. 2018, 102, 811–817. [CrossRef] Zuo, R.; Xiong, Y.; Wang, J.; Carranza, E.J.M. Deep learning and its application in geochemical mapping. Earth-Sci. Rev. 2019, 192, 1–14. [CrossRef] Iglesias, J.C.; Álvarez; Santos, R.B.M.; Paciornik, S. Deep learning discrimination of quartz and resin in optical microscopy images of minerals. Miner. Eng. 2019, 138, 79–85. [CrossRef] Hinton, G.E.; Salakhutdinov, R.R. Reducing the Dimensionality of Data with Neural Networks. Science 2006, 313, 504–507. [CrossRef] Hinton, G.; Mohamed, A.-R.; Jaitly, N.; Vanhoucke, V.; Kingsbury, B.; Deng, L.; Yu, D.; Dahl, G.; Senior, A.; Nguyen, P.; et al. Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups. IEEE Signal Process. Mag. 2012, 29, 82–97. [CrossRef] Gong, M.; Yang, H.; Zhang, P. Feature learning and change feature classification based on deep learning for ternary change detection in SAR images. ISPRS J. Photogramm. Remote Sens. 2017, 129, 212–225. [CrossRef] Pham, C.C.; Jeon, J.W. Robust object proposals re-ranking for object detection in autonomous driving using convolutional neural networks. Signal Process. Image Commun. 2017, 53, 110–122. [CrossRef] Esteva, A.; Kuprel, B.; Novoa, R.A.; Ko, J.; Swetter, S.M.; Blau, H.M.; Thrun, S. Dermatologist-level classification of skin cancer with deep neural networks. Nature 2017, 542, 115–118. [CrossRef] Szegedy, C.; Vanhoucke, V.; Ioffe, S.; Shlens, J.; Wojna, Z. Rethinking the Inception Architecture for Computer Vision. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 26 June–1 July 2016; pp. 2818–2826. Qureshi, A.S.; Khan, A.; Zameer, A.; Usman, A. Wind power prediction using deep neural network based meta regression and transfer learning. Appl. Soft Comput. 2017, 58, 742–755. [CrossRef] Xu, G.; Zhu, X.; Fu, D.; Dong, J.; Xiao, X. Automatic land cover classification of geo-tagged field photos by deep learning. Environ. Model. Softw. 2017, 91, 127–134. [CrossRef] Xu, B.; Shen, S.; Shen, F.; Zhao, J. Locally linear SVMs based on boundary anchor points encoding. Neural Netw. 2019, 117, 274–284. [CrossRef] Pellant, C. Rocks and Minerals—Smithsonian Handbook; DK Publisher: New York, NY, USA, 2002. Minerals 2019, 9, 516 42. 43. 17 of 17 The Geological Museum of China. Available online: http://www.gmc.org.cn (accessed on 28 March 2019). International Telecommunication Union. Available online: https://www.itu.int/rec/R-REC-BT.601-7-201103I/en (accessed on 12 March 2019). © 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).