2021 IEEE 16th International Conference on Industrial and Information Systems (ICIIS) | 978-1-6654-2637-4/21/$31.00 ©2021 IEEE | DOI: 10.1109/ICIIS53135.2021.9660698 Machine Learning Based Emotion Level Assessment Lumini Wickremesinghe1 , Dakheela Madanayake1 , Anuradha Karunasena1 , and Pradeepa Samarasinghe1 1 Faculty of Computing, Sri Lanka Institute of Information Technology, Sri Lanka {luminiwickramasinghe@gmail.com,dpmadanayake@gmail.com,anuradha.k@sliit.lk,pradeepa.s@sliit.lk} Abstract—With recent advancements of technology, identification of emotions of humans via facial recognition is done with the application of numerous methods including machine learning and deep learning. In this paper, machine learning techniques are applied for identifying different levels of emotions of individuals for unannotated video clips using Facial Action Coding System. In order to archive the above, first, two methods were experimented to obtain a labeled image data set to train classification models where in the first method, clustering of images was done using Action Units(AU) identified from literature and the emotion levels of the images were determined through the resulted clusters and images are labeled according to the cluster they belonged to. In the second method, the image set is analyzed explicitly to identify AUs contributing to emotions rather than relying on those identified in literature and then the clustering of image set was done using those identified AUs to label the images similar to the first method. The two labeled data sets were used to train classification models with Random Forest, Support Vector Machine and K-Nearest Neighbour algorithms separately.Classification models showed better accuracy with data set produced using the second method. An overall F1 score, accuracy, precision and recall of 87% was obtained for the best classification model which is developed using the Random Forest algorithm to identify levels of emotions. Identifying the AU combinations related to emotions and developing a classification model for identifying levels of emotions are the major contributions of this paper. The results of this research would be especially useful to identify levels of emotions of individuals who are having issues in verbal communication. Index Terms—Facial Action Coding System, Action Units, Emotion levels, Clustering, Classification I. I NTRODUCTION The response of a person to an internal stimuli or an external phenomenon would result in an emotion expression. Identification and analysis of emotions therefore, would guide to understand the reasons behind the responses for the stimuli or the phenomenon. Human emotions are often expressed through a number of verbal and non-verbal cues. Non-verbal cues such as gestures and facial expressions are especially helpful to understand emotions of adults and small children who are having difficulties in expressing their emotions through speech. Facial expressions, speech, physiological signals, gestures and other multi-modal information are used with varying computational techniques to analyze personal emotions [1], [2]. Recently much consideration has been drawn to identify the emotions through facial expressions. One of the most popular tools employed in the context human emotion recognition is the Facial Action Coding 978-1-6654-2637-4/21/$31.00 ©2021 IEEE System (FACS) [3]. The FACS is a human-observerbased system designed to describe subtle changes in facial features. Action Units(AUs) are the fundamental actions of individual muscles or groups of muscles that are anatomically related to contraction or relaxation of specific facial muscles. Facial AUs provide an important cue for facial expression recognition. FACS consists of 44 AUs, including those for head and eye positions. Recently much emphasis has been paid on developing Facial Emotion Expression (FEE) analysis systems based on images [4]– [6] as well as videos [7]–[9], using different techniques such as machine learning and deep learning [10], [11]. Though much late research has been on Deep Neural Network (DNN) based systems, as such research cannot be mapped to clinical justification, still FACS based analysis is preferred for developing FEE systems [12]. When considering AU based FEE analysis systems one of the most important and fundamental features of FACSbased emotion recognition is to identify and display the contribution of each activated AU to a particular emotion. In analyzing the relationships between AUs and facial emotions, different techniques have been used such as using human specialists [13], [14] and Specific Affect Coding System (SPAFF) [15]. In SPAFF, an observational research was performed to explore upper and lower facial AUs and used to illustrate a set of frequently used facial expressions. For example, AUs 1, 2, 5, 6, 12, 23, 24 and 25 were used to indicate the enthusiasm emotion while AUs 1, 6, 15 and 17 were used individually or as combinations to differentiate sad from other emotions. The study presented the mapping of AUs with a total of 5 positive facial expressions and 12 negative expressions and presented a logical guide for the recognition of emotional facial expressions. Moreover, classification techniques were also popularly used to map AUs with the emotion expressions in existing literature. For example, [16] used Support Vector Machine (SVM), Extreme Gradient Boosting (Xgboost) and DNN along with Min-max Normalization in their study. Their work provided the mapping of the AU combinations to seven emotions where AUs 6, 7, 12 and 25 were the most prominent combination of AUs for the happy emotion, while AUs 1, 4, 15 and 17 were for the sad and AUs 1, 2, 5, 25 and 26 were the most related for the surprise expressions respectively. Apart from these mentioned techniques, statistical approaches were also used in deriving AU-Emotion relationships. In [17], relationships of AUs to expressions were obtained using a form of relation matrix which was derived using a concept called discriminative power statistical 289 Authorized licensed use limited to: SLIIT - Sri Lanka Institute of Information Technology. Downloaded on January 23,2022 at 15:11:50 UTC from IEEE Xplore. Restrictions apply. analysis. Results of this research revealed positive and negative relationships in relation to the expressions: as an example, for happy emotion, AUs 6, 7, 12 and 26 showed positive associations and AUs 1, 2, 5 and 9 showed negative associations. Similarly, positive associations of AUs 1, 4, 10, 15, 17 and AUs 1, 2, 5, 26, 27 were shown for sad and surprise emotions respectively. A supervised neural network was then used to determine the emotion type from the derived AUs. Existing studies as given above classifies only the emotion type based on a given video or an image. When videos are considered, most of the existing databases have categorized the videos based on the emotion type, though one emotion type may have a set of images ranging from neutral level to the highest level of emotion intensity. As an example, CK+ database [18] has folders consisting of images from the lowest level to the highest level of happy, sadness, surprise, angry, contempt, disgust and fear emotions. Though the current research has not developed ways to consider the level, identifying the variation of emotion intensity level is crucial for behaviour analysis given that certain disorders such as Autism are early identified by analyzing the level of emotion expressions [19]. In addressing this gap, we develop a novel technique in this research to identify both, the emotion type and the level for a given image. In carrying out this research, we first experimented with the recommended mapping between AU and emotions in the literature. The review given above evidences that though there are many techniques used in mapping AUs to emotion types, there are still inconsistencies between them. Thus we considered a set of AUs which were common for a given emotion in the literature for our first method given in Section II-A. As the outcome of this method did not result in high accuracy, a novel technique explained in Section II-B was developed using Exploratory Factor Analysis (EFA) which outperformed in the classification accuracy. EFA is a statistical technique which helps in reducing large number of indicator variables into limited set of factors based on correlations between variables. The rest of the paper is organized by detailing out the two methodologies we adopted in Section II followed by the results demonstration and analysis of the results in Section III. The analysis is completed by summarizing the key outcomes in Section III-D. II. M ETHODOLOGY For the purpose of this research, two video databases were used where each video shows how facial expression of an individual changes from neutral to an expression of a specific emotion such as happy, sad and surprise. The Extended Cohn-Kanade (CK+) data-set contains 593 video sequences from a total of 123 different subjects, ranging from 18 to 50 years of age with a variety of genders and heritage. Each video shows a facial shift from the neutral expression to a targeted peak expression, recorded at 30 frames per second (FPS). For example, Fig. 1 shows three different levels of happy emotion expressed by an individual. The second data set used in the research is the BAUM1 data set which contains 1184 multi-modal facial video Fig. 1: Levels of Happy Emotions of an Individual clips collected from 31 subjects. The 1184 video clips contain impulsive facial expressions and speech of 13 emotional and mental states [20]. The Sections II-A and II-B explain the process followed in order to develop models in identifying the varying levels of emotions for a given video. A. Method I The first method explained in this section is based on identifying the level of emotion based on the suggested AUs for a given emotion in the literature. Fig. 2: Emotion Classification Process through Method I As per the process shown in Fig. 2, the videos relating to emotions happy, sad and surprise were first segmented to frames resulting in three data sets containing images from lowest to highest emotion levels. AU intensities were then generated with respect to the three data sets using OpenFace 2.0 [21]. OpenFace is a facial behavior analysis toolkit capable of generating a set of numerical data which indicates existence of AUs as well as their respective intensities for facial expressions. Intensities were generated with respect to seventeen AUs numbered 1, 2, 4-7, 9, 10, 12, 14, 15, 17, 20, 23, 25, 26 and 45 representing inner brow raiser, outer brow raiser, brow lowerer, upper lid raiser, cheek raiser, lid tightener, nose wrinkler, upper lip raiser, lip corner puller, dimpler, lip corner depressor, chin raiser, lip stretched, lip tightener, lips part, jaw drop and blink respectively. As shown in Section I, literature shows inconsistent results on AU contribution to a specific emotion. To avoid these inconsistencies, a common set of AUs given in past studies were considered for a specific emotion. For example, the common set for happy emotion consists of AUs {6, 7, 12 and 25} while the common AU group for sad and surprise emotions are {1, 15, 17} and {1, 5, 26} respectively. Considering these common AU sets, the three generated AU intensity data sets for happy, sad and surprise emotions were clustered separately using KMeans clustering algorithm. The purpose of clustering as above is to group image frames showing same emotion levels together and to label them into clusters with the level they belonged to. So that a labeled data set is available for the classification model. Manual analysis of images belonging to resulted clusters revealed that images showing neutral, moderate and peak 290 Authorized licensed use limited to: SLIIT - Sri Lanka Institute of Information Technology. Downloaded on January 23,2022 at 15:11:50 UTC from IEEE Xplore. Restrictions apply. levels of an emotion have been grouped together. Therefore, each image was given a label ’neutral’,’moderate’ and ’peak’ depending on the cluster it belonged to. Labeling the images based on the results of the clustering was preferred over manual labeling of images in a video clip.It is difficult to separate images manually with the presence of only subtle differences between images which could be barely differentiated by human eye and because of that the current method enables more systematic separation of images based on their features to groups. Once the entries corresponding to all image frames were labeled through clustering, a classification model was trained and tested using the labeled data set to classify a given image to the relevant emotion type and level. Different classification algorithms including Support Vector Machine(SVM), Random Forest (RF) and KNN (Knearest neighbour) were used to develop the model and accuracy related to each algorithm was obtained. However, as this method was built based on the past research recommendations and as low accuracy was observed through this model on identifying the levels of emotions, another method detailed in Section II-B was followed. B. Method II The main difference in the method detailed here is automated techniques to identify the contributing AUs and their level of contributions for emotions, rather than relying on the contributing AU recommendations from the literature. Method II for clustering images to groups. In comparison to Method I, performing EFA and t-tests were done in Method II with the aim of improving the clustering of images by selecting the most appropriate AUs to identify a level of an emotion and thereby leading to develop a more reliable labeled data set for classification which could result in identifying levels of emotions of a given image with better accuracy. As discussed above, images showing happy, sad and surprise emotions from the database were clustered using K-Means clustering by only considering AU combinations which have shown variations in intensities with the variations of emotions during the t-test. By using the above method, the images related to each specific emotion were grouped into three clusters and those clusters consisted of images related to neutral (no emotion), peak and moderate levels of that emotion. The images in each cluster were then given the label, neutral, peak or moderate based on the cluster the image belonged to. The above method was used to label all the images of the data set into seven classes, namely, neutral, moderately-happy, peak-happy, moderately-sad, peak-sad, moderately-surprised and peaksurprised. Finally, a classification model was developed using the labeled data set created as above. The training, testing split was 70% to 30%. Classification algorithms RF, SVM and KNN were used on the data and accuracy obtained were compared in Section III. III. R ESULTS AND D ISCUSSION A. Results of Method I Fig. 3: Emotion Classification Process through Method II Similar to Method 1, as shown in Fig. 3, first, the video data were segmented into three sets of images and AU intensities corresponding to those images were generated using the OpenFace. Exploratory Factor Analysis (EFA) technique [22] is then used on the generated AU intensities which resulted in three factors with a clear demarcation of contributing AUs. The analysis of factors with respect to the AUs revealed that the factors corresponded to happy, sad and surprise emotions. Furthermore to EFA, additional tests were performed to explore whether intensities of AUs related to an emotion as found in EFA shows any difference when there are variations of emotion levels (e.g. moderately happy and very happy) in images. This was carried out by performing independent t-tests using image sets drawn from the image data sets used in this research which shows variations of an emotion. The results of the t-tests revealed that while some AUs shows differences in terms of intensities when the level of emotion varies, some others do not. Only those AUs which shows differences were considered in Fig. 4: Classification Report for Random Forest Classifier: Method I As mentioned in Section II-A, clustering of images related to an emotion was done considering AU combinations which are said to be related to the emotion in literature. The images were labeled based on the cluster they belonged to and the resulted labeled data were used for developing a classification model to identify levels of emotions. F1 Score is the average value of recall and precision to evaluate the proposed approach accurately. Multiple Machine Learning(ML) models were used to develop the model and the RF classifier resulted the 291 Authorized licensed use limited to: SLIIT - Sri Lanka Institute of Information Technology. Downloaded on January 23,2022 at 15:11:50 UTC from IEEE Xplore. Restrictions apply. Fig. 5: Confusion Matrix for Random Forest Classifier: Method I highest F1-score which is 76% with the train:test split of 70:30. Classification report and confusion matrix resulted for RF classifier is shown in Fig. 4 and Fig. 5. On further analysis of the confusion matrix, the diagonal has higher weights for peak emotions and neutral emotions while there are few miss-classifications in moderate class of each emotions. Many of these samples were miss-classified into their own peak or neutral emotion. B. Results of Method II 1) Results of EFA: In method II, the AU intensity data generated for the image set were analyzed using EFA to explore the underlying factor structure and AUs related to the factors. Prior to extraction of factors using EFA, Kaiser-Meyer-Olkin (KMO) Measure of Sampling Adequacy and Bartlett test of spherecity was conducted to check the suitability of data for factor analysis. [23] The results showed that the KMO value was 0.797, and the significance of Bartlett’s sphericity was 0, which indicated that the data could be analysed using factor analysis. EFA is then conducted on the data with the Maximum Likelihood Estimation as the extraction technique. The visual representation using the scree plot confirmed that there are three factors within the data. Furthermore, AUs related to the three factors were also identified with a cutoff factor loading of 0.4. The factors and their respective AUs found during EFA are shown in Table I. By reviewing TABLE I: Results of EFA Factor Factor 1 Factor 2 Factor 3 Related AUs AU6, AU10, AU12, AU14, AU25 AU1, AU2, AU5, AU26 AU4, AU15, AU17 the AUs in literature related to emotions and analysis of AUs, it was found that Factor 1 in the above table could be related to emotion Happy. Although AU6, AU12 and AU25 are the popularly known AUs in relation to the emotion happy in literature [15], [16], [24], exploring images in the data set revealed and that AU10 and AU14 which correspond to upper lip raiser and dimpler are also visible related to images showing happy emotion in the data set. The factor 2 in the table could be related to the emotion surprise, whereas, factor 3 on the other hand could be related to the emotion sad. 2) Results of Independent t-test: Following identification of AUs related to emotions as above, next, independent t-tests are performed to find whether intensities of AUs related to an emotion vary with the level of emotion. The results of the t-test revealed that all AUs related to an emotion does not show significant variations with the variation of the emotion. For example, even though after EFA it is concluded that AUs 4, 15 and 17 were related to the sad emotion, the independent t-test revealed that only intensities of AU15 and AU17 show significant variations between moderately sad and peak sad images. Similarly AU1 and AU5 were found to show significant variations between moderately surprised and peak surprised images, whereas, all AUs found related to the Happy emotion (i.e AU6, AU10, AU12, AU14 and AU25) showed significant variations between moderately happy and peak happy images. 3) Cluster Analysis: The results of EFA and t-tests were used to identify the AUs which are best suitable for clustering images related to an emotion considering their level of emotion. For example, images showing sad emotion were clustered using AU15 and AU17 only, since, during t-test it was found that only those two AUs showed variations when the level of emotion varied. In this research, clustering of images is done using K-means clustering technique which is a distance-based clustering method. Selecting AUs which have intensities varying with the emotion level only for clustering is done so that only significant features which could be used to cluster images more effectively are considered and thereby leading to better clustering of images belonging to the same level of emotion. Per each emotion, in the above manner three clusters were generated. The clusters resulted by grouping the images belonging to five individuals for surprise emotion is shown in Fig. 6. Content of the resulted clusters were analyzed manually and it was found that images related to no specific emotion (neutral), moderate level of an emotion and peak level of an emotion were clustered together. The images belonging to the above three clusters were then given the labels, neutral, moderate or peak. For example, images showing the happy emotion, were given the label ‘neutral’, ‘moderately happy’ and ‘peak happy’ depending on the cluster each image belonged to. Following the above procedure, all images in the data set were labeled with seven class names which are, neutral, moderately-happy, peak-happy, moderately-sad, peak-sad, moderately-surprised and peaksurprised. 4) Classification Model: Once the image data in the data set were labeled following the clustering as explained in III-B3, the resulted data set was used for classification. SVM, KNN, Logistic Regression (LR) and RF classification algorithms were used on the data set. Metrics 292 Authorized licensed use limited to: SLIIT - Sri Lanka Institute of Information Technology. Downloaded on January 23,2022 at 15:11:50 UTC from IEEE Xplore. Restrictions apply. Fig. 6: Clustering Visualization for Surprise Emotion precision, recall and F1 score were used to compare the performance of the four classification algorithms. Classification report and confusion matrix obtained for RF classifier are shown in Fig. 7 and 8 respectively. Fig. 8: Confusion Matrix for Random Forest Classifier: Method II such that the classification model could learn better by using it. TABLE II: Comparison of Surprise Emotion classification Image Method I Label Method II Label Moderate Neutral Peak Moderate Peak Peak TABLE III: Comparison of Happy Emotion Classification Image Method I Label Method II Label Fig. 7: Classification Report for Random Forest Classifier: Method II As seen in Fig 4 and Fig 5, the emotion levels were classified with adequate accuracy in Method II except for moderate sad emotion level, where miss-classified entries were relatively higher than the other levels. Furthermore, Method II classification resulted in an overall F1-score, accuracy, precision and recall of 87% and showed the best performance with the train:test split of 70:30. C. Discussion When comparing the results of Method I presented in Fig 4 and Fig 5 with Method II results given in Fig 7 and Fig 8, it is evident that the classifier developed in Method II shows better accuracy in identifying the levels of emotions. The reason for the above would be that the data set used in method II is more systematically labeled Neutral Neutral Neutral Moderate Moderate Peak Classification models learns to predict class labels by understanding the relationships between the features and the labels given for training. If such data are mislabeled, however, the model might not perform well in predicting classes. In Method II of the research two additional steps are used to improve the clustering, leading to better labeling of the data set. For example, EFA is used to ensure that no important features related to an emotion are overlooked, whereas, t-tests are used to identify only AUs which are suitable to identify varying emotion levels. Since all important and significant features are used for clustering, Method II has resulted in a more accurately labeled data set. Manual analysis of data sets resulted for Method I and Method II also revealed that the data in Method II are much more accurately clustered and therefore, better labeled. For example, Table II, Table III and Table IV show how three sets of images corresponding to surprise, happy and sad emotions were labeled respectively using 293 Authorized licensed use limited to: SLIIT - Sri Lanka Institute of Information Technology. Downloaded on January 23,2022 at 15:11:50 UTC from IEEE Xplore. Restrictions apply. Method I and Method II. As seen in Table II, Method I has mislabeled neutral and moderately surprised emotion levels, while similar miss-classification results can be seen in Table III and Table IV. TABLE IV: Comparison of Sad Emotion Classification Image Method I Label Method II Label Neutral Neutral Neutral Moderate Peak Moderate Peak Peak D. Conclusion and Future Works Existing research reveals numerous efforts made on identifying different emotions shown by individuals such as happy and sad by analyzing videos and images. There is however, a limited research on identifying levels of emotion such as moderately happy and peak happy. In this research, two methods were experimented to identify levels of emotions for a given image or a video. The first method clustered the image set showing different levels of emotions using the action units recommended in the literature, labeled the images based on the clusters they belonged to and used the labeled data to train and test number of classification algorithms to develop a classification model to identify levels of emotions. This method however, did not result in an adequate level of accuracy. Therefore, an alternative method was used where identifying AUs which are related to emotions in the data set via EFA method and identifying the AUs related to an emotion which vary with the levels of emotions via independent t-test were done before the steps of clustering and classification. The classification models in Method II performed better than Method I in terms of accuracy, precision and F1-score. The reason for the above would be that in method II additional steps are taken to improve the clustering and thereby labeling the image set better. As in this research the emotion type and level classifications were achieved for happy, sad and surprise emotions, this work will be further extended to other emotion types. Broadening this work, we plan to develop models to identify emotion levels of growing children and identify the patterns of AU contributions with age. Acknowledgements - This research was supported by the Accelerating Higher Education Expansion and Development (AHEAD) Operation of the Ministry of Higher Education of Sri Lanka funded by the World Bank (https://ahead.lk/result-area-3/). R EFERENCES [1] A. Saxena, A. Khanna, and D. Gupta, “Emotion recognition and detection methods: A comprehensive survey,” Journal of Artificial Intelligence and Systems, vol. 2, no. 1, pp. 53–79, 2020. [2] C. Busso, Z. Deng, S. Yildirim, M. Bulut, C. M. Lee, A. Kazemzadeh, S. Lee, U. Neumann, and S. Narayanan, “Analysis of emotion recognition using facial expressions, speech and multimodal information,” in Proceedings of the 6th international conference on Multimodal interfaces, 2004, pp. 205–211. [3] P. Ekman, “Are there basic emotions?” Psychological Review, vol. 99, pp. 550–553, 1992. [4] V. Jacintha, J. Simon, S. Tamilarasu, R. Thamizhmani, J. Nagarajan et al., “A review on facial emotion recognition techniques,” in Proceedings of the International Conference on Communication and Signal Processing. IEEE, 2019, pp. 0517–0521. [5] M. Xiaoxi, L. Weisi, H. Dongyan, D. Minghui, and H. Li, “Facial emotion recognition,” in Proceedings of the IEEE 2nd International Conference on Signal and Image Processing. IEEE, 2017, pp. 77– 81. [6] H. Siqueira, S. Magg, and S. Wermter, “Efficient facial feature learning with wide ensemble-based convolutional neural networks,” Proceedings of the Conference on Artificial Intelligence, vol. 34, no. 04, pp. 5800–5809, Apr. 2020. [Online]. Available: https://ojs.aaai.org/index.php/AAAI/article/view/6037 [7] Y. Fan, X. Lu, D. Li, and Y. Liu, “Video-based emotion recognition using cnn-rnn and c3d hybrid networks,” in Proceedings of the 18th ACM International Conference on Multimodal Interaction, 2016, pp. 445–450. [8] D. Meng, X. Peng, K. Wang, and Y. Qiao, “Frame attention networks for facial expression recognition in videos,” in Proceedings of the IEEE International Conference on Image Processing. IEEE, 2019, pp. 3866–3870. [9] S. J. Ahn, J. Bailenson, J. Fox, and M. Jabon, “20 using automated facial expression analysis for emotion and behavior prediction,” The Routledge handbook of emotions and mass media, p. 349, 2010. [10] E. Pranav, S. Kamal, C. S. Chandran, and M. Supriya, “Facial emotion recognition using deep convolutional neural network,” in Proceedings of the 6th International conference on advanced computing and communication Systems. IEEE, 2020, pp. 317– 320. [11] S. Li and W. Deng, “Deep facial expression recognition: A survey,” IEEE Transactions on Affective Computing, 2020. [12] M. Nadeeshani, A. Jayaweera, and P. Samarasinghe, “Facial emotion prediction through action units and deep learning,” in Proceedings of the 2nd International Conference on Advancements in Computing, vol. 1. IEEE, 2020, pp. 293–298. [13] A. A. Rizzo, U. Neumann, R. Enciso, D. Fidaleo, and J. Noh, “Performance-driven facial animation: basic research on human judgments of emotional state in facial avatars,” CyberPsychology & Behavior, vol. 4, no. 4, pp. 471–487, 2001. [14] C. G. Kohler, E. A. Martin, N. Stolar, F. S. Barrett, R. Verma, C. Brensinger, W. Bilker, R. E. Gur, and R. C. Gur, “Static posed and evoked facial expressions of emotions in schizophrenia,” Schizophrenia Research, vol. 105, no. 1-3, pp. 49–60, 2008. [15] J. A. Coan and J. M. Gottman, “The specific affect coding system (spaff),” Handbook of emotion elicitation and assessment, vol. 267, 2007. [16] J. Yang, F. Zhang, B. Chen, and S. U. Khan, “Facial expression recognition based on facial action unit,” in Proceedings of the Tenth International Green and Sustainable Computing Conference. IEEE, 2019, pp. 1–6. [17] S. Velusamy, H. Kannan, B. Anand, A. Sharma, and B. Navathe, “A method to infer emotions from facial action units,” in Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE, 2011, pp. 2028–2031. [18] P. Lucey, J. F. Cohn, T. Kanade, J. Saragih, Z. Ambadar, and I. Matthews, “The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression,” in 2010 ieee computer society conference on computer vision and pattern recognition-workshops. IEEE, 2010, pp. 94–101. [19] C. T. Keating and J. L. Cook, “Facial Expression Production and Recognition in Autism Spectrum Disorders: A Shifting Landscape,” Child and Adolescent Psychiatric Clinics, 2020. [20] S. Zhalehpour, O. Onder, Z. Akhtar, and C. E. Erdem, “Baum-1: A spontaneous audio-visual face database of affective and mental states,” IEEE Transactions on Affective Computing, vol. 8, no. 3, pp. 300–313, 2016. [21] T. Baltrusaitis, A. Zadeh, Y. C. Lim, and L.-P. Morency, “Openface 2.0: Facial behavior analysis toolkit,” in Proceedings of the 13th IEEE International Conference on Automatic Face & Gesture Recognition. IEEE, 2018, pp. 59–66. [22] B. Williams, A. Onsman, and T. Brown, “Exploratory factor analysis: A five-step guide for novices,” Australasian journal of paramedicine, vol. 8, no. 3, 2010. [23] M. Effendi, E. M. Matore, M. F. M. Noh, M. A. Zainal, and E. R. M. Matore, “Establishing factorial validity in raven advanced progressive matrices (rapm) in measuring iq from polytechnic students’ ability using exploratory factor analysis (efa),” Proceedings of Mechanical Engineering Research Day, vol. 2020, pp. 248–250, 2020. [24] L. Zhang, A. Hossain, and M. Jiang, “Intelligent facial action and emotion recognition for humanoid robots,” in Proceedings of the International Joint Conference on Neural Networks. IEEE, 2014, pp. 739–746. 294 Authorized licensed use limited to: SLIIT - Sri Lanka Institute of Information Technology. Downloaded on January 23,2022 at 15:11:50 UTC from IEEE Xplore. Restrictions apply.