2020 8th IEEE International Conference on Biomedical Robotics and Biomechatronics (BioRob) New York, USA. Nov 29 - Dec 1, 2020 Performance Evaluation of Pattern Recognition Algorithms for Upper Limb Prosthetic Applications A. Marinelli, Student Member, IEEE, M. Semprini, Member, IEEE, M. Canepa, L. Lombardi, S. Stedman, A. Dellacasa Bellingegni, M. Chiappalone, Member, IEEE, M. Laffranchi, Member, IEEE, E. Gruppioni, Member, IEEE, L. De Michieli, Member, IEEE, N. Boccardo preventing users from perceiving the prosthesis as a true substitute of the missing limb [3]. Recently, many groups have focused on improving the control of multi DoF myoelectric hand prostheses [4-6]. These studies are generally focused on improving pattern recognition algorithms for detecting the intended movement, in some cases even reaching accuracy scores greater than 95% [7]. However, these promising outcomes are only achieved in lab environments and are not translated into useful everyday life applications mainly because these systems are not robust in scenarios which differ from that of their training, with a consequent considerable drop in performance [2]. Often, a simple change of arm posture makes the prosthesis useless [8]. Therefore there is an urgent need for robust and reliable decoders, able to cope with the potential sources of variability in the input data [9-11]. Some groups attempted to solve these issues by providing decoders with supplementary input data, using the approach of data abundance as in [12]. To promote usage in daily life scenario, here we propose to improve the decoder accuracy without relying on additional sources of input data, but through optimization of pattern recognition from the available data using information solely coming from the EMG sensors embedded in the prosthesis. We also aimed at reducing the number of EMG sensors. We tested and optimized several state-of-the-art classifiers: NonLinear Logistic Regression (NLR) [7], Regularized LeastSquare (RLS) [13], Artificial Neural Network (ANN) [14], Support Vector Machine (SVM) [15], and Linear Discriminant Analysis (LDA) [7], for simultaneously decoding multi-joint hand configurations from up to 6 EMG electrodes placed on the arm. Moreover, NLR, RLS and LDA combined pattern recognition with evaluation of the likelihood of the decoded joint positions, thus enabling a confidence-based rejection criterion, called “abstention”, which reduced decoding errors [16]. We here show that accurate tuning of algorithm parameters and the inclusion of the abstention feature, strongly increased algorithms accuracy and in particular of the NLR, which outperformed the others. This was observed not only on healthy subject data, but also on upper limb amputees controlling a virtual hand. Abstract— Poly-articulated, myoelectric hand prostheses reproduce complex multi-degree of freedom movements, which are fundamental to effectively assist upper limb amputees in the execution of daily life activities. In this scenario, the control system consists in a pattern recognition algorithm translating the recorded electromyographic (EMG) activity into joint movements. However, the low decoding performance typically reached by the control system results in poor stability of the prosthetic device. In order to solve this issue, here we tested several state-of-the-art classifiers for decoding multi-joint hand movements from electromyographic recordings of arm muscles, collected from healthy subjects. Specifically, we tested: NonLinear Logistic Regression (NLR), Regularized Least-Square, Artificial Neural Network, Support Vector Machine, and Linear Discriminant Analysis. We aimed at minimizing the number of EMG electrodes (6 maximum) by optimizing each classifier in terms of the F1Score, and then we compared the performance of the classifiers. We found that the NLR algorithm achieved the best results with only 3 EMG electrodes. The optimized algorithms were then tested on three right arm amputees controlling a virtual hand. We obtained that algorithm’s performance was comparable with that obtained from healthy subjects. In particular, the NLR classifier achieved 99% correct classification for all the patients, indicating its potential effective use in prosthetic applications. I. INTRODUCTION Poly-articulated myoelectric hand prostheses are characterized by a high number of degrees of freedom (DoF). A crucial feature for their functionality and usability is their controllability. Indeed, low usage intuitiveness, often due to the poor ergonomics of the control system [1], lies among the main causes for prosthesis abandonment. Currently, typical myoelectric hand prostheses rely on the use of two electromyographic (EMG) sensors, placed on flexor and extensor muscles of the wrist, respectively. EMG sensors detect the difference in electric potential produced by the underlying muscles, and the control policy regulates the prosthesis velocity such that it is proportional to the muscle contraction [2]. However, this does not allow simultaneous multi-joint control of the prosthesis as it is based on the control of one joint at the time. It is not very intuitive, and the resulting movement is far from being smooth and natural, *Research supported by INAIL, grant PPR-AS 2017-2020. A. Marinelli, M. Semprini, M. Canepa, L. Lombardi, S. Stedman, M. Chiappalone, M. Laffranchi, L. De Michieli and N. Boccardo are with Rehab Technologies IIT-INAL Lab, Istituto Italiano di Tecnologia, Genova 16163 Italy (Tel.: 0039-010-28961; e-mails: andrea.marinelli@iit.it, marianna.semprini@iit.it, michele.canepa@iit.it, lorenzo.lombardi@iit.it, 978-1-7281-5907-2/20/$31.00 ©2020 IEEE samuel.stedman@iit.it, michela.chiappalone@iit.it, matteo.laffranchi@iit.it, lorenzo.demichieli@iit.it, nicolo.boccardo@iit.it). A. Marinelli is also with the Department of Informatics, Bioengineering, Robotics and systems Engineering, University of Genova, 16145 Italy. E. Gruppioni and A. Dellacasa Bellingegni are with the Prosthetic Centre INAIL, Vigorso di Budrio, Bologna 40054 Italy (e-mails: a.dellacasabellingegni@inail.it, e.gruppioni@inail.it). 471 Authorized licensed use limited to: Auckland University of Technology. Downloaded on November 03,2020 at 03:37:50 UTC from IEEE Xplore. Restrictions apply. A Rest Closure Pronation Opening B Supination sEMG Signals (V) Hand Opening/ Closure Wrist Pronation/ Supination Time (s) Figure 1. A typical example of EMG activity and related joint movement. A: EMG activations and virtual hand speed during different hand and wrist movements. B: DoFs of the controlled virtual hand. section II.F). To this end, we customized the EMG Data Acquisition & Training Software (EDATS), developed by Centro Protesi INAIL of Budrio [7] (Fig. 2A). This acquisition system collected EMG data for off-line training of classifiers, implemented in MATLAB (MathWorks). Once each classifier was trained, subjects were allowed to freely perform any of the 4 movements, which was decoded by the resulting best performing classifier and replicated in real-time by the virtual hand, whose velocity was linearly dependent on the root mean square (RMS) of EMG signals. II. MATERIAL AND METHODS A. Subjects We recruited 10 able-bodied, right-handed subjects (6 males, age 36 ± 9 years) and 3 amputated subjects (monolateral, right trans-radial amputation of dominant limb, expert active prostheses users). All subjects provided written informed consent. The study conformed to the standard of the Declaration of Helsinki and was approved by the ethical committees of Bologna-Imola (CP-PPRAS1/1-01). D. Training and testing of the classifiers model The analysed classifiers (see section II.F) were supervised algorithms and thus needed a specific calibration procedure to estimate the best set of internal parameters for further on-line use. The calibration described here, regards the optimization of internal parameters of each classifier, and hence is not to be confused with the further optimization of the algorithm according to other hyperparameters (see section II.G). Input data for classifiers was composed by single samples of EMG data. For each classifier, the data was divided into test set (TS), training set (TR) and validation set (VS). For the B. Experimental Protocol Six commercial EMG electrodes (13E200 AC, Otto Bock) were embedded into a custom-made elastic brace placed around forearm or stump. This allowed to collect electrical activity from 6 relevant muscle groups involved in grasping and prone/supination movements (Fig. 1). Muscles involved are: Extensor Carpi Radialis Longus Muscle (EMG0), Palmaris Longus Muscle and Flexor Carpi Ulnaris Muscle (EMG1), Extensor Digitorum Muscle (EMG2), Flexor Carpi Radialis Muscle (EMG3), Extensor Carpi Ulnaris Muscle (EMG4), and Brachio-Radial Muscle (EMG5). After electrodes placement, subjects were positioned in front of a monitor displaying a virtual hand with an active wrist (Fig. 1B). We asked them to sequentially perform hand opening/closing and wrist pronation/supination for 10 times. We also collected 16 repetitions of hand at rest (duration 2s, sampling frequency 1kHz). C. EMG Signal Processing EMG signals were recorded by custom EMG processing board programmed by Code Composer software (Texas Instruments). The EMG processing board performed A/D conversion and then Bluetooth transmission of data to a dedicated PC (Dell Xps15, Intel i7, 16Gb Ram, Windows 10). Here single samples of the EMG signals related to movements and rest, were used to off-line train several classifiers (see Figure 2. Experimental Setup. A: a power supply, the EMG processing board, the sEMG armband, the virtual hand and the EDATS software. B: a healthy subject performing in Real-Time the virtual hand control. C: an amputee performing in Real-Time the virtual hand control. 472 Authorized licensed use limited to: Auckland University of Technology. Downloaded on November 03,2020 at 03:37:50 UTC from IEEE Xplore. Restrictions apply. TABLE I. NLR STRUCTURE OF POLYNOMIAL FEATURES FOR EACH D. D Figure 3. Block diagram of the experimental setup. algorithms NLR, RLS, ANN and SVM, the TS was obtained after a down-sampling operation from 1 kHz to 25 Hz. This operation was performed in order to reduce TR size and the computational time required to create the classifier model, as in [7]. The TR was composed by 70% of remaining data, while the VS (not used for the LDA algorithm), was finally composed by 30% of remaining data. For the LDA, the dataset was only split into TS (30% of the entire dataset) and TR (the remaining 70%). Through a custom MATLAB API, the parameters of the selected pattern recognition model were tuned off-line on the TR. A validation phase followed on the VS for tuning the hyperparameters to prevent overfitting (step 1 in Fig. 3. Lastly, the performance of the model was assessed on the TS. Each classifier was optimized according to its model parameters with a computational time ranging from few seconds to few minutes, depending on the algorithm. An evaluation phase followed in which the calibrated algorithm was used to decode hand motion on-line using a real-time PC based PR simulator (step 2 in Fig. 3). The resulting outcomes were then uploaded (step 3 in Fig. 3) to the EMG processing board (Tiva Cortex-M4) for final on-line classification (step 4 in Fig. 3), during which users were able to freely control the virtual hand. In this phase the sampling frequency was 300Hz, thus the classification was updated every 3ms. Description Example 1 Linear case (LR) x1, x2, x3, x4, x5, x6 2 max 2nd degree 3 max 3nd degree 4 max 4nd degree 5 max 5nd degree 6 max 6nd degree x1, …, x6, x1x2, x1x3, …, x5x6, x12, x22, …, x62 x1, …, x62, x1x2x3, …, x4x5x6, x13, …, x63 x1, …, x63, x1x2x3x4, …, x3x4x5x6, x14, …, x64 x1, …, x64, x1x2x3x4x5, …, x2x3x4x5x6, x15, …, x65 x1, …, x65, x1x2x3x4x5x6, x16, …, x66 7 max 7nd degree x1, …, x66, x17, x27, x37, x47, x57, x67 of classes likelihood. During validation, we tested the F1Score performance of each classifier with respect to the number of EMG electrodes. This analysis was used to estimate the minimum number of electrodes. Also, we optimized other parameters for NLR and ANN classifiers. For NLR, we also considered the role of the degree (D) of complexity. This parameter, ranging from 1 to 7, codified a structure of polynomial features, as reported in TABLE I. For each D, the obtained polynomial features consisted in: all the values obtained by the power’s series of each input until the degree defined by the parameter D (xi1,…xiD; i = 1…6 number of electrodes), and all combinations obtained through a combination without repetition of a maximum number of elements equivalent to the degree defined by D. Instead, for ANN classifier, we first evaluated the role of the number of hidden layers by setting the number of neurons to 30 (the maximum), and we then optimized the number of neurons, using the optimal number of hidden layers. In addition, for the NLR, RLS and LDA regression algorithms we introduced a “truth index”, based on Likelihood threshold [11]. The Likelihood maximizes the efficiency of each algorithm: we tested different values of threshold of likelihood from 70 to 100% in order to guarantee the classification of the voluntary movements only. The threshold was optimized for each dataset and each algorithm, by evaluating the F1Score obtained. Movements discarded from classification were considered as “abstentions” and did not produce any movement. E. Virtual hand The virtual hand was created with the 3D design program Blender (Blender Foundation). The trajectories of the opening and closing of the hand, as well as the pronation and supination of the wrist (Fig.1A), were implemented coherently with the real joint axes and related joint movements of the human hand (50th percentile) [17]. The synergetic behaviour of all the fingers involved in the movement was mapped by means of a Unity (Unity Technologies) based software. A final middle-ware software implementation connected the virtual environment (moving the virtual hand) to the EDATS software by means of TCP/IP. G. Optimization and testing of the best configuration of hyperparameters The dataset acquired from the healthy population was used to find the best hyperparameters (i.e. the set of electrodes, degree of optimal complexity, number of hidden layers, and number of neurons) for the pattern recognition algorithms. The TS (section II.D) was used to evaluate the performance of each classifier according to each specific set of hyperparameters in terms of F1Score, which takes into account the rate of false and true positives and of false negatives [18]. A performance comparison of the algorithms was then evaluated in terms of classification (the % of correctly decoded movements, i.e. accuracy), F1Score, and abstention (the % of non-assigned movements). The best configuration F. Pattern Recognition algorithms In order to perform EMG pattern recognition, we tested a wide range of supervised machine learning algorithms: NLR [7], RLS [13], ANN [14], SVM [15], and LDA [7]; for each algorithm we defined the model and their optimization as in the cited references. ANN and SVM are classifiers and thus their output is the decoded movement class; instead, the other algorithms are regressors and thus their output is an estimate 473 Authorized licensed use limited to: Auckland University of Technology. Downloaded on November 03,2020 at 03:37:50 UTC from IEEE Xplore. Restrictions apply. Figure 4. F1Score obtained by NLR using different number of electrodes and fixing maximum value of D to 7. NS: not significant. Figure 5. F1Score obtained by NLR using different maximum value of parameter D and fixing number of electrodes to 3. NS: not significant. of each classifier was then used for training the algorithms on the amputees’ dataset. We also assessed whether the obtained scores were comparable with those obtained by the healthy. Statistical analysis was performed with the Wilcoxon signed rank test and the Bonferroni correction [19]. Ulnaris Muscle (EMG1), and the Extensor Carpi Ulnaris Muscle (EMG4) was enough to reach the same performance as with the maximum number of electrodes, Wilcoxon signed rank test p = 0.084. Fig. 4 shows that for NLR, a significant decrease of F1Score was obtained when the number of electrodes was reduced from three to two, Wilcoxon signed rank test p = 0.0039. For the other classifiers, the number of electrodes required to maximize the performance was greater than 3, and was respectively 4 for the RLS, 6 for ANN and SVM, and 5 for LDA, as indicated in TABLE II. III. RESULTS A. Effect of EMG electrodes number on performance We first investigated, for each algorithm, the minimum number of EMG electrodes needed to maximize performance, expressed as non-statistical difference between the distributions of F1Score obtained from the TS. Starting from the full configuration (6 EMG electrodes) we progressively reduced the number of electrodes by removing those placed on smaller muscles, according to the following order: EMG5, EMG2, EMG3, EMG4. For the NLR algorithm, we found that a configuration of three electrodes placed on the Extensor Carpi Radialis Longus Muscle (EMG0), Palmaris Longus Muscle and Flexor Carpi B. Effect of D parameter on NLR performance For the NLR algorithm, we swept the maximum value of the parameter D, which regulates the maximum polynomial degree, from 1 to 7 and accordingly evaluated the performance in terms of F1Score. As shown in Fig. 5, we found that D = 2 maximized the performance. C. Effect of network architecture on ANN performance For the ANN classifier, we first tested the role of the TABLE II: F1SCORE OBTAINED FOR EACH CLASSIFIER WITH A DIFFERENT NUMBER OF ELECTRODES. In bold are indicated the number of electrodes which saturated the performance of each classifier. Algorithm NLR(D=7) RLS ANN(L=10, N=30) SVM LDA EMG [#] 2 3 4 5 6 2 3 4 5 6 2 3 4 5 6 2 3 4 5 6 2 3 4 5 6 F1Score [%] 82.7 ± 19.7 94.9 ± 10.2 98.2 ± 5.1 99.9 ± 0.2 99.9 ± 0.2 25.0 ± 13.9 50.3 ± 13.0 60.3 ± 10.8 59.9 ± 11.0 57.1 ± 11.6 70.7 ± 5.5 75.7 ± 3.7 77.7 ± 4.8 79.0 ± 4.6 81.7 ± 4.8 70.4 ± 5.5 76.0 ± 3.8 78.6 ± 4.0 80.6 ± 4.3 83.6 ± 4.4 89.1 ± 11.6 96.4 ± 1.5 97.8 ± 1.3 98.5 ± 1.5 98.7 ± 1.4 Figure 6: Optimization of ANN with 6 EMG electrodes. A: F1Score obtained using different maximum number of hidden layer and fixing maximum number of neurons to 30. B: F1Score obtained using different maximum number of neurons and fixing maximum number of hidden layers according to previous analysis. NS: not significant. 474 Authorized licensed use limited to: Auckland University of Technology. Downloaded on November 03,2020 at 03:37:50 UTC from IEEE Xplore. Restrictions apply. hidden layers on performance. We fixed the maximum number of neurons to 30 and computed the F1Score for each maximum number of hidden layers (ranging from 1 to 10). We found that L = 6 hidden layers represented the optimal choice, as the F1Score does not differ from that obtained with a larger number of layers (Fig. 6A). We then fixed L = 6 hidden layers and evaluated the performance of the ANN varying the maximum number of neurons in the following range: [1, 5, 10, 15, 20, 23, 24, 25, 26, 27, 28, 29, 30]. We found that with N = 24 maximum number of neurons, performance is maximized (Fig. 6B). TABLE III: CLASSIFIERS PERFORMANCE SCORES OBTAINED WITH THE HEALTHY POPULATION DATA. Data are reported as mean ± standard deviation across subjects. In bold are reported the best scores according to each evaluated indicator (classification, F1Score, and abstention). Algorithm EMG [#] Classification [%] F1Score [%] Abstention [%] NLR(D=2) RLS ANN(L=6, N=24) SVM LDA 3 4 6 6 5 99.9 ± 0.1 62.6 ± 11.0 80.3 ± 4.8 83.5 ± 4.6 98.5 ± 1.4 97.3 ± 7.9 60.3 ± 10.8 80.2 ± 4.7 83.6 ± 4.4 98.5 ± 1.5 66.9 ± 7.4 43.4 ± 8.6 39.5 ± 4.0 recognition. Moreover, NLR led to more consistent results, as indicated by the small standard deviation. With respect to the F1Score, LDA obtained the highest value, although with no statistical difference with NLR (Fig. 7B, TABLE III). NLR also obtained highest percentage of abstention (Fig. 7C). D. Comparison of algorithms performances Performances in terms of F1Score of the NLR, RLS, ANN, SVM, and LDA methodologies were compared through a Wilcoxon Signed-Rank test for a different number of electrodes. We found no statistical difference of F1Score over different numbers of electrodes. TABLE III summarizes the performance scores in terms of classification, F1Score, and abstention for each classifier with the optimized number of electrodes and hyperparameters, as previously determined. Fig. 7A reports the classification score obtained by each classifier. With only 3 EMG electrodes, NLR obtained the highest score, even higher than that obtained by LDA, which is the gold standard for applications requiring pattern E. Algorithms evaluation on the amputees’ dataset We tested the classifiers on three trans-radial amputees, using an optimized configuration, as obtained from the analysis of the healthy dataset. TABLE IV shows the values of classification, F1Score, and abstention obtained from patients. Amputees obtained scores matching those obtained by healthy subjects: NLR obtained the highest classification score, followed by LDA. F1Scores were highest for LDA, followed by NLR. NLR obtained greater abstentions. A IV. DISCUSSION We tested several pattern recognition algorithms for decoding hand movements and we identified NRL as the one producing best performance. Indeed, our results demonstrate that NLR is the only algorithm, among those tested, which reached the highest classification performance with only three EMG electrodes. This is particularly desirable in the context of hand prosthetics, as placing each EMG sensor requires a corresponding hole in the socket. Therefore, the smaller the number of electrodes, the smaller the possibility of undermining the socket robustness as well as the stability of the entire prosthetic system. This is crucial for amputees whose residual arm is proximal, resulting in an internal lamination of the socket that is drastically reduced. Reducing B TABLE IV: CLASSIFIERS PERFORMANCE SCORES OBTAINED BY bold are reported the best scores according to each indicator (classification, F1Score, and abstention) for each patient. AMPUTEES. In P. C 1 2 3 Figure 7. Algorithm comparison. A: Fraction of correct classification obtained by each classifier. B: F1Score obtained by each classifier. C: Fraction of abstention obtained by each classifier. NS: not significant. Algorithm NLR RLS ANN SVM LDA NLR RLS ANN SVM LDA NLR RLS ANN SVM LDA EMG [#] 3 4 6 6 5 3 4 6 6 5 3 4 6 6 5 Classification [%] 99.0 39.0 70.5 75.1 96.0 99.9 60.5 72.3 73.3 94.7 99.9 60.6 80.3 84.1 96.5 F1Score [%] 82.9 36.1 71,3 76.0 96.0 85.5 54.1 73.8 75.0 94.5 93.4 53.5 80.5 84.4 96.4 Abstention [%] 77.6 47.4 39.7 75.0 42.9 34.4 72.3 39.1 35.3 475 Authorized licensed use limited to: Auckland University of Technology. Downloaded on November 03,2020 at 03:37:50 UTC from IEEE Xplore. Restrictions apply. the number of electrodes therefore, increases the physical robustness of the prosthesis once donned on the residual limb. Moreover, fewer electrodes substantially reduce the prosthesis cost. All these features may increase the number of patients adopting multi-electrodes control strategies. With respect to NLR algorithm, we also found that keeping a small degree of polynomial complexity (D = 2) maintains the same level of performance as with higher degrees. This affects the computational burden, as algorithm can still achieve high performance, but more efficiently and in shorter times, a highly desirable feature when designing embedded devices, such as myoelectric prostheses. We were also able to simplify network architecture for ANN classifier, by optimizing the maximum number of hidden layers and neurons, and setting them to 6 and 24 respectively. When comparing classifiers in their optimized form, we found that, also in this case, the NLR reached highest classification and F1Score. NLR also obtained a greater number of abstentions. However, this apparent weakness is counterbalanced by the high classification frequency set. In fact, implementation of abstention is generally aimed at reducing the number of classification artifacts, but often comes at the cost of adding delays in the control loop, because the abstention percentage limits the number of decoded movements. Generally classifiers work with a time window of 200 ms (5 Hz), meaning that every second, 5 movements are decoded [20]. A high percentage of abstention may thus result in a latency between movement intention and actual movement of the controlled device. However, our classifiers are implemented with frequency of 300 Hz, meaning that 300 movements per second are decoded and thus even with a high abstention percentage, the control loop is not affected by noticeable delays, and users therefore are not perceiving any latency between movement intentions and resulting action. Overall, our analysis revealed that the NLR classifier is the best option for pattern recognition application in the framework of EMG signal acquisitions. We confirmed this result with upper limb trans-radial amputees, who were able to successfully control a virtual hand by controlling the residual muscles of the stump. Clearly, we need to extend the study to a wider population of amputees and we also need to confirm these promising results with clinical trials on a physical hand prosthesis. However, as verbally described by the amputees, the NLR algorithm allows them to reliably translate real-time movement intentions into actions. We plan to assess online performance of the classifier with a target achievement control (TAC) test, which measures the ability to move a virtual arm into a target posture [21]. REFERENCES [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] V. CONCLUSION [18] We tested several pattern recognition algorithms for hand movement classification using EMG data collected from healthy subjects. We identified NLR as the best performing classifier with the minimum number of electrodes and thus as optimal solution for real-time control of prosthetic hand. This hypothesis was further supported by some tests on trans-radial amputees controlling a virtual hand. Additional investigations and bigger amputee datasets are needed to evaluate the NLR outcome for real-time control of a prosthetic hand. [19] [20] [21] D. Proaño-Guevara, J. Procel-Feijóo, J. Zhingre-Balcazar, and L. Serpa-Andrade, "Biomimetical arm prosthesis: A new proposal," in International Conference on Applied Human Factors and Ergonomics, 2017: Springer, pp. 549-558. A. D. Roche, B. Lakey, I. Mendez, I. Vujaklija, D. Farina, and O. C. Aszmann, "Clinical Perspectives in Upper Limb Prostheses: An Update," Current Surgery Reports, vol. 7, no. 3, p. 5, 2019. A. Chadwell, L. Kenney, S. Thies, A. Galpin, and J. Head, "The reality of myoelectric prostheses: understanding what makes these devices difficult for some users to control," Frontiers in neurorobotics, vol. 10, p. 7, 2016. M. Sartori, G. Durandau, S. Došen, and D. Farina, "Robust simultaneous myoelectric control of multiple degrees of freedom in wrist-hand prostheses by real-time neuromusculoskeletal modeling," Journal of neural engineering, vol. 15, no. 6, p. 066026, 2018. J. M. Hahne, M. A. Schweisfurth, M. Koppe, and D. Farina, "Simultaneous control of multiple functions of bionic hand prostheses: Performance and robustness in end users," Science Robotics, vol. 3, no. 19, p. eaat3630, 2018. T. Kapelner, F. Negro, O. C. Aszmann, and D. Farina, "Decoding motor unit activity from forearm muscles: perspectives for myoelectric control," IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 26, no. 1, pp. 244-251, 2017. A. Dellacasa Bellingegni et al., "NLR, MLP, SVM, and LDA: a comparative analysis on EMG data from people with trans-radial amputation," J Neuroeng Rehabil, vol. 14, no. 1, p. 82, Aug 14 2017, doi: 10.1186/s12984-017-0290-6. Y. Geng, P. Zhou, and G. Li, "Toward attenuating the impact of arm positions on electromyography pattern-recognition based motion classification in transradial amputees," Journal of neuroengineering and rehabilitation, vol. 9, no. 1, p. 74, 2012. A. J. Young, L. J. Hargrove, and T. A. Kuiken, "The effects of electrode size and orientation on the sensitivity of myoelectric pattern recognition systems to electrode shift," (in eng), IEEE Trans Biomed Eng, vol. 58, no. 9, pp. 2537-44, Sep 2011, doi: 10.1109/TBME.2011.2159216. B. Wan et al., "Study on fatigue feature from forearm SEMG signal based on wavelet analysis," in 2010 IEEE International Conference on Robotics and Biomimetics, 2010: IEEE, pp. 1229-1232. S. Amsüss, P. M. Goebel, N. Jiang, B. Graimann, L. Paredes, and D. Farina, "Self-correcting pattern recognition system of surface EMG signals for upper limb prosthesis control," IEEE Transactions on Biomedical Engineering, vol. 61, no. 4, pp. 1167-1176, 2013. I. Kyranou, S. Vijayakumar, and M. S. Erden, "Causes of performance degradation in non-invasive electromyographic pattern recognition in upper limb prostheses," Frontiers in neurorobotics, vol. 12, p. 58, 2018. H. Sun and Q. Wu, "Regularized least square regression with dependent samples," Advances in Computational Mathematics, vol. 32, no. 2, pp. 175-189, 2010. S. Dreiseitl and L. Ohno-Machado, "Logistic regression and artificial neural network classification models: a methodology review," Journal of biomedical informatics, vol. 35, no. 5-6, pp. 352-359, 2002. C.-C. Chang and C.-J. Lin, "LIBSVM: A library for support vector machines," ACM transactions on intelligent systems and technology (TIST), vol. 2, no. 3, pp. 1-27, 2011. J. W. Robertson, K. B. Englehart, and E. J. Scheme, "Effects of confidence-based rejection on usability and error in pattern recognitionbased myoelectric control," IEEE journal of biomedical and health informatics, vol. 23, no. 5, pp. 2002-2008, 2018. A. R. Tilley, The measure of man and woman: human factors in design. John Wiley & Sons, 2001. D. M. Powers, "Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation," 2011. J. Demšar, "Statistical comparisons of classifiers over multiple data sets," Journal of Machine learning research, vol. 7, no. Jan, pp. 1-30, 2006. R. Boostani and M. H. Moradi, "Evaluation of the forearm EMG signal features for the control of a prosthetic hand," Physiological measurement, vol. 24, no. 2, p. 309, 2003. A. M. Simon, L. J. Hargrove, B. A. Lock, and T. A. Kuiken, "The target achievement control test: Evaluating real-time myoelectric pattern recognition control of a multifunctional upper-limb prosthesis," Journal of rehabilitation research and development, vol. 48, no. 6, p. 619, 2011. 476 Authorized licensed use limited to: Auckland University of Technology. Downloaded on November 03,2020 at 03:37:50 UTC from IEEE Xplore. Restrictions apply.