Keynote: IEEE Int’l Conf. on Online Analysis and Computing Science (ICOACS’16), Chongqing, China, May 28-29, 2016 Easy with Eve: Real time multi-channel emotion detection for effective learning Abdolhossein Sarrafzadeh , Prof., PhD Head of Department of Computing, Director of Centre of Computational Intelligence for Cyber Security (CICS) Department of Computing, Faculty of Creative Industries and Business, Unitec Institute of Technology, New Zealand Homepage: www.unitec.ac.nz/about-us/contact-us/staff-directory/prof-hossein-sarrafzadeh Phone: +64 9 815 4321 ext. 6040 Email: [email protected] ABSTRACT Many software systems would significantly improve performance if they could adapt to the emotional state of the user, for example if Intelligent Tutoring Systems, ATM’s and ticketing machines could recognize when users were confused, suspicious, frustrated or angry they could provide remedial help so improving the service. Research leading to the development of Easy with Eve, an Affective System in the domain of mathematics (an Affective Tutoring System- ATS) will be presented. Eve adapts to users by detecting their emotion through facial expression and gesture analysis, and can display emotion herself via a lifelike animated agent called Eve which is currently being tested in a school in New Zealand. Eve enjoys a facial expression recognition component that was developed in-house. A gesture analysis component has been developed and will soon be integrated with Eve. Eve’s adaptations are guided by a case-based method for adapting to user’s states; this method uses data that was generated by an observational study involving 3000 video clips of human participants. The observational study, the casebased method, the life-like agent and the ATS itself and its facial expression and gesture analysis capabilities will be briefly presented. Keywords: Affective Tutoring Systems, Lifelike Agents, Emotion Detection, Human Computer Interaction, Affective Computing ABOUT THE KEYNOTE SPEAKER Dr. Hossein Sarrafzadeh is Professor and Head of Department of Computing at Unitec Institute of Technology. He is well linked to the ICT industry both nationally and internationally, and has strong links to the international research and academic community. Leveraging these strong networks with industry, academia, and with government, particularly in the field of IT security and computational intelligence, he has attracted funding to his department and his institution. He understands global IT trends, needs and opportunities and has a firm grasp of the capability in NZ. Dr Sarrafzadeh has developed one of the world's first real-time facial expression and gesture recognition systems. He has also developed a novel vision based targeted advertising system which has been patented in New Zealand and internationally. Dr Sarrafzadeh has led the establishment of New Zealand’s first cyber security centre in partnership with the National Institute of Information and Communications Technology of Japan and has also been instrumental in the founding and development of the Centre for Computational Intelligence for Environmental Engineering and the Internet of Things (in partnership with Wuhan University and LJDY (IT Company), in China and NIWA (NZ)). Professor Sarrafzadeh is known for his work in the areas of intelligent systems, cyber security, IoT, adaptive user interfaces and software engineering. He has developed one of the world’s first realtime facial expression and gesture recognition systems, and also a novel vision based targeted advertising system patented both in New Zealand and internationally. He has published over 140 research papers and supervised over 40 PhD and Masters theses. He is frequently invited as a keynote speaker at international conferences.