2016 5th International Conference on Informatics, Electronics and Vision (ICIEV) 780 Real Time EEG Based Automatic Brainwave Regulation by Music Alamgir Hossan A.M.Mahmud Chowdhury Department of Applied Physics, Electronics and Communication Engineering University of Chittagong Chittagong, Bangladesh ah.apece@cu.ac.bd Department of Applied Physics, Electronics and Communication Engineering University of Chittagong Chittagong, Bangladesh rizveemahmud@gmail.com Abstract— In this paper, we proposed an approach to automatically regulate the mood or brainwave of the paralyzed or sensory impaired or psychologically sick people. A MATLAB based algorithm has been developed to analyze the brain wave signals obtained from a real time EEG data acquisition system corresponding to different moods of the subject people. The program translated the brain wave signals into command to select and play suitable music tracks according to the state of the brain. These music or sound tracks, selected based on the choice of the target people, acted in turn as a feedback to continuously adjust the subject’s mood as desired. Relaxation music, subcontinental soft patriotic, Rabindra Sangeet & motivational songs were used as example for relaxation, entertainment and warm-up respectively. Sample stored EEG data of different patients having different moods were used to verify the feasibility of the approach and the obtained results ensured its effectiveness. Keywords- Brainwave regulation; BCI; Real time EEG data acquisition; Music Therapy; Neurofeedback. I. INTRODUCTION Automatic brainwave regulation is an extremely useful technique, especially for mentally disordered, completely paralyzed or severely disabled people who cannot physically do anything for their mental comfort unlike normal human being. For different internal and external reasons, their mood can change any time. Most of the time, neither they can express their emotion to their attendant, nor always, their near people can understand what to do for these people, how they feel, what type of music they need to or want to listen. Further, nowadays, it is hardly being possible for their near and dear people to attend them all the time. They get lonely, sometimes unhappy or restless and some other times drowsy untimely. We have attempted to make these unlucky people get relief from these sort of unpleasant mental states by automatically playing time-suitable music, such as relaxation music during excited or restless mood while motivational music for drowsy mood and so on. Interaction of neurons in the brain takes place in the form of electro-chemical signals creating the brain electric and magnetic fields. Based on the frequency and amplitude range, these electric fields are classified into five different bandlimited signal patterns, known as the brain waves, namely delta (δ), theta (θ), alpha (α), beta (β) and gamma (γ) in ascending frequency range [1]. Different brain wave patterns directly correspond to the different states of the brain. In fact, the mood, emotions, feelings or any mental status are simply reflection of these brainwaves [2]. These brain wave patterns change according to one’s feeling, thought and work. Internal and external stimuli can also significantly change these brain waves [3]. Brain waves are sensed and transduced by placing electrodes on the scalp. Traditionally, invasive and noninvasive electroencephalogram (EEG) techniques have been used to record these brainwaves for the diagnosis of some brain affecting diseases such as epilepsy, brain tumors, mental disorders, sleep pattern change, dementia, Alzheimer’s Parkinson disease etc. [4]. These techniques are massive wired and not user friendly. But with the advent of portable, costeffective, wireless EEG recording devices, such as Emotive [5], and tremendous developments in the neuro signal processing tools [6], it has been possible to utilize these brain waves in real time, to directly communicate and control external devices and machines [7], which has initiated a potential technology called brain computer interface (BCI) [8]. Numerous BCI systems, such as EEG-based brain controlled wheelchair, spelling device, cursor, web browser, assistive mobile robots, vehicles, home appliances, etc. have recently been reported in literature, especially to assist paralyzed or severely disabled people, a detail review of these systems can be found in [9]-[11]. In this work, we used brain waves, obtained from wireless EEG data acquisition system and analyzed with MATLAB, to select and play different music tracks stored in a digital device which in turn act as an audio feedback to the subjects to regulate their mood. We chose music as feedback keeping in mind that music can not only adjust mood but also ease pain, increase immune ability and has overall a great influence in human mind [12]. A number of music tracks of different categories, namely, relaxation, soft and motivational music were selected corresponding to different brain wave patterns by a MATLAB program. For example, when high frequency brain waves, beta and gamma (excited mental state) had been found, relaxation music tracks were played which in turn helped the subject to gain relaxed mental state. Similarly, for low frequency brain waves, theta (drowsiness), motivational or high scale music tracks were played to influence the subject to 978-1-5090-1269-5/16/$31.00 ©2016 IEEE Authorized licensed use limited to: POLITECHNIKI WARSZAWSKIEJ. Downloaded on December 03,2023 at 09:30:05 UTC from IEEE Xplore. Restrictions apply. 781 warm-up or get active. In case of alpha brain wave pattern, which correspond to the more relaxed state of the brain, soft or melodious music tracks were played to let the subject have a sustainable relaxed state. This process continued until a delta brain wave pattern (deep sleep) was found when the music player remained apparently off. Music tracks were selected studying the choice or personal profile of the subjects and could be changed as desired. The rest of the paper is organized as follows. In Section II, a brief review of brain waves and its relation with mental states is given. In Section II, we discuss the EEG signal acquiring, processing and buffering techniques briefly and then describe the complete algorithm development and coding techniques. Performance verification of the designed program is summarized and discussed in Section III. In Section IV, we conclude this paper discussing the significance, limitations and future research directions of this work. II. BRAINWAVES AND ITS RELATION WITH HUMAN MINDS Human brain is made up of billions of brain cells known as neurons. These brain cells use synchronized electrical pulses to communicate with each other. The combination of electrical activity of the brain produce wave like cyclic rhythms commonly known as brainwaves [13]. These waves are detected and recorded by EEG equipment by placing a number of electrodes on the different parts of the scalp. The amplitude, frequency and phase of these waves depend on the location of the electrodes [14]. For example, at the cranial surface, the amplitude ranges from 1 to 100μV at low frequencies while at the cerebrum, it can be 10 times stronger. Typical brainwave signals for a normal adult are illustrated in Fig. 1. The frequency bands are usually classified into five categories as follows: Delta (δ) waves Theta (θ) waves Alpha (α) waves Beta (β) waves Gamma (γ) waves 0.5-4 Hz 4-8 Hz 8-13 Hz 13-22 Hz 22-30 Hz and above Delta (δ) waves are usually low frequency and high amplitude signals. These types of signal are usually observed during deep dreamless sleep and extremely deepest meditation. When people remain in this state, they loss total external awareness. These are also dominant waves among infants. Theta (θ) waves are slightly high frequency and low amplitude than delta waves. Theta waves are usually associated with reduced consciousness, drowsiness or light sleep. Alpha (α) waves are of around 10μV peak to peak amplitude and the most stable and balanced states. This is associated with physical and mental relaxation but a complete awareness of surrounding. Alpha waves help overall mind body coordination, calmness, alertness and learning. Figure 1. Brain waves pattern on a normul adult Beta (β) waves are less than 20μV peak to peak and indicate high states of wakefulness. When we remain highly alert, focused, or get agitated, tensed, afraid, perform calculations, we experience more beta waves. These wave are also associated with some mental disorders such as anxiety, insomnia, and OCD etc. [15]. Gamma (γ) waves are the highest frequency brain waves and linked with highly disordered or restless mental states. For various reasons human mood or related brain waves change. Normal people do a lot to regulate their brain waves or adjust their moods naturally and unnaturally. Such, as meditation, listening to music, watching television, relaxation or deep breathing practice, yoga, praying etc. However, physically and mentally disabled people cannot do these for their mood adjustment. III. PROPOSED METHOD The system we proposed, is composed of signal acquisition, buffering, MATLAB based processing and communication, and audio feedback blocks. A schematic of the overall functional flow of the realtime music-based brain wave regulation system is shown in Fig. 2. Real time Data Buffering System Real time EEG Data Acquisition System Music Player MATLAB Spectral Analysis Classification and External Command Relaxation Music Soft Music Motivational Music Subject Figure 2. Schematic of the proposed approach Authorized licensed use limited to: POLITECHNIKI WARSZAWSKIEJ. Downloaded on December 03,2023 at 09:30:05 UTC from IEEE Xplore. Restrictions apply. 782 In this section, we describe the functions of these blocks and procedure used to develop MATLAB based algorithm in a step by step way. A. EEG Signal Acquisition, Preprocessing and Buffering EEG data could be collected in real time from subject’s scalp by using different commercially available, cost-effective user-friendly portable Neuro-headset, such as Emotive, Xwave, Muse etc. [16]. For better resolution and higher bit rate Emotive which collects data using 14 electrodes located at different positions of the scalp (AF3, F7, F3, FC5, T7, P7, O1, O2, P8, T8, FC6, F4, F8, AF4) as suggested by the American Electroencephalographic Society(AES) [17] with a sampling rate of 128 Hz, 0.2-45Hz bandwidth, A/D converter with 16bit resolution could be used [18]. The recorded signals are transferred via Bluetooth. For removing artifacts resulted from surroundings such as power line noise, acquired signal could be preprocessed by using different filters including high pass, low pass, band pass and notch filters [7]. These primarily processed data could directly be interfaced by MATLAB in real time by calling a dynamically-linked library(dll) routine. However, it is difficult to continuously stream and process data in real time since it requires some time for acquiring data and computing those data. To overcome this, a buffering software such as FieldTrip, DataSuite could be used [19]. Further, some MATLAB based efficient tools, such as EEGLAB, BCILAB, BCI2000, OpenVibe, BioSig etc. are widely used nowadays for these sorts of BCI applications [20]. However, in lack of real time data acquisition system, we used sample EEG data similar to the output of these real time systems collected from PhysioNet [21] and DEAP databases [22]. The dataset we used were collected from six patients at different condition for a number of short durations with a 48 channel EEG device using standard 10/20 system at 512Hz sampling rate. We imported these data in MATLAB workspace and processed according to the algorithm discussed in next section, to translate these brainwave signals into command to control the audio tracks. B. Algorithm Development and Coding An algorithm to implement the proposed approach was developed as described in this section. Various samples of typical single channel EEG data, acquired from different people having different modes and same person for different modes, were loaded to MATLAB for analysis by using simple load command. The loaded data were further filtered with Butterworth filters to remove unwanted noises induced by powerlines, muscle activities etc. and then spectral analysis was carried out using fft algorithm. The frequency corresponding to the peak amplitude was found out using simple looping and logical commands to determine which brain wave band (δ, θ, α, β, γ) was dominant. MATLAB audioread command was used to play music tracks from the music library using soundcard. Different music tracks under three main categories, namely, relaxation, soft and motivational music, were selected regarding subject’s taste and preferences. These tracks were saved in wav format in the current MATLAB directory. The number of music tracks could be changed desirably. In real time applications, these music tracks would dynamically be changed based on the dominant band of the subject’s brainwaves which in turn would adjust the subject’s mood as audio feedback. The decision boundaries were defined in the program as depicted in the flowchart below. Fig. 3. illustrates the flowchart of the designed algorithm and Fig. 4. depicts a typical arrangement of music library. MATLAB version 8.5.0.197613 (R2015a) was used in 64 bit windows OS platform. Figure 3. Flowchart of the algorithm designed Authorized licensed use limited to: POLITECHNIKI WARSZAWSKIEJ. Downloaded on December 03,2023 at 09:30:05 UTC from IEEE Xplore. Restrictions apply. 783 different patients of different mental states. Results for six such sample data are summarized in TABLE I. When these samples were loaded to the MATLAB program we developed, it was observed that the music tracks corresponding to dominant band were playing expectedly in the audio player as mentioned in the table. For dominant δ band (during deep sleep), no music was playing. In real time system, these music tracks would act as audio feedback to the real subjects and adjust their moods thereby. The spectral resolution of different bands, along with the raw EEG signal of such a typical sample of 20ms duration is illustrated in Fig. 5. Music library • • • • • • • • • • • • Relaxation Music Track 1 Track 2 -----Track n Soft Music Track 1 Track 2 ---------Track n Motivational Music Track 1 Track 2 ---------Track n TABLE I. Figure 4. Arrangement of music library IV. PERFORMANCE EVALUATION To test the effectiveness of the approach, relative power spectral densities were calculated using Fast Fourier Transform(FFT) algorithm for a large number of single channel(F4) raw EEG segments(epochs) collected from RELATIVE SPECTRAL DENSITIES OF BRAINWAVES Sam ple δ Wave θ Wave α Wave β Wave γ Wave Music Played 1 5% 10% 15% 50% 20% Relaxation 2 8% 17% 45% 20% 10% Soft 3 10% 60% 15% 10% 5% Motivational 4 20% 28% 30% 20% 2% Soft 5 60% 20% 10% 9% 1% No Music 6 15% 17% 48% 12% 8% Soft Figure 5. Relative Spectral densities of different brainwaves Authorized licensed use limited to: POLITECHNIKI WARSZAWSKIEJ. Downloaded on December 03,2023 at 09:30:05 UTC from IEEE Xplore. Restrictions apply. 784 V. DISCUSSION AND CONCLUSION In this work, we studied an approach to adjust the mood of different physically and mentally disabled people by music. The feasibility and effectiveness of the approach were verified. Not only music, any kind of entertainment, along with or instead of music, can be selected for the library. Though we proposed the approach for real time applications, we could not implement it in real time because we could not afford to have real time EEG acquisition system at the time of experiment. However, we verified the system’s performance by offline data which ensured the effectiveness of our approach. We continue working to make it a fully real time system for real patients. Our proposed system can be potentially used not only in hospitals but also at home level for the aforementioned patients. This approach can further be extended for automatic mood regulation of any people in real time if the range of the real time EEG signal acquisition system is further extended and developed. The signals we used were collected with few channels thus covering a small area of the brain, more realistic brain signals collected through many channels that cover large area would give more accurate insight of the brain and then a more appropriate feedback could be given. REFERENCES [1] [2] [3] [4] [5] [6] PL Nunez, Ramesh Srinivasan. Electric fields of the brain: the neurophysics of EEG. Oxford University Press, USA, 2006. J Panksepp, Affective neuroscience: The foundations of human and animal emotions. Oxford university press, 1998. Y. Liu, O. Sourina and M. K. Nguyen, "Real-Time EEG-Based Human Emotion Recognition and Visualization," Cyberworlds (CW), 2010 International Conference on, Singapore, 2010, pp. 262-269. P. Hoole et al., "Autism, EEG and brain electromagnetics research," Biomedical Engineering and Sciences (IECBES), 2012 IEEE EMBS Conference on, Langkawi, 2012, pp. 541-543. Andrew Campbell et al. "NeuroPhone: brain-mobile phone interface using a wireless EEG headset", Proceedings of the second ACM SIGCOMM workshop on Networking, systems, and applications on mobile handhelds, ACM, pp.3-8, 2010. S Sanei, JA Chambers, EEG signal processing. John Wiley & Sons, 2013. [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] Christos Papadelis et al. "Using brain waves to control computers and machines", Advanced Human-Computer Interaction, vol. 2013. New York: Hindawi Publishing Corporation, pp.1-2, 2013. TM Vaughan et al. "Brain-computer interface technology: a review of the Second International Meeting", IEEE transactions on neural systems and rehabilitation engineering: a publication of the IEEE Engineering in Medicine and Biology Society 11, No. 2, pp. 94-109. 2003. Bi, Luzheng, Xin-An Fan, Yili Liu, "EEG-based brain-controlled mobile robots: a survey", IEEE Transactions on Human-Machine Systems, Vol.43, No. 2, pp. 161-176, 2013. JR Millán et al. "Combining brain–computer interfaces and assistive technologies: state-of-the-art and challenges", Frontiers in neuroscience 4, p. 161, 2010. A. Nijholt et al., "Brain-Computer Interfacing for Intelligent Systems," in IEEE Intelligent Systems, vol. 23, no. 3, pp. 72-79, May-June 2008. Li Xue, “Study on music and emotionregulation”, Journal of Jilin College of The Arts, no. 1, pp.7-10. 2015. G Buzsaki, Rhythms of the Brain. Oxford University Press, 2006. DW Croswell, "The evolution of biomedical equipment technology", Journal of clinical engineering Vol. 20. No. 3, pp. 230, 1995. KR Popper, JC Eccles, The self and its brain. Springer Science & Business Media, 2012. Dipali Bansal et al. "Real Time Acquisition and Analysis of Neural Response for Rehabilitative Control", International Journal of Electrical, Robotics, Electronics and Communications Engineering 8, no. 5, pp. 697-701, 2014. American electroencephalographic society guidelines for standard electrode position nomenclature," Journal of Clinical Neurophysiology, vol. 8, pp. 200-202, 1991 Y. Liu and O. Sourina, "EEG Databases for Emotion Recognition," Cyberworlds (CW), 2013 International Conference on, Yokohama, 2013, pp. 302-309. Arnaud Delorme et al. "MATLAB-based tools for BCI research", BrainComputer Interfaces. Springer London, pp. 241-259, 2010. Arnaud Delorme et al. "EEGLAB, SIFT, NFT, BCILAB, and ERICA: new tools for advanced EEG processing", Computational intelligence and neuroscience 2011, p.1, 2011. G. B. Moody, R. G. Mark and A. L. Goldberger, "PhysioNet: a Webbased resource for the study of physiologic signals," in IEEE Engineering in Medicine and Biology Magazine, vol. 20, no. 3, pp. 7075, May-June 2001. S. Koelstra, et al., "DEAP: A Database for Emotion Analysis; Using Physiological Signals," Affective Computing, IEEE Transactions on, vol. 3, pp. 18-31, 2012. Authorized licensed use limited to: POLITECHNIKI WARSZAWSKIEJ. Downloaded on December 03,2023 at 09:30:05 UTC from IEEE Xplore. Restrictions apply.