Virtual Assistants for Mobile
Devices Using Voice and Gesture
Technologies
Kishore Prahallad (IIIT-H), Radhika Mamidi (IIIT-H), Anoop
Namboodri (IIIT-H), Sandhya Kode (IIIT-H), Shobha Reddy
(Methodist college of Engg.), N Balaji (VNR college of
Engg).
Need for futuristic interfaces
•
•
Existing keyboard/touch based interfaces are not handsfree
o Hard to workwith, and are not handsfree
o Usage is limited to literate and abled-persons
Proposal - a futuristic interface
o Enables literate and illiterate, differently-abled to use the
mobile devices
o Communication, information retrieval and entertainment
o Uses voice and gesture based technologies for a multi-modal
mobile interaction and computing
Virtual assistant
•
•
•
•
•
Voice commands
o Call my doctor
o Sufficient knowledge of user and context built in
Converse (dialog) with the device for information access
o Could you check whether train tickets are available to Tirupati
tomorrow?
Gesture based recognition
o hand-"write" SMS/Emails
Technologies: speech recognition, natural understanding,
expressive speech synthesis, gesture recognition, affective state
modeling
Applications: multimodal interfaces for information access
(socionity), differently-abled, teaching and learning.
VNRVJIET : Design of Multimodal Communication
Interface
•
•
Objective : The objective of this research proposal is to design a Multimodal human
computer interface software which helps in addressing the communication & interaction
needs of People with Autism
The Software must support all possible modalities like text, voice, touch & eye gestures in
providing required assistance to people with Autism in the areas of
— Communication, Speech Development, Social Interaction, Computer Aided Learning,
Assessment of Learning skills
Faculty Team
Research Engineer / Student
Team
External Advisor
Dr. N.Balaji - Expertise: Embedded
Systems, Signal Processing
Mr. Naveen,M.Tech - Research
Engineer in the domain of
Wireless Sensor Networks
Mr. SVN Narayana Rao, Sr. Engg Mgr,
HP India
Dr. Y.Padma Sai - Expertise : Biomedical
Signal Processing
Ms. P. Anoosha, MS(USA),
Communications
Dr. L.Padma Sree – Expertise : Intrusion
detection
Research Scholars from
ECE,CSE&EEE
Dr. P Srihari – Expertise: Communication
Engineering
Affective HCI: Role of Speech Prosody
•
To integrate human emotions in intelligent HCI for improved interpersonal
communication:
- An interface between human and computer sensitive to humans affective state .
-
The interface sense, process and interpret the prosody features like pitch, stress,
rhythm and speech rate, etc.
Accordingly, the recipient mobile indicates an icon which changes in real time
according to the emotional state of the user.
Investigator: Methodist College Of Engineering & Technology
-
Faculty Team
Student Team
Dr. Shobha Reddy
Research Associates (PhD)
Mr. Pawar Vinod, (PhD)
Jr. Research Assistants (MTech)
Mr. Ramesh Babu , MTech
Mr. Srikanth, Mtech
Mrs. Sushma, (PhD)
Mr. Sravan Kumar, MTech
Mr. Syed Ather, MTech