Unlocking Human Potential through Technological Innovation –
Interdisciplinary Research at Georgia Institute of Technology
Ellen Yi-Luen Do
College of Architecture and School of Interactive Computing,
GVU Center, Health Systems Institute, and Center for Music Technology
Georgia Institute of Technology, Atlanta, GA 30332, USA
ellendo@gatech.edu
1. Be Well
What’s the first sentence you would say when you greet a friend (besides Hi, Hello or Howdy)? A
couple examples come to mind: How are you? How have you been? Good Morning! Nice to see
you! No doubt you can come up with a couple more that are similar in the spirit. There are of
course cultural and situational differences and people may express their greetings both audibly
and physically, such as handshaking, hugging, cheek kissing, or gesturing a wave or a high-five.
How about the parting words? Good-bye! See you later! Have a nice day! Good luck! Take care!
As evidenced in the social greeting and parting rituals, we human beings intentionally
communicate our awareness of each other’s presence and show interest in other people’s welfare
or happiness. The idea of wellness is related to human potential. What is wellness? Wellness is
the presence of wellbeing. Wellness is about being active, alive and vital. It concerns individuals,
communities and our environments.
Wellness has multiple dimensions. A popular notion of the six dimensions of wellness is
consisted of physical, emotional, occupational, social, intellectual and spiritual (Hettler 1976).
The physical dimension concerns diet, nutrition and physical activities. The emotional dimension
recognizes awareness and acceptance of feelings and behaviors. The occupational dimension
realizes achievement and enrichment through work. The social dimension encourages
contribution to the environment, community and the world. The intellectual dimension seeks
creative and stimulating activities. The spiritual dimension recognizes the search for values,
meaning and purpose in life.
Keynote Paper - National Science Council sponsored “Intelligent, Quality, Good Life” conference, Jan 20, 2009
at National Taiwan University, Taiwan, hosted by National Taiwan University's INSIGHT Center
(Center of INnovation and Synergy for IntelliGent Home Technology), Taipei, Taiwan
Achieving wellness is a Grand Challenge. We are concerned about the quality of life for
ourselves and for our society. As human beings we want to develop and cultivate our untapped
potential for a happy, healthy, creative and fulfilling life. Technological innovation may be just
the key to unlock human potential for the Holy Grail of wellness.
The Georgia Institute of Technology aspires to the commitment to improving the human
condition through advanced science and technology. As a top technological university with
genuine concerns about human conditions, Georgia Tech has created a culture of possibilities
fostering the growth of interdisciplinary research centers such as the GVU Center, the Health
Systems Institute and the Center for Music Technology. As a faculty member with a joint
appointment in the College of Architecture and College of Computing, I have enjoyed being an
active member in these research centers and participated in various research initiatives.
The research and innovating projects produced by these centers are too numerous to list them all.
To briefly demonstrate the spirit and the scope of the types of the research efforts, the rest of this
paper will focus on three areas of interest – health, awareness, and entertainment, each illustrated
by a couple projects, to form part of the picture of “unlocking human potential through
technological innovation.”
2. Health: for the old and the young, with capturing, recording and notification
Besides supporting healthcare services and capabilities ranging from surgery planning to
diagnosing and treating chronic disease, many opportunities exist to help people live a more
carefree and independent life.
The Digital Family Portrait (Mynatt 2001) helps family members at a distance to “keep an eye”
out for their love ones in a casual, lightweight manner. Figure 1 shows a picture of Grandma
displayed together with other family portraits. The picture is surrounded by the images of the
butterflies that change daily, reflecting some portion of Grandma’s activities recorded by the nonobtrusive sensors installed in the house that is an “Aware Home,” a living laboratory for
ubiquitous computing research that is a two-story single-family house built with sensing
infrastructure that is capable of knowing information about itself and the whereabouts and
activities of its inhabitants (Cory 1999).
Keynote Paper - National Science Council sponsored “Intelligent, Quality, Good Life” conference, Jan 20, 2009
at National Taiwan University, Taiwan, hosted by National Taiwan University's INSIGHT Center
(Center of INnovation and Synergy for IntelliGent Home Technology), Taipei, Taiwan
Figure 1. Digital Family Portrait of Grandma’s activities as butterfly icons
Although for privacy reasons, one may appreciate not being constantly under the surveillance
camera, there are times that a recording and monitoring service may come in handy. Take Cook’s
Collage (Tran 2007) for example, a capture system installed on the kitchen countertop that
provides visual summary of recent cooking activity serves as a memory aid. Imagine you are in
the middle of making a cake and you stop to answer the phone. When you get back, you wonder
if you have already put in either three or four cups of flour in your mixing bowl. The display
shows visual snapshots arranged as a series of comic strip panels for you to touch and review past
events. Figure 2 shows a person touching Cook's Collage display to check to see how many cups
of flour he already put into the bowl.
Figure 2. Cook’s Collage captures cooking activities for later review
Living well at home is a concern for people of all ages. Children with asthma enjoy playing
Keynote Paper - National Science Council sponsored “Intelligent, Quality, Good Life” conference, Jan 20, 2009
outdoors
asUniversity,
much as Taiwan,
others.hosted
Parents
are concerned
about their
children’s
at National
Taiwan
by National
Taiwan University's
INSIGHT
Center wellbeing even
(Center of INnovation and Synergy for IntelliGent Home Technology), Taipei, Taiwan
when
they are out of sight. With the Pervasive Remote Asthma Monitoring (Chhabra 2007) added to a
cell phone or a cute animal pendant worn on their necks, their coughing and wheezing can be
recorded no matter where they are. This monitoring could alert caregivers remotely and enable
them to quickly supply medical attention in the event of an emergency. Figure 3 shows that, with
embedded electronics and software, the voice can be processed through digital signal processing,
sent through the network and incorporated into patients’ electronic medical records.
Figure 3. Pervasive Asthma Monitoring helps detect patients’ coughing and wheezing patterns
If you are disabled and confined to a wheelchair at home, Personal Robotic Assistant (Kemp
2008) can turn your world into a clickable interface with a laser pointer. If you want your robotic
assistant to pick up a toy on the floor and deliver it to your friend on the other side of the room,
all you need to do is to point the green laser at the object first, and then at the person. Your
robotic helping hand will then follow your command and deliver the object. Figure 4 illustrates
how the robot would perceive objects and a person, instructed through green laser pointer.
Keynote Paper - National Science Council sponsored “Intelligent, Quality, Good Life” conference, Jan 20, 2009
at National Taiwan University, Taiwan, hosted by National Taiwan University's INSIGHT Center
(Center of INnovation and Synergy for IntelliGent Home Technology), Taipei, Taiwan
Figure 4. Instructing the Personal Robotic Assistant by clicking on highlighted objects with a laser pointer
3. Awareness: of self & the environment, for physical, emotional, social & spiritual wellness
Information technology can serve as a medium to help us connect with ourselves, with other
people, and with our environment. Applications in a hospital setting may be designed to help
relieve anxiety by the patients and their family members. Applications at a household that inform
people about energy consumption patterns may help residents monitor their energy use and
modify their behavior patterns. Application in a prayer practice may even help people connect
with their spiritual needs.
Being sick is a no fun. It is uncomfortable and stressful. While confined in a hospital bed, a
patient may feel overwhelmed and alone. Being able to learn about one’s own illness and medical
procedures, or to communicate with clinicians, friends and families may help alleviate the stress.
The Patient Interactive Communication and Learning System (PILS) simplifies the patient's
hospital communication experience by combining communication tools into one easy-to-use
system mounted on the patient bed. As shown in Figure 5, PILS has a one-touch call button that
connects to nurse’s video phone. Family video conferencing, educational video, entertainment
media, and vital signs information are all just “one-touch away.” The entertainment system also
doubles as display for medical records so that the clinicians could use it to explain medical
conditions and treatment options to the patients and concerned family members.
Keynote Paper - National Science Council sponsored “Intelligent, Quality, Good Life” conference, Jan 20, 2009
at National Taiwan University, Taiwan, hosted by National Taiwan University's INSIGHT Center
(Center of INnovation and Synergy for IntelliGent Home Technology), Taipei, Taiwan
Figure 5. Patient Interactive Communication & Learning System mounted on the bedside connects patients,
care givers & family members at home, monitors vital signs and provides learning opportunities
Or consider energy awareness – we are all concerned about climate change and our impact on the
environment and the world. Would you like to be aware of the energy consumption at your home
and change your usage pattern? Would you adopt an electronic pet for this purpose? The Energy
Puppet (Abdelmohsen 2008) is an ambient display device that provides peripheral awareness of
energy consumption for individual home appliances. The display produces different “petlike”
behavioral reactions according to energy use patterns of the appliances to give homeowners an
indication of their energy consumption. The puppet would raise its “arms” in victory to display
normal consumption rate (see Figure 6), or its “eyes” would change color to red and “roar” to
warn the homeowners when the specific appliance reaches high consumption rates.
Figure 6. Energy Puppet raises its arm to celebrate low energy consumption in the home.
Sun Dial (Wyche 2008) is an application that supports Muslims’ prayer practices by showing the
Keynote Paper - National Science Council sponsored “Intelligent, Quality, Good Life” conference, Jan 20, 2009
natural
environmental
cues hosted
of thebymovement
of the
sun through
theCenter
sky pictorially in
at National
Taiwan
University, Taiwan,
National Taiwan
University's
INSIGHT
(Center of INnovation and Synergy for IntelliGent Home Technology), Taipei, Taiwan
the screen of
a mobile phone. The ritual involves several cycles and prayer takes place during a “window of
opportunity,” instead of an exact time. Therefore, an alarm clock function is not appropriate. With
a simulated sun movement through the sky on a cell phone, even people in an interior space with
no view to the sky can perceive the different phases of the day for prayer opportunities. This
project provides an example of how prayer as an activity can be supported with technology.
Figure 7 shows the sketch used in the study with the Muslim community and the resulting screen
shot for Sun Dial showing a mosque in profile and the sun locations in the blue sky background.
Figure 7. Sun Dial provides environmental cues on a cell phone for prayer opportunities.
4. Entertainment – music and technology
Music is an important thing in life. We hear it everywhere: in elevators, vehicles, concert halls
and shopping malls. Music is an art form that helps us express ourselves, and our feelings. Music
is fun. An upbeat happy music can cheer you up. A soft melody can soothe you. Appreciation of
music is more than just entertainment. It also has social, cultural, and emotional implications.
Music helps promote wellness, manage stress, alleviate pain, express feelings and improve
communication (Campbell 2000, AMTA 2008).
Imagine finding a musician that who can jam with a group of players and produce inspiring
rhythms in various varying speed and intensity that go beyond human player’s ability? Haile is an
interactive robotic percussionist that can listen to live players, analyze their music in real-time,
and use the results of the analysis to play back in an improvisational manner. Besides being able
to imitate human musicians, Haile can sense, analyze, and react to perform with acoustic diversity
and dynamic range, and with varying velocities between the two arms that are difficult for human
players. The experience of playing with an interactive robotic percussionist may facilitate a
Keynote Paper - National Science Council sponsored “Intelligent, Quality, Good Life” conference, Jan 20, 2009
at National Taiwan University, Taiwan, hosted by National Taiwan University's INSIGHT Center
(Center of INnovation and Synergy for IntelliGent Home Technology), Taipei, Taiwan
musical experience that is inspiring and encourages novel expressions and interactions. Figure 8
shows a drumming session with the interactive robotic percussionist.
Figure 8. Drumming with an interactive robotic percussionist
Too busy and can’t find time to practice piano? How about wearing a glove that can help you
learn to play music while you’re at your desk or on the move. The Piano Touch (Huang 2007)
project provides a new way for people to learn to play the piano. Wirelessly synchronized with an
iPod, cell phone or other music playing device, the Piano Touch is a light-weight glove outfitted
with little vibration motors to cue the musicians about which finger they need use to play the next
note. Figure 10 shows a converted golf glove with vibration electronics playing on a lighted
keyboard. A pilot study shows that students learned the songs that they were practicing with the
Piano Touch glove much better than the songs that they were practicing without the glove. We
are currently working on making Piano Touch an engaging, pervasive hand rehabilitation aid for
patients who suffer loss of functionality of their hands.
Figure 10. Piano Touch is a lightweight glove that cues which fingers to play.
Keynote Paper - National Science Council sponsored “Intelligent, Quality, Good Life” conference, Jan 20, 2009
at National Taiwan University, Taiwan, hosted by National Taiwan University's INSIGHT Center
(Center of INnovation and Synergy for IntelliGent Home Technology), Taipei, Taiwan
5. May the Force be with You!
In the beginning of this paper we discuss the meanings of our greeting and parting phrases. We
have covered topics of human potential and the dimensions of wellness. We have seen interesting
technological innovation projects addressing different aspects of wellbeing in our everyday lives about health, awareness and entertainment. These projects would not have been possible without
the culture of interdisciplinary collaboration, the synthesis of research across many domains, and
the aspiration to support fundamental human needs. Let’s move forward together towards a bright
and exciting future!
All good things must come to an end. Let me quote from an old Irish Blessing here: “May the
road rise up to meet you, may the wind be always at your back. May the sun shine warm upon
your face and the rain fall softly on your fields.” And until we meet again, my friend, be well.
6. References
Abdelmohsen, Sherif Morad and Ellen Yi-Luen Do (2008) “Energy Puppet: An Ambient
Awareness Interface for Home Energy Consumption,” in SID 08, 7th International Workshop on
Social Intelligence Design, Designing socially aware interactions, Dec 3-5, Universidad de Puerto
Rico, San Juan, PR http://cdr.uprrp.edu/SID2008/default.htm
AMTA (2008) American Music Therapy Association, http://www.musictherapy.org/
Campbell, Don (2000) The Mozart Effect for Children: Awakening Your Child's Mind, Health
and Creativity With Music, William Morrow Publishing
Chhabra, J., Asmi, A., Ragavan, L, (2007). Pervasive Asthma Monitoring, in Pediatric Center of
the Future class, taught by Do, E. YL., Zimring, C., Cowan, D., Lamb, G. Lindgren, J., Jones, J.,
http://www.hsi.gatech.edu/pedcenter
Fischer, T., Davis, T, Vargas, C. (2006) Patient Interactive Communication and Learning System
http://cool.coa.gatech.edu/patientroom, the Patient Room of the Future class at Georgia Tech,
taught by Do, E. YL, Zimring, C. Cowan, D., Lamb, G., Kahn, S., Mullick, A., Winegardend, C.
Hettler, Bill, (1976) Six Dimensions of Wellness, National Wellness Institute (www.nwi.org),
and http://www.hettler.com/sixdimen.htm
Huang, Kevin, Ellen Yi-Luen Do, Thad Starner, (2008) “PianoTouch: A Wearable Haptic Piano
Instruction System For Passive Learning of Piano Skills,” in ISWC 2008, 12th IEEE International
Symposium on Wearable Computers, http://www.iswc.net/, pp 41-44, Sep 28 - Oct 1, Pittsburgh,
Pennsylavania
Kemp, Charles C., Cressel Anderson, Hai Nguyen, Alex Trevor, and Zhe Xu, (2008) A Pointand-Click Interface for the Real World: Laser Designation of Objects for Mobile Manipulation,
3rdPaper
ACM/IEEE
on Human-Robot
(HRI),
241-248
Keynote
- National International
Science CouncilConference
sponsored “Intelligent,
Quality, Good Interaction
Life” conference,
Jan 20,
2009
at National Taiwan University, Taiwan, hosted by National Taiwan University's INSIGHT Center
(Center of INnovation and Synergy for IntelliGent Home Technology), Taipei, Taiwan
Kidd, Cory D., Robert J. Orr, Gregory D. Abowd, Christopher G. Atkeson, Irfan A. Essa, Blair
MacIntyre, Elizabeth Mynatt, Thad E. Starner and Wendy Newstetter. (1999) "The Aware Home:
A Living Laboratory for Ubiquitous Computing Research," in the Proceedings of the Second
International Workshop on Cooperative Buildings - CoBuild'99. Position paper, October 1999.
http://awarehome.imtc.gatech.edu/publications
Mynatt, E.D., Rowan, J., Craighill, S. and Jacobs, A. (2001). "Digital family portraits: Providing
peace of mind for extended family members." In Proceedings of the ACM Conference on Human
Factors in Computing Systems (CHI 2001). Seattle, Washington: ACM Press, pp. 333-340
Tran, Q., Calcaterra, G., Mynatt, E. (2007) "Using Memory Aid to Build Memory Independence."
Proceedings of HCII: Human Computer Interaction International, 959-965
Weinberg, Gil and Scott Driscoll (2007) The interactive robotic percussionist: new developments
in form, mechanics, perception and interaction design, Proceedings of the ACM/IEEE
international conference on Human-robot interaction, Arlington, Virginia, USA, Pages: 97 - 104
Wyche, Susan, Kelly E. Caine, Benjamin Davison, Micheal Arteaga, Rebecca E. Grinter, (2008)
Sun Dial: Exploring Techno-Spiritual Design through a Mobile Islamic Call to Prayer
Application, CHI extended abstract: 3411-3416
Keynote Paper - National Science Council sponsored “Intelligent, Quality, Good Life” conference, Jan 20, 2009
at National Taiwan University, Taiwan, hosted by National Taiwan University's INSIGHT Center
(Center of INnovation and Synergy for IntelliGent Home Technology), Taipei, Taiwan