Mobile Medication Assistant for the Elderly
António Teixeira, Flávio Ferreira, Nuno Almeida, Ana Filipa Rosa
Department of Electronics, Telecommunications and Informatics (DETI)
Institute of Electronics Engineering and Telematics of Aveiro (IEETA)
Health School (ESSUA)
University of Aveiro
Portugal
S4S Presentation
Method
• An iterative/cyclic method composed by
– Requirements Analysis
•
•
•
•
•
Get Personas, Problem and Vision
Brainstorming (with people from different domains)
Identify Persona expectations
Construct Context Scenarios
Identify requirements
– Developing the application
– Testing the application
– Evaluation with users
S4S Presentation
2
Motivation
• The elderly present high levels of non-adherence
to medication
• High number of errors in taking medication,
concerns about side effects, lack of motivation,
memory issues
Goal: Mobile application for seniors to:
• improve adherence to medication and minimize errors
• serve as a testbed for research in the best uses of
different modalities with a particular focus on speech.
S4S Presentation
3
Persona & Persona Expectations
Persona: Primary
Name: Emília Rodrigues
Age: 62 years old
Mrs. Emília lives in Coimbra with her husband Filipe Rodrigues. She’s
housewife.
Mrs.Emília doesn’t know very well to work with electronic devices.
She is right handwriter.
She is diabetic and has arthritis in her right superior member. Her
health condition requires a regular and diary medication.
Mrs. Emília has the habit to call her daughter during the dinner
preparation. However she has some difficulty in doing the two
activities at the same time due to her limitation in the right superior
member.
She has weekly appointments in her local health centre for
surveillance.
Mrs. Emília would like to buy equipment that facilitates the contact
with her daughter. Something that allow her to control her
medication that she forgets to take.
S4S Presentation
Persona Expectations
•Able to use the same application with
arthrisis
• Allow speech interaction
• Help prevent gaps in medication
• Report the medication and side effects
• Could use the App even with poor ability
to work with electronic devices
4
Requirements
Data Requirements
User Requirements
• When to take a medication
• Inform in a casual way
• Why to take a medication
• Use speech for everything
• Medication side effects
• Credible and reliable
• Medication expiration date
• Avoid content overload and small
icons
• Maximum dose
• Personalization
• Distributed data
• Adaptation
• User profile
• Clinical information
S4S Presentation
• Extra care with language,
dimension and touch
5
System Architecture
• Windows Phone 7
• Web services to overweight features
• Several modules to control modalities and intelligence
User
Face Detection
S4S Presentation
6
Speech Service
SpeechWS
Add
Grammar
Grammar
Repository
Get
Grammars
ASR
TTS
S4S Presentation
Microsoft
Speech
Platform
• Add Grammar: allow the
input of a new grammar for
the service in GRXML or
ARPA format
• Get Grammar: gets the list
of all grammars in the
service and their names
• ASR: translates na audio
stream to text, using the
microsoft speech platform
• TTS: converts text to
speech
7
Image Service
• Aims to generate a synthesized image of a pill/packages
– Inestead of having a database with all the images
• Based on pill/package discription it return an image
• Information needed:
–
–
–
–
Pill/Package shape
Pill/Package pattern
Pill/Package colors
Pill/Package name
S4S Presentation
8
User, NLG and FaceDetector Service
• The User service that manage the user medication data and plan
• The User service provide the push notification service to display
alerts
• The NLG (natural language generation) service that automatically
generates sentences with the provided information
• The Face Detector service that detects faces in a
image and returns the size of the closer face
S4S Presentation
9
Main Features
• Alerts
– to remember users about
medication schedules
• Help
– on common medication related
questions
•
•
•
•
•
•
Help
Guide
Options
I forgot to take my medication?
What are the side effects of this medication?
Medication insertion and management by third
parties.
Configurable interaction
How to use application guide and tips
User knowledge adaption
S4S Presentation
Alerts
10
Multimodal Input
• Touch
– Accepted for all operations
• Speech
– MSFT Speech Plat. (MSP)
– Can be more intuitive and
faster than touch
What to do if a medication
intake was forgotten?
Choose:
TOUCH
1 ) “I forgot my
Medication”
2) “I forgot a medication"
3) Select the forgotten
medicine
VOICE
User says:
“I forgot to take X”
S4S Presentation
What are the side effects
of this medication?
Choose:
TOUCH
1 ) Select the right
option
VOICE
User says:
“What are the side
effects?”
11
Multimodal Output
• Graphics and Text
– Panoramic view
– Medication images generation
– Medication details
• Speech
– TTS (e.g. “The side effects of medication are…”)
– Use of Natural Lang Generation
Medication name
Quantity
Application method
Intake interval
MOBACC 2013
“You should take one tablet of Aspirin”
12
Adaption to Context
• User distance to mobile phone
– Relevant display elements resized if
user moves phone closer/farther
– Deal with vision problems (it will
depend of user – myopia or
astigmatism)
• Ambient noise level
– Sound sampled while user is not
speaking
– Volume level adjusted accordingly
MOBACC 2013
13
Further work
• Improve zoom adaption algorithm and add it to more views
• Implement the dark mode of the application
• Improve the auto voice detect algorithm
• Improve the speech interaction
• Improve “how to use” features
• Add features
– Auto adaptable help system
– Advice for more daily problems about medication intake
S4S Presentation
14
Thank you for your
attention.
S4S Presentation
15