Diabetes Forecast System
Final Report
Brian Mitchell
(retlif.maps@gmail.com)
Ben Parker
(bwparker@gmail.com)
Ken Chen
(kenken1982@hotmail.com)
Alberto Flores
(aaflores@gmail.com)
December 6th, 2005
Abstract
Glucose meters and Diabetes management are important subjects of attention for
people diagnosed with type I diabetes. The process of monitoring one's own blood
glucose with a glucose meter is ofter referred as self-monitoring of blood glucose or
"SMBG". There exist a significant amount of tools available today that provide several
alternatives to this process.
The Diabetes Forecast System (DFS) is a tool designed to provide real-time
predictions of blood sugar levels (BGL). Forecasts are based on readings from a current
diet, latest insulin doses, current workout schedules as well as current sugar levels. These
inputs are utilized to produced a forecast of BGL for a specific period of time in the
future. According to our task analysis and requirements, we propose additional features
that unlike other devices available in the market, improve the user experience as well as
the frequency of user input.
Credits
1. Brian Mitchell – current chief architect behind DFS. His skills in Flash as well as his
great knowledge of diabetes makes them a key member in providing support as well as
feedback for improvements. He was the lead person for deliverable number two were
he led the rest of the team in obtaining and filtering references for the development of
this product.
2. Ben Parker - one of the developers for the DFS team. His active commitment and
experience developing interfaces at his work-place provided great feedback during our
regular meetings. Ben took the lead on two deliverables: Compiling the user tasks and
compiling the results of the usability tests. He also designed most of the flow of DFS
as well as compiled other ideas to improve team work.
3. Ken Chen - provided a great touch of diversity and foreign feedback to DFS. His
contributions are found in the History module of DFS. Ken is also a developer for the
DFS team and took the lead on the fourth deliverable as he prepared and led the team
in building an effective task list and questionnaire.
4. Alberto Flores - the driving force behind the last phase of the project (the final report
and the website) and in the production of great ideas throughout the semester. He also
took the lead in designing prototypes for DFS during the initial development process
and organized meetings to improve communication process among team members.
Introduction
With type 1 diabetes the body does not produce insulin. A person with type 1 diabetes,
therefore, is tasked with simulating the insulin production of a healthy pancreas through
insulin shots. The balance between sugar intake and insulin is delicate and can be very
difficult with mistakes causing potentially deadly results. If one takes too much insulin
they can become hypoglycemic (low blood sugar levels) or hyperglycemic (high blood
sugar levels). The tools currently available on the market give a person with diabetes the
ability to accurately measure their current blood sugar levels which is a tremendous help
in keeping blood sugar regulated. However, that measurement is just a snapshot in time
and does not give the user any indication to where their glucose level is heading nor does
it help the user decide what actions should be taken to correct their condition if/when it
becomes necessary.
Our proposed system will offer our users this advantage. It will take into account the
four major factors in glucose levels (carbohydrate intake, insulin intake, level of activity,
and current blood glucose level) to give the user a prediction of where their blood glucose
level will be in the near future and it will help them make healthy decisions to keep it in
the normal range. The device would even notify the user if they have forgotten to test
their glucose level recently and may be in danger of falling outside healthy limits.
We hope to create a tool that targets diabetic users of almost all ages, sexes and races.
Ideally it would be a tool with "universal usability" among type I diabetics. One of the
member's of our group has type 1 diabetes and served as one of our main subjects for
real-life tasks that diabetics need to perform to monitor and maintain their BGL.
According to our research, we found that “SMBG “ is a significant challenge for many
users. Our study showed that the majority of people with type I diabetes have a good
sense of awareness about their health condition. Their knowledge about nutritional facts
are well above the average person. This condition also provides most potential users the
confidence to rely a lot more in their own experience than in a device which can assist in
self management of current conditions. Our goal is to provide a companion to such user,
attemping to make a device that is appealing and practical to the user.
Presentation of Design
Our final design is the result of quite a lot of work, which is described in the
"Development Process" webpage.
In order to explain the main work flow of DFS we present a transition diagram for the
final version (3.0). This diagram was revised several times and improved after receiving
important comments and results from our usability tests.
Figure 1 – DFS transition diagram
Careful consideration was given to a simple interface. This is well described in Figure
1. Here we can see how there are basic functions that are needed and process are parallel
(similar) in all operations (saved pop-up).
The splash screen starts things off after you press the begin button (which will not be a
part of the final product). Users have the ability to turn it off if they'd like to from the
settings screen.
Figure 2 – Splash screen
The next thing you see is the home screen. From this screen (and almost all the others
- see Figure 1) you can get any of the other major screens defined by the links on the
toolbar and the settings screen which has a link in the bottom right corner of the screen.
On the home screen you will find the prediction, which does not actually work because
we couldn't get a prediction model running. We investigated Markov chains, but these
are complicated (they are the basis of Google's PageRank algorithm) and would have
taken a long time to implement. We would have liked to have gotten the graph working
as well, but could not, but that would be one of the next things to implement. One
additional thing to note is the help menu, which can be accessed from any screen and
provides useful feedback about what to do on any given screen. Below is a shot of the
home screen and one of the help screen.
Figure 3 – Forecast screen
Our usability test showed the excitement for this feature. It is the main purpose of our
service, yet the most difficult to implement as well. Our findings also show that different
attempts to compute forecast based on human input have been made, nevertheless these
have also noted the lack on consistency in user input. Our findings indicate that users
found this screen appealing. Full implementation of this feature was preferred but also the
complexity of this computation was also understood by most subjects.
One important feature that was also considered was the ability to upload your data to
the DFS device. Since this is the primary function of forecasting, this button stands next
to the forecast chart where users can add input and thus assist in maintaining an accurate
reading. DFS will also support different formats (xls, csv, rtf, pdf) for exporting data to
different clients (PC, email, cell-phones). Our goal in providing this king of
communication is to assist users in share their information if allowed in order to provide
medical assistance if needed.
Figure 4 – Home help screen
As explained in Figure 1, all our screen provide a help screen that fades in and out as it
appears on the screen (see Figure 4). Basic functionality and purpose of the function is
displayed in the screen. This “fading” pop-up system also proved to be interesting to
many users as well.
We go now to the next screen: the area where you have your blood sugar level
measured. The idea is to put a glucometer test strip into the pda, then apply a drop of
blood to the strip, as you do with any glucometer. However, obviously we couldn't do
that for usability testing so we created a false randomized reading generator. Just click on
"get BGL" and you're given a value between 60 and 300, I believe. This data is stored,
and can be viewed from the history screen.
Figure 5 – BGL input screen
The next screen is the diet screen. It uses an accordion menu to break up food choices
by food group. Foods are added to the list from the settings screen and the carbohydrate
values are added at that time. You can get to this screen by pressing the "new food"
button. If you add an item by mistake, you remove it by selecting it from the column on
the right and pressing the lower green button. Add a food by selecting on the left and
pressing the upper green button (Figure 6). Save your selections into the pda by pressing
"save".
Figure 6 – Diet input scr een
Have you exercised? Exercise is a vital input that can affect BGL measurements
significantly. We identified this fact and added a feature that most user found as a very
helpful one. This has an effect on future BGL's and would be used in a working
prediction algorithm. Select the type of exercise you have done and the number of
minutes that you've worked out, then press "save" (see Figure 7). If you don't see the type
of exercise you've done, please add an exercise by pressing "new exercise".
Figure 8 – Exercise screen
Now, each time you inject yourself, record it (Figure 9). You can keep track of slow and
fast acting insulin with DFS. Use the sliding bars, which still need a little work, or the
"numeric stepper" devices as Flash calls them to make the entry perfect down to the unit.
Figure 9 – Insulin screen
So now you want to see what you've done. The history screen shows previous meals,
insulin injections, work outs, and BGL readings. You can modify anything, as well, so if
you are recording something late, then come here, double click on the entry and then
change it! (see Figure 10)
Figure 10 – History screen
The settings screen has the most options, of course (Figure 11). Set up basic info such
as age and weight, but you can also add or subtract foods and exercises. One of our
commentors suggested adding a "return to default" button, and it was a great idea. We
also have a setting to stop showing the default screen upon start up, to save time in
getting to what you need. We would like to add more themes and language translations,
so the setup for these is there, but they are ineffective.
Figure 11 – Settings screen
Development Process
Our final version (3.0) is the product of many suggestions of improvemts as well as
different prototypes (presented below). Different features that were originally designed,
but never implemented are purposely left for future developments.
Prototype I - DFS Iconic Prototype
The iconic prototype incorporates all of the features that we want to include: recording
exercise, diet, insulin and blood glucose levels (BGL), but does this mainly with icons
instead of words. Predictions and measurements of readings cannot be pictures however,
and must be described in words.
General Features
The display will automatically turn off after 2 minutes. The unit is not waterproof;
please keep it in the case it comes with. There are pockets in the case for lancets, syringes
and two 10 mL unit bottles of insulin. It doesn't look obvious how deletions should be
done, but it's easy. The output fields at the bottom of some screens are actually editable,
in other words, the items can be removed. To do so, simply double tap and drag the item
with the included stylus from the text box to some point outside that box and it will
automatically disappear from the list.
HIPAA (Health Insurance Portability and Accountability Act) guidelines prohibit the
transmission of medical documents without the owner's permission. For this reason,
downloading data from the PDA requires a password. If users should find that they've
misplaced the device they can login to the DFS Website and have the PDA remotely
locked: the company will send it a message that will freeze the screen and encrypt the
data inside. If found again the company will send a message to the device decrypting the
data.
BGL Screen
You can get a reading from any screen. Simply insert a test strip into the top right
corner of the meter and the BGL screen will come up. No need to return to the main
screen and find the right screen to go to from there. An icon showing a drop of blood will
come up indicating that you should use a lancet to prick your finger now. It takes 5
seconds for the system to figure out your BGL once you add a droplet to the test strip.
This will be displayed on the screen. After 30 seconds, the display will return to the main
screen. Coding your glucometer is normally required based on test strips (numbers can
range from 1 - 49, depending on the meter). Test strips come coded with one of these
numbers and users are responsible for entering this number in the glucometer before they
use any of the strips. If they forget to do this, however, the reading will not be truly
accurate. The DFS meter codes strips automatically.
Main screen
Displayed are the most recent 5 BGLs. Below this, the readings taken within the past
24 hours are graphed. At the very bottom of the screen is listed any warnings for low
readings (this is usually anything < 60, but the number can be manipulated in "Setup")
and high readings (anything > 150, again this is adjustable)
Setup screen
The button at the top left of the Main screen, "Setup", brings you to the Setup screen
where you can adjust several variables, such as the range limits for normal blood glucose
levels and what should happen when your blood sugar level is critically low or high
(these values are also adjustable, but cannot be set < 40 or > 350). Responses to critical
levels could include wirelessly notifying the local police department by calling 911, or
sending an urgent call to a relative or friend. This option is adjustable in the Setup section
as well. Things such as brightness, contrast, color settings, and sound can be manipulated
here as well. The date and time can be set in this section.
Foods can be added along with their nutritional content here. The average serving that
you would eat should be included so that you don't need to enter the size of a portion on
your plate. This may lead to inaccuracies and a loss of interest in entering foods eaten if it
requires that much work. If you eat twice or three times what you normally do, just add
the food again as many times as is needed.
Exercise machines that the DFS meter can interact with should be added here. The
DFS system comes with a CD that includes drivers, or small pieces of software that
communicate between the DFS meter and the electronic device, such as a treadmill,
pedometer, or heart rate meter. Connecting to such a device provides the meter with the
most accurate information possible. You should indicate here the type of insulin(s) that
you are using. This will allow the meter to make the most accurate BGL forecasts.
If you accidentally submit a record that had misinformation in it, you would want to
delete it. However, giving such privileges has a disadvantage; that being the malicious or
unintended deletion of data by others should the device fall into the wrong hands. You
can set permissions for the current (and only "logged in") user here. Choose not to give
delete permissions if you do not make many mistakes with data entry, and can live with
an occasional bad record. Know, though, that permissions can be changed at any point in
case an egregious error is made on an entered record.
Downloading
Information can be downloaded from the PDA to your computer can be done from this
screen. This requires a password to comply with HIPAA. Many screens have a "When"
and "Done" button. The "When" button allows you to select an earlier (but not later time)
time for when the action occurred, like eating. if for example, you ate two pieces of fruit
three hours ago, but forgot to enter the items at that time, you can include them later, but
choose the "When" button and include the right time for when they were done. The
"When" button does not need to be tapped if the action to be performed occurred or will
occur within 10 minutes prior or will happen in the next 10 minutes.
Food screen
Nutrient info for more than seven thousand foods are included, downloaded from the
USDA nutrient database (http://www.nal.usda.gov/fnic/foodcomp/search/). Select foods
that you normally eat and they will be added to the output screen at the bottom of the
interface. This output box can be manipulated, i.e., the items listed here can be doubletapped and dragged outside the box by the stylus to remove the item from the list of foods
eaten. Either choose the colored diagrams for the types of foods you would eat, or choose
specific foods from the list on the right. The list of foods consists of the four most chosen
foods from the database of foods. This list may change as the types and frequencies of
foods you eat changes.
Exercise screen
Include the amount and intensity of exercise that you do. Since much of diabetes
involves making decisions based on approximate information, we suggest the method of
entering approximate workout times and intensity levels as an acceptable means of
recording exercise habits. Asking users to enter average heart rates and average mile
splits for runners, among many other numbers, would be asking too much we feel and is
not necessary to making an accurate forecast of future BGL's. The option to download
this accurate info from a machine is available and would make predictions more accurate
than they could be.
Insulin screen
Simply slide the marker along the rail to indicate the amount of each type of insulin
you plan on taking. That's all there is to it, except to know that you press "Done" when
the previous task is completed.
Laboratory screen
This screen allows you to look into the past or make an educated guess about the
future. Choose a microscope slide to learn about your previous entries concerning diet,
exercise, insulin intake or BGL readings. You will be allowed to see a graph of readings
from up to 30 days in the past for each of these four domains. At the bottom of this
screen, see forecasts for the next 12 hours. Drag the marker along the rail to the hour in
the future that you want to learn more about. The further out you go, the less accurate the
forecast given will be. There is no "DONE" or "SUBMIT" button here, once you let go of
the marker after dragging it, the prediction will be made from where it is now located.
Prototype II - DFS Personal edition
DFS Personal edition consists of a main layout used in every screen. It contains four
main buttons at the bottom to move among the main services offered by DFS. These are:
 My Forecast
 My Insulin
 My Diet
 My Journal
At the top left corner, there is a customizable add-on to the interface named "My DFS"
where each user will be able to performed costumed-based operations to a particular unit
(e.g. changing themes, exporting data to an external device, show other configurations,
etc). At the top right corner, there is a status bar, which by default is in green. It will only
change to color red when blood glucose levels are under a dangerous margins. This will
serve as an alert to the user. Additional vibration functionality to the device is available
via "My DFS" settings configuration.
Main Screen
The main screen displays the version number and serves as the main screen to ensure
that all services have properly loaded into the system. This indicates that the system is
ready to continue. The version number is useful to indicate the type of operating system
in use by DFS as it's currently configured in the unit. This will serve for customer service
information and updates.
My Diet Screen
This screen will display information about a particular product that the user of DFS is
about to take (or has taken). This will allow users to enter information that are relevant to
keep a more accurate reading and forecast for DFS. Once a selection is done, users need
to enter the product, quantity and units. Then, users will need to click on "Add" to submit
the entry. Additionally, information about the product in question is described for
purposes only relevant to DFS and calculating BGL's.
My Forecast
This screen is the main interface for DFS defining brand. It provides an interface to
display BGL's in a configurable period of time. The specific level status is described by a
color where red is always a sign of danger. At the bottom, a complete description of the
current status is display where current average, max, min and hit percentage (actual vs
forecasted results) numbers are displayed. BGL average, max and min are numbers for
the current day. The hit percentage is information that refers to the amount of accuracy in
comparing forecasted results and actual BGL level. Error margins are based on inaccurate
entries in the "My diet" section which could provide inaccurate readings. The Stat line (in
the graph) shows real-time information. When the margins go beyond a specific bound,
the interfaces displays an alert (red background). It can optionally vibrate and email a list
of recipients (e.g. doctors, 911, family, etc) when configured in "My DFS"
My Insulin
This screen displays the insulin doses needed and makes recommendations based on
fast and slow acting. Moving the arrow level on the measuring bars, displays (and
automatically moves the other acting type) It also provides a "Needed coverage"
percentage which displays the amount of insulin which is needed an how much of it is
covered with the selection. This is a measurement aid for users to be assured of the
correct doses when in need. It will also allow to record the entry to ensure accuracy of
readings and forecast.
Usability Test Process
Our goal when running the test was to have the users run a series of tasks in a very
relaxed environment. We broke our tests into a five step process. We began with a simple
introduction of ourselves and our systems. The users then took a written pre-survey that
included basic personal information such as age, weight and when they discovered
diabetes. At that point we had them run through a series of pre-determined tasks and
recorded thier comments, suggestions and actions. We gave them a post-survey that asked
them for their thoughts and opinions on the experience and finished the process by
allowing to voice and additional comments they may have.
We pooled our users from the people we knew in our day to day life. They included
men and women, ages 21-60, of a eclectic cultural background. One of the tests was
actually administered over the internet to user in another country! We also attempted to
interview users who were patients of an endocrinologist that we know, but because of
HIPAA regulations, we could not find out who those potential users might be.
Here is the usability test and pre/post-tests that we asked our testers to complete:
Diabetes Forecast System (DFS)
UTG: Usability Test Guide – For DFS Agents’ Eyes Only
Introduce/Thank
Start out by introducing yourself and thanking the tester for coming to help you. This
may be a good time to offer them something to eat or drink. Try to make them feel as
comfortable as possible and let them know that you appreciate them donating their time
to you.
Explain/Outline
Explain to them what our system does, or more specifically what its supposed to do.
Let them know that we hope to make the interface better and welcome their input into any
potential drawbacks of the current design as well as suggestions for improvements. Make
it clear that the system is being tested, not them; problem are expected/invited and they
signal a fault on our side not a deficiency of the user. Warn them that some part of the
interface are not complete and they may not be able to FULLY complete all tasks and that
is ok. Briefly outline the process that they are about to engage in: A written pre-test, a
series of observed tasks and finally a written post-test. Ask them if they have any
questions and once their questions have been answered ask them to sign the paper.
Pre-test(Page 2)
Hand them the pre-test questionnaire and writing utensil and ask them to fill it out.
Allow them to skip any questions if they feel uncomfortable and you may provide as
much assistance as they require, just make sure the answers are their own.
Run Tasks(Page 3 & 4)
Start the system and give the tester the driver’s seat. Allow them a moment to look at
the screen and become comfortable. Giving them as much time as they require, read each
task to them and ask them to carry it out. DO NOT read the steps of the tasks to them, we
are trying to see if the system is designed well enough that it can be used without explicit
directions. At this time take COPIOUS notes on what they accomplish, what confused
them, what mistakes they made, any comments they make, what they seem to like/dislike,
ANYTHING of relevance. If they become completely stumped with a task you may show
them the NEXT step they must take, then allow them to move on from there – again
observe and record their actions and comments.
Post-test(Page 5)
Once they have finished all the tasks, give them the post test and writing utensil and
ask them to answer the questions. This should run similar to the pre-test.
Closing
Thank them again for their time and ask them if there is anything they would like to
add or comment about. Ask for final thoughts or suggestions. Thank them again and
allow them to continue their day.
Pre-test Questions
1. What is your age?
2. What is you native language?
3. Do you consider yourself techno-savvy?
4. How long have you had diabetes?
5. When you learned that you had Type I diabetes, what were some of the most difficult
things to do? Why?
6. How well do you know your condition and what steps you need to take to maintain
balance?
7. Do you think a tool that helps you to forecast your BGL over the next few hours would
be useful?
8. Are you a vegetarian?
9. Do you exercise a lot? If so, how many hours do you average exercise a week?
10. How often do you record your BGL?
11. How relevant for you is to know your BGL? (From a scale of 1-10)
12. What are the most and least frequent functions you use with your BGL testing device?
Why?
13. What are some of the most difficult experiences you have had when getting a
reading?
Tasks
Reminder: This test focus on the usability of the device, there is no hidden or implied test
on the user. The following tasks assume that you are using the DFS with normal healthy
condition. It is your first time using this device, and you are trying to perform several
tasks using the basic features of the device.
Task 1: Change a Default Setting.
You are a user in a Spanish speaking country and you need to change the language setting
so that you can read the device. Please find the change the language on the device.
1. Turn on the device and enter the setup screen.
2. Locate the Language setting
3. Change the setting from English to Spanish
4. Exit the setup after saving the setting.
Task 2: Update Personal Information.
Go to the settings menu and update your personal information.
1. This one is hopefully fairly self explanatory.
Task 3: Test Your Blood Glucose Level
You are doing the routine task of obtaining your BGL with this device, how would you go
about testing you blood. (Explain the hardware of the device and tell the user to tell you
what steps they would take physically..ie. taking blood and where they would put it etc.)
1. Turn on the device.
2. Go to the get blood section.
3. Insert the testing strip to put blood into system
4. Wait for reading
Task 4: Check the Last Record of BGL
You want to see the BGL that you tested yesterday.
1. Turn on the device.
2. Go to the history section.
3. From the list of dates and time, pick the specific one you want to see.
Task 5: Enter Food Information and Insulin
You just ate breakfast and took both long and short insulin. Test your BGL, enter both the
BGL reading and food information into the system, and get a forecast for when you need
to check again.
1. Perform a Simple BGL Test
2. Enter Food menu
3. Enter Food data and exit to main menu
4. Enter Insulin Menu
5. Enter Insulin Data then press Forecast button. Did the result show up? Do you
understand what the reading says? i.e. Did the system provide useful suggestion and
prediction that can help you know when to check your BGL again?
Task 6: Export your data to your Computer
You want to send the result of the forecast to your doctor by sending the information to
his email address. Suppose the device is wireless enabled.
1. Turn on the device
2. Click Export on the main screen.
Task 7: Edit Exercise List
You like to participate in Water Polo. This is not an option on the exercise list. Please
update the list so that it includes Water Polo.
Post-test Questions
1. Did you like the device? If you are given a scale from 1 (least satisfied) to 10 (most
satisfied), what would you rate this device?
2. Was doing data entry easy and efficient or was it complex and time-consuming?
3. Were the graphics appropriate? Did you find the text sizes and font styles easy to read?
4. Was the task time consuming, tedious or difficult?
5. Was the prediction informative and helpful or was it too vague?
6. Were you satisfied with the number of screens that you had to pass through to get to
the screen you wanted?
7. Were the instructions or the icons presented by DFS ambiguous, confusing or easy to
follow?
8. Were you satisfied with the feedback you were presented after entering data?
9. Were you allowed to fix errors (e.g. change the mistyped information)?
10. Do you think that you would remember how to do similar tasks as the ones you have
performed in the future? If not, please state which kind of task that would require you
most practice.
11. What are the most and least useful parts of the device in your opinion? Please tell us
the reasons as well.
This concludes our task descriptions and tests that we administered before and after the
tasks were done. Now some observations regarding the process of finding subjects,
completing the tests, and the results of our work!
Overall Summary
The most definitive statement that can be made about usability tests is that they take
more time than expected. Between finishing the prototype and working around schedules
this deliverable took almost three times as long as expected. Tight and solid scheduling is
important because one delay can have a domino effect that leaves the group with dry
periods of waiting followed by late nights spent furiously scampering to get things done.
We felt we got a lot out of our usability tests, with a number of problems coming to
light that seem blatantly obvious in hindsight. The most common and noticeable problem
was the ambiguity of the icons on our toolbar, which resulted in almost every user either
wasting time pondering over their usage and meaning or wasting clicks navigating down
digital dead-ends. The second most common issue is almost embarrassing: the way we
display our predictions is far too vague to be truly useful. That prediction is the major
piece of innovation with our product and fixing this issue is of the highest importance to
us. We also discovered bugs in our implementation and were offered some very useful
suggestions about how to streamline data entry. We plan on implementing as many of the
suggestions as possible because we found we agreed with the users in most cases.
The actual running of the tests was a learning experience. We were able to run some
very fun tests with the technology at our disposal, including running the prototype right
off a webpage and doing an entire test with someone in another country through text
messaging and video conferencing! All of the user’s were eager to help, but it is very
important to start getting as much feedback as soon as possible. We found that as the
experiments wore on the users got more and more comfortable expressing their opinions
to us and offering suggestions. If given the opportunity to run the tests again, we’d add an
emphasis to verbal questions and answers at the beginning of the test and devise some
critical thinking or creativity evoking questions just to open communication lines as much
as possible before we move onto questions and tasks that we consider critical.
Our interpretations of problems with DFS based on usability test observations
Below are some results of our tests. These are problems that were revealed about our
interface by observing people while they completed the tasks that we developed. We
rated them in terms of importance (problem severity) and their complexity (measured by
time it took users to overcome the problems).
“Importance” indicates the overall severity of the problem, and is rated on a scale from 1
(unimportant) to 5 (most important). “Time to solve” is an indication of how long it took
subjects to deal with the listed problem, and is rated on a scale from 1 (not time
consuming) to 5 (very time consuming).
Problems…
… and suggestions to improve them from users
Importance
Time to solve
Confusing icons.
Color and symbolic meaning are misleading.
Maybe use fewer colors and more standard icons.
Text pop-ups on roll-over
5
5
Buttons that don't work.
Buttons don't have any functionality and
don't give any useful feedback.
5
3
Prediction is vague.
Prediction is not specific and gives no
advice to help prevent a predicted severe BGL.
5
2
Help button
Users had trouble finding the help button and
rarely used it.
We don’t offer helpful instructions on navigation
3
2
Entering food (or new food type) data.
Adding a food to the list of foods or just
recording foods eaten.
4
5
Send users to history screen after data entry.
Currently, users go to home screen after
data entry.
2
3
Eliminate extra steps
Send users to list editing screens from exercise
and diet instead of routing it through settings.
2
2
Test subjects and their opinions of DFS
**Please note that all names have been changed to protect confidentiality.
James
Jim is a 54 year old High School principal with Type II diabetes. English is his native
language. He’s had diabetes for the past 2.5 years. He says the most difficult part of
dealing with diabetes is the continuous monitoring needed from day to day. Before the
test he showed interest in using a tool that could help him predict his daily BGL. He
averages less than 3 hours of exercise a week.
The test took place at Jim’s house, at his home computer. I was pleasantly surprised to
find that he had difficulty with almost every task. As I watched him perform the tasks I
noticed that each task (or sub-task for larger ones) could be broken down into two
sections. First, navigation. Second, manipulation. Navigation being the process needed
to start the task and manipulation being the task itself. The manipulation portion of the
test went off without a hitch and he expressed pleasure with much of the interface with
regards to data entry and task completion (with a few minor exceptions that were more
the result of an unfinished product than faulty interface design). However, the navigation
portion of the tasks was atrocious. Our main mode of navigation is a toolbar across the
top of the screen. Jim expressed extreme displeasure with this; not with the idea, but
rather with our specific implementation. Whereas before the usability test all the icons
seemed unique and descriptive, afterwards I’m left wondering how we could not have
seen the obvious pitfalls. He felt there is ambiguity between several icons and not a
satisfactory description of what they do. He frequently commented that the icons needed
descriptive text boxes to appear upon rollover, because icons are more prone to
misinterpretation. There is also the case of the “edit list” buttons from the diet and
exercise screens; these buttons take you to settings, from which you have to take one
more step to edit the lists. They should take you straight to the screens to edit the lists.
He also expressed dissatisfaction with the vagueness of the prediction.
Dave
Dave is a 21 year old computer science student with Type I diabetes, and he uses an
insulin pump. English is his native language. He has had diabetes for 18 years. When
first diagnosed he found it most difficult not eating the same things that other kids do. He
is not a vegetarian. He wrote that he knows how to deal with his diabetes "very well."
Dave exercises only a little bit through regular activities. He records his blood glucose
levels (BGL's) a few times per week, and that it is relevant to him to know his BGL. He
uses his own glucometer mostly to transfer readings to his insulin pump, but uses the
BGL memory feature least of all. Dave considers himself "techno-savvy" and would be
interested in using a device that forecasts blood glucose levels over a period of several
hours. His native language is English.
We met at the Student Union on the University of Maryland, College Park campus, in a
study room which was quiet, to do the test. He finished the tasks quickly and without
asking any questions. It took him about five minutes. He is a talented programmer and
familiar with usability testing so I think he understood what he needed to do and was able
to work it out with no problems. He was satisfied by the icons and the text (size and font)
in the interface, and was able to navigate the menus easily. The most useful part of the
device in his opinion was the history feature and the graph of the most recent BGL's. The
least useful feature was the slide bar approach on the insulin screen. Dave liked the
device and gave it a '7' out of a scale from 1 to 10 (10 being "most satisfied" with the
interface). The easiest task for him was getting a BGL and the hardest ones were entering
food that he had eaten and recording exercise. The predictions he found informative but
they don't "suggest what to do" to prevent a high or low reading. He suggested that the
system go right to the history screens once data is entered about diet, exercise, or insulin
doses.
Steve
Steve is a 38 year old man with Type II diabetes, and he has Down Syndrome. English is
his native language. He has had diabetes for 5 or 6 years. When he was first diagnosed,
he said that he had trouble taking off extra weight, and that it was hard to exercise
regularly. Currently, he goes to the gym almost everyday for 30 to 45 minutes. He is not
vegetarian. Steve knows his condition fairly well, saying that the food he eats turns into
sugar, and that he tries not to eat a lot of sugary snacks. When asked if he is "technosavvy", he said he's "so-so, not that great" with using computers. When I met with him,
before we started, he was using his computer. I noticed during the test that he had no
problems using the mouse or keyboard to complete the tasks. He said that he would like
to have a device that predicted his BGL's, but only tests his BGL three times a week,
which is typical for someone with Type II diabetes. Despite this, he felt it very important
to know his BGL, because if it was too high, he would be rushed to the hospital.
I met Steve at his home and we did the test at the table in his dining room. Steve had a
lot of questions. He seemed lost at many points during the testing. I verbally assisted
him with each task. He did have a good memory about the screens after learning what
each one did. I think that he felt he was not doing what he should have been doing when
he attempted to complete a task correctly, but faulty design on the part of the interface
prevented this. For example, in task 7, we request that users enter a new exercise type,
then save it. The 'save' button does not give users any feedback, however, and it doesn't
actually save the exercise to the database. This was an oversight on the designer's part.
Steve finished the testing in a little less than 15 minutes. He liked the device, but felt it
was time consuming. The icons were "straight-forward" in their meaning but the text was
a little small. There were a lot of different screens in his opinion. He was okay with the
feedback presented and thought that he would be able to fix mistakes if he had made any.
His confidence was high about his ability to remember how to do the tasks we asked him
to perform. If the tasks had taken any longer I think Steve would have become bored and
may have asked to stop. It seemed that he was becoming very impatient toward the end
of the task list, which made me wonder about the usefulness of such a device to someone
who has a cognitive disability.
Julia
Julia is a 26 year old doctoral student in genetics, and she doesn't have diabetes so has
never used a glucometer. English is her native language. She is familiar with computers,
but says that she is not "techno-savvy". On a scale from 1 to 10 (10 being "most
satisfied" with the interface), she rated the system a "6, with high potential. I feel when
all pulled together and all functions working it will be a very nice interface." She found
data entry easy to do and efficient. The graphics and icons seemed appropriate, except
that the icon for the BGL screen was gray, which she took to be a feature that wasn't
available, because it was "grayed out". The prediction was felt to be too vague, she
would like to get a numerical value. She was satisfied with the number of screens that
had to be worked through to get to the right one, but didn't appreciate one screen which
didn't provide any feedback after she tried saving a new type of food to the food list.
Julia couldn't tell whether or not she would be able to easily fix mistakes, because she
didn't make any she told me.
During the test I noticed that she hunted around for the right screens to complete the
tasks. She asked a few questions of me and I responded after letting her look a little more
carefully to discover the answer herself. I think once she learned what the icons meant, or
what screens they led her to, she remembered them well. She spoke to herself as she
worked and I was able to hear that she was looking for the right answer but was a little
stuck. She didn't quite understand the insulin screen, but for someone who doesn't have
diabetes, that response seemed appropriate. Working on our tasks took her about 10
minutes.
Father Barry
Father Barry is a 31 year old Franciscan priest who has Type I diabetes. He was
diagnosed 23 years ago. He said that he started giving himself injections when he was 8
(many children have their parents do this). His hardest struggle with diabetes is sticking
to a diet low in carbohydrates. English is his native language. He does not consider
himself "techno-savvy". In regards to his knowledge about his own diabetes, he told me,
"I chase a low blood sugar with sugar, and use Humalog [insulin] when readings are
high." Considering that our system makes a BGL prediction, he said that he would look
at the prediction and then do a test. If the test matched the prediction on several
occasions he would probably use it.
He finished the tasks in about 10 minutes and had questions throughout the tasks. When
he began, he tried touching the screen, but the device is only interactive through the
mouse or touchpad on my laptop. He used the touchpad. He found it somewhat difficult
understanding the icons and asked about these. The icons representing the settings screen
and the history screen were confusing to him, the others he found to be "somewhat
ambiguous". I didn't realize that some of the tasks and interface features did not match
up. For example, one task asks the users to press the 'forecast' button, but the button on
the screen that should have been pressed was 'save'. I realized that better communication
across our team would have prevented confusion like this. I wondered if Fr. Barry was
becoming frustrated because of interface - task mismatches. As the person I tested with
the greatest experience with glucometers (23 years), I took his opinion seriously when he
gave the interface a 4 or 5 on a scale from 1 to 10 (10 being "most satisfied" with the
device). He said data entry was complex and time consuming, and the feedback after
saving it was "somewhat unclear". The BGL prediction he found a little vague. Asked
about whether he would be able to save changes, he said that he wasn't sure because he
hadn't made any data entry errors during the testing.
Celia
Celia is a 30 year-old single woman who learned of her condition of type I diabetes not
too long ago. She is originally from El Salvador, however she has been leaving in the
Washington DC area for the past 15 years. Although she speaks English fluently, she
appreciates any sensitive information (related to her health) in Spanish. It was a very
unexpected experience to learn of her condition and she is willing to learn and use
everything she can in order to take care of her body.
Prior to her learning of her condition, Celia enjoyed life doing different things such as
dancing (her preferred hobby) as well as helping others doing community service. As she
started to feel extremely tired very often, she decided to start some tests. Blood tests gave
notice of her condition. At this point she decided to learn as much as she can about proper
and balanced diet. She also tried to work closely with doctors in order to keep herself in
good shape since she has great goals in her life.
Celia is a school teacher for an elementary school in Montgomery County. She enjoys
working with children and would like to keep her condition as secret as possible, however
she understand that things are going to change for her for now on. Although she find
technology tools (computers, PDAs, ipods, etc) fascinating, she does not consider herself
computer savvy. Despite this, she feel very comfortable in using basic tools commonly
found on the internet (such as pressing button, data entry, etc). She find different cultures
very fascinating and enjoys spending time with people from other latin american
countries and learn from them. These has led her to learn about different types of food.
She prefers desserts more than anything and has a great taste when preparing them. This,
however has been a great barrier to cross since she knows she needs to be more careful
now that she needs to eat more balanced from now on.
Celia expressed her impression about the idea of forecasting. She enjoyed very much the
testing process, although she would have hope for complete functionality. She also found
the “diet” record feature very neat (with arrows to add to your daily intake). She clearly
noted that feature as the most userfriendly and easier to use part of the system.
Monica
Monica is an administrative assistant for a Contruction Company. She handles incoming
calls as well as some book-keeping for the company. She is very familiar with two-three
friends who have type I diabetes. Although she is not personally familiar with diabetes,
she is very mindful of the condition of her friends and at times has conversed with them
to learn more about it. One thing that called her attention was the fact that people with
type I diabetes had to learn to be more disciplined with their diet.
Monica showed that it was a useful tool as long as the user is willing to keep on entering
data constantly. She understood pretty quickly that DFS depends on human interaction to
provide an accurate reading and found that the interface (in general) was pretty well
organized and made a lot of sense.
Jennifer
Jennifer is a college freshman in Taiwan. She was born with type I diabetes. When she
was young, she did not have to worry too much about her BGL and insulin since most of
the issues were taken care of by her parents. After she went to middle school, she began
to learn to use BGL meter herself and calculate how much corresponding insulin she had
to take.
Jennifer seems to be very confident about the device even though it is her first time of
using it. There was no occasion that she needed to seek out help from the device or from
the observer. The flow of her operation over different menus were very smooth. She did
not have any problem performing the assigned tasks. It seems that the user was satisfied
with the interface of the device.
Her case showed that for some users the interface for the DFS system was designed
properly such that it is easy to follow and operate different functions. Even though her
first language is not English, the words chosen for the menu were straightforward and
easy to understand that she did not mistake any content or feedback. Because the user
does not live in America, this usability test was conducted through presenting the
interface online and combine instant messaging program along with a video camera and
microphone to provide the observer chance to monitor the real reaction of the user.
Conclusions & Future Work
Final Status
The current version of DFS (3.0) is only a preliminary version that shows the feasible
interface of the device; most critical functions are working except the prediction model
and actual reading of the blood glucose level (BGL). Users can record their exercise,
diet, and BGL reading (assume they used other device than DFS because DFS is not
physically capable of reading BGL now) and save them for future use, which is the
prediction. Ideally, the prediction will change every time with different type of meal,
different period of exercise, and different amount of dose of insulin. This is why DFS
records all the data in each section and time after performing the tasks. The data in the
History section (represented by the magnifying glass) is supposed to be used to make the
prediction of BGL level over time. Right now, the DFS 3.0 has a random prediction that
changes every time the exercise, diet, insulin, and BGL reading is recorded; this is to
show that in the real working version the prediction should have actually changed. As for
now, DFS 3.0 is not ready for public release.
Future Work and Recommendations
The future possible tasks for DFS would be mainly the actual implementation of the
prediction model and the ability of reading BGL for the device. We believe that the
Markov chains would be a feasible option for the prediction model. Currently, Markov
chains are used for predicting weather conditions, and are the basis for Google's
PageRank algorithm (http://en.wikipedia.org/wiki/Markov_chain). With further study and
alternation, DFS would adopt a working model and be able to predict users’ BGL.
Several different versions of DFS's interface have been developed, right now we have
the data needed for translating DFS from English to Spanish, Chinese, and Japanese. Here
is an example chart of corresponding terms in English and Chinese/Japanese (see Figure
12).
As for recommendations, the myLife group has developed a very nice project that
tracks personal diet information; we would like to cooperate with them to further improve
the diet tracking function for the DFS device. We were also impressed by the DDAS
(Diabetes Diabetic Analysis Software) group's project. They developed an interface on
the web that users could upload information to about diet, exercise and insulin intake (not
to mention BGL readings) in order to pass this along to doctors. What's required is to
have a glucometer such as ours to record those types of information in order to be
uploaded. Also, it is crucial to conduct more usability tests in the future as we add new
capabilities such as new language supports.
Figure 12 – Translation guide for DFS features
Acknowledgements
We'd like to thank the following people for making this project possible:
Dr. Matthew Kim (consult)
Dr. Ben Shneiderman (professor)
Georg Apitz (TA)
Ari Rasekh (critique)
Chris Parlette (critique)
Mike Delacruz (critique)
Randy Jeune (critique)
And all our usability test users...
Lastly, but not least important, we want to thank one another for the hard work and
good teamwork that we showed during the semester. If our team had suffered because of
poor communication or a lack of interest on anyone's part, DFS would have been much
different!
References
The following are references used to developed our working product.








"Home Blood Glucose Prediction: Validation, Safety, and Efficacy Testing in
Clinical Diabetes" Albisser, A. Michael, Ph.D. - Diabetes Technology &
Therapeutics 7 (2005)
"A Graphical User Interface for Diabetes Management That Integrates Glucose
Prediction and Decision Support" Albisser, A. Michael, Ph.D. - Diabetes
Technology & Therapeutics 7 (2005)
"Type 1 diabetes: new perspectives on disease pathogenesis and treatment."
Atkinson, Mark A; Eisenbarth, George S. - Deparment of Pathology, Immunology,
and Laboratory Medicine, College of Medicine, University of Florida, Gainesville
(pdf)
"Computerising a guideline for the management of diabetes" Barahona, Pedro et
al. - International Journal of Medical informatics 64 (2001): 275-284.
"Self-Monitoring of Blood Glucose: The Basics." Benjamin, Evan MD. - Clinical
Diabetes 20 (2002) (pdf)
"Limitations of Conventional Methods of Self-Monitoring of Blood Glucose"
Boland, Elizabeth, MSN APRN, CDE, Monsod, Teresa, MD, Delucia, Maria, MS,
et al. - Yale University School of Medicine, Yale University, New Haven (online)
"Information technology and computer-based decision support in diabetic
management" Carson, E.R. et al. - Computer Methods and Programs in
Biomedicine 32 (1990): 179 - 188.
"Providing Secure Access to Confidential Patient Information Detailing Diabetic
Condition" Chadwick, D.W., M.D., New, J.P., M.D., McDowall, D.M., M.D. University of Kent. 3 October 2005. (pdf)
















"A computer system for interpreting blood glucose data" Deutsch, T., Gergely, T.,
Trunov, V. - Computer Methods and Programs in Biomedicine 76 (2004): 41 - 51.
"Type 1 diabetes: recent developments" Devendra, D.; Liu, E. Eisenbarth, G. Barbara Davis Center for Childhood Diabetes, Univ. of Colorado Health Sciences
Center, Denver. (online)
"Correlation of Fingerstick Blood Glucose Measurements with GlucoWatch
Biographer Glucose Results in Young Subjects with Type 1 Diabetes." Garg, S.
MD; Potts, R. PHD; Ackerman, N. PHD; et al. - (pdf)
"Self-measurement of blood glucose. Accuracy of self reported data and
adherence to recommended regimen." Gonder-Frederick, L.A.; Julian, D.M.; Cox,
D.J., et al. (online)
"DiasNet - an Internet Tool for Communication and Education in Diabetes."
Hejlesen, Ole K, Plougmann, Soren, Cavan, David A. - Aalborg University. 3
October 2005. (pdf)
"Self-Monitoring of Blood Glucose: Language and financial barriers in a
managed care population with diabetes." Karter, A. PHD, Ferrara, A. MD. PHD;
Darbinian, J. MPH. - Epidemiology Health Services Psychosocial Research. (pdf)
"Usability in the real world: assessing medical information technologies in
patients' homes." Kaufman, David R. et al. - Journal of Biomedical Informatics 36
(2003): 45 - 60.
"Episodes of Severe Hypoglycemia in Type 1 Diabetes Are Preceded and
Followed within 48 Hours by Measurable Disturbances in Blood Glucose."
Kovatchev, B.; Cox, D., Farhy, L., et al. - University of Virginia Health System
and National Science Foundation Center for Biological Timing. (online)
"Computerised Decision-Support Tools in Diabetes Care: Hurdles to
Implementation." Lehmann, Eldon D., M.B. B.S., B.Sc. - DIABETES
TECHNOLOGY & THERAPEUTICS 6 (2004): 422 - 429.
"The freeware AIDA interactive educational diabetes simulator http://www.2aida.org - (1) A download survey for AIDA v4.0." Lehmann, Eldon
D. - Med Sci Monit 7 (2001): 504-515.
"Information Technology in Clinical Diabetes Care-A Look to the Future."
Lehmann, Eldon D., M.B. B.S., B.Sc. - DIABETES TECHNOLOGY &
THERAPEUTICS 6 (2004): 755 - 759.
"Collection of Information Directly from Patients through an Adaptive HumanComputer Interface." Lobach, David F., MD, PhD, MS; Arbanas, Jennifer M.,
MS; Mishra, Dharani D. MS, et al. - Division of Clinical Informatics, Department
of Community and Family Medicine, Duke University. 3 October 2005. (pdf)
"Self-Control: A Physician's Guide to Blood Glucose Monitoring in the
Management of Diabetes." Mayfield, MD, MPH; Havas, S. MD, MPH, MS
AAFP. (pdf)
"Six Challenges in Measuring the Quality of Health Care" McGlynn, E. - Health
Affairs 16 (1997). (pdf)
"Interface Design for Health Care Environments: The Role of Cognitive Science."
Patel, Vimla L. and Kushniruk, Andre W. - Cognitive Studies in Medicine,
McGill University, Montreal, Quebec.
"Infection and diabetes: the case for glucose control." Rayfield EJ, Ault MJ,
Keusch GT, et al. (online)





"Evaluation of Portable Blood Glucose Meters - Problems and
Recommendations." Slingerland, R.; Miedema, K.; Isala Klinieken, et al. (pdf)
"Using the ADAP learning algorithm to forecast the onset of diabetes mellitus."
Smith, J W; Everhart, J E; Dickson, W.C. - Annual Symposium on Computer
Applications in Medical Care, Washington, DC USA. 6-9 Nov. 1988. pp. 261265. 1988. (online)
"Recent advances in treatment of youth with type 1 diabetes: better care through
technology." Tamborlane, W.V.; Bonfig, W.; Boland, E. (online)
"Newer Portable Glucose Meters - Analytical Improvement Compared with
Previous Generation Devices?" Weitgasser, R.; Gappmayer, B.; Pichler, M. Salzburg General Hospital, Austria. (online)
"SAM: Speech-Aware Applications in Medicine to Support Structured Data
Entry." Wormek, A.K., DVM, Ingenerf, J., Ph.D., Orthner, H.F., Ph.D. American Medical Informatics Society. (pdf)