Diabetes Forecast System Final Report Brian Mitchell (retlif.maps@gmail.com) Ben Parker (bwparker@gmail.com) Ken Chen (kenken1982@hotmail.com) Alberto Flores (aaflores@gmail.com) December 6th, 2005 Abstract Glucose meters and Diabetes management are important subjects of attention for people diagnosed with type I diabetes. The process of monitoring one's own blood glucose with a glucose meter is ofter referred as self-monitoring of blood glucose or "SMBG". There exist a significant amount of tools available today that provide several alternatives to this process. The Diabetes Forecast System (DFS) is a tool designed to provide real-time predictions of blood sugar levels (BGL). Forecasts are based on readings from a current diet, latest insulin doses, current workout schedules as well as current sugar levels. These inputs are utilized to produced a forecast of BGL for a specific period of time in the future. According to our task analysis and requirements, we propose additional features that unlike other devices available in the market, improve the user experience as well as the frequency of user input. Credits 1. Brian Mitchell – current chief architect behind DFS. His skills in Flash as well as his great knowledge of diabetes makes them a key member in providing support as well as feedback for improvements. He was the lead person for deliverable number two were he led the rest of the team in obtaining and filtering references for the development of this product. 2. Ben Parker - one of the developers for the DFS team. His active commitment and experience developing interfaces at his work-place provided great feedback during our regular meetings. Ben took the lead on two deliverables: Compiling the user tasks and compiling the results of the usability tests. He also designed most of the flow of DFS as well as compiled other ideas to improve team work. 3. Ken Chen - provided a great touch of diversity and foreign feedback to DFS. His contributions are found in the History module of DFS. Ken is also a developer for the DFS team and took the lead on the fourth deliverable as he prepared and led the team in building an effective task list and questionnaire. 4. Alberto Flores - the driving force behind the last phase of the project (the final report and the website) and in the production of great ideas throughout the semester. He also took the lead in designing prototypes for DFS during the initial development process and organized meetings to improve communication process among team members. Introduction With type 1 diabetes the body does not produce insulin. A person with type 1 diabetes, therefore, is tasked with simulating the insulin production of a healthy pancreas through insulin shots. The balance between sugar intake and insulin is delicate and can be very difficult with mistakes causing potentially deadly results. If one takes too much insulin they can become hypoglycemic (low blood sugar levels) or hyperglycemic (high blood sugar levels). The tools currently available on the market give a person with diabetes the ability to accurately measure their current blood sugar levels which is a tremendous help in keeping blood sugar regulated. However, that measurement is just a snapshot in time and does not give the user any indication to where their glucose level is heading nor does it help the user decide what actions should be taken to correct their condition if/when it becomes necessary. Our proposed system will offer our users this advantage. It will take into account the four major factors in glucose levels (carbohydrate intake, insulin intake, level of activity, and current blood glucose level) to give the user a prediction of where their blood glucose level will be in the near future and it will help them make healthy decisions to keep it in the normal range. The device would even notify the user if they have forgotten to test their glucose level recently and may be in danger of falling outside healthy limits. We hope to create a tool that targets diabetic users of almost all ages, sexes and races. Ideally it would be a tool with "universal usability" among type I diabetics. One of the member's of our group has type 1 diabetes and served as one of our main subjects for real-life tasks that diabetics need to perform to monitor and maintain their BGL. According to our research, we found that “SMBG “ is a significant challenge for many users. Our study showed that the majority of people with type I diabetes have a good sense of awareness about their health condition. Their knowledge about nutritional facts are well above the average person. This condition also provides most potential users the confidence to rely a lot more in their own experience than in a device which can assist in self management of current conditions. Our goal is to provide a companion to such user, attemping to make a device that is appealing and practical to the user. Presentation of Design Our final design is the result of quite a lot of work, which is described in the "Development Process" webpage. In order to explain the main work flow of DFS we present a transition diagram for the final version (3.0). This diagram was revised several times and improved after receiving important comments and results from our usability tests. Figure 1 – DFS transition diagram Careful consideration was given to a simple interface. This is well described in Figure 1. Here we can see how there are basic functions that are needed and process are parallel (similar) in all operations (saved pop-up). The splash screen starts things off after you press the begin button (which will not be a part of the final product). Users have the ability to turn it off if they'd like to from the settings screen. Figure 2 – Splash screen The next thing you see is the home screen. From this screen (and almost all the others - see Figure 1) you can get any of the other major screens defined by the links on the toolbar and the settings screen which has a link in the bottom right corner of the screen. On the home screen you will find the prediction, which does not actually work because we couldn't get a prediction model running. We investigated Markov chains, but these are complicated (they are the basis of Google's PageRank algorithm) and would have taken a long time to implement. We would have liked to have gotten the graph working as well, but could not, but that would be one of the next things to implement. One additional thing to note is the help menu, which can be accessed from any screen and provides useful feedback about what to do on any given screen. Below is a shot of the home screen and one of the help screen. Figure 3 – Forecast screen Our usability test showed the excitement for this feature. It is the main purpose of our service, yet the most difficult to implement as well. Our findings also show that different attempts to compute forecast based on human input have been made, nevertheless these have also noted the lack on consistency in user input. Our findings indicate that users found this screen appealing. Full implementation of this feature was preferred but also the complexity of this computation was also understood by most subjects. One important feature that was also considered was the ability to upload your data to the DFS device. Since this is the primary function of forecasting, this button stands next to the forecast chart where users can add input and thus assist in maintaining an accurate reading. DFS will also support different formats (xls, csv, rtf, pdf) for exporting data to different clients (PC, email, cell-phones). Our goal in providing this king of communication is to assist users in share their information if allowed in order to provide medical assistance if needed. Figure 4 – Home help screen As explained in Figure 1, all our screen provide a help screen that fades in and out as it appears on the screen (see Figure 4). Basic functionality and purpose of the function is displayed in the screen. This “fading” pop-up system also proved to be interesting to many users as well. We go now to the next screen: the area where you have your blood sugar level measured. The idea is to put a glucometer test strip into the pda, then apply a drop of blood to the strip, as you do with any glucometer. However, obviously we couldn't do that for usability testing so we created a false randomized reading generator. Just click on "get BGL" and you're given a value between 60 and 300, I believe. This data is stored, and can be viewed from the history screen. Figure 5 – BGL input screen The next screen is the diet screen. It uses an accordion menu to break up food choices by food group. Foods are added to the list from the settings screen and the carbohydrate values are added at that time. You can get to this screen by pressing the "new food" button. If you add an item by mistake, you remove it by selecting it from the column on the right and pressing the lower green button. Add a food by selecting on the left and pressing the upper green button (Figure 6). Save your selections into the pda by pressing "save". Figure 6 – Diet input scr een Have you exercised? Exercise is a vital input that can affect BGL measurements significantly. We identified this fact and added a feature that most user found as a very helpful one. This has an effect on future BGL's and would be used in a working prediction algorithm. Select the type of exercise you have done and the number of minutes that you've worked out, then press "save" (see Figure 7). If you don't see the type of exercise you've done, please add an exercise by pressing "new exercise". Figure 8 – Exercise screen Now, each time you inject yourself, record it (Figure 9). You can keep track of slow and fast acting insulin with DFS. Use the sliding bars, which still need a little work, or the "numeric stepper" devices as Flash calls them to make the entry perfect down to the unit. Figure 9 – Insulin screen So now you want to see what you've done. The history screen shows previous meals, insulin injections, work outs, and BGL readings. You can modify anything, as well, so if you are recording something late, then come here, double click on the entry and then change it! (see Figure 10) Figure 10 – History screen The settings screen has the most options, of course (Figure 11). Set up basic info such as age and weight, but you can also add or subtract foods and exercises. One of our commentors suggested adding a "return to default" button, and it was a great idea. We also have a setting to stop showing the default screen upon start up, to save time in getting to what you need. We would like to add more themes and language translations, so the setup for these is there, but they are ineffective. Figure 11 – Settings screen Development Process Our final version (3.0) is the product of many suggestions of improvemts as well as different prototypes (presented below). Different features that were originally designed, but never implemented are purposely left for future developments. Prototype I - DFS Iconic Prototype The iconic prototype incorporates all of the features that we want to include: recording exercise, diet, insulin and blood glucose levels (BGL), but does this mainly with icons instead of words. Predictions and measurements of readings cannot be pictures however, and must be described in words. General Features The display will automatically turn off after 2 minutes. The unit is not waterproof; please keep it in the case it comes with. There are pockets in the case for lancets, syringes and two 10 mL unit bottles of insulin. It doesn't look obvious how deletions should be done, but it's easy. The output fields at the bottom of some screens are actually editable, in other words, the items can be removed. To do so, simply double tap and drag the item with the included stylus from the text box to some point outside that box and it will automatically disappear from the list. HIPAA (Health Insurance Portability and Accountability Act) guidelines prohibit the transmission of medical documents without the owner's permission. For this reason, downloading data from the PDA requires a password. If users should find that they've misplaced the device they can login to the DFS Website and have the PDA remotely locked: the company will send it a message that will freeze the screen and encrypt the data inside. If found again the company will send a message to the device decrypting the data. BGL Screen You can get a reading from any screen. Simply insert a test strip into the top right corner of the meter and the BGL screen will come up. No need to return to the main screen and find the right screen to go to from there. An icon showing a drop of blood will come up indicating that you should use a lancet to prick your finger now. It takes 5 seconds for the system to figure out your BGL once you add a droplet to the test strip. This will be displayed on the screen. After 30 seconds, the display will return to the main screen. Coding your glucometer is normally required based on test strips (numbers can range from 1 - 49, depending on the meter). Test strips come coded with one of these numbers and users are responsible for entering this number in the glucometer before they use any of the strips. If they forget to do this, however, the reading will not be truly accurate. The DFS meter codes strips automatically. Main screen Displayed are the most recent 5 BGLs. Below this, the readings taken within the past 24 hours are graphed. At the very bottom of the screen is listed any warnings for low readings (this is usually anything < 60, but the number can be manipulated in "Setup") and high readings (anything > 150, again this is adjustable) Setup screen The button at the top left of the Main screen, "Setup", brings you to the Setup screen where you can adjust several variables, such as the range limits for normal blood glucose levels and what should happen when your blood sugar level is critically low or high (these values are also adjustable, but cannot be set < 40 or > 350). Responses to critical levels could include wirelessly notifying the local police department by calling 911, or sending an urgent call to a relative or friend. This option is adjustable in the Setup section as well. Things such as brightness, contrast, color settings, and sound can be manipulated here as well. The date and time can be set in this section. Foods can be added along with their nutritional content here. The average serving that you would eat should be included so that you don't need to enter the size of a portion on your plate. This may lead to inaccuracies and a loss of interest in entering foods eaten if it requires that much work. If you eat twice or three times what you normally do, just add the food again as many times as is needed. Exercise machines that the DFS meter can interact with should be added here. The DFS system comes with a CD that includes drivers, or small pieces of software that communicate between the DFS meter and the electronic device, such as a treadmill, pedometer, or heart rate meter. Connecting to such a device provides the meter with the most accurate information possible. You should indicate here the type of insulin(s) that you are using. This will allow the meter to make the most accurate BGL forecasts. If you accidentally submit a record that had misinformation in it, you would want to delete it. However, giving such privileges has a disadvantage; that being the malicious or unintended deletion of data by others should the device fall into the wrong hands. You can set permissions for the current (and only "logged in") user here. Choose not to give delete permissions if you do not make many mistakes with data entry, and can live with an occasional bad record. Know, though, that permissions can be changed at any point in case an egregious error is made on an entered record. Downloading Information can be downloaded from the PDA to your computer can be done from this screen. This requires a password to comply with HIPAA. Many screens have a "When" and "Done" button. The "When" button allows you to select an earlier (but not later time) time for when the action occurred, like eating. if for example, you ate two pieces of fruit three hours ago, but forgot to enter the items at that time, you can include them later, but choose the "When" button and include the right time for when they were done. The "When" button does not need to be tapped if the action to be performed occurred or will occur within 10 minutes prior or will happen in the next 10 minutes. Food screen Nutrient info for more than seven thousand foods are included, downloaded from the USDA nutrient database (http://www.nal.usda.gov/fnic/foodcomp/search/). Select foods that you normally eat and they will be added to the output screen at the bottom of the interface. This output box can be manipulated, i.e., the items listed here can be doubletapped and dragged outside the box by the stylus to remove the item from the list of foods eaten. Either choose the colored diagrams for the types of foods you would eat, or choose specific foods from the list on the right. The list of foods consists of the four most chosen foods from the database of foods. This list may change as the types and frequencies of foods you eat changes. Exercise screen Include the amount and intensity of exercise that you do. Since much of diabetes involves making decisions based on approximate information, we suggest the method of entering approximate workout times and intensity levels as an acceptable means of recording exercise habits. Asking users to enter average heart rates and average mile splits for runners, among many other numbers, would be asking too much we feel and is not necessary to making an accurate forecast of future BGL's. The option to download this accurate info from a machine is available and would make predictions more accurate than they could be. Insulin screen Simply slide the marker along the rail to indicate the amount of each type of insulin you plan on taking. That's all there is to it, except to know that you press "Done" when the previous task is completed. Laboratory screen This screen allows you to look into the past or make an educated guess about the future. Choose a microscope slide to learn about your previous entries concerning diet, exercise, insulin intake or BGL readings. You will be allowed to see a graph of readings from up to 30 days in the past for each of these four domains. At the bottom of this screen, see forecasts for the next 12 hours. Drag the marker along the rail to the hour in the future that you want to learn more about. The further out you go, the less accurate the forecast given will be. There is no "DONE" or "SUBMIT" button here, once you let go of the marker after dragging it, the prediction will be made from where it is now located. Prototype II - DFS Personal edition DFS Personal edition consists of a main layout used in every screen. It contains four main buttons at the bottom to move among the main services offered by DFS. These are: My Forecast My Insulin My Diet My Journal At the top left corner, there is a customizable add-on to the interface named "My DFS" where each user will be able to performed costumed-based operations to a particular unit (e.g. changing themes, exporting data to an external device, show other configurations, etc). At the top right corner, there is a status bar, which by default is in green. It will only change to color red when blood glucose levels are under a dangerous margins. This will serve as an alert to the user. Additional vibration functionality to the device is available via "My DFS" settings configuration. Main Screen The main screen displays the version number and serves as the main screen to ensure that all services have properly loaded into the system. This indicates that the system is ready to continue. The version number is useful to indicate the type of operating system in use by DFS as it's currently configured in the unit. This will serve for customer service information and updates. My Diet Screen This screen will display information about a particular product that the user of DFS is about to take (or has taken). This will allow users to enter information that are relevant to keep a more accurate reading and forecast for DFS. Once a selection is done, users need to enter the product, quantity and units. Then, users will need to click on "Add" to submit the entry. Additionally, information about the product in question is described for purposes only relevant to DFS and calculating BGL's. My Forecast This screen is the main interface for DFS defining brand. It provides an interface to display BGL's in a configurable period of time. The specific level status is described by a color where red is always a sign of danger. At the bottom, a complete description of the current status is display where current average, max, min and hit percentage (actual vs forecasted results) numbers are displayed. BGL average, max and min are numbers for the current day. The hit percentage is information that refers to the amount of accuracy in comparing forecasted results and actual BGL level. Error margins are based on inaccurate entries in the "My diet" section which could provide inaccurate readings. The Stat line (in the graph) shows real-time information. When the margins go beyond a specific bound, the interfaces displays an alert (red background). It can optionally vibrate and email a list of recipients (e.g. doctors, 911, family, etc) when configured in "My DFS" My Insulin This screen displays the insulin doses needed and makes recommendations based on fast and slow acting. Moving the arrow level on the measuring bars, displays (and automatically moves the other acting type) It also provides a "Needed coverage" percentage which displays the amount of insulin which is needed an how much of it is covered with the selection. This is a measurement aid for users to be assured of the correct doses when in need. It will also allow to record the entry to ensure accuracy of readings and forecast. Usability Test Process Our goal when running the test was to have the users run a series of tasks in a very relaxed environment. We broke our tests into a five step process. We began with a simple introduction of ourselves and our systems. The users then took a written pre-survey that included basic personal information such as age, weight and when they discovered diabetes. At that point we had them run through a series of pre-determined tasks and recorded thier comments, suggestions and actions. We gave them a post-survey that asked them for their thoughts and opinions on the experience and finished the process by allowing to voice and additional comments they may have. We pooled our users from the people we knew in our day to day life. They included men and women, ages 21-60, of a eclectic cultural background. One of the tests was actually administered over the internet to user in another country! We also attempted to interview users who were patients of an endocrinologist that we know, but because of HIPAA regulations, we could not find out who those potential users might be. Here is the usability test and pre/post-tests that we asked our testers to complete: Diabetes Forecast System (DFS) UTG: Usability Test Guide – For DFS Agents’ Eyes Only Introduce/Thank Start out by introducing yourself and thanking the tester for coming to help you. This may be a good time to offer them something to eat or drink. Try to make them feel as comfortable as possible and let them know that you appreciate them donating their time to you. Explain/Outline Explain to them what our system does, or more specifically what its supposed to do. Let them know that we hope to make the interface better and welcome their input into any potential drawbacks of the current design as well as suggestions for improvements. Make it clear that the system is being tested, not them; problem are expected/invited and they signal a fault on our side not a deficiency of the user. Warn them that some part of the interface are not complete and they may not be able to FULLY complete all tasks and that is ok. Briefly outline the process that they are about to engage in: A written pre-test, a series of observed tasks and finally a written post-test. Ask them if they have any questions and once their questions have been answered ask them to sign the paper. Pre-test(Page 2) Hand them the pre-test questionnaire and writing utensil and ask them to fill it out. Allow them to skip any questions if they feel uncomfortable and you may provide as much assistance as they require, just make sure the answers are their own. Run Tasks(Page 3 & 4) Start the system and give the tester the driver’s seat. Allow them a moment to look at the screen and become comfortable. Giving them as much time as they require, read each task to them and ask them to carry it out. DO NOT read the steps of the tasks to them, we are trying to see if the system is designed well enough that it can be used without explicit directions. At this time take COPIOUS notes on what they accomplish, what confused them, what mistakes they made, any comments they make, what they seem to like/dislike, ANYTHING of relevance. If they become completely stumped with a task you may show them the NEXT step they must take, then allow them to move on from there – again observe and record their actions and comments. Post-test(Page 5) Once they have finished all the tasks, give them the post test and writing utensil and ask them to answer the questions. This should run similar to the pre-test. Closing Thank them again for their time and ask them if there is anything they would like to add or comment about. Ask for final thoughts or suggestions. Thank them again and allow them to continue their day. Pre-test Questions 1. What is your age? 2. What is you native language? 3. Do you consider yourself techno-savvy? 4. How long have you had diabetes? 5. When you learned that you had Type I diabetes, what were some of the most difficult things to do? Why? 6. How well do you know your condition and what steps you need to take to maintain balance? 7. Do you think a tool that helps you to forecast your BGL over the next few hours would be useful? 8. Are you a vegetarian? 9. Do you exercise a lot? If so, how many hours do you average exercise a week? 10. How often do you record your BGL? 11. How relevant for you is to know your BGL? (From a scale of 1-10) 12. What are the most and least frequent functions you use with your BGL testing device? Why? 13. What are some of the most difficult experiences you have had when getting a reading? Tasks Reminder: This test focus on the usability of the device, there is no hidden or implied test on the user. The following tasks assume that you are using the DFS with normal healthy condition. It is your first time using this device, and you are trying to perform several tasks using the basic features of the device. Task 1: Change a Default Setting. You are a user in a Spanish speaking country and you need to change the language setting so that you can read the device. Please find the change the language on the device. 1. Turn on the device and enter the setup screen. 2. Locate the Language setting 3. Change the setting from English to Spanish 4. Exit the setup after saving the setting. Task 2: Update Personal Information. Go to the settings menu and update your personal information. 1. This one is hopefully fairly self explanatory. Task 3: Test Your Blood Glucose Level You are doing the routine task of obtaining your BGL with this device, how would you go about testing you blood. (Explain the hardware of the device and tell the user to tell you what steps they would take physically..ie. taking blood and where they would put it etc.) 1. Turn on the device. 2. Go to the get blood section. 3. Insert the testing strip to put blood into system 4. Wait for reading Task 4: Check the Last Record of BGL You want to see the BGL that you tested yesterday. 1. Turn on the device. 2. Go to the history section. 3. From the list of dates and time, pick the specific one you want to see. Task 5: Enter Food Information and Insulin You just ate breakfast and took both long and short insulin. Test your BGL, enter both the BGL reading and food information into the system, and get a forecast for when you need to check again. 1. Perform a Simple BGL Test 2. Enter Food menu 3. Enter Food data and exit to main menu 4. Enter Insulin Menu 5. Enter Insulin Data then press Forecast button. Did the result show up? Do you understand what the reading says? i.e. Did the system provide useful suggestion and prediction that can help you know when to check your BGL again? Task 6: Export your data to your Computer You want to send the result of the forecast to your doctor by sending the information to his email address. Suppose the device is wireless enabled. 1. Turn on the device 2. Click Export on the main screen. Task 7: Edit Exercise List You like to participate in Water Polo. This is not an option on the exercise list. Please update the list so that it includes Water Polo. Post-test Questions 1. Did you like the device? If you are given a scale from 1 (least satisfied) to 10 (most satisfied), what would you rate this device? 2. Was doing data entry easy and efficient or was it complex and time-consuming? 3. Were the graphics appropriate? Did you find the text sizes and font styles easy to read? 4. Was the task time consuming, tedious or difficult? 5. Was the prediction informative and helpful or was it too vague? 6. Were you satisfied with the number of screens that you had to pass through to get to the screen you wanted? 7. Were the instructions or the icons presented by DFS ambiguous, confusing or easy to follow? 8. Were you satisfied with the feedback you were presented after entering data? 9. Were you allowed to fix errors (e.g. change the mistyped information)? 10. Do you think that you would remember how to do similar tasks as the ones you have performed in the future? If not, please state which kind of task that would require you most practice. 11. What are the most and least useful parts of the device in your opinion? Please tell us the reasons as well. This concludes our task descriptions and tests that we administered before and after the tasks were done. Now some observations regarding the process of finding subjects, completing the tests, and the results of our work! Overall Summary The most definitive statement that can be made about usability tests is that they take more time than expected. Between finishing the prototype and working around schedules this deliverable took almost three times as long as expected. Tight and solid scheduling is important because one delay can have a domino effect that leaves the group with dry periods of waiting followed by late nights spent furiously scampering to get things done. We felt we got a lot out of our usability tests, with a number of problems coming to light that seem blatantly obvious in hindsight. The most common and noticeable problem was the ambiguity of the icons on our toolbar, which resulted in almost every user either wasting time pondering over their usage and meaning or wasting clicks navigating down digital dead-ends. The second most common issue is almost embarrassing: the way we display our predictions is far too vague to be truly useful. That prediction is the major piece of innovation with our product and fixing this issue is of the highest importance to us. We also discovered bugs in our implementation and were offered some very useful suggestions about how to streamline data entry. We plan on implementing as many of the suggestions as possible because we found we agreed with the users in most cases. The actual running of the tests was a learning experience. We were able to run some very fun tests with the technology at our disposal, including running the prototype right off a webpage and doing an entire test with someone in another country through text messaging and video conferencing! All of the user’s were eager to help, but it is very important to start getting as much feedback as soon as possible. We found that as the experiments wore on the users got more and more comfortable expressing their opinions to us and offering suggestions. If given the opportunity to run the tests again, we’d add an emphasis to verbal questions and answers at the beginning of the test and devise some critical thinking or creativity evoking questions just to open communication lines as much as possible before we move onto questions and tasks that we consider critical. Our interpretations of problems with DFS based on usability test observations Below are some results of our tests. These are problems that were revealed about our interface by observing people while they completed the tasks that we developed. We rated them in terms of importance (problem severity) and their complexity (measured by time it took users to overcome the problems). “Importance” indicates the overall severity of the problem, and is rated on a scale from 1 (unimportant) to 5 (most important). “Time to solve” is an indication of how long it took subjects to deal with the listed problem, and is rated on a scale from 1 (not time consuming) to 5 (very time consuming). Problems… … and suggestions to improve them from users Importance Time to solve Confusing icons. Color and symbolic meaning are misleading. Maybe use fewer colors and more standard icons. Text pop-ups on roll-over 5 5 Buttons that don't work. Buttons don't have any functionality and don't give any useful feedback. 5 3 Prediction is vague. Prediction is not specific and gives no advice to help prevent a predicted severe BGL. 5 2 Help button Users had trouble finding the help button and rarely used it. We don’t offer helpful instructions on navigation 3 2 Entering food (or new food type) data. Adding a food to the list of foods or just recording foods eaten. 4 5 Send users to history screen after data entry. Currently, users go to home screen after data entry. 2 3 Eliminate extra steps Send users to list editing screens from exercise and diet instead of routing it through settings. 2 2 Test subjects and their opinions of DFS **Please note that all names have been changed to protect confidentiality. James Jim is a 54 year old High School principal with Type II diabetes. English is his native language. He’s had diabetes for the past 2.5 years. He says the most difficult part of dealing with diabetes is the continuous monitoring needed from day to day. Before the test he showed interest in using a tool that could help him predict his daily BGL. He averages less than 3 hours of exercise a week. The test took place at Jim’s house, at his home computer. I was pleasantly surprised to find that he had difficulty with almost every task. As I watched him perform the tasks I noticed that each task (or sub-task for larger ones) could be broken down into two sections. First, navigation. Second, manipulation. Navigation being the process needed to start the task and manipulation being the task itself. The manipulation portion of the test went off without a hitch and he expressed pleasure with much of the interface with regards to data entry and task completion (with a few minor exceptions that were more the result of an unfinished product than faulty interface design). However, the navigation portion of the tasks was atrocious. Our main mode of navigation is a toolbar across the top of the screen. Jim expressed extreme displeasure with this; not with the idea, but rather with our specific implementation. Whereas before the usability test all the icons seemed unique and descriptive, afterwards I’m left wondering how we could not have seen the obvious pitfalls. He felt there is ambiguity between several icons and not a satisfactory description of what they do. He frequently commented that the icons needed descriptive text boxes to appear upon rollover, because icons are more prone to misinterpretation. There is also the case of the “edit list” buttons from the diet and exercise screens; these buttons take you to settings, from which you have to take one more step to edit the lists. They should take you straight to the screens to edit the lists. He also expressed dissatisfaction with the vagueness of the prediction. Dave Dave is a 21 year old computer science student with Type I diabetes, and he uses an insulin pump. English is his native language. He has had diabetes for 18 years. When first diagnosed he found it most difficult not eating the same things that other kids do. He is not a vegetarian. He wrote that he knows how to deal with his diabetes "very well." Dave exercises only a little bit through regular activities. He records his blood glucose levels (BGL's) a few times per week, and that it is relevant to him to know his BGL. He uses his own glucometer mostly to transfer readings to his insulin pump, but uses the BGL memory feature least of all. Dave considers himself "techno-savvy" and would be interested in using a device that forecasts blood glucose levels over a period of several hours. His native language is English. We met at the Student Union on the University of Maryland, College Park campus, in a study room which was quiet, to do the test. He finished the tasks quickly and without asking any questions. It took him about five minutes. He is a talented programmer and familiar with usability testing so I think he understood what he needed to do and was able to work it out with no problems. He was satisfied by the icons and the text (size and font) in the interface, and was able to navigate the menus easily. The most useful part of the device in his opinion was the history feature and the graph of the most recent BGL's. The least useful feature was the slide bar approach on the insulin screen. Dave liked the device and gave it a '7' out of a scale from 1 to 10 (10 being "most satisfied" with the interface). The easiest task for him was getting a BGL and the hardest ones were entering food that he had eaten and recording exercise. The predictions he found informative but they don't "suggest what to do" to prevent a high or low reading. He suggested that the system go right to the history screens once data is entered about diet, exercise, or insulin doses. Steve Steve is a 38 year old man with Type II diabetes, and he has Down Syndrome. English is his native language. He has had diabetes for 5 or 6 years. When he was first diagnosed, he said that he had trouble taking off extra weight, and that it was hard to exercise regularly. Currently, he goes to the gym almost everyday for 30 to 45 minutes. He is not vegetarian. Steve knows his condition fairly well, saying that the food he eats turns into sugar, and that he tries not to eat a lot of sugary snacks. When asked if he is "technosavvy", he said he's "so-so, not that great" with using computers. When I met with him, before we started, he was using his computer. I noticed during the test that he had no problems using the mouse or keyboard to complete the tasks. He said that he would like to have a device that predicted his BGL's, but only tests his BGL three times a week, which is typical for someone with Type II diabetes. Despite this, he felt it very important to know his BGL, because if it was too high, he would be rushed to the hospital. I met Steve at his home and we did the test at the table in his dining room. Steve had a lot of questions. He seemed lost at many points during the testing. I verbally assisted him with each task. He did have a good memory about the screens after learning what each one did. I think that he felt he was not doing what he should have been doing when he attempted to complete a task correctly, but faulty design on the part of the interface prevented this. For example, in task 7, we request that users enter a new exercise type, then save it. The 'save' button does not give users any feedback, however, and it doesn't actually save the exercise to the database. This was an oversight on the designer's part. Steve finished the testing in a little less than 15 minutes. He liked the device, but felt it was time consuming. The icons were "straight-forward" in their meaning but the text was a little small. There were a lot of different screens in his opinion. He was okay with the feedback presented and thought that he would be able to fix mistakes if he had made any. His confidence was high about his ability to remember how to do the tasks we asked him to perform. If the tasks had taken any longer I think Steve would have become bored and may have asked to stop. It seemed that he was becoming very impatient toward the end of the task list, which made me wonder about the usefulness of such a device to someone who has a cognitive disability. Julia Julia is a 26 year old doctoral student in genetics, and she doesn't have diabetes so has never used a glucometer. English is her native language. She is familiar with computers, but says that she is not "techno-savvy". On a scale from 1 to 10 (10 being "most satisfied" with the interface), she rated the system a "6, with high potential. I feel when all pulled together and all functions working it will be a very nice interface." She found data entry easy to do and efficient. The graphics and icons seemed appropriate, except that the icon for the BGL screen was gray, which she took to be a feature that wasn't available, because it was "grayed out". The prediction was felt to be too vague, she would like to get a numerical value. She was satisfied with the number of screens that had to be worked through to get to the right one, but didn't appreciate one screen which didn't provide any feedback after she tried saving a new type of food to the food list. Julia couldn't tell whether or not she would be able to easily fix mistakes, because she didn't make any she told me. During the test I noticed that she hunted around for the right screens to complete the tasks. She asked a few questions of me and I responded after letting her look a little more carefully to discover the answer herself. I think once she learned what the icons meant, or what screens they led her to, she remembered them well. She spoke to herself as she worked and I was able to hear that she was looking for the right answer but was a little stuck. She didn't quite understand the insulin screen, but for someone who doesn't have diabetes, that response seemed appropriate. Working on our tasks took her about 10 minutes. Father Barry Father Barry is a 31 year old Franciscan priest who has Type I diabetes. He was diagnosed 23 years ago. He said that he started giving himself injections when he was 8 (many children have their parents do this). His hardest struggle with diabetes is sticking to a diet low in carbohydrates. English is his native language. He does not consider himself "techno-savvy". In regards to his knowledge about his own diabetes, he told me, "I chase a low blood sugar with sugar, and use Humalog [insulin] when readings are high." Considering that our system makes a BGL prediction, he said that he would look at the prediction and then do a test. If the test matched the prediction on several occasions he would probably use it. He finished the tasks in about 10 minutes and had questions throughout the tasks. When he began, he tried touching the screen, but the device is only interactive through the mouse or touchpad on my laptop. He used the touchpad. He found it somewhat difficult understanding the icons and asked about these. The icons representing the settings screen and the history screen were confusing to him, the others he found to be "somewhat ambiguous". I didn't realize that some of the tasks and interface features did not match up. For example, one task asks the users to press the 'forecast' button, but the button on the screen that should have been pressed was 'save'. I realized that better communication across our team would have prevented confusion like this. I wondered if Fr. Barry was becoming frustrated because of interface - task mismatches. As the person I tested with the greatest experience with glucometers (23 years), I took his opinion seriously when he gave the interface a 4 or 5 on a scale from 1 to 10 (10 being "most satisfied" with the device). He said data entry was complex and time consuming, and the feedback after saving it was "somewhat unclear". The BGL prediction he found a little vague. Asked about whether he would be able to save changes, he said that he wasn't sure because he hadn't made any data entry errors during the testing. Celia Celia is a 30 year-old single woman who learned of her condition of type I diabetes not too long ago. She is originally from El Salvador, however she has been leaving in the Washington DC area for the past 15 years. Although she speaks English fluently, she appreciates any sensitive information (related to her health) in Spanish. It was a very unexpected experience to learn of her condition and she is willing to learn and use everything she can in order to take care of her body. Prior to her learning of her condition, Celia enjoyed life doing different things such as dancing (her preferred hobby) as well as helping others doing community service. As she started to feel extremely tired very often, she decided to start some tests. Blood tests gave notice of her condition. At this point she decided to learn as much as she can about proper and balanced diet. She also tried to work closely with doctors in order to keep herself in good shape since she has great goals in her life. Celia is a school teacher for an elementary school in Montgomery County. She enjoys working with children and would like to keep her condition as secret as possible, however she understand that things are going to change for her for now on. Although she find technology tools (computers, PDAs, ipods, etc) fascinating, she does not consider herself computer savvy. Despite this, she feel very comfortable in using basic tools commonly found on the internet (such as pressing button, data entry, etc). She find different cultures very fascinating and enjoys spending time with people from other latin american countries and learn from them. These has led her to learn about different types of food. She prefers desserts more than anything and has a great taste when preparing them. This, however has been a great barrier to cross since she knows she needs to be more careful now that she needs to eat more balanced from now on. Celia expressed her impression about the idea of forecasting. She enjoyed very much the testing process, although she would have hope for complete functionality. She also found the “diet” record feature very neat (with arrows to add to your daily intake). She clearly noted that feature as the most userfriendly and easier to use part of the system. Monica Monica is an administrative assistant for a Contruction Company. She handles incoming calls as well as some book-keeping for the company. She is very familiar with two-three friends who have type I diabetes. Although she is not personally familiar with diabetes, she is very mindful of the condition of her friends and at times has conversed with them to learn more about it. One thing that called her attention was the fact that people with type I diabetes had to learn to be more disciplined with their diet. Monica showed that it was a useful tool as long as the user is willing to keep on entering data constantly. She understood pretty quickly that DFS depends on human interaction to provide an accurate reading and found that the interface (in general) was pretty well organized and made a lot of sense. Jennifer Jennifer is a college freshman in Taiwan. She was born with type I diabetes. When she was young, she did not have to worry too much about her BGL and insulin since most of the issues were taken care of by her parents. After she went to middle school, she began to learn to use BGL meter herself and calculate how much corresponding insulin she had to take. Jennifer seems to be very confident about the device even though it is her first time of using it. There was no occasion that she needed to seek out help from the device or from the observer. The flow of her operation over different menus were very smooth. She did not have any problem performing the assigned tasks. It seems that the user was satisfied with the interface of the device. Her case showed that for some users the interface for the DFS system was designed properly such that it is easy to follow and operate different functions. Even though her first language is not English, the words chosen for the menu were straightforward and easy to understand that she did not mistake any content or feedback. Because the user does not live in America, this usability test was conducted through presenting the interface online and combine instant messaging program along with a video camera and microphone to provide the observer chance to monitor the real reaction of the user. Conclusions & Future Work Final Status The current version of DFS (3.0) is only a preliminary version that shows the feasible interface of the device; most critical functions are working except the prediction model and actual reading of the blood glucose level (BGL). Users can record their exercise, diet, and BGL reading (assume they used other device than DFS because DFS is not physically capable of reading BGL now) and save them for future use, which is the prediction. Ideally, the prediction will change every time with different type of meal, different period of exercise, and different amount of dose of insulin. This is why DFS records all the data in each section and time after performing the tasks. The data in the History section (represented by the magnifying glass) is supposed to be used to make the prediction of BGL level over time. Right now, the DFS 3.0 has a random prediction that changes every time the exercise, diet, insulin, and BGL reading is recorded; this is to show that in the real working version the prediction should have actually changed. As for now, DFS 3.0 is not ready for public release. Future Work and Recommendations The future possible tasks for DFS would be mainly the actual implementation of the prediction model and the ability of reading BGL for the device. We believe that the Markov chains would be a feasible option for the prediction model. Currently, Markov chains are used for predicting weather conditions, and are the basis for Google's PageRank algorithm (http://en.wikipedia.org/wiki/Markov_chain). With further study and alternation, DFS would adopt a working model and be able to predict users’ BGL. Several different versions of DFS's interface have been developed, right now we have the data needed for translating DFS from English to Spanish, Chinese, and Japanese. Here is an example chart of corresponding terms in English and Chinese/Japanese (see Figure 12). As for recommendations, the myLife group has developed a very nice project that tracks personal diet information; we would like to cooperate with them to further improve the diet tracking function for the DFS device. We were also impressed by the DDAS (Diabetes Diabetic Analysis Software) group's project. They developed an interface on the web that users could upload information to about diet, exercise and insulin intake (not to mention BGL readings) in order to pass this along to doctors. What's required is to have a glucometer such as ours to record those types of information in order to be uploaded. Also, it is crucial to conduct more usability tests in the future as we add new capabilities such as new language supports. Figure 12 – Translation guide for DFS features Acknowledgements We'd like to thank the following people for making this project possible: Dr. Matthew Kim (consult) Dr. Ben Shneiderman (professor) Georg Apitz (TA) Ari Rasekh (critique) Chris Parlette (critique) Mike Delacruz (critique) Randy Jeune (critique) And all our usability test users... Lastly, but not least important, we want to thank one another for the hard work and good teamwork that we showed during the semester. If our team had suffered because of poor communication or a lack of interest on anyone's part, DFS would have been much different! References The following are references used to developed our working product. "Home Blood Glucose Prediction: Validation, Safety, and Efficacy Testing in Clinical Diabetes" Albisser, A. Michael, Ph.D. - Diabetes Technology & Therapeutics 7 (2005) "A Graphical User Interface for Diabetes Management That Integrates Glucose Prediction and Decision Support" Albisser, A. Michael, Ph.D. - Diabetes Technology & Therapeutics 7 (2005) "Type 1 diabetes: new perspectives on disease pathogenesis and treatment." Atkinson, Mark A; Eisenbarth, George S. - Deparment of Pathology, Immunology, and Laboratory Medicine, College of Medicine, University of Florida, Gainesville (pdf) "Computerising a guideline for the management of diabetes" Barahona, Pedro et al. - International Journal of Medical informatics 64 (2001): 275-284. "Self-Monitoring of Blood Glucose: The Basics." Benjamin, Evan MD. - Clinical Diabetes 20 (2002) (pdf) "Limitations of Conventional Methods of Self-Monitoring of Blood Glucose" Boland, Elizabeth, MSN APRN, CDE, Monsod, Teresa, MD, Delucia, Maria, MS, et al. - Yale University School of Medicine, Yale University, New Haven (online) "Information technology and computer-based decision support in diabetic management" Carson, E.R. et al. - Computer Methods and Programs in Biomedicine 32 (1990): 179 - 188. "Providing Secure Access to Confidential Patient Information Detailing Diabetic Condition" Chadwick, D.W., M.D., New, J.P., M.D., McDowall, D.M., M.D. University of Kent. 3 October 2005. (pdf) "A computer system for interpreting blood glucose data" Deutsch, T., Gergely, T., Trunov, V. - Computer Methods and Programs in Biomedicine 76 (2004): 41 - 51. "Type 1 diabetes: recent developments" Devendra, D.; Liu, E. Eisenbarth, G. Barbara Davis Center for Childhood Diabetes, Univ. of Colorado Health Sciences Center, Denver. (online) "Correlation of Fingerstick Blood Glucose Measurements with GlucoWatch Biographer Glucose Results in Young Subjects with Type 1 Diabetes." Garg, S. MD; Potts, R. PHD; Ackerman, N. PHD; et al. - (pdf) "Self-measurement of blood glucose. Accuracy of self reported data and adherence to recommended regimen." Gonder-Frederick, L.A.; Julian, D.M.; Cox, D.J., et al. (online) "DiasNet - an Internet Tool for Communication and Education in Diabetes." Hejlesen, Ole K, Plougmann, Soren, Cavan, David A. - Aalborg University. 3 October 2005. (pdf) "Self-Monitoring of Blood Glucose: Language and financial barriers in a managed care population with diabetes." Karter, A. PHD, Ferrara, A. MD. PHD; Darbinian, J. MPH. - Epidemiology Health Services Psychosocial Research. (pdf) "Usability in the real world: assessing medical information technologies in patients' homes." Kaufman, David R. et al. - Journal of Biomedical Informatics 36 (2003): 45 - 60. "Episodes of Severe Hypoglycemia in Type 1 Diabetes Are Preceded and Followed within 48 Hours by Measurable Disturbances in Blood Glucose." Kovatchev, B.; Cox, D., Farhy, L., et al. - University of Virginia Health System and National Science Foundation Center for Biological Timing. (online) "Computerised Decision-Support Tools in Diabetes Care: Hurdles to Implementation." Lehmann, Eldon D., M.B. B.S., B.Sc. - DIABETES TECHNOLOGY & THERAPEUTICS 6 (2004): 422 - 429. "The freeware AIDA interactive educational diabetes simulator http://www.2aida.org - (1) A download survey for AIDA v4.0." Lehmann, Eldon D. - Med Sci Monit 7 (2001): 504-515. "Information Technology in Clinical Diabetes Care-A Look to the Future." Lehmann, Eldon D., M.B. B.S., B.Sc. - DIABETES TECHNOLOGY & THERAPEUTICS 6 (2004): 755 - 759. "Collection of Information Directly from Patients through an Adaptive HumanComputer Interface." Lobach, David F., MD, PhD, MS; Arbanas, Jennifer M., MS; Mishra, Dharani D. MS, et al. - Division of Clinical Informatics, Department of Community and Family Medicine, Duke University. 3 October 2005. (pdf) "Self-Control: A Physician's Guide to Blood Glucose Monitoring in the Management of Diabetes." Mayfield, MD, MPH; Havas, S. MD, MPH, MS AAFP. (pdf) "Six Challenges in Measuring the Quality of Health Care" McGlynn, E. - Health Affairs 16 (1997). (pdf) "Interface Design for Health Care Environments: The Role of Cognitive Science." Patel, Vimla L. and Kushniruk, Andre W. - Cognitive Studies in Medicine, McGill University, Montreal, Quebec. "Infection and diabetes: the case for glucose control." Rayfield EJ, Ault MJ, Keusch GT, et al. (online) "Evaluation of Portable Blood Glucose Meters - Problems and Recommendations." Slingerland, R.; Miedema, K.; Isala Klinieken, et al. (pdf) "Using the ADAP learning algorithm to forecast the onset of diabetes mellitus." Smith, J W; Everhart, J E; Dickson, W.C. - Annual Symposium on Computer Applications in Medical Care, Washington, DC USA. 6-9 Nov. 1988. pp. 261265. 1988. (online) "Recent advances in treatment of youth with type 1 diabetes: better care through technology." Tamborlane, W.V.; Bonfig, W.; Boland, E. (online) "Newer Portable Glucose Meters - Analytical Improvement Compared with Previous Generation Devices?" Weitgasser, R.; Gappmayer, B.; Pichler, M. Salzburg General Hospital, Austria. (online) "SAM: Speech-Aware Applications in Medicine to Support Structured Data Entry." Wormek, A.K., DVM, Ingenerf, J., Ph.D., Orthner, H.F., Ph.D. American Medical Informatics Society. (pdf)