Final Report V.I .G.I .L.A.N.T. Visually Impaired Guidance Implementing Localization and Navigation Technologies ECE 4007 Senior Design Project Section L02, VIGILANT Anani Afanou Arpan Ghosh Daniel Newsome Ken Orji Submitted April 29, 2009 TABLE OF CONTENTS EXECUTIVE SUMMARY .......................................................................................................................... 2 1. INTRODUCTION .................................................................................................................................. 3 1.1 OBJECTIVE ............................................................................................................................................ 3 1.2 MOTIVATION ......................................................................................................................................... 3 1.3 BACKGROUND ....................................................................................................................................... 4 Kalman Filtering .................................................................................................................................... 4 GPS Device for the Visually-Impaired ................................................................................................... 5 2. PROJECT DESCRIPTION AND GOALS ........................................................................................... 5 3. TECHNICAL SPECIFICATIONS ........................................................................................................ 6 4. DESIGN APPROACH AND DETAILS ................................................................................................ 7 4.1 DESIGN APPROACH ................................................................................................................................ 7 User Interface and Directional Feedback .............................................................................................. 8 Navigational Technology ........................................................................................................................ 9 4.2 CODES AND STANDARDS...................................................................................................................... 10 4.3 CONSTRAINTS, ALTERNATIVES, AND TRADE-OFFS ............................................................................... 11 Constraints ........................................................................................................................................... 11 Alternatives ........................................................................................................................................... 12 Trade-offs ............................................................................................................................................. 12 5. SCHEDULE, TASKS, AND MILESTONES ...................................................................................... 13 6. PROJECT DEMOSTRATION ............................................................................................................ 14 7. MARKETING AND COST ANALYSIS ............................................................................................. 14 7.1 MARKETING ANALYSIS ....................................................................................................................... 14 7.2 COST ANALYSIS ................................................................................................................................... 15 8. SUMMARY AND CONCLUSIONS .................................................................................................... 16 9. REFERENCES ...................................................................................................................................... 17 APPENDIX A : HTC G1 SMART-PHONE SPECIFICATIONS ......................................................... 18 APPENDIX B : VIGILANT GANTT CHART ....................................................................................... 20 VIGILANT (ECE 4007 L02) 1 EXECUTIVE SUMMARY The purpose of the VIGILANT project was to develop a prototype of a smart-phone-based navigational device for use by the visually-impaired pedestrians that would aid them in maneuvering from one location to another on the Georgia Tech campus. An application running on the Google G1 mobile phone's Android platform decodes the received GPS coordinates from satellites along with differential GPS data from a base station atop the TSRB building located on the Georgia Tech campus. An additional laptop was needed to act as an intermediary between the differential GPS (DGPS) receiver and the Google G1 phone. Using DGPS significantly enhances the accuracy of traditional GPS systems which is essential for pedestrian applications. Two-position Kalman filtering was used to remove noise from the DGPS correction positions. The Georgia Tech wireless network, LAWN, was used for communication between the base station, the laptop, and the mobile device. The VIGILANT navigation device uses a convenient touch-screen for an easily accessible, userfriendly interface that visually-impaired users can effortlessly operate. The device uses a scrollable, audio-based interface with a comprehensive list of locations extracted from the online Georgia Tech campus map to allow visually-impaired users to input their destination. Once a destination is selected, our application uses Google Maps on the G1 phone to obtain the shortest walking path and the corresponding directions between the user's current location and the selected destination. The directions will be read out and updated in real-time based on the user’s position. Additional features of the product include route recalculation in case of user error and user orientation at starting location to ensure the user will start on the correct path and stay on that path. Overall, VIGILANT provides a cost-effective, lightweight, and easily portable navigational solution for visually-impaired pedestrians. VIGILANT (ECE 4007 L02) 2 V.I. G. I.L.A.N .T. 1. INTRODUCTION The VIGILANT navigation device uses a convenient touch-screen for an easily accessible and user-friendly audio interface that visually-impaired users can effortlessly operate. VIGILANT provides a cost-effective, light-weight, and easily portable navigational solution for visually-impaired pedestrians in need of a functional navigation device. 1.1 Objective The purpose of the VIGILANT navigation system is to provide directions to guide and aid a visually-impaired pedestrian in travelling from one location on the Georgia Tech campus to another. This navigation system uses differential GPS to pin-point the location of the user. A two-position Kalman filter was programmed on the Android operating system of a Google G1 mobile phone to remove noise from the differential GPS data measurements to accurately estimate the user’s position. By using the programmed G1 smart-phone as a GPS device, a visually-impaired pedestrian will have a lightweight personal navigator which reads out turn-by-turn directions. 1.2 Motivation Current GPS solutions for the blind are costly, ranging from $700 to $1600; VIGILANT after production, will cost as little as $450. Assuming that the user already has the Google G1 phone, the previous price includes a DGPS receiver and the VIGILANT software package. Alternatives like the Sendoro GPS system run on a bulky BrailleNote PDA while the VIGILANT system runs a small smartphone, which is an everyday device. This makes VIGILANT a lightweight, portable, non-intrusive VIGILANT (ECE 4007 L02) 3 solution to the alternatives, and will draw little attention to the visually-impaired pedestrian. The VIGILANT system aspires to make everyday navigation simple and easy for the visually-impaired without the need for bulky hardware [1]. The above mentioned solutions also use traditional satellite GPS which can give an error of 10-20 meters in its position estimate, especially in urban locations where there is interference from numerous sources like buildings. Pedestrian navigation demands position estimates with a significantly less magnitude of error. VIGILANT employs differential GPS, to correct offsets in traditional GPS, and Kalman filtering, which removes noise from position estimates, in order to improve the accuracy of user’s position by an order of magnitude. VIGILANT focuses on the visually-impaired, because new technology is at times notorious for alienating the visually-impaired. This pattern can be seen in the surge of touch screen gadgets which clearly are marketed for the sighted, with little concern for the blind. VIGILANT is a system that will use new technology to improve the everyday life of the visually-impaired [2]. 1.3 Background Kalman Filtering The Kalman filter algorithm was developed by Rudolph Kalman for spacecraft navigation for the Apollo space program [3]. The Kalman filter has further been implemented for all forms of navigation including land, marine, and aerospace. The Kalman filter estimates the state of a linear system by approximating the variables of a wide range of data. The estimate obtained from the Kalman filter can be used in tracking the position of a user from a GPS device. A two-position Kalman filter was used to filter the incoming DGPS data VIGILANT (ECE 4007 L02) 4 GPS Device for the Visually-Impaired GPS devices have been around for decades, but most of the products are geared towards the “average” user. These particular devices sometimes have entirely visual interfaces with no sound output at all. Making a user-friendly interface for the visually impaired is the key to creating a successful product. Sendero Group, a GPS company, has recently developed a talking map system for the visually-impaired called Way To Go. Way To Go uses BrailleNote and VoiceNote mPower software to automatically create routes for walking or riding in a vehicle along with distance and direction to the desired location [4]. Using an optional Braille interface, visually impaired users can easily operate the device and receive directions for their destination travel. A System for Wearable Audio Navigation (SWAN) is being developed to serve as a navigation and orientation aid for persons temporarily or permanently visually-impaired [5]. SWAN uses a personal computer along with advanced filtering techniques to safely direct the pedestrian user to their destination using an audio-only output via headphones. 2. PROJECT DESCRIPTION AND GOALS The purpose of the project was to produce a navigational device for use by the visually-impaired that aids them in travelling from one location on the Georgia Tech campus to another. The interface device was chosen to be a Google G1 smart-phone. An application was written to run on its Android platform which uses Kalman filtering to remove noise from corrected differential GPS position estimates. The use of differential GPS provides highly accurate positioning of the user on the mapping system. An VIGILANT (ECE 4007 L02) 5 additional lightweight DGPS receiver was required to receive the differential GPS data. The Georgia Tech wireless network, LAWN, was used to transmit the GPS data to the mobile device. A scrollable, audio-based interface with a pre-defined list of locations found on the Georgia Tech campus was designed to run on the G1 phone to allow the user to input their destination without having to physically type the information on the device. External headphones can be used to listen to the turn-byturn directions of the system to the user. Additional features of the product include route recalculation in case of user error and user orientation at the starting location to ensure the user will begin his/her trek in the correct direction and stay on the designated path. The goals of the final product are as follows: Receive differential GPS corrections to offset errors in satellite GPS. Remove noise from DGPS data by running a Kalman filter in software. Provide and dynamically update turn-by-turn navigational instructions. Supply an audio-based, user friendly interface. Appeal to visually-impaired pedestrians as a lightweight, portable, and costeffective navigational tool. 3. TECHNICAL SPECIFICATION VIGILANT was implemented using a DGPS receiver and a smart-phone which runs the Google Android platform. A laptop acts as an intermediary between the DGPS receiver and the G1 phone to send and receive data corrections and estimates. The smart-phone will run the core of our application, VIGILANT (ECE 4007 L02) 6 including the user interface and providing directional audio feedback to the user. The HTC G1 smartphone was chosen, and its specifications are given in Appendix A. The DGPS chip set receives ephemeral satellite correction data from a base station, processes this information, and sends a corrected estimate of the user's position to the smart-phone after removing measurement noise using a two-position Kalman filter. The SOKKIA GIR 1600 differential GPS receiver has been chosen, and its specifications are listed below in Table 1. Table 1. Specifications for the DGPS Chip Set Model Brand SOKKIA Series GIR 1600 Performance & Features Position Accuracy Sub-meter Update Rate 20 Hz Interface Bluetooth, Serial Operating Time Above 5 hrs. Weight 1 lb. 4. DESIGN APPROACH AND DETAILS 4.1 Design Approach VIGILANT was implemented in two key parts. The first was the user interface which includes directional feedback. The second part was the navigational technologies implemented in the back-end which pinpoint the location of the user. VIGILANT (ECE 4007 L02) 7 User Interface and Directional Feedback VIGILANT was implemented on the HTC G1 smart-phone powered by the Google Android platform. Android is an open-source operating system and software platform for mobile devices which allows developers to write their own code and applications for these devices in the Java language. The team chose the Android platform as it is open-source, has a superior API with enriched libraries compared to other smart-phone platforms, and is possible to obtain root-level access for writing drivers and more privileged applications. Using the Android API also makes our application independent of the smartphone manufacturer. Currently the HTC G1 is the only commercially available smart-phone which supports the Android platform. Keeping in mind the target audience for this product, the user interface was designed to be extremely simple and accessible. The interface consists of a scrollable list containing the names of buildings and other places on the GT campus which was imported from the GT campus map. The user can navigate this list using the G1's touch-screen interface or side-scrolling buttons. Optional keyboard support can also be used to enter destination. The application also provides voice feedback to the user by verbally outputting the name of the selected building/place in the list. The user can to listen to all audio feedback through any standard pair of headphones or the external speakerphone on the mobile device. Once a destination is selected, our custom application uses the Google Maps application on the G1 phone to obtain the shortest walking path and the corresponding directions between the user's current location and the selected destination. The G1 has the ability to connect to any 802.11 a/b/g wireless network and uses the GT wireless LAWN to operate Google Maps. Our application voices text directional instructions using the text to speech libraries in Android. Based on the user’s current location, these instructions will be updated. VIGILANT (ECE 4007 L02) 8 Navigational Technology Differential GPS (DGPS) is the primary navigational technology for this application. A base station situated on the roof of the Tech Square Research Building broadcasts ephemeral satellite corrections for the coordinates received by the GPS in the G1 smart-phone. A server was coded to enable this broadcast to occur over the GT Wireless LAWN. The broadcast corrections are received by a DGPS receiver chip set which is connected to a laptop via Bluetooth communication. The laptop acts as an intermediary between the receiver and the G1 in our prototype. The final version of the product would utilize a Bluetooth connection to eliminate this intermediary. Figure 1 depicts the high-level view of the “final” version of the VIGILANT navigation system, in which the DGPS receiver connects directly to the Google G1 phone via Bluetooth. The laptop processes the corrections, applies them to the raw GPS data, and sends them to the G1 phone over the GT LAWN wireless network. Figure 1. High-level view of “final” version of VIGILANT navigation system. DGPS improves the accuracy of the user's position estimate over standard GPS by an order of magnitude. Once DGPS was implemented, a Kalman filter was coded to recursively refine the estimate VIGILANT (ECE 4007 L02) 9 of the user's position by removing noise from the GPS coordinate readings. The DGPS coordinates, after being passed through the Kalman filter, is the final position estimate used by our way finding application. Figure 2 illustrates the overall hierarchy of the VIGILANT prototype that was designed including the use of the laptop as an intermediary between the DGPS receiver and the Google G1 phone. Figure 2. VIGILANT prototype high-level design. 4.2 Codes and Standards We implemented this product on widely used and accepted technology standards to make it as portable and flexible as possible. VIGILANT was implemented using the following technology standards: The IEEE 802.11 a/g wireless communication standard, commonly known as wireless LAN (WLAN), operates in the 5 GHz frequency band with a maximum data rate of 54 Mbits/second and a VIGILANT (ECE 4007 L02) 10 throughput in the mid-20 Mbits/second range [6]. This is the most widely used WLAN standard and makes it very easy to port our application to other campuses or locations which have such a network enabled. The Global Positioning System (GPS) is a global navigation satellite system that uses satellites to transmit precise microwave signals thereby allowing GPS receivers on Earth to determine their current location, velocity, and the time [6]. There is no way to make our smart-phone global due to service provider limitations; however the GPS on the phone will continue to function anywhere on Earth. Android is an open-source software development platform and operating system for mobile devices based on the Linux kernel [6]. The open-source aspect of Android is attracting several mobile phone manufacturers who are coming out with smart-phones powered by this platform within the next year [7]. We have chosen the android platform in an attempt to make our application independent of the phone manufacturer. 4.3 Constraints, Alternatives, and Trade-offs Although the concept for the VIGILANT navigation system is a fairly new idea, the actual design of the device had a few constraints and trade-offs in order for the device to properly function. Alternative designs and implementations for different aspects of the project were also considered. Constraints With our primary customers being visually-impaired pedestrians, we are limited to audio response to provide directional feedback to the user. The user interface was designed in an extremely minimalistic and accessible manner so that it could be operated by a visually-impaired person. The HTC G1 is the VIGILANT (ECE 4007 L02) 11 only commercially available smart-phone that currently supports the Android platform, making it the ideal choice for this prototype. This implementation of VIGILANT was limited to the Georgia Tech campus as it is the only DGPS base station in the area which we can get access to as developers. However, it can be ported to other locations covered by DGPS by only changing the list of destinations that show up in the user interface. This prototype of VIGILANT requires that the user to carry around a laptop to interface between the G1 smart-phone and the DGPS chip set. This intermediary can easily be removed by the manufacturers or professional developers who have root level access to the phone and can write drivers for serial communication over the G1’s mini-USB port or for Bluetooth add-ons. Alternatives If the Kalman filtering approach had failed to remove the noise from the DGPS readings, the team intended to design a custom filter which integrated readings from the G1's accelerometer to determine position and thereby correct the DGPS coordinates over time. Bluetooth was also explored as an alternative communication link between the laptop and the G1 smart-phone if communication over GT wireless hit a road block. Currently there is no API for Bluetooth in Android. Trade-offs VIGILANT's unique selling point lies in its implementation on a pocket-size, lightweight and mobile device that visually-impaired pedestrians would be able to carry on them on a daily basis. An implementation using standard GPS is possible with just the smart-phone as required hardware. However, the team has chosen to use DGPS which requires the user to also carry a laptop in a backpack VIGILANT (ECE 4007 L02) 12 On the other hand, this improves accuracy over standard GPS by an order of magnitude which is very important for navigation on a pedestrian scale. As mentioned before, the inconvenience of carrying the laptop around can be removed very easily in the future by the manufacturers or developers of the smartphone, who have root level access, by writing mini-USB to serial drivers or additional drivers for the support of Bluetooth add-ons so the phone can communicate directly to the DGPS chip set. 5. SCHEDULE, TASKS, AND MILESTONES The design group was divided into teams of two, and each team was assigned a section of the project. Table 2 shows the tasks, start and end dates, durations, priority, and person(s) responsible for the task. The important project milestones are marked in bold, in the priority column. A Gantt chart depicting the schedule, tasks, and milestones can be seen in Appendix B. VIGILANT (ECE 4007 L02) 13 Table 2. VIGILANT work schedule 6. PROJECT DEMONSTRATION Upon completion of the project, the VIGILANT navigation system was capable of delivering accurate directions to the user for use in way-finding. To demonstrate our working product, we will simply blindfold a member of the team, and that person will use only the VIGILANT navigation system to find their way from a starting position to a destination on the GT campus. The results of the demonstration will be based upon how accurately the device navigates the user to the chosen destination and how easily the device operates without the use of visual stimulations. VIGILANT (ECE 4007 L02) 14 7. MARKETING AND COST ANALYSIS 7.1 Marketing Analysis VIGILANT can be marketed to any region or area that has a Differential GPS base station. With the increase in base stations, as well as DGPS base stations kits, systems like VIGILANT will see a rise in demand in the near future [8]. There is currently one product in the market that is similar to VIGILANT. This product is the Sendero GPS system. This system is designed to be used with BrailleNote PDA, a portable note taker for the visually impaired. Like VIGILANT, the BrailleNote GPS offers travel direction to the user, as well as audible location information. The downside of the Sendero GPS is that it only works with the BrailleNote which is quite costly at about $5500; VIGILANT works on smartphones, which price at about $400 [9]. Unlike VIGILANT, the BrialleNote GPS system does not use differential GPS to get a more accurate reading of the user’s position. The BrailleNote GPS system, including the PDA, is quite bulky and can be considered an inconvenience. VIGILANT is designed to work on a smart-phone, which is light and nonintrusive. The advantage that VIGILANT has over its competitor is that it is cheaper, more compact, and has a simple user interface that can be easily accessed by the visually-impaired [2]. 7.2 Cost Analysis The total cost of the VIGILANT prototype is $460 in parts, shown in Table 4, and $10,500 in labor. The main cost of VIGILANT is dependent on the price of the smart-phone it runs on. The VIGILANT prototype will be running on the T-Mobile G1 phone, priced at $179.99 with a new two year T-Mobile contract and $399.99 without the contract [9]. The price of the T-Mobile G1 phone is excluded from the total product cost, assuming that the user already has already purchased the phone. The labor VIGILANT (ECE 4007 L02) 15 cost for creating the system is about $10,500. This price was calculated assuming that four engineers designed and built the system in approximately 300 hours at a wage of $35 an hour. After this initial cost of production, VIGILANT can be sold as a software package for a small fee of $49.99 through “Google Market”, a home for applications made using Google Android. The user who purchases VIGILANT will still have to acquire a DGPS receiver. A very good DGPS receiver is priced at approximately $400. This price is still cheaper than the $1,649 price of the Sendero GPS System. Table 3 shows the cost breakdown of all additional components needed to implement the VIGILANT system. Table 3. Component price list Component Price DGPS Receiver VIGILANT software $ 400.00 $ 49.99 TOTAL COST = $ 449.99 package 8. SUMMARY AND CONCLUSIONS All devices including the Google G1 phone, the DGPS receiver, the DGPS base station, and the laptop have been incorporated to communicate together via either the GT LAWN wireless network or Bluetooth communication. All essential code including that for the design of the audio-based userinterface, the retrieving of Google Maps route coordinates, and the two-position Kalman filter has been coded in the Java language for use on the Android API. All code has been integrated together to form a final, functioning navigation device. An additional feature of the VIGILANT product includes corrected user orientation at the starting location to ensure that the user is facing in the correct direction before any navigation begins. This is VIGILANT (ECE 4007 L02) 16 implemented by using a heading value that is taken from the phone’s internal compass to direct the user to walk in the correct initial direction. Another added feature is route recalculation. If the user takes a wrong turn or misses a turn, VIGILANT supplies a new set of directions to get the user back on the original path. Also if the user deviates slightly from the path, VIGILANT provides warnings to quickly get the user back on track. To ensure the best possible results from the final device, testing of the VIGILANT navigation system continued until the final project demonstration deadline in order to tweak the Kalman filter parameters as well as other parts of the system’s code. Upon the final demonstration, the team showed that the use of the mobile phone’s internal compass would successfully point the user in the correct starting direction and that the navigation system would successfully send and receive DGPS coordinates through the Kalman filter to obtain the user’s estimated position; however, a timing lag between the sending and receiving of that information was shown to be problematic. In order to achieve a fully functioning navigation device, timing parameters must be tweaked to achieve perfect synchronization. 9. REFERENCES [1] B. Hachey, “GPS Systems for the blind: Great Navigation Aids, but Not for the Faint of Heart,” October, 2008. [Online]. Available: http://www.carroll.org/2008/10/02/gps-systems-for-the-blindgreat-navigation-aids-but-not-for-the-faint-of-heart. [Accessed Feb. 01, 2009]. [2] S. Carew, “Touch-screen gadgets alienate blind,” January, 2009. [Online]. Available: http://www.reuters.com/article/newsOne/idUSTRE5080T320090109. [Accessed Feb. 01, 2009]. [3] D. Simon, “Kalman Filtering,” Embedded Systems Programming, [Online Magazine], (2001 June), Available: http://academic.csuohio.edu/simond/courses/eec644/kalman.pdf VIGILANT (ECE 4007 L02) 17 [4] “Way To Go: Accessible Talking Maps for the Blind,” Sendero Online Store, [Online Document], Available HTTP: http://www.senderogroup.com/shopwtg.htm [5] J. Wilson, B. N. Walker, J. Lindsay, C. Cambias, F. Dellaert, “SWAN: System for Wearable Audio Navigation,” Proceedings of the 11th International Symposium on Wearable Computers (2007 Oct.), [Online Document], Available HTTP: http://sonify.psych.gatech.edu/~walkerb/publications/pdfs/2007ISWC-Wilson-et-al-submitted.pdf [6] Wikipedia, [Online Document], Available HTTP: http://www.wikipedia.org [7] “Multiple Android phones expected in 2009” InformationWeek, [Online Document], Available HTTP: http://www.informationweek.com/news/personal_tech/smartphones/showArticle.jhtml?articleID= 212501692 [8] TAPR, “DGPS Reference Station Assembly Manual,” DGPS Datasheet, Mar. 1998. [9] L. Holson and M. Helft, “Smartphone Is Expected via Google,” New York Times, 03 Aug., p. 34, 2008. VIGILANT (ECE 4007 L02) 18 APPENDIX A HTC G1 Smart-Phone Specifications VIGILANT (ECE 4007 L02) 19 Model HTC Brand Series G1 Performance & Features Modes Weight Dimensions Form Factor Battery Display Platform / OS Processor Memory GPS / Location Bluetooth USB Wi-Fi Packet Data WAP / Web Browser Touch Screen Memory Card Slot GPS / Location GSM 850 / GSM 900 / GSM 1800 / GSM 1900 WCDMA 1700 / WCDMA 2100 5.6 oz (159 g) 4.62" x 2.19" x 0.67" (117 x 56 x 17 mm) Slide Internal Antenna Talk: 5.8 hours max. (350 minutes) Standby: 402 hours max. (16.8 days) WCDMA mode 1150 mAh LiIon Type: LCD (Color TFT/TFD) Resolution: 320 x 480 pixels 3.2" diagonal Colors: 65,536 (16-bit) with color dithering Android 528 MHz Qualcomm MSM7201A 71 MB (internal memory available to user for storage) Type: A-GPS supports LBS, includes Google Maps with Street View Supported Profiles: HSP, HFP 1.5 version 2.0+EDR mini-USB connector Version(s): 802.11 b,g Input Side Keys Primary input method: Finger capacitive Card Type: microSD (TransFlash) 1 GB card included / supports up to 8 GB Type: A-GPS supports LBS, includes Google Maps with Street View Technology: HSDPA 7.2 Packet Data WAP / Web Browser Browser Software: Google Android (based on WebKit) with Google Gears Touch Screen VIGILANT (ECE 4007 L02) Primary input method: Finger capacitive. 20 APPENDIX B VIGILANT Gantt Chart VIGILANT (ECE 4007 L02) 21 VIGILANT (ECE 4007 L02) 22