Pedro Rivera Torres
Ammar Almani
Srrah Algheithy
Thad Guriel
Andy Duann
Hao Chen


PDR Revision
Functional decomposition (hardware)
 Hardware subcomponents/implementation

Functional decomposition (software)
 Software subcomponents/implementation



Current project status
Goals (milestone 1, 2, expo)
Technicalities (schedule, budget)
Thad Guriel

Sam Siewart
 Google IO interface

Jim Pilkington
 Assistive Technology Specialist
 Colorado Department of Human Services
 Division of Vocational Rehabilitation

Catherine Kilcommons
 Assistive technology lab coordinator
 Disability services - University of Colorado Boulder
Thad Guriel
User
Audio Output
RF
Receiver
User Info
Android
Maps Info
CPU
RF
Comm
IR
Receiver
IR
Navigation
Keypad
Thad Guriel
Thad Guriel
Thad Guriel
Hardware
Device
Navigation
IR Receiver
Communication
Transmitter
Door
Microprocessor
Receiver
Audio
TTS 256
Navigation
Phone analog
IR Transmitter
Thad Guriel
Communication
Transmitter
Receiver
Query signal sent
from device Xbee
Xbee checks for signal
Xbee establishes
communication with
device
No
Is there a
signal?
Yes
Xbee sends interrupt
signal to processor
Andy Duann
Xbees establish
communication
Processor wakes up
Processor kicks
out sign data
Processor
turns on IR
Talk to device
XBee
Start clock
Send data
Xbee cyclic
sleep
Turn off IR
Processor goes
to sleep
Andy Duann
Andy Duann
Cyclic sleep
Cycle
wake up
No
Was
there a
signal?
Interrupt from
processor
Yes
Xbee
establishes
communication
Send signal to
wake door
XBee
Transmit
data
Andy Duann
Andy Duann
No audio signal
uP outputs logic
low to switches
Turns off
synthesizer switch
Turns on Android
switch
Andy Duann
Andy Duann
Phone audio
Comparator
outputs logic high
uP turns off
synthesizer
interrupts
Outputs logic low
to switches
Comparator goes
low when phone
audio gone
Andy Duann
Andy Duann
Synthesizer audio
Turns android off
Turns synthesizer
on
Synthesizer sends
interrupt request
when data present
uP sets logic high
to switches
uP checks android
interrupt
No
Is it low?
Yes
uP sends interrupt
enable to
synthesizer
Andy Duann
Andy Duann
Andy Duann
Andy Duann
Andy Duann


Locate building entrance after hand-off from
GPS
Read signs from a distance and determine its
relative direction
Hao Chen
GPS tells user they are in the
vicinity of Engineering
entrance, and “Engineering
Building; to your northeast”,
for example
Device tells user “Engineering
lobby; to your north” and
“elevators; to your West”.
Hao Chen


3 infrared LED’s are
mounted on the entrance,
to send directional
information at a wide
angle
The information of the
building is transmitted
with XBee
Hao Chen



5 IR receivers are mounted on the device
IR receivers are sensitive to 950 nm
wavelengths and 38 kHz carrier only
The direction of the entrance can be
determined based on which IR receiver(s)
received the signal
Hao Chen
2
1

3
Reader
4
5
In the above picture, receiver 4 picks up the
signal while the rest don’t, therefore the
microprocessor determines the sign is at
northeast direction
Hao Chen




Motivation: battery runs out quickly if IR
LED’s are on all the time
Solution: only turn transmitter on when user
close
Device uses XBee to trigger transmitter
The transmitter turns off after set amount of
time
Hao Chen
User
approaches
entrance,
transmitter
turns on
XBee
transmits
data to device
Device
determines
direction of
entrance
Device
outputs
information
and direction
through
speaker
Sign
automatically
turns off after
set time
Hao Chen
Srrah Algheithy
Level 0
Software
Level 1
Android
Level 2
Android app
Level 3
GPS/location
information
Android ADK
Directional
information
Keypad
PIC24FGB110
Android to
ADK USB
interface
USB
initialization
Xbee
information
Srrah Algheithy
Android app
to access
GPS and
Compass
USB interface
Android phone
2.3.4
gingerbread
Android ADK
Srrah Algheithy
Continuously
given
information
about
surroundings
• Building names
• Street names
Current
location
continuously
updated
Navigate to
points of
interest
• Bus stops
Srrah Algheithy
Android
Location
Manager
Geocoding
GPS
App
Location
Listener
Proximity
Alert
Srrah Algheithy

requestLocationUpdates()
 Location provider
 Time between updates
 Distance between updates
 LocationListener()

ItemizedOverlay<Overlay Item>
 OverlayItem()
▪ Specify and name points of interest

Use built in compass for direction
Srrah Algheithy






Communicate effectively with the keypad system.
Coordinate Keypad output with Android apps.
Control internal functionalities of the Android
phone
Manage operation and information coming form
Xbee
Coordinate Xbee info to turn on IR sensors.
Manage the audio system.
Ammar Almani


Using PIC24F j256 GB110
Features:
 16-bit: similar to DSP
 Still low power (3.3V)
 Programming Memory: 256 KB
 Full Speed USB operation in Device
 USB on-The-Go
 3 ADCs: 500K conversion / second.
Ammar Almani
Primary
Secondary
Main()
Bluetooth
Connection
Audio
Voice Recognition
(Android)
Android/GPS
Keypad
Sensors
Xbee info
Ammar Almani


Coordination Module
Using the keypad encoder to generate interrupts
when a button is pressed.
Module
Keypad_IR()
Inputs
-An interrupt signal from chip
- 4 binary digits
Outputs
- The appropriate command to
both Android phone and Xbee
Functionality
To coordinate between user
and peripherals.
Test Procedure
Test to control android app
through PIC24F android chip
Ammar Almani

Input/output module
Module
gps_Info()
Inputs
- Destination specified by user
through keypad
Outputs
- Control audio system to issue
guidance commands
Functionality
Takes destination as input uses GPS
in Android to output guidance voice
commands
Test Procedure
More than one
Ammar Almani

Composite module
Module
handle_sensors()
Inputs
-Receive signal from GPS to turn on
sensors
-If analog needed use ADCs
Outputs
- Output a specific direction for
audio system
Functionality
Takes IR information from five
sensors and use transmitted data
to perceive the sign’s location .
Test Procedure
- Once HW ready, PIC24 kit is used
to test.
Ammar Almani

Input Module
Module
receive_xbee()
Inputs
-Wake up xbee signal from GPS
Outputs
-information in buffer to the audio
function
Functionality
Takes data from xbee, store it in a
buffer, make sure all data is
transmitted; and then send it to the
audio function.
Test Procedure
- Use UART utility to test function
in 2 levels
Pedro Rivera

A system that outputs audio signals from both
Android device and text-to-speech chip
Module
output_audio()
Inputs
-Synthesizer interrupt request
-Android Interrupt
Outputs
-Logical output to analog audio
switches
Functionality
Takes both data from xbee and
main and an audio interrupts form
android; then decides the priority
and behave accordingly.
Test Procedure
- More than one test
Pedro Rivera
Primary
Secondary
Bluetooth
gps_info
Audio
Kyepad_I
R
Voice_recognition
Xbee_receive
Sensors
Extra sensors
Pedro Rivera

Requirements for Open Accessory
Hardware
 Making PIC the host
▪ Being host entails:
-Initiating communication
-Knowing what all devices are doing
Pedro Rivera

Open accessory Handshake (When
accessory is connected, the accessory
will perform these steps):
 Send “Get Protocol” command
 Send device info to identify the
accessory to the Android Device
 Send “Start” Command
 The Android device should re-enumerate
in accessory mode and launch an app
Pedro Rivera

USB Accessory Handshake, part 2
•
Accessory mode if:
•
•
Vendor ID is 0x18D1 (Google)
Product ID is 0x2D00 or 0x2D01
•
Read configuration descriptors
• Look for first bulk IN and first bulk OUT endpoints
• Set configuration to 1
• Endpoints are now ready for communication
Pedro Rivera
Pedro Rivera
Hardware
Software
Audio circuit built & testing in
progress
Android app for location written
(testing in progress)
Keypad circuit built & tested
ADC driver working for up to 80KHz
IR built & testing in progress
Interrupt is pending
XBee testing in progress
UART driver pending
Thad Guriel
Hardware
Software
Successfully implements the “wake up” of
signs when user is near
App provides current location,
destination, arrival, direction
Successfully generates 38 kHz modulated
infrared signal and receives it at a 5 meter
distance
USB driver working
Successfully limits the range of XBEE to 10
meters and able to transmit information
from one XBEE to the other.
Keypad, press a button on keypad and it
controls something on the android
PCB is designed and ordered
Xbee receives signal, gets information and
saves it (doesn’t have to be correct)
UART driver working
Thad Guriel
Hardware
Software
3 IR LEDs are used to complete the signs
Interfacing between keypad and GPS
All 5 IR receivers are mounted on the device
& successfully determines the direction of
the sign
App gives directions to destination
Received XBEE sign information is
outputted through speaker
Xbee functioning
PCB revision built
Audio under testing
Thad Guriel


All chips are mounted on PCB, device is fully
functioning.
The project is demoed through video
recording.
Thad Guriel
Task
Thad
Hao
Srrah
Andy
Pedro
Ammar
GPS App
Primary
Secondary
Secondary
USB interface
Secondary
Primary
Secondary
Microprocessor
Secondary
Secondary
Primary
Communication
Primary
Secondary
Secondary
IR
Secondary
Primary
Secondary
Power
Secondary
Secondary
Primary
Keypad
Secondary
Secondary
Audio
Primary
Secondary
PCB
Primary
Primary
Hardware
Software
---------
Secondary
Secondary
Primary
Primary
Primary
Secondary
Secondary
Primary
Primary
Secondary
Thad Guriel
Primary
Secondary
Part
Cost
PIC test board
$60
XBee
$50
Speakjet
$24.95
TTS 256
$21.95
Audio Amp
$3
Keypad encoder
$10
Voltage regulator
$2
Flip flop
$2
IR
$10
Arduino
$30
Headphone jack
$5
Total
-$218.90
UROP funding
$1000
Remaining budget
$781.1
Thad Guriel










Wall e (slide 5) - http://www.robotbooks.com/robot_toys_II.htm
User (slide ) - http://platinumjellybean.tumblr.com/post/1005345702/this-is-a-stick-man
Device for visually impaired (slide 6) - http://www.humanware.ca/web/en/newsletter/240.htm
Microchip (slide 36) - http://www.edn.com/article/520757Microchip_targets_active_current_reduction_in_PIC24F_expansion.php
Android logo (slide 33 & 34) – http://blogs.vertigo.com/personal/ericl/Blog/Lists/Photos/Forms/DispForm.aspx?ID=12
Android logo (slide 32) – http://www.digitaltrends.com/mobile/do-developers-prefer-apples-or-androids/
Gingerbread Android logo (slide 31) – http://www.vpsems.com/android-gingerbread.html
Visually impaired navigation (slide 22) - http://www.teu.ac.jp/hashino/research/index.html
Clip art (slide 27, 26, 24 ,23 & 56) – Microsoft PowerPoint clipart
Android/USB – http://www.everythingusb.com/android-open-accessory-kit-21147.html
Thad Guriel
Andy Duan
Thad Guriel