Interactive Prototype Report
Group 3
Henry Su - Did the Camera, Manage, and Translation mode activities (all closely
related), wrote up the overview of the UI, added to/organized the "what was left out"
and "wizard of oz" parts of the Prototype Overview, and did the Readme.
Jeremy Syn - Implemented the Favorites Translation interface, as well as working on
Prototype report
Michael So - Did the options menu screen, made the layout xml files for Manage and
Translation mode activities,did the Tasks section and Problem and Solution Overview
in the Report
Eric Chung - Implemented the cropping activity and wrote on the crop parts in the
doc.
Problem and Solution Overview
The problem that some people have when traveling overseas is that they do not have
a strong grasp of the native language. They may feel really lost and confused without
the availability of proper language aides. Without the ability to read signs or
communicate with the natives, one may feel insecure traveling in that kind of
environment. This problem could even extend to visiting local areas and shops where
the locals or workers do not speak fluent English.
The solution we are proposing is a
mobile application that allows you to take a picture of a sign in a foreign language,
and the application will translate it into a language of your choice. For example, you
are an English speaker who is on vacation in China; you can use our mobile
application to take a picture of a sign in Chinese and translate the Chinese to English.
This solution is an approach to create a comfortable way to travel in an unknown,
foreign area.
Tasks
Easy - Choosing from a variety of possible translations
The first task is to make use of the multiple translations capability of our application.
When looking up a translation for a certain image, we provided the option of looking
at alternative translations in the case that the meaning of the words were ambiguous
and hard to understand in that particular context. In order to look at the multiple
translations, the user would have to go through the normal process of taking a picture
and pressing the translate function to receive the translation of the image. Then
below the image would appear the translations of the text.
Moderate - Save result for future reference
The second task is to save a particular image into the favorites list. We provided the
capability to be able to store images with their corresponding translations into the
phone for when you want to look at them again at a later time. This may happen
when you want to look for something, but you don't remember what the sign looks
like. You can look for the image from your list based on the translation and look for
the signs that look like the sign in the image. To do this, when on the screen with the
image and its translations, the user would click the menu button and select the save
option. Then the application would acknowledge the user that it has been saved to
the favorites list. To bring up the favorites list, click the menu button at any time and
select the options menu. In that menu, the favorites list would be the first on the list.
Hard - Selecting region of picture to translate
The third task is to take a picture of a sign and then crop out unnecessary objects that
may interfere with the text recognition program. This would be useful in situations
where the camera is maximally zoomed in and couldn't quite center the text, with a
lot of extra things in the background such as trees and buildings. To perform this
task, the user would take the picture and in the next screen before pressing the
translate button, go to the menu options and select the "crop" option. A cursor would
then appear on the screen and user can move the cursor with the arrow buttons.
When the middle button is selected the cropping begins, and when pressed again
completes the cropped area. The user can then press the translate button to translate
just the cropped area.
Revised Interface Design

We took out the "Retake" options in the menu options list because we figured it
would be intuitive for the user to just press the "Back" button to go back and
retake the image. So instead of having a "Retake" option, we have "Back" fulfill
that task.

Also, in crop, the bottom bar was taken out. In Android's camera mode, there is
no bottom bar there either; this gives the user the entire screen to look at and
take a picture. So, in crop mode, we allowed the user to use the entire screen to
crop. This means there are no on screen instructions, but we figured the user,
after seeing the cursor appear on the screen, would use the arrow keys to move
it to a corner of the desired cropping rectangle, then attempt to use the "select"
button (the center dpad button), see the cursor flip, use the arrow keys to move
the cursor again to make the rectangle bigger, and then press select again when
the rectangle was big enough. It seems intuitive enough, especially without a
touch screen.

We took away the "Menu" icon on the bottom bar of the screen because the
emulator already comes with a menu button so it should be intuitive for the user
to simply click that menu button instead.

In our Favorite Translations, we ordered it in alphabetical order and took away
the numbers because it would interfere with the search function.

We changed the image saving dialog from "Saved!" to "Saved to Favorites" to
make it clearer as to where the image is being saved to.
Sketches of unimplemented portions of the interface
Storyboard
Figure 1
Application starts in the camera mode.
Figure 2
Press Center button to take picture
and enter manage mode.
Figure 3
Goto Menu options and select "Crop".
Figure 5
Use arrow keys to move cursor to
desired position.
Figure 4
Cursor being at upper left corner.
Figure 6
Complete the box to select the area
of interest.
Figure 7
Press Center key to select cropped
area.
Figure 9
Choose the “Save” option from the
menu to save to favorites.
Figure 11
Choose Options from the Menu to
bring up the options menu.
Figure 8
Press Center key to translate image.
Figure 10
Save dialog alerts user.
Figure 12
Select Favorites.
Figure 13
List of favorite translations.
Figure 14
When selected shows you the saved
translation.
Prototype Overview
Overview of the UI implemented:
Our application starts out in Camera mode, as shown in Figure-1. Here, the user can
press the Menu button to change the From and To translation languages, as well as
access the Options screen. There is also a Zoom feature, done by pressing the "+"
and "-" keys. As with most phone cameras, the user presses the center button to take
the picture. Taking the picture also leads us to the Manage mode, shown in Figure-2.
In the Manage mode, you have the additional Menu option to Crop the picture, in
order to select the region you want to have translated. Once the cropping is done, we
enter the Manage mode again, but this time showing the newly-cropped image. This
is shown in Figure-7 below. Pressing the center button again will translate the text
from the image (only the part within the cropping box, if specified), and move the
application to the Translate mode (Figure-8). Here, the Menu loses the Crop option,
but gains the Save option and the Send option. There may be multiple possible
translations, which are all given below the picture. The user may Save the
translation/picture to the Favorites list by pressing Menu->Save. An alert box pops
up, informing the user that the picture has been saved (see Figure-10). Note that
whichever translation is highlighted (if there were more than one possible translation)
is the one that's saved to Favorites. To get out of the dialog box, just press Back. To
view the Favorites list, hit Menu->Options from Camera, Manage, or Translate mode,
then click Favorites once in the Options menu. The Favorites list is shown in Figure13. Then, scroll down to the desired translation, and hit the center button to arrive at
the picture and translation(s) as it first appeared in Translate mode.
What was left out and why:
1. In the Menu in all three modes, the From: and To: translation language selector
was not implemented yet. This is because it was not a necessary part of the three
tasks.
2. Similarly, the Zoom feature was not implemented as it wasn't part of the three
tasks.
3. The Send option in the Menu from Translate mode is not implemented yet, because
it was not part of the three tasks; when implemented, it would allow the user to send
the translation/image to someone on his/her contact list.
4. In the translate mode, we did not implement the side arrows that let the user know
if there are more possible translations down the list below the bottom of the screen.
This is in part because of its difficulty, but also because it was not necessary for the
demo, which only features 2 possible translations, which could fit in the screen space
already.
5. On the Options screen, the items "2. Tourist Guide", "3. Select Languages", and "4.
Add/Remove Dictionaries" do nothing when clicked on. This was done because they
were not necessary Activities to accomplish the three tasks. Only "1. Favorites" was
necessary for the case of the moderate task.
6. On the Favorites screen, the search box is non-functional. It is supposed to autoscroll down the list of translations as the user types, to make looking up a translation
easier. We did not implement this for the prototype because there were not enough
saved entries to make the search box necessary.
Any wizard of oz techniques that are required to make it work:
1. In our Camera mode, we are using a static image because it would be very difficult
to actually simulate the process of taking an image on the emulator, which is a nonmobile device. Thus, the application starts with a single preloaded image, and
pressing the center button to "take" the picture merely transitions the application to
Manage mode; no picture is actually taken.
2. Because we are using a static image for camera mode, we are using that same
image for all other modes, including cropping. Also, after we "crop" the image, we
decided that instead of spending a lot of time trying to figure out how to transfer that
cropped picture back to the main activity, we would simply switch to a pre-cropped
version of the picture. This way, we don't spend too much time on implementation
that probably won't work when we actually implement it anyway.
3. The From: and To: languages shown in Manage mode were hard-coded in: as
mentioned above, the From/To translation language selectors were not yet
implemented.
4. No actual translation went on when the user presses the center (translate) button
in Manage mode: we just hard-coded into the code the translations that the actual
translator might have gotten. The final application would likely have to utilize a
picture to text converter, and then a translator to translate the text.
5. We provided a dialogue box that informs the user that an image has been saved to
Favorites when they select the option to save a particular translation, but we didn't
implement the actual picture-saving mechanism yet.