Group 1: WAITer (Waiter Assistant Information Technology)
http://ratbert.bmrc.berkeley.edu/courseware/cs160/fall01/Projects/Group1/
Andrei Boutyline (role: PDA Programming)
Neetin Gulati (role: Testing)
Ha Nguyen (role: Documentation)
Randy Shoopman (role: Web Programming)
Low-fi Prototyping and Usability Testing
Introduction and Mission Statement
Brief Project Summary:
WAITer (Waiter Assistant Information Technology) is a graphical application for PDAs
that will increase the efficiency of the processes in the life cycle of a restaurant order,
which begins with a server taking an order and ends when a customer pays for the bill.
Our application will achieve this goal by automating specific tasks and providing realtime status of orders, a wireless messaging utility for efficient employee communication,
and billing capabilities.
Purpose and Rationale of Usability Testing:
In the low-fi prototyping and usability testing phase of our project, our purpose is to
evaluate the quality of our preliminary graphical interface, which we have modeled using
paper cut-outs. To achieve our objective, we will select three restaurant servers to
represent our target audience and have them each work through three tasks. These tasks
have been carefully selected to be representative of our system functionality. By taking
notes and encouraging our subjects to verbalize their thoughts, our team will be able to
identify some of the weaknesses and strengths of our interface. At the conclusion of this
project phase, we will be prepared to make interface improvements and eventually begin
developing a hi-fidelity prototype.
Project Mission Statement:
Figure 1
Although the purpose of this project does not include
developing a fully-functional WAITer application, our
team is dedicated to learning the skills involved in
designing a successful graphical user interface for this
application. In doing so, we acknowledge the value of
user-feedback and are committed to integrating userfeedback in our design decisions.
Prototype Description
Prototype Construction:
Our paper prototype was designed using
primarily note card stock in various colors (specifically,
purple, blue, and red). We used the colors to
distinguish between the three interview tasks, which
proved to be very helpful in making the ‘computer’ run
smoothly. We cut our screens so that they were the
same size as standard PDA screens. The main screen is
the exact size of the screen of the HP Jornada. It
consists of the global toolbar and a clickable diagram of
all the tables in the restaurant (see figure 1). Since the
global toolbar is standard, the other screens are overlaid
over the table area of the main screen, reducing
redundancy and improving the overall visual experience
since less screen area is changed when navigating the
prototype.
Tables are an important top level object in
WAITer. Many actions are preformed in the context of
a specific table. Therefore the user needs to be
constantly aware of which table they are operating on.
In order to make this clear, each overlay screen has a transparent tab on the left which
highlights the correct table on the global toolbar screen. This is illustrated on the Table
Home screen (see figure 2) which has two vertical black marks on the transparent tab that
overlay table 1 on the global toolbar that show the user that he is dealing with
Figure 3
the first table.
We used peel-away stacks of post-it notes for
input fields so that the user could write directly on the
prototype without us having to erase the input and
damage the prototype. This also had the added
advantage of making input fields easier to distinguish
from the non-input areas. One such screen that uses
this technique is the messenger screen (see figure 3),
where post-its are used in the message entry area.
Small message boxes, drop-down menus, and
some screen updates were also done using transparency
overlays. We wrote the dialogs/values on small pieces
of card stock which we taped to transparencies that can
be placed over the card stock screens. The
transparencies helped keep these small card stock pieces formatted and lined up properly.
For example, in the messaging system, when a user clicks “Send”, the system notifies the
user with a green “Message Sent” box (see figure 4) that is implemented in this fashion.
One last technique we used for one line updates was card stock attached to
transparent fingers. Using the fingers we could fairly easily insert the
Figure 4
Figure 2
correct values into the correct location on the screen.
An example of one of our fingers is shown in figure 5.
Also, some of our screens use text highlighting and
blinking to alert or focus the users attention. This was
achieved using a laser pen.
Figure 5
Architecture Considerations:
WAITer is an application that is assumed to be
part of suite of applications for restaurant automation.
It is the server’s interface to the system. The full
system would probably include interfaces for
configuration, the kitchen/cooks, managers, bus boys/girls, host/hostess, and possibly
even the customers. In order to limit the scope of this project, we have purposely focused
our attention to only the waiter’s point of view. At the same time, though, we have
considered the most likely interaction between the various systems and
Figure 6
tried to design with that in mind.
Another point is that WAITer should be
considered a client application. It is assumed that there
is a data server in the back of the restaurant somewhere
or hosted off-site that stores the real time data. For
example, the menu is stored centrally on this server and
is pulled over the wireless network into the client
PDAs. This assumption is reasonable, because storing
redundant copies of the menu would lead to consistency
and update problems, to say the least. So, in general,
WAITer only provides the client interface to the data on
the restaurant server.
Prototype Function:
The global toolbar (see figure 1) provides top level navigation and access to the
most important areas of the application. The vertical part, from top to bottom, contains
the current system time, a link back to the main screen (home icon), a link to the waiters
personal information (person icon), a scrollable, clickable listing of the tables, and a back
link (left arrow icon). The horizontal part, from left to right (excluding
Figure 7
the left arrow), contains a forward link, a link to the messaging system
(mail icon), a link to the help system (question mark
icon), and an exit link. In figure 1, you can also see the
numbered circles and rectangles, which represent links
to data pertaining to specific restaurant tables. The
screen displayed in figure 2 is a homepage for a specific
table which is displayed when a table link is clicked. In
this screen, the server can view an order summary
(Order button), place an order (Menu button), and
process a bill (Bill button). The screen displayed in
figure 6 shows the typical layout for a billing screen,
which is shown when the “Bill” button is clicked.
Finally, figure 7 displays the main menu screen, from
which an order can be taken. This screen is displayed
upon the clicking of the “Menu” button.
Method
Participants:
Our decision to perform the usability testing on people having experience as
restaurant servers was trivial, since we previously identified this group as WAITer's
target audience. However, finding participants that met this criterion and had the
willingness and availability to be our test subjects was rather challenging. Eventually,
we were able to find three college students having varying degrees of experience with the
restaurant industry. We were referred to our first participant by one of our group
member's friend. This participant, who we will refer to as Subject1, is a senior at Cal
with previous experience working at a Denny's restaurant in her hometown. Subject2,
our second participant, is a neighbor of one of our group members. Out of desperation,
Ha randomly knocked on her neighbor's door and asked if anyone living there had
experience as a server and would be willing to participate in our usability testing.
Luckily, Subject2 had experience with an ordering system at a pizza parlor and was kind
enough to help out our team. Finally, we were able to find our last participant by asking
a waitress who was one of the subjects of our contextual inquiry. We will refer to her as
Subject3.
Environment:
The environments in which we performed our usability testing were simply
chosen out of convenience. In all three of our tests, we chose locations that were both
convenient for our test subjects and had a large table were our paper prototype could be
laid out. We performed our tests with Subject1 and Subject2 in the kitchen of one of our
group members. This environment was in close proximity to the homes of both
participants and had a large kitchen table. The test for Subject3 was performed in a
lounge in Wheeler Hall on the UC Berkeley campus. This location had a large table and
was near a classroom where Subject3 attends one of her lectures. The equipment setup
for all three tests was the same. Basically, the index card representing the main screen of
our application was placed on the table, directly in front of where the subject was seated.
The rest of the paper prototype (i.e. other index cards, transparent panels, paper dropdown lists, etc.) was laid out on the side of the table, in close proximity to the group
member playing the role as computer. The pieces of the prototype were organized into
three groups, where each group represented the materials needed for a particular task. In
addition to our paper prototype, we used a mechanical pencil that the test participants
used as a device pointer and a writing tool.
Tasks:
Each test subject was asked to perform three tasks that are representative of our
system's functionality. They were asked to perform an easy task first, and then a
moderate one, and finally a difficult task. Each of the tasks was written on an index card
that was handed to the test subjects. Please refer to the appendix for a picture of the
actual cards used. The contents of these cards are listed below.
Task1: Easy
A customer has a complaint and wishes to speak with a manager. Contact Bob
and ask him to go to table 5.
Task2: Moderate
Bob, a customer at table 1 complains that his steak was overcooked. He doesn't
want another steak so you need to subtract it from his bill. In addition, in order to
comply with the restaurant policies, the customer is entitled to a free dessert. Bob
wishes to have the apple pie so place the order, but do not add the charge to his
bill.
Task3: Difficult
Four customers have been seated at table1. They wish to order the following:
person 1: a coke
person 2: an Evian water
person 3: a coke
person 4: a turkey burger with no mayonnaise
Testing Procedure:
In performing our usability testing, our group followed a very formal and
preplanned procedure. The details of this procedure are described in a testing script,
which can be viewed in the appendix. In general, a greeter welcomed the test subject,
briefly described the project that our group was working on, and had the test subject sign
an acknowledgement of confidentiality. The facilitator then explained to the subject the
idea and purpose of a paper prototype and gave the test subject a brief explanation of the
main features in our prototype. This included explaining the function of each icon on the
main toolbar. Next, the team member acting as the computer gave a brief demonstration
of how functionality will be simulated with the paper prototype. For example, the
computer person would point to a button with a pencil and then place an index card
representing a different screen over the previous screen to simulate a screen change.
Finally, the facilitator presented the test subject with an index card describing one of the
three tasks. The facilitator then encouraged the test subject to think out loud while the
computer person manipulated the paper prototype to simulate actions and the observer
took notes on the test subjects comments and reactions. This process was repeated for
each of the three tasks. In performing the usability testing with Subject1 and Subject2,
the members of our group performed the following roles: Andrei was the greeter, Neetin
was the computer, Ha was the facilitator, and Randy was the observer. Due to the
limited availability of Subject3 and scheduling conflicts among our group members, only
Randy and Andrei administered the third usability test. Randy played the roles of the
greeter and computer while Andrei acted as the facilitator and observer.
Test Measures:
Prior to conducting the usability testing, our group discussed the test measures of
the experiment. We decided that we should concentrate on how the test subject
interacted with and responded to our system in general. More specifically, we planned on
observing how the test subject responds to the general flow of the screens, how our
features are presented in the interface, how quickly tasks can be accomplished, and the
intuitiveness of our GUI design. In addition to these high-level observations, our test
measures include more specific details such clarity in our wording, the intuitiveness of
our graphical icons, and the organization of our menus. Finally, our group decided it
would be important to take notes on the errors made by test subject in performing the
tasks and any incidents where the subject took more time than we expected to perform a
specific task. Although we agreed that all members of the group will look for these test
measures during the test, it would be the main responsibility of the observer to take
detailed notes in accordance with these test measures.
Results
The results of our usability testing gave our team valuable insight in regards to
the strengths and weaknesses of our preliminary GUI design. The notes that were taken
by the observers in the three tests can be found in the appendix. In summary, the
usability testing resulted in 6 major observations. The first problem in our GUI was the
lack of clarity between the functions of the “order” button and the “menu” button that
appeared in windows relating to taking an order. It was not obvious to the test subjects
that the “order” button would take them to a screen that displays an order summary for a
table while the “menu” button would take them to a screen that displays the restaurant’s
menu. This problem was observed by all three subjects. Since these two buttons are
central to our application, our team gave this problem a severity rating of 3 (i.e. major
usability problem). The second observation made relates directly to the first. Due to the
confusion of over the two buttons, all three test subjects had severe timing delays in their
performance when deciding which of the two buttons should be pressed. The third
observation resulting from these tests was Subject2’s dissatisfaction with the flow of our
beverage-ordering process. He felt that having to order each customer’s beverage
separately took too long. Subject3, on the other hand, felt that our ordering processes
were sufficiently fast. Our team gave this problem a severity rating of 2 (i.e. minor
usability problem). The fourth observation that we made was Subject2’s dissatisfaction
with placing a free dessert order, as specified in task 2. He felt that when a price
adjustment is made in response to a customer complaint, our application should
automatically prompt the waiter to enter a complementary dessert for the customer.
Since this process of giving free desserts would not apply to all restaurants using
WAITer, our team gave this complaint a severity rating of 0 (i.e. not a problem). The
fifth observation relates to a complaint made by Subject3. She voiced a concern about
how the menu was not flexible enough to handle customizations. Since this is a feature
that will be in the final product and is unrelated to the three tasks of the usability testing,
our team gave this observation a severity rating of 0. Finally, Subject1 and Subject2 had
some confusion with the order summary screen that appeared when the “order” button
was clicked. In this screen, there is a text area that is not editable that is used by the
application to display a summary of the current order. Both Subject1 and Subject2 tried
to write in this text area. Our team gave this observation a severity rating of 0 since we
feel that in the software GUI, it will be quite obvious that this area is not for user input.
We realize that it is hard to show this distinction on a paper prototype.
Discussion
Having completed the usability testing, our team feels that this was a great
learning experience and will have great impact in the development of our graphical
interface design. In general, the members of our team learned how to build a low-fidelity
prototype that focuses on the main features of an application and ignores the low-level
details such as font size, screen colors, specific icon images, etc. In addition, our team
learned the process of identifying effective test subjects and the process involved in
conducting a productive test. In addition to these general lessons, our team learned a lot
in regards to the strengths and weaknesses of our preliminary design. For example, we
learned that our subjects generally found our interface to be intuitive and easy to learn.
They also thought that WAITer would be a useful tool for restaurant servers and would
be welcomed in the restaurant industry. In addition to these strengths, our team also
realized that our design contained weaknesses and required some modifications. The
most important change that we will make is clarifying the functionality of the “Order”
and the “Menu” buttons. We will probably change “Order” to “View Order Summary”
or something similar using less words. In addition, we are considering removing the
association of ordered items to specific customers seated at the same table. This feature
would make splitting bills quite easy. However, the trade-off is that it slows down the
ordering process. Finally, our team is considering adding an alternate ordering process
for expert users. Specifically, servers will be able to by-pass the existing menu screens
and simply place their orders by using short-hand notations via direct pen-input. This
feature could drastically reduce the time it takes to place an order.
In recognizing the great benefits of usability testing, our team also realizes its
limitations. For example, this experiment could not reveal whether restaurant owners
would be willing to invest in the WAITer system. In addition, usability testing could not
reveal the interaction that would take place among servers and kitchen staff. The success
of this interaction would be central to the effectiveness of our application. Finally, we
realize that these experiments were unable to prove that our system actually increases the
efficiency of servers.
Appendix
http://ratbert.bmrc.berkeley.edu/courseware/cs160/fall01/Projects/Group1/assign/appendix.html