Usability Study on e-Commerce Web Sites
Submitted by Austen R. Moore, Instructional Technology
For Dr. Liu’s Design Strategies for New Media
April 17, 2005
Table of Contents
Introduction …………………………………………………………………….3
Site Descriptions ……………………………………………………………......3
Methodology ……………………………………………………………………7
Participants………………………………………………………………7
User Tasks ………………………………………………………………8
Data Collection.………………………………………………………….8
Testing Facilities………………………………………………………...10
Data and Results……………………………………………………………….10
Ebay.com………………………………………………………………..10
Overstock.com…………………………………………………………..13
Yahoo Shopping (shopping.yahoo.com)………………………………..16
Discussion………………………………………………………………………19
Information Design Issues………………………………………………19
Interface Design Issues………………………………………………….21
Interaction Design Issues………………………………………………..21
Summary……………………………………………………………………….22
References Used………………………………………………………………..23
Related Links…………………………………………………………………..23
Appendices……………………………………………………………………..24
Author Information……………………………………………………………27
Usability Study on e-Commerce Web Sites
2
by Austen R. Moore
Introduction
In the past 10 years it has become more and more commonplace for consumers to
make their everyday purchases online. During this revolution of access and information
companies seemingly sprung up overnight to fill the purported demand that Internet
technology had created. Web sites that delivered everything from home loans to groceries
quickly took advantage of this new information age. Then suddenly, just like it began the
industry went bust. There were simply too many sites and too few consumers to make
them all profitable. It would seem that only the strongest survived the years that followed,
and with them new methodologies would be formed about what works in web design.
The purpose of this report is to test three popular commerce sites that are
exclusively represented on the World Wide Web. I hope to determine the particular
commonalities in their usability and design to help future designers build upon what
works best. The data for this report was gathered by observing three separate users who
attempted to make purchases on each site. Users were asked to think aloud during the
testing, and were given a brief survey afterward to document their thoughts about using
each site.
Site Descriptions
The sites that were selected for this study were found by using the popular online
tool Google.com, using the terms “e-commerce” and “online shopping” to gather results.
The results were screened to using the following factors:
-
Is the site an entirely web based project, or is it simply an online version
of a store, such as Target.com?
It was also preferable to use a site that has a good reputation, and is
known as a resource in this area.
Users had to be able to make purchases online and use secure check-outs
for their credit card information.
The sites had to have a built in “Search” feature to enable multiple
means of browsing.
The sites chosen for this project were:
Usability Study on e-Commerce Web Sites
3
by Austen R. Moore
Figure 1. Screen shot of ebay’s home page
eBay is an online community of buyers and sellers from small businesses and
individuals. While eBay does not house or store any of the items listed here, it works as a
sort of middle man to connect people and the interesting items they are looking for. Just
about anything someone might want can be found in either one of ebay’s auctions or
from one of their many sellers’ online stores. eBay calls itself “The World’s
Marketplace”, and with over a hundred million registered members and local sites in over
twenty different countries, it seems that it truly is. EBay lists items from trading cards
and antiques to automobiles and even land.
EBay’s site looks pretty well structured, however with all of the options primarily
in text hyperlinks, one might wonder about how the users will know where to search first.
It seems that the search functionality is one of the most prominent features on the home
page, as the text area is quite large and attention is immediately drawn to it.
Usability Study on e-Commerce Web Sites
4
by Austen R. Moore
Figure 2. Screen shot of Overstock’s home page
Overstock.com is an Internet leader for name brands at clearance prices. They
purport to offer top name brand items at 40-80% the retail price every week. Their goods
come through wholesalers and retailers and Overstock even offers many of the same
warranties that retailers do. Overstock buys many of the overproduced items that come
off the shelves every year at liquidation prices, and then is able to sell them online at a
fraction of the original cost.
The Overstock home page is neat and organized with plenty of eye-catching
pictures. Tabs are used much like on popular sites like Amazon, and users can quickly
find their way around. There are many appealing aspects to this site as well, such as low
shipping costs and areas with extremely marked down products.
Usability Study on e-Commerce Web Sites
5
by Austen R. Moore
Figure 3. Yahoo Shopping’s home page
Yahoo Shopping may not be what it’s name implies. Indeed, a user does not
actually do any shopping from Yahoo Shopping. In actuality this site is like a search
engine for thousands of other retail sites and displays the information directly from them.
While there is no indication of this on the Home Page, the sites description from Google
states, “Yahoo Shopping is the best place to compare products, compare prices, read
reviews and merchant ratings.” Yahoo is typically considered a trusted source for all
kinds of information.
Yahoo’s site looks pretty plain from the outset. While there is use of the familiar
tabs system, the most eye-catching side of the page comes from this list of advertisers.
There are a few simple graphics to indicate what to search by, and the left side bar has
indications of visual hierarchy.
Usability Study on e-Commerce Web Sites
6
by Austen R. Moore
Methodology
Participants
Three participants took part in this usability study. (Table 1) All three participants
were between the ages of 24-30 and worked full time. All three also stated that their first
experiences using the Internet came early in college. There were two females and one
male participating in this study. Participants all work full time and have access to the
Internet from both their jobs and their homes. Every user also had previous experience
making purchases online, although none of the users had purchased from Overstock or
Yahoo shopping before, and only one user had shopped on ebay prior to this test.Users
indicating buying books, gifts, clothes, tools, and even diapers from the Internet in their
previous experiences. All user indicated that they primarily use the Internet for finding
information, such as phone numbers, maps, and movie times. Two users said that the
purchases they choose to make online are for things that they don’t feel they have to
inspect before buying, such as a car. The other user indicated that they only prefer to buy
small things online. Finally the users had various reasons for the sites they chose to shop
from. One user said that they relied on the reputation of the site and if it appears to have
secure transaction abilities in place. Another also indicated that a reputation through
positive word-of-mouth usually helps them to decide where to shop, while the final user
simply shops wherever they can find the best deals on price and shipping.
Table 1. User Profiles
Users
Age
Gender
Occupation
Internet access
First Use
Primary Uses
User 1
26
Female
Programmer
Home/Office
College
Email/Research
User 2
24
Female
Chiropractic Asst.
Home/Office
College
Finding Phone
Numbers/Locations
User 3
27
Male
Engineer
Home/Office
College
Reference
Materials/Shopping
Previous Online
Purchases
Clothes, Books
Books, Gifts
Diapers, Tools, Books,
Software
Useful Sites
Austin360, Google,
Amazon
Yahoo Yellow Pages,
Google
Google, Yahoo Maps,
Bible Gateway.net
Usability Study on e-Commerce Web Sites
7
by Austen R. Moore
User Tasks
Users were introduced to the selected sites one at a time. For each user the order
of the sites presented was different to minimize thoughts and actions becoming biased
due to testing fatigue. The users were given three tasks for each site, which took
approximately 30 minutes to complete. The tasks are listed below.
Task 1(5 min): Site Impressions - Users were given approximately 3 minutes to freely
explore the site they were visiting. After 3 minutes were up users were asked to stop and
answering the following questions:
1. What did you first look for on the site?
2. Were there any elements to the site you felt like you couldn’t find?
3. What would you most likely use this site for?
This first task helps to give the researcher an idea for how a user would react when first
coming to the new site, especially with no direct objective. This determines if and how
the site directs the user’s attention, and if this is distracting or effective.
Task 2(10 min): Specific Task – Users were then given similar tasks for each site so that
I could observe their actions when performing the tasks. The tasks were as follows:
Ebay: “Your grandfather wants to relive his marching band glory days but it looks
like he lost his old trombone when he moved. Can you find him a slide trombone
for under around $100 including shipping?”
Overstock: “A co-worker bragged to you that they bought a Sony digital camera
for under $100. It sounds too good to be true, but if they are telling the truth you
might just get one for yourself.”
Yahoo Shopping: ”You’ve been asked to help your younger brother buy a pair of
basketball shoes on the web. He only has $75 to spend and he wears a size 9. He
really wants red shoes to match his jersey, although this is not essential.”
This task really tests the sites ability to provide desired information to the user. Whether
the user is browsing by topic or quickly using a search function to narrow things down,
the site must provide an easy means for the user to find what they are after.
Task 3(10-15 min): Open Ended Task – Users are supplied with a task that truly gives
them opportunities to explore the sites and virtually shop for themselves. The tasks were
as follows:
Ebay: “It is time to redecorate your home office! You have exactly $500 to set
yourself up an invigorating new work space.”
Usability Study on e-Commerce Web Sites
8
by Austen R. Moore
Overstock: “You just set up a home theatre, now all you need are your favorite
movies to test it out. Get your top 5 list ready and order away!”
Yahoo Shopping: “You never imagined it could happen, but you actually got in
shape for the summer. Treat yourself to some new outfits.”
This final task is far more open than the previous one. The user has more opportunity to
look for things they might actually want, and in many cases they may not actually know
what that is yet. The site must be able to keep the user’s interest while they are browsing
different areas and even offer suggestions based on the search criteria.
Once all three sites and task were completed, users were given a final survey to rate their
satisfaction shopping with the online commerce sites. This survey can be found at the end
of this report.
Data Collection
Assessment Tools: The users were provided with the following materials:
-Consent Form (Appendix A)
-Pretest Questionnaire (Appendix B)
-Post-Test Survey (Appendix C) x3
Procedures: The researcher had previously prepared the usability testing packets
for each user. To start the researcher explained the basic premise for the test and
asked the participant whether they had any questions or needed clarification. The
users were then asked to read and sign the consent form, and then to complete the
pretest questionnaire. Once these were completed the user began the study by
going to each website and completing the stated tasks for each one.
Participant Observation: During the testing the researcher sat just to the back
right of each user. Users were encouraged to think aloud, or narrate their actions
as the researcher took notes. Often users asked questions to themselves while
working but never to the researcher. The researcher took notes regarding the
navigation and thoughts expressed during each exercise. If needed, the researcher
would ask the user to clarify an action or something that was said to gain better
understanding.
Date Analysis: The data that was gathered from the pre and posttests was
analyzed and synthesized into charts using Microsoft Excel. The researcher also
organized the notes taken during the tests as well as any open ended responses
given by the users and either added these to the charts or used these observations
to determine dominant themes or perceptions the users had regarding the ecommerce sites.
Usability Study on e-Commerce Web Sites
9
by Austen R. Moore
Testing Facilities
All users participating in the study did so at the researcher’s home. The study was
conducted in a small office on a Dell desktop PC. The PC ran on Windows XP and users
visited the sites using Microsoft Explorer 6.0. The web sites were accessed using a highspeed Internet connection. Users sat in a desk chair with the researcher sitting to their
back right.
Data and Results
The usability testing for each site was broken in three tasks, which all three users
completed in approximately one and one half hours, with each user spending about thirty
minutes on each site. The users were asked to give their first impressions of each site
after spending about three to five minutes looking around. (Task 1). Upon completion of
the first task the participants were given a specific shopping objective (Task 2) which
should have taken no more than a few minutes to complete. Each user narrated their
actions and decisions as best they could while the research documented them. Then the
users were given a more open-ended objective (Task 3) that was also narrated and
documented by the researcher. In synthesizing the data the researcher compiled the users’
comments and post-test answers (Appendix 3) for the following site impressions.
Scores ranging from 1 to 2 were Very Unsatisfied
Scores ranging from 2.3 to 3 were Unsatisfied
Scores ranging from 3.3 to 3.7 Barely Satisfied
Scores ranging from 4 to 4.7 were Satisfied
Scores ranging from 5 to 6 were Very Satisfied
-First Impressions
When first coming to this site all of the users felt somewhat lost without being
given an objective. One user scrolled down the site quickly to see if anything caught his
eye and indicated that due to the lack of pictures nothing did. The other users looked first
to see what was available to purchase and one of the almost immediately went into the
section for featured items. All users felt comfortable with the site but some indicated
without graphics to give them ideas for what to search for they didn’t know where to
begin. Due to the detail and specialty that the site seemed to indicate, two of the users
said they felt like they would most likely come here if they were looking for something
that was hard to find or rare. The third user indicated that they may only come here if
they wanted to get an idea about prices.
-User Scores
Usability Study on e-Commerce Web Sites
10
by Austen R. Moore
Table 2. Average User Score for Ebay
Average
1.Finding desired areas of site
2. Quality of information for products
3. Ease of reading the text
4. Site's search capability
5. Appearance of site
6. Speed of pages loading
7. Order forms
8. Delivery and ordering information
9. Entertainment value
10. Overall shopping experience
11. Product Satisfaction
12. Questions Answered
13. Products Relevancy
12. Likeliness to return
4.7
4.7
5.3
5.7
5.3
4.7
4
4.7
4
4.7
4.7
4.7
4.7
5.3
Indications
Satisfied
Satisfied
Very Satisfied
Very Satisfied
Very Satisfied
Satisfied
Satisfied
Satisfied
Satisfied
Satisfied
Satisfied
Satisfied
Satisfied
Very Likely
-Score Breakdown
The users in this test gave the site fairly positive ratings in each of the questioned
areas. Because ebay is primarily know as an auction site, the users found the most
dissatisfaction in the areas controlled by the individual sellers. This may be because of
lack of sufficient detail for certain items, no pictures, or confusing ordering information.
This is most likely the reason for slightly lower Order form scores. The users
overwhelmingly enjoyed the site’s search capability as they had no problems finding
what they were looking for, and refining their searches from there. Also
-Additional Details
Users each found the products they desired efficiently and without asking too
many questions. All used the Search function to find most of the items, and all of the
users also refined their searchers when they received too many results. Based on
observation it became obvious that the users never looked at any auctions that did not
provide a picture of the items listed(Figure 5). Some preferred the “Buy it Now” option to
actually pursuing an auction, as they would rather know now that they had the item in
question (Figure 6).
The only comments made that indicated any dissatisfaction were in regards to the
amount of text presented on many of the pages. The users all felt this site was a good site
if you came to it with a specific item in mind.
Usability Study on e-Commerce Web Sites
11
by Austen R. Moore
Figure 4. ebay’s product page: Users often said there was too much to read
Usability Study on e-Commerce Web Sites
12
by Austen R. Moore
Figure 5. A look at eBay’s musical instruments section: visual hierarchy and relevant
advertisements for the page
Figure 6. A display of a search for “Trombones”: Users only looked listings with
pictures, and some preferred the “Buy it Now”option
-First Impressions
When first entering this site the users acted just as they did on the ebay site. One
user went straight to the clearance bin, one looked to the navigation bar to assess their
options, and one scrolled down to see if any pictures caught their attention. Only one user
indicated that they had any difficulty searching, while the others had no problems. The
user who experienced difficulty was solely relying on the navigation bar to find an item,
without typing the item into the search area. The users all indicated that they could see
themselves using this site just to find out prices for things, with one specifically stating
that they would probably come here to find good deals, and another that they might come
here for books or home furnishings.
-User Scores
Usability Study on e-Commerce Web Sites
13
by Austen R. Moore
Table 3. Average User Scores for Overstock
Average
1.Finding desired areas of site
2. Quality of information for products
3. Ease of reading the text
4. Site's search capability
5. Appearance of site
6. Speed of pages loading
7. Order forms
8. Delivery and ordering information
9. Entertainment value
10. Overall shopping experience
11. Product Satisfaction
12. Questions Answered
13. Products Relevancy
12. Likeliness to return
4.3
5
5.3
4.3
4.7
5.3
4.7
4.7
4
4.3
4.7
4.3
4
3.7
Indications
Satisfied
Very Satisfied
Very Satisfied
Satisfied
Satisfied
Very Satisfied
Satisfied
Satisfied
Satisfied
Satisfied
Satisfied
Satisfied
Satisfied
Maybe
-Score Breakdown
The scores for Overstock.com indicate that users were relatively satisfied with
their experience on this site. The product information was exceptional as the site owners
enter it rather than individual sellers like on eBay. Pages loaded swiftly and were not
bogged down and the text wasn’t over used. Overall users seemed to like the way the site
is set up however the most indicated that they were not as likely to return to this site in
the future. This might simply be because the users did not find the deals on Overstock to
be as good as other sites.
-Additional Details
Users all seemed to be satisfied with their experience on this site and some even
used the site’s “top sellers” option to help them determine their own preferences. Two
users noted that when leaving the shopping cart by using the “Continue Shopping”
button, they were not returned to the previous page they were on, but instead were sent
back to the top of the section they had been on. This misstep caused them to have to
retrace several steps to return to where they had been. The third user simply used the
“Back” button on the browser. One user also apparently found a mistake on the site
where when he clicked “View more results” the site sent him back to the home page.
Another area that a user commented on was the inconsistency on the home page when
clicking on links. There were six pictures indicating sections and four of the six were
actually links into those sections, however when the user tried to click on the camera
picture nothing happened. (Figure 6)
Usability Study on e-Commerce Web Sites
14
by Austen R. Moore
Figure 7. Overstock’s home page: Only 4 of the 6 pictures that represent sections were
actually links. This represents inconsistency on this site.
Figure 8. Search Display: Clear pictures and item descriptions
Usability Study on e-Commerce Web Sites
15
by Austen R. Moore
-First Impressions
When first entering this site the users again followed their same methods. One
user read the top to determine what was on the site, another scrolled down to see the
pictures, and another was immediately drawn in by a picture to check out the shoes. None
of the users indicated any difficulty in working with the search utilities during this first
impression, however some would later. One user felt like this would be a good site for
comparing prices, while another was unsure because they felt that this site had fewer
items to look at than other sites. The third user felt like this might be a good site to find
out what new items were available in stores.
-User Scores
Table 4. Average User Scores for Yahoo Shopping
Average
1.Finding desired areas of site
2. Quality of information for products
3. Ease of reading the text
4. Site's search capability
5. Appearance of site
6. Speed of pages loading
7. Order forms
8. Delivery and ordering information
9. Entertainment value
10. Overall shopping experience
11. Product Satisfaction
12. Questions Answered
13. Products Relevancy
12. Likeliness to return
3.7
4
5
4
3.7
5
4
4
2.7
3
3
3
3
2
Indications
Barely Satisfied
Satisfied
Very Satisfied
Satisfied
Barely Satisfied
Very Satisfied
Satisfied
Satisfied
Unsatisfied
Unsatisfied
Unsatisfied
Unsatisfied
Unsatisfied
Unlikely to Return
-Score Breakdown
The users had less kind things to say about Yahoo Shopping. Besides the naturally
high scores for pages loading and readable text, the users were generally unsatisfied.
Most had a difficult time finding exactly what they wanted every time and found the site
to look less than professional. While the quality of information for the products was
found satisfactory, the sheer numbers of products that are given and the quality of some
of those products raised doubts from a couple of users. One user found that they liked the
Star icons representing product and site quality were quite useful, but was annoyed when
they chose to switch views from list to grid and the stars disappeared (Figures 9 & 10).
Another failed to even realize that there were other pages to look through, as a large box
Usability Study on e-Commerce Web Sites
16
by Austen R. Moore
of advertisements separated the list from the page number selections (Figure 11). The
final users did not even realize that the site was sending him to other websites until the
third task when he tried to check out and he could not find the shopping cart. Overall the
users found this site frustrating to use and all were unlikely to return.
Figure 9. Product display page: A user was frustrated when the star ratings did not
appear in this view.
Usability Study on e-Commerce Web Sites
17
by Austen R. Moore
Figure 10. The same results from Figure 9, but in list view rather than grid.
Figure 11. Search Result on Yahoo Shopping: A user did not realize there were more
pages to see because of the large block of advertisements on the page.
Usability Study on e-Commerce Web Sites
18
by Austen R. Moore
Discussion
Overall the users did not have too much trouble accomplishing the tasks that they
were given. While eBay did not always have the highest scores, it was definitely the
users’ favorite in terms of its selection and search capabilities. Overstock was liked and
did well in most areas, but based on the users comments it is presumed that felt it had less
to offer them, and the deals did not seem that special. Yahoo Shopping was the biggest
disappointment in terms of a shopping experience and outside of simple price
comparisons, it is doubtful any of the users will visit it again. In the following section
these sites will be looked at from the perspectives of information, interface, and
interactions designs.
Information Design
These sites all did their best to provide the desired information for the users. As
one user commented, “I only buy things online that I don’t need to inspect”. It is this idea
that has designers providing as much detailed information about the products as they can.
Unfortunately on eBay the information provided is often up to the seller who
posted the item. On at least two occasions the user left auctions that did not provide all of
the information they were looking for. EBay was also the site where the users felt the
most overwhelmed by the text. Due to the massive number of products listed on eBay, all
of the users commented that when coming to eBay it was best to have something in mind
before searching. (Figure 12) To help compensate for this there is a small sense of visual
hierarchy applied to the pages to help the user’s eyes to see the distinctions.
Figure 12. Ebay’s use of text can be daunting.
Usability Study on e-Commerce Web Sites
19
by Austen R. Moore
Overstock had what was likely the best information design. By using tabs to
organize section and distinctive fonts to draw attention, all of the users easily found
where to go. The layout on the display pages was also a good example of information
design as items were place in a grid pattern to be easily compared and relevant
information about cost and savings in presented. (Figure 13). Information was well
displayed and consistent throughout the site.
Figure 13. Overstock’s layout for search results.
On Yahoo Shopping a lot of information was also provided. As Yahoo
Shopping’s primary purpose is to enable the user to compare prices and ratings, it would
seem that these areas are lacking. The list view on the results page provides a lot of
information about each product, but there is no room left to show many items for
comparisons. The grid view would be helpful for these comparisons, but some of the
information the users were using goes away, such as the star ratings for the products.
(Figures 9 & 10) Perhaps the largest oversight is that the site did not give the users a clear
idea on what the site is about. None of the users knew that they could not actually buy
from this site until they actually started selecting items, and there is no indication that this
is how Yahoo Shopping works.
Overall users were looking for prices, pictures, and for a design that helped them
to determine where they were and where to find things. Overstock.com provided the best
example of this by using various font sizes and contrasting colors consistently throughout
its site, as well as sharp pictures and brief but useful blurbs of information.
Usability Study on e-Commerce Web Sites
20
by Austen R. Moore
Interface Design
Users were not met by anything new or innovative by the sites they were
presented, however the commonality of these sites helped the users to feel confident
when initially moving throughout each site. All three sites used tabs to indicate section
titles as well as presenting the Search bars just below them at the top of the page. Menus
were almost always on the left side of the screen, while advertisements, if present, were
to the right of the screen. (Figures 1, 2 & 3). Menus are more or less consistent and on all
sites the level of specificity for links increased as the user navigated through. Ebay’s
menu occasionally expanded to fill the screen if the number of options were to great to
fill the sidebar but seemed to be understood readily by the users. (Figure 5) When given a
choice nearly all of the users clicked a picture before clicking hypertext to navigate the
sites. Overstock and Yahoo Shopping were best in providing pictures with there
selections, while eBay relied mostly on text(Figure 5). All of these sites kept consistent
color schemes and it is apparent why these e-Commerce sites have excelled. Users had
few problems finding what they were looking for using word searches and through basic
browsing.
Interaction Design
Each of the sites offered plenty of tools for users to find the products they were
looking for. One useful tool that users found was an option to calculate shipping costs on
eBay. Two users used this successfully when shopping for items, while the third did not
attempt it. Also on eBay, one user did complain that some results, while relevant, were
still not what he was looking for. For example when searching for “desks”, he also got
results for “desk lamps”. Ebay seemed to take the most clicks to get somewhere when
browsing, although this is most likely due to the number of categories that the millions of
auctions fall into. The users did not complain about this, but they did feel like eBay was
more objective based than a site to casual browse. The only problem indicated on any of
the sites as far as interaction seemed to be a glitch on Overstock where the user was
returned to the home page when they clicked on the “View More” option. Most users also
used the sorting functions to list items by price or by manufacturer. There were no
problems with the usage of the sort functions.
Usability Study on e-Commerce Web Sites
21
by Austen R. Moore
Summary
E-Commerce is a huge industry and making every day purchases online is
growing more and more common. Through this study it is evident that users have
developed certain preferences and expectations for e-Commerce sites and the way they
present information. As discovered in this study, even one bad experience on a website
can lose the site business that will be difficult to impossible to win back. Based on the
research that has been conducted in this study it is possible to make the following
conclusions about online consumers and e-commerce web sites.
-
Offer several means to sort information, such as price, brand, or size.
Search bars are a must; invest in good search functionality.
Distinguish advertisements from the rest of the site.
Pictures will always draw a users attention before plain text will.
Distinctive fonts help to highlight sections.
Saving reminders for users to return to previous pages is desired.
When comparing prices a grid formation works best.
Offer clear information on what the site is about.
Consistency in the site design is preferred over innovation.
Usability Study on e-Commerce Web Sites
22
by Austen R. Moore
References Used
Krug, S. (2000). Don’t Make Me Think! A Common Sense Approach to Web Usability.
Nielson, J., Snyder, C., Molich, R., and Farrell, S. (2000) Methodology. Retrieved from
Nielson Norman Group Web site. http://www.nngroup.com/reports/ecommerce/
Preece, J, Rogers, Y, & Sharp, H. (2002) Interaction Design: Beyond Human-Computer
Interation. John Wiley & Sons.
Related Links
http://www.useit.com
http://id-books.com
http://www.google.com
http://www.ebay.com
http://www.overstock.com
http://shopping.yahoo.com
Usability Study on e-Commerce Web Sites
23
by Austen R. Moore
Appendix A
Consent Form
Date:_______
Study Administrator is:______________
Participant is:______________
Introduction
You have been scheduled to participate in a usability study about how websites meet your
needs. Our purpose in conducting this study is to understand what makes some websites
easier to use than others. The results of this study will NOT be published in professional
reports. Please keep in mind that this is a test of websites; we are not testing you! In the
session, we'll ask you to visit some websites and look for various things. You will not be
required to actually buy anything or to enter any personal or credit card information
online. All information we collect concerning your participation in the session is
confidential. We will not videotape or audiotape the session.
Participation is voluntary. To the best of our knowledge, there are no physical or
psychological risks associated with participating in this study. During the session, the
study administrator will assist you and answer any questions. You may take short
breaks as needed and may stop your participation in the study at any time.
It takes about 30 min.- 1 hour to complete testing each website.
Statement of Informed Consent
I have read the description of the study and of my rights as a participant. I agree to
participate in the study.
Signature _______________________________
Date _______________________________
Usability Study on e-Commerce Web Sites
24
by Austen R. Moore
Appendix B
Pre-Test Questionnaire
User id: __________
Date: ____________
Background Questionnaire
This information will be kept strictly confidential.
1. Your age: 18-23
24-30
2. Gender: Male
Female
31-35
36-45
46-60
over 60
3. Occupation __________________________
4. How do you access the Internet?
Home
Office
Both
5. When did you first start using the Internet? ___________________________________
6. What do you primarily use the Internet for?
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
7. Have you ever made purchases online? If so, what kind of purchases did you make?
________________________________________________________________________
________________________________________________________________________
8. What types of purchases would you consider making online?
________________________________________________________________________
________________________________________________________________________
9. How would you decide on where to make purchases online?
________________________________________________________________________
________________________________________________________________________
10. Please list any sites that you feel are particularly useful.
________________________________________________
________________________________________________
________________________________________________
________________________________________________
Usability Study on e-Commerce Web Sites
25
by Austen R. Moore
Appendix C
Post Test Feedback Form
User id: __________
Date: ____________
Site: ___________________________
Please rate your satisfaction with the site you just completed by circling the appropriate
number for each question. After you finish evaluating the other sites, you may still come
back and change your answers on this form.
very unsatisified
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1.Finding desired areas of site
2. Quality of information for products
3. Ease of reading the text
4. Site's search capability
5. Appearance of site
6. Speed of pages loading
7. Order forms
8. Delivery and ordering information
9. Entertainment value
10. Overall shopping experience
3
3
3
3
3
3
3
3
3
3
4
4
4
4
4
4
4
4
4
4
very satisfied
5
6
5
6
5
6
5
6
5
6
5
6
5
6
5
6
5
6
5
6
11. How satisfied are you with the products you selected based on what you were asked
to do?
Not very 1
2
3
4
5
6 very
12. How satisfied are you that this site answered all the questions you had?
Not very
1
2
3
4
5
6 very
5
6 very
5
6 very
13. How relevant are the products on this site to you?
Not very
1
2
3
4
14. How likely are you to return to this site on your own?
Not very
1
2
Usability Study on e-Commerce Web Sites
3
4
26
by Austen R. Moore
Author Information
This paper is written by Ryan Moore for the course EDC385G
Design & Strategies for New Media at the University of Texas –Austin
Usability Study on e-Commerce Web Sites
27
by Austen R. Moore