Usability Test: Paint Visualizers
Research Component
Adina Efron
November 25, 2013
EDC385G Designs & Strategies for New Media
The University of Texas at Austin
Dr. Min Liu
Table of Contents
Introduction .......................................................................................................................... 4
Background ................................................................................................................................................................................ 4
Website Descriptions ............................................................................................................................................................. 5
A. Sherwin-Williams .....................................................................................................................................................................5
B. Glidden ..........................................................................................................................................................................................7
C. Benjamin Moore ........................................................................................................................................................................9
Purpose ...................................................................................................................................................................................... 10
Methodology....................................................................................................................... 10
Research Questions .............................................................................................................................................................. 10
How the Sites Were Selected ............................................................................................................................................ 11
Testing Method ....................................................................................................................................................................... 13
Participant Information ...................................................................................................................................................... 13
Instruments.............................................................................................................................................................................. 14
Tasks ........................................................................................................................................................................................... 15
Data and Results ................................................................................................................. 16
First Impressions ................................................................................................................................................................... 16
Usability Test Recordings and Think Alouds ............................................................................................................. 16
1. Sherwin-Williams .................................................................................................................................................................. 16
2. Glidden........................................................................................................................................................................................ 20
3. Benjamin Moore ..................................................................................................................................................................... 23
Post-Test Questionnaire Results ..................................................................................................................................... 27
Likert Scale Results.................................................................................................................................................................... 27
Oral Debriefing Results ............................................................................................................................................................ 29
Which Site Did the Participant Prefer/Dislike?............................................................................................................ 31
Discussion ........................................................................................................................... 33
Information .............................................................................................................................................................................. 33
Finding the Visualizer .............................................................................................................................................................. 33
Operating the Visualizer ......................................................................................................................................................... 35
Interface .................................................................................................................................................................................... 35
Navigation ..................................................................................................................................................................................... 35
Interpreting Application Interface Elements ................................................................................................................ 36
Realistic Scenes v. Simple Walls........................................................................................................................................... 40
Interaction ................................................................................................................................................................................ 40
False Visualizers ......................................................................................................................................................................... 41
Virtual Room Elements ............................................................................................................................................................ 42
Color Application ........................................................................................................................................................................ 42
Summary ............................................................................................................................. 43
Information .............................................................................................................................................................................. 44
Interface .................................................................................................................................................................................... 44
Interaction ................................................................................................................................................................................ 44
References .......................................................................................................................... 45
Background Questionnaire .................................................................................................. 46
2
Post-Test Questionnaire ...................................................................................................... 47
Introduction Script .............................................................................................................. 49
About the Author ................................................................................................................ 50
3
Introduction
Background
Paint visualizers help consumers try out paint colors in a virtual environment before
physically going to the paint store to buy samples or gallons of paint. Almost every paint
website features a paint visualizer within the site, and these applications provide a
unique opportunity for paint brands to make a connection with potential customers. By
allowing users to experiment with paints colors, companies create a valuable path
engaging customers with their products. (See Fig. 1.)
Fig. 1 Benjamin Moore’s Personal Color Viewer allows the user to apply paint swatch colors to a virtual
kitchen.
Paint visualizer applications have a playful, exploratory tone. The user can choose a
room (e.g., bedroom, bathroom) and then experiment with applying colors to the walls,
utilizing drag and drop functionality and other interactive selection methods. One may
argue that paint visualizer applications, in addition to other color selection tools offered
by many companies, serve to increase brand awareness and loyalty. Many paint
companies provide mobile versions of the paint visualizers to serve users who would
prefer to use mobile devices. The visualizer applications usually include a sharing
feature to various social media. If the user “paints” a room and shares it on Facebook,
that posting becomes free marketing for the paint company (see Fig. 2). Furthermore,
the visualizer might be the first significant interaction a consumer has with the paint
company’s products, so a positive or negative experience could shape whether a user
decides to walk into a brick-and-mortar store to buy paint.
4
Fig. 2 Sharing feature allows for social media interaction.
Because of the marketing potential of these interactive applications, it would be wise for
companies to ensure successful usability for its users. Technical usability entails the
“general usability of a tool for a user—how easily users can complete a task with the
least number of obstacles” (Stevenson and Liu 3). The purpose of this study is to
explore the usability of the paint visualizers, in terms of the ability of users to carry out
common tasks associated with this type of application.
Website Descriptions
All three companies chosen (Sherwin-Williams, Glidden, and Benjamin Moore) feature
some kind of paint visualizer or color palette program in their homeowner paint site. The
three companies are major, high-performing paint brands with broad visibility and wide
availability. The sites not only offer paint visualizer programs but a variety of helpful
color selection tools and advice as well. However, the focus of this usability test will just
be the paint visualizer application.
A. Sherwin-Williams
URL: http://www.sherwin-williams.com
Sherwin-Williams was founded in 1866. The company claims to be the largest producer
of paints and coatings in the U.S. and among the largest producers in the world. The
company holds more than 500 patents. As of third quarter 2013, The Sherwin-Williams
Company net sales were $2.85 billion. According to Yahoo! Finance, the company’s
branded products are sold through a chain of about 4,100 company-operated stores
5
and facilities; products are also sold through mass merchandisers, home centers,
independent paint dealers, and other retailers and distributors.
The Center of Excellence (excellence.sherwin.com) says the company has been
“creating terrific professional and do-it-yourself tools for color style for more than a
century.” One of the company’s color consultant’s, Harry H. Scheid, advised the 1939
Columbia University conference on bus construction, which resulted in yellow being
chosen for safety for all school buses.
Sherwin-Williams offers products to a variety of consumer and industry segments,
including homeowners, contractors, and architects, as well as products geared toward
marine, automotive, product, and aerospace industries. The company offers over 1,500
colors for homeowner use.
Fig. 3 Sherwin-Williams homepage
6
Fig. 4 Sherwin-Williams’ Paint Color Visualizer
B. Glidden
URL: http://www.glidden.com
Glidden traces its history back to 1875 when founder Francis Harrington Glidden started
a varnish-making business in Cleveland. Glidden is the U.S. flagship brand for PPG, the
world’s largest paint and coatings company, according to the media page of the site.
Glidden claims to be the most widely accessible paint brand in the U.S., with almost
10,000 outlets carrying the brand on their shelves. PPG Industries, the corporation that
owns Glidden, report net sales of $4 billion, with its Performance Coatings segment
earning a net of $1.6 billion (PPG Press Release PDF).
7
Fig. 5 Glidden homepage
Fig. 6 Glidden’s Color Palette tool
8
C. Benjamin Moore
URL: http://www.benjaminmoore.com
Benjamin Moore was started in 1883 and is committed to “producing the highest-quality
paints and finishes in the industry.” In 2000, Benjamin Moore joined the Berkshire
Hathaway family of companies, which is under the direction of Warren Buffet. Berkshire
Hathaway had net earnings of $5.05 billion in the third quarter of 2013.
According to the company’s press site, “J.D. Power and Associates 2013 Interior Paint
Satisfaction Study ranked Benjamin Moore highest in consumer satisfaction, marking
the third consecutive year and fourth time the company has earned this rating. Benjamin
Moore paints deliver authentic Benjamin Moore colors and are available exclusively
from Benjamin Moore's network of paint and decorating dealers.” Through its
association with Berkshire Hathaway and as a result of its ratings, Benjamin Moore
appears to have a high-end, quality-focused brand image. More than 3,400 colors are
featured in the company’s paint catalog.
Fig. 7 Benjamin Moore homepage
9
Fig. 8 Benjamin Moore’s Personal Color Viewer
Purpose
The purpose of the usability test is to gather data about the ease of use of the three
paint visualizers at the Sherwin Williams, Glidden, and Benjamin Moore sites.
The goals of the study are to:
 Determine which paint visualizer is the easiest to learn and use.
 Identify which elements of the applications impede user goals.
 Identify which elements of the applications help users accomplish their goals.
Methodology
Research Questions




How easily can users find, launch, and use the visualizer application?
Which features of the visualizer application were confusing or difficult to use?
Which of the three visualizers is the easiest to use?
Which visualizer do users prefer?
The following chart (see Fig. 9) explains how data was collected to answer the research
questions and is taken from Handbook of Usability Testing (Rubin and Chisnell 167).
10
Research Question
How easily can users
find, launch, and use
the visualizer
application?
Which features of the
visualizer application
were confusing or
difficult to use?
Which of the three
visualizers is the
easiest to use?
Which visualizer do
users prefer?
Data Collected
Method of Collection
 Number of clicks
 Silverback recording
required to find the
application (creates click
visualizer and launch
“splashes” so clicks are
it.
easy to track and count)
 User explanations
 Think Alouds
regarding ease of use
 Post-Test Questionnaire
of the visualizer (Think
Alouds, Post-Test
Questionnaire).
 Identification of user
 Silverback application
errors through analysis
 Post-Test Questionnaire
of screencasts.
 Think Alouds
 Post-Test
Questionnaire
responses.
 Think Aloud
responses.
 Oral debriefing.
 Post-Test
 Post-Test Questionnaire
Questionnaire
 Think Alouds
responses.
 Think Aloud
responses.
 Post-Test
 Post-Test Questionnaire
Questionnaire
responses.
Fig. 9 Research Question/Data Collection Table
How the Sites Were Selected
This usability test is a comparison test, with the goal of determining which alternative is
easiest to use and learn, as well as identifying advantages and disadvantages of
different designs (Rubin and Chisnell 37).
Rubin and Chisnell recommend that comparison tests include “wildly differing”
alternatives rather than similar options for two reasons:
 The (hypothetical) design team would be forced to reconsider their ways of
thinking and possibly redefine and improve a product in “fundamental ways.”
 Drastically differing alternatives are more difficult to compare because they share
similar frameworks and conceptual models. Therefore, the user must carefully
consider which design is better in relation to the task performed (Rubin and
Chisnell 38).
11
To start out, the following sites were analyzed in order to determine which three sites
were the most different from each other. A comparison chart was constructed to isolate
the three most “different” sites. The chart represents a general picture of what each
visualizer is like and by no means encompasses the many distinct interface and
interaction elements present in each.
Where is the
visualizer
located in
the main
site?
What is the
visualizer
link called?
Where does
the
visualizer
launch?
What is the
environment
like?
How many
interior
scenes are
there?
Sherwin
Williams
Glidden
Valspar
Mythic
Benjamin
Moore
From the main
page, navigate
to the
Homeowner
link under
“Paints, Stains,
Colors, &
Supplies.” Then
under the
“Color” tab,
click “Try on
Colors.”
Color Visualizer
Under the
“Color” tab
in the
header
Under “Explore
Colors”
On the main
page, under
“Color
Palette”
On the main
page, click
“Explore Color”
tab.
Room
Painter
Virtual Painter
Paint Your
Room Online
In a separate
window
Within the
body of the
page
Within the body
of the page
Mythic
Room
Visualizer
Within the
body of the
page
User can
choose different
realistic interior
and exterior
scenes to test
colors on.
Colors in left
panel; scene is
in the right
panel.
User views
artificial
walls in a
virtual, nondescript
room. A
carousel of
color
categories
(like “Top 10
Home Office
Colors”) is
located in
the center of
the virtual
“wall.”
1
User must pick
interior or
exterior first.
Then the user
can choose
from different
realistic
scenes. User
must click
“Change Color”
pull up color
swatches in a
separate page.
User views
a corner of a
virtual, nondescript
room. All
paint color
options are
laid out in a
grid to the
right of the
room.
User views
scene in the left
panel; user can
choose colors
from families of
color swatches.
User can reduce
colors to 5 chips
or 1 chip at a
time.
53
1
100
54
Within the body
of the page
Some sites, like Olympic and Behr, were disqualified because they require users to
upload their own photos to the site in order to use the paint visualizer. This test focuses
on sites that provide sample rooms in the interface for quick visualization paint colors.
12
The following sites were chosen for the test because of a distinct characteristic that set
it apart from others:
 Sherwin Williams – Visualizer launches in separate window
 Glidden – Features a carousel of color categories
 Benjamin Moore – Includes 100 interior scenes (more than the other sites) and
features the 5- or 1-chip color options
Testing Method
The usability test is designed as within-subjects with some counterbalancing. Each of
the three chosen participants will perform the same two scenarios, but not all
participants will test all three sites; each website will have each scenario performed on it
(see Fig. 10). The scenarios will be similar but different enough so participants don’t feel
like they are doing the exact same task. Participants will be allowed to explore the site
for 2–3 minutes before completing the scenario.
Sherwin-Williams
Participant 1 X Scenario 2
Participant 2 X Scenario 1
Participant 3
Glidden
X Scenario 2
X Scenario 1
Benjamin Moore
X Scenario 1
X Scenario 2
Fig. 10 Sites tested by each participant
To avoid transfer of learning and bias, each participant should take an extended break
between testing sites. This allows the user to rest and to possibly forget what was
learned while using the previous site (Rubin and Chisnell 75). However, in this usability
test, the participant was asked to complete two short scenarios within the same session.
After completing the first scenario on the first site assigned, the participant was given
the opportunity to take a short break if they wanted to. Since one of the research
questions is, “Which visualizer do users prefer?”, it seemed appropriate to allow the
participants to use each of the two sites assigned to them within a short period of time
so they could express their preferences. The value of collecting more authentic
preferential data outweighed the danger of bias and transfer of learning.
Participant Information
Rubin and Chisnell write that background questionnaires provide information about the
participants that will help the test designer understand “their behavior and performance
during a test.” A background questionnaire was administered through Google forms to
all three users prior to testing.
The questionnaire was intended to reveal participant experience, attitudes, and
preferences that are related to performance (Ruben and Chisnell 162). The
questionnaire also identified an overview of general user characteristics.
13
Fig. 11 shows the results of the questionnaire given to all three users. The Background
Questionnaire used in this study may be found in Appendix A.
Have you ever used a web-based
paint visualizer program?
How would you rate your computer
skill?
I am interested in home décor and
design. (1=Strongly Disagree;
5=Strongly Agree)
How many hours per day do you use
the Internet (excluding work and/or
school)?
Age
Gender
Activities commonly completed on
the Internet.
Participant 1 Participant 2 Participant 3
No
No
No
Intermediate Intermediate Advanced
4
4
3
1-3
1-3
More than 5
31
Female
Social
networking,
Email
29
Female
Shopping,
Social
networking,
Email
33
Male
Shopping,
Social
networking,
Email
Fig. 11 Results of Background Questionnaire
Instruments
The instruments used in the test included:
 Background and Post-Test Questionnaires
 The observer’s MacBook Pro laptop
 Silverback screen capturing application
 Paper and pen
 Word processing application (Google drive)
 iPhone Voice Memos app to record user Think Alouds and responses
The participants completed Background Questionnaires administered through Google
forms prior to the usability test. Each participant operated the site using the observer’s
MacBook Pro laptop; the participant’s clicks and actions were recorded with the
Silverback screen capturing application. Video of the users was not collected. The
observer recorded notes about the participants’ actions using paper and pen as well as
a word processing application; the observer also used the Voice Memos app on the
iPhone to record user Think Alouds and responses.
14
Tasks
Rubin and Chisnell as well as Nielsen et al. suggest devising realistic scenarios for
participants (Rubin and Chisnell 182; Nielsen et al. 13). “Good tasks accurately
represent actual use,” writes Nielsen et al. in E-Commerce User Experience:
Methodology (13). The tasks should also take advantage of the participants’ motivations.
“The trick for task designers is to create scenarios compelling enough to motivate users
to behave as if they were working on a task of their own” (Nielsen et al. 14). The
following tasks were adapted from Nielsen et al.’s Methodology (15).
Task 1 – First Impressions
The participants were given two to three minutes to explore each site, with no particular
scenario or task given. After three minutes, they were asked: “Does the site seem easy
to navigate?” Their answers were recorded and later compared with the test results.
Task 2 – Realistic Scenarios
After the first impression exercise, the participant was given a scenario to complete. The
participant had two scenarios to complete (one for each site tested).
Scenario 1
You are thinking of painting a room in your house. You would like to test the available
colors out on a sample room online before going to the store to buy paint. Find a sample
virtual room and apply a color of your choice. Then save the room for later reference.
Scenario 2
Your friend wants to paint his living room and he needs help picking a color. Find a
sample living room and apply a color you think he would like. Then email the room to
him. (Email the room to adinaefron@gmail.com.)
The participant had 20 minutes to complete each task. If after 20 minutes the participant
needed more time, it was allowed.
After completing the test on each site, the participant was asked to complete a PostTest Questionnaire. The primary purpose of a post-test questionnaire is to “gather
preference information from the participants in order to clarify and deepen your
understanding of the product’s strengths and weaknesses” (Rubin and Chisnell 192).
The questionnaire was designed to elicit opinions about the product’s ease of use. An
informal post-test oral debriefing was also conducted to further clarify user preference
information. The Post-Test Questionnaire can be found in Appendix B.
15
Data and Results
First Impressions
After the participants’ First Impressions, in which they browsed the site for 2–3 minutes,
an oral debriefing was conducted to answer the question, “Does the site seem easy to
navigate?” The following table summarizes participants’ responses.
Question: “Does the site seem easy to navigate?”
Response
Site #1 – Sherwin-Williams
Participant 1 “I’m unclear as to how to get back to the homepage…Within each
page it’s clearly defined and easy to navigate, but when you move
from page to page, I’m kind of unclear of how to get back to where I
was.”
Participant 2 “No, it seemed very busy.”
Site #2 – Glidden
Participant 2 “Yes”
Participant 3 “Yes”
Site #3 – Benjamin Moore
Participant 1 “Yes…the titles of the [navigator] tabs described the content of
what was in that particular page.”
Participant 3 “No, I’d give it a 2/5…It’s too crowded.”
Usability Test Recordings and Think Alouds
While the questionnaires and oral debriefings provide participant opinions before and
after completing the scenarios, the usability test recordings and Think Alouds provide
valuable information about the participants’ behavior and thoughts while they are
actively using the site.
This section includes highlights from each participant’s usability test session, shedding
some light on problems users might have with the paint visualizer applications. When
relevant, comments made aloud (“Think Alouds”) are included. The section is organized
by site, then by participant.
1. Sherwin-Williams
Participant 1 – Scenario 2
Number of clicks it took to launch visualizer: 4
Time it took to complete scenario: 9 minutes
16
After choosing a room, the participant clicked on colors in the palette for about a minute
and did not notice that she hadn’t applied any of the colors to wall. She thought she had
completed the task.
She got confused by the colors in the palette, as well. She said thought some of the
colors were coordinating colors when they really weren’t. (See Fig. 12.)
The participant was prompted to register before being able to email the scene;
registration caused the participant some trouble. Registration occurs in a new tab (see
Fig. 13, page 18), and after registering, it took about four minutes to realize she had to
click on the already opened tab to get back to her virtual room. Then, when she tried to
save her palette to “My Colors,” she received a message saying she had to confirm her
account, which caused further delays. The registration also was delayed by loading
problems on the site.
Fig. 12 The participant thought the circled colors were coordinating colors, but they weren’t.
17
Fig. 13 Registration opens in a new window (see circled area), which causes delay and frustration later
on.
Shortly after finding her way back to the visualizer tab, the participant realized she could
drag and drop the colors onto the walls and other elements. She found the “Share
scene” button fairly easily (see Fig. 14, page 19). Sending the email was easy as well.
18
Fig. 14 Registration opens in a new window, which causes delay and frustration later on.
Participant 2 – Scenario 1
Number of clicks it took to launch visualizer: 8
Time it took to complete scenario: 11 minutes, 48 seconds
When she searched for “sample virtual room,” the top results did not link to the
visualizer application, which could pose a problem for people who aren’t sure what the
program is called but are trying to describe it.
After she clicked on “Find & Explore Colors,” a logical place to go, she had trouble
finding the visualizer. The Color Visualizer link is located in a tiny font in the left sidebar,
but it’s not clearly visible in the options in the main area of the page. She ended up
clicking on different links and ultimately searched for “visualizer,” which launched the
program.
The visualizer seems to hang out in the left area of the page on most pages, but that
doesn’t seem to be where someone would look for it. It would have been helpful if one
of the main elements of the body were the visualizer link. See Fig. 15, page 20.
19
Fig. 15 “Launch Color Visualizer” is visible in the left sidebar, but that’s not necessarily where people
would look for it.
She didn’t have any difficulty applying colors to the walls and navigating the different
paint chip palettes in the left area of the visualizer. She had the same difficulty of
registering and finding her painted room as Participant 1. After registering, she was
taken to an overview page that summarized her personal information (where she lived,
nearest paint store). She asked, “So, how do I get back to where I was?” From the
registration page, she searched for visualizer, which started the visualizer all over again.
At first, didn’t realize she had to click back to the other open tab on the browser to find
her previous work.
When she realized she had to click on the 1st browser tab, she had to log in. Then she
received a message about confirming her account, so she had to stop and log in to her
email. This is a big inconvenience for such a minor activity, which is supposed to be fun
and experimental. After verifying, it was not clear how to refresh her room without
starting all over, which she ended up doing. “If it was me…I would have given up by
now,” she sighed.
2. Glidden
Participant 2 – Scenario 2
Number of clicks it took to launch visualizer: 12
Time it took to complete scenario: 5 minutes
From the home page, she clicked “Find Ideas Fast,” which took her to a page called
“Colors By Theme.” From there, she clicked the “Ideas” header tab, and she started
exploring static images of rooms in the “Inspiration” area. It seems she thought this is
20
where the virtual rooms would be located. She ended up looking at one picture that
seemed like it could be altered because of some red tagging. See Fig. 16.
Fig. 16 A sample room in the “Inspiration” area that seems like it can be altered but is really static.
The participant decided to search for “visualizer,” and that resulted in the link for the
visualizer to appear in the top results. This was how the participant was able to launch
the visualizer.
Once in the visualizer, the participant did not need much time to figure out how to drag
and drop colors to the walls. In case the user is unsure, the directions are prominently
featured above the color grid. See Fig. 17, page 22.
The participant appeared to be enjoying playing with the colors and picking different
color palettes. At one point she said, “Cool, very cool.” Saving the room she painted did
not pose too much of a challenge, although at first she thought the “icons aren’t very
clear.” Within less than 30 seconds, she had successfully shared her painted room.
21
Fig. 17 A simple interface plus prominently displayed instructions made for easy use. The instructions
read, “Pick a color, then drag and drop it where you want it.”
Participant 3 – Scenario 1
Number of clicks it took to launch visualizer: 1
Time it took to complete scenario: 3 minutes, 20 seconds
Participant 3 found Glidden’s color visualizer (“Room Painter”) in one click, whereas
participant 2 required 12 clicks. See Fig. 18, page 23. This is an example of where
having more users test the site would be valuable in understanding the usability of the
site.
However, once in the visualizer room, it took the participant about a minute to figure out
exactly how to apply color to the wall through drag and drop. The participant did seem a
little bit confused by the “Flip the Chip” feature in the right area of the interface. He
clicked on it several times but didn’t seem to use it as intended, which is to apply
coordinating colors.
The photographs within the grid of colors seemed a little pointless to the participant. He
joked about wanting “kitty paper.” The photos are probably meant to evoke real-life
objects as color inspiration, but they fell flat with the participant. See Fig. 19, page 23.
The participant did not encounter any difficulties in sharing the site. He used the
observer’s Facebook account to log in and save the room.
22
Fig. 18 Participant 3 found the “Room Viewer” in no time!
Is this a cat or
a paint color?
Fig. 19 Why is there a photo of a cat among the color swatches? The participant was confused.
3. Benjamin Moore
Participant 1 – Scenario 2
Number of clicks it took to launch visualizer: 5
Time it took to complete scenario: 6 minutes
23
Finding the visualizer did not pose a problem for the participant, but she did run into
some trouble applying colors. She selected a color combination form the Color
Collection drop down. When she clicked on the box holding the color combination, the
colors were applied to the wall in a predetermined manner. At first she wasn’t sure how
the colors were applied, and she didn’t know how to change them. See Fig. 20.
After clicking on the
Color Combination,
colors were applied.
Fig. 20 When the participant clicked on the circled color combination in the left side of the page, the
colors were automatically applied to different parts of the room. But she didn’t know how or why they were
applied.
After she was satisfied with the colors in the room, she clicked Save. After clicking Save,
a pop-up window said she needed to be signed into My Favorites to use the feature.
She agreed to register. It’s worth noting that the popup box said, “Your current session
will not be lost.” This is a feature the Sherwin-Williams visualizer was lacking.
The participant did accidentally enter her email address and desired password in the
Login area of the screen, which is reserved for people who already have accounts. But
she quickly realized she needed to register first and corrected her mistake.
After the registration process, her painted room appeared and she was able to save her
scene. See Fig. 21, page 25.
24
Fig. 21 After registering and logging in, her room was saved, as promised.
There was some confusion after clicking Save—a popup asked her if she wanted to
update her existing color combination or save it as a new color combination. The
hesitancy of her mouse clicks indicated she was not sure how to answer. She left the
default information as it was and clicked “Save.” See Fig. 22.
Fig. 22 A slightly perplexing popup box appears after attempting to Save the room, but the disruption is
minimal.
25
Participant 3 – Scenario 1
Number of clicks it took to launch visualizer: 50
Time it took to complete scenario: 7 minutes
Participant 3 had a difficult time navigating to Benjamin Moore’s Personal Color Viewer.
His frustrating journey took him through several areas of the site. The path he took to
finding and launching the visualizer was:
1. Clicked Painting Ideas header tab
2. Clicked Explore Color header tab
3. Clicked Connect Your Rooms with Color sidebar link
4. Clicked Learn How header tab
5. Clicked Homepage (clicked on upper left logo)
6. Clicked Color Gallery link (proceeded to click on color swatches)
7. Clicked Rooms by Color Link (browsed through different rooms)
8. Hit back button
9. Clicked on Living Room
10. Clicked on Rooms by color tab
11. Clicked on color options in room
12. Clicked on Find Products header tab
13. Clicked on For Your Home header link
14. Clicked on Site Map
15. Clicked on Interior Paint Chooser (filled out questionnaire for choosing paint)
16. Finally found the Personal Color Viewer link in the left sidebar
The participant clicked on very logical sounding links (e.g., Connect Your Rooms with
Color, Rooms by Color), but none of them took him to the paint visualizer. He clicked
through more than 15 pages of the site, and yet the Personal Color View did not appear
in a visible area on any of those pages. When the participant was encouraged to think
aloud during his search for the visualizer, he said, “The site sucks…there’s too much
going on. I can’t find where it wants me to [use the virtual room].” When I asked him if
he knew what he was looking for was called, he said, “Not really, but I’m going to go to
the site map.” The Site Map ultimately helped him find the visualizer, but only after filling
out a lengthy questionnaire about choosing paint. “It’s about time I found that!” he
exclaimed. “I always go to the site map when I can’t find something.”
Once he was presented with the “desk” interface (see Fig. 23, page 27), the participant
was able to navigate through the visualizer smoothly.
26
Fig. 23 The “desk” interface that introduces the user to different options of the visualizer.
In the virtual room of the Personal Color Viewer, the participant clicked on a color in the
presented swatches—as a result, the color was applied to a wall. But when he clicked
on another color, for some reason nothing happened. There weren’t any instructions
visible in the Color Viewer window, so the participant had to use trial and error to figure
out how to apply colors.
He easily found the “EMAIL” button in the lower left hand area of the interface and was
able to quickly email the scene.
Post-Test Questionnaire Results
After completing each scenario for a particular site, the participant answered a PostTest Questionnaire that included several questions graded on the Likert scale as well as
an Oral Debriefing. The Post-Test Questionnaire was intended to collect data about
participant opinions as well as verbal responses to provide preferential data.
After completing the second scenario on the second site, each participant answered an
additional set of questions. The participant was asked which site he or she preferred
and why, and which site he or she disliked and why.
Likert Scale Results
For the Likert scale portion of the questionnaire, participants were asked to rate their
opinions about the site:
Please select the rating that best describes your opinion.
27
Strongly
Disagree
Disagree
Neutral/
Undecided
Agree
Strongly
Agree
1
2
3
4
5
The following section summarizes results from the Likert scale portion of the
questionnaire, organized by site, participants organized by column. The average
between the two participants is in the right-most column.
1. Sherwin-Williams
Site Feedback
The site was easy to navigate.
It was easy to find the paint visualizer
application.
It was easy to launch the paint
visualizer application.
It was easy to use the paint visualizer
application.
I enjoyed using the paint visualizer
application.
It was easy to complete the task I
was given.
Likert Scale Responses
Participant 1 Participant 2
(Scenario 2)
(Scenario 1)
2
2
5
1
Average
Rating
2
3
5
4
4.5
4
4
4
5
2
3.5
4
1
2.5
2. Glidden
Site Feedback
The site was easy to navigate.
It was easy to find the paint visualizer
application.
It was easy to launch the paint
visualizer application.
It was easy to use the paint visualizer
application.
I enjoyed using the paint visualizer
application.
It was easy to complete the task I
was given.
Likert Scale Responses
Participant 2 Participant 3
(Scenario 2)
(Scenario 1)
5
5
5
5
Average
Rating
5
5
5
5
5
4
5
4.5
3
3
3
5
5
5
3. Benjamin Moore
Site Feedback
28
Likert Scale Responses
Participant 1 Participant 3
Average
The site was easy to navigate.
It was easy to find the paint visualizer
application.
It was easy to launch the paint
visualizer application.
It was easy to use the paint visualizer
application.
I enjoyed using the paint visualizer
application.
It was easy to complete the task I
was given.
(Scenario 1)
5
4
(Scenario 2)
1
1
Rating
3
2.5
5
1
3
4
1
2.5
5
1
3
4
1
2.5
The following table summarizes the average ratings given to each site for comparison.
The highest average in each site is bolded and underlined to emphases which site was
rated highest in each category.
Site Feedback
The site was easy to navigate.
It was easy to find the paint visualizer
application.
It was easy to launch the paint
visualizer application.
It was easy to use the paint visualizer
application.
I enjoyed using the paint visualizer
application.
It was easy to complete the task I
was given.
Likert Scale Responses
1. Sherwin2. Glidden
Williams
2
5
3
5
3. Benjamin
Moore
3
2.5
4.5
5
3
4
4.5
2.5
3.5
3
3
2.5
5
2.5
Based on the results of the Likert scale questions, Glidden mostly got the highest
positive scores after participants completed scenarios. The two participants who used
Benjamin Moore had the most wildly contrasting opinions—one participant liked it, while
the other had a very bad experience using it. Sherwin-Williams seemed be the least
polarizing of the sites, but it got poorer ratings than Glidden.
Oral Debriefing Results
The Oral Debriefing following the completion of each scenario provided qualitative data
about the participants’ opinions. The following section summarizes the feedback
collected during post-scenario completion oral debriefing, organized by site.
Comments in this section have been tagged with one of the three I’s (INFORMATION,
INTERACTION, OR INTERFACE).
29
1. Sherwin-Williams
Participants
Question
Participant 1
(Scenario 2)
Which specific
 Got confused about whether
features of the
colors were grouped together as
site were
coordinating colors or if they
confusing?
were separate. INTERFACE
 Didn’t realize she had to drag
and drop to apply the paint
colors. INTERACTION
 The lines separating the
“paintable” walls were
confusing. INTERFACE
Participant 2
(Scenario 1)
Registering—she assumed when she
registered, it would “magically take
me back to the previous page, which
would be the visualizer, but it did
not.” Once she got back to the right
tab, she had to verify her email, and
then go back to the visualizer. She
expected that once she refreshed
the page, it would save what she
created, but she had to start over.
INTERACTION
Which specific
features of the
site were difficult
to use?
None


The suggested “coordinated”
colors were ugly, in her opinion.
She wasn’t able to paint the
things she wanted to paint in the
picture. She was frustrated that
pillows and curtains all had to be
the same color. INTERACTION
2. Glidden
Question
Which specific
features of the
site were
confusing?
Participants
Participant 2
(Scenario 2)
 Thought the site was easy to
use, but she didn’t realize she
had to drag colors to paint the
walls. She admitted there were
instructions, but she did not
read them and “assumed she
knew how to use it.”
INFORMATION, INTERACTION


Participant 3
(Scenario 1)
“At first I didn’t see the option
for more colors underneath the
main palette. Other than that
though it was simple.” INTERFACE
Had trouble applying colors to
walls; he thought he could click
on the wall and then click on a
color rather than drag and drop.
INTERACTION
Which specific
features of the
site were difficult
to use?
She didn’t find anything too
difficult. She said feedback was
“good” and helped her figure out
how to use the visualizer.
INFORMATION
30
None
3. Benjamin Moore
Participants
Question
Participant 1
(Scenario 1)
Which specific
 When asked to choose a room,
features of the
she didn’t see anything on the
site were
site that clearly showed her how
confusing?
to do that. She had to click
around to find a room to paint.

INFORMATION, INTERFACE


INFORMATION

Participant 3
(Scenario 2)
“I couldn't find the app.”
She couldn’t see all the colors;
she also wasn’t sure how to
apply colors to different parts of
wall/room. INFORMATION,
“Too much going on in the
menus.” INTERFACE
Thought the tab “Explore Color”
didn’t mean anything; thought
the tabs were confusing.
INFORMATION
INTERACTION
Which specific
features of the
site were difficult
to use?
None


Couldn’t find the app until he
went to the site map. INTERFACE
The app itself was “useless.” He
couldn’t apply enough colors to
see what went with what.
INTERACTION

The lack of depth in the photos
made it hard to visualize what
the room would look like.
INFORMATION

The colors swatches took up too
much space on the screen;
would have liked to seen more
of the room as the main part of
the page. “They could have
made more efficient use of the
space. The interface was busy.”
He didn’t notice other colors
because there was so much
going on. INTERFACE
Which Site Did the Participant Prefer/Dislike?
After completing a different, but similar scenario, on two sites, the participant was asked
which site he or she preferred. The following section presents qualitative information
about preferences, organized by participant. These comments are tagged with the three
I’s.
31
Participant 1: Sherwin-Williams v. Benjamin Moore
Which of the two visualizers did you prefer? Why?
The participant said she liked Sherwin-Williams more because it was more
customizable.
She was expecting to be able to choose where to pick colors on the Benjamin Moore
site but couldn’t. Sherwin-Williams gave her “more customizable options.” INTERACTION
Which visualizer did you dislike? Why?
She didn’t necessarily dislike either of them and felt they both “worked the same way.”
She didn’t like Benjamin Moore because she couldn’t decide which color went where;
she admits she may not have been able to figure that out. She didn’t find any
customizing options. INTERACTION, INFORMATION
Winner: Sherwin-Williams
Participant 2: Sherwin-Williams v. Glidden
Which of the two visualizers did you prefer? Why?
“If I have to pick one, I’d pick Sherwin-Williams because it has furniture in it. So I can
see how it looks like with furniture.” She felt the Glidden site was just focused on color
coordination. INTERFACE
Which visualizer did you dislike? Why?
She disliked Sherwin-Williams because of the troublesome login and registration
process. She wasn’t able to get back to the room she painted after registering. She felt
the activity was supposed to be fun, but the site wouldn’t take her back to her painted
room. She said if it were a real activity, “I would just quit.” INTERFACE
Winner: Draw
Participant 3: Glidden v. Benjamin Moore
Which of the two visualizers did you prefer? Why?
Glidden for two reasons: 1) it was easier to find the application and 2) the application
was more “intuitive.” INFORMATION, INTERACTION
At first the Benjamin Moore site looked “prettier,” but the functionality didn’t follow, so he
would choose Glidden as a preference. “Once you get to the app [on Benjamin Moore],
the prettiness was gone.” INTERACTION, INTERFACE
Which visualizer did you dislike? Why?
He didn’t like that he had to give his personal information or login with Facebook to
complete the task on Glidden. He liked that he could give a fake email on the Benjamin
Moore site. INTERACTION
Winner: Glidden
32
Discussion
How did the paint company sites and their visualizers perform in terms of the 3 I’s of
design—Information, Interaction, and Interface? Based on the results from the usability
test, one can draw conclusions about how each site fared in each category, considering
the main elements involved in site design, navigation, information organization, and
functionality.
Information
The information needed by participants included what the visualizer was called, where it
was located, and how to operate it.
Finding the Visualizer
The name of the visualizer could be an important factor in whether the user can find the
application. Sherwin-Williams calls its application “Paint Color Visualizer,” which is selfevident and accurately describes the application. However, Benjamin Moore calls its
tool the “Personal Color Viewer,” which may not give enough information about whether
it actual performs the visualizer functionality. Ambiguous naming might be why
Participant 3 had so much trouble finding the tool in the Benjamin Moore site (it took him
50 clicks to get there). Paint companies should consider whether the tool they’re
presenting to the public is named in a way that clearly explains to the user what it does.
The site architecture and location of links could also pose challenges to users looking
for information. Benjamin Moore seemed to violate the rules of good information design
throughout their site. For example, both participants who used Benjamin Moore
complained that they couldn’t find the app and that the tabs were confusing. A busy
interface and ambiguous language in the tabs prevented participants from finding the
visualizer app. By contrast, the same participant who took 50 clicks to find the Benjamin
Moore visualizer only needed one click to find the Glidden “Room Painter.” Once the
participant clicked on the logical “Explore Color” tab on the Benjamin Moore homepage,
the link “Paint Your Room Online” (which takes the user to the Personal Color Viewer)
was located in tiny letters in a crowded sidebar on the left of the page (see Figs. 24 and
25, page 34).
33
The “Room
Painter” is one
step down from
the navigator
tab “Color”
Fig. 24 It only takes one click to get to the Glidden “Room Painter,” which is a very clear description for
the tool.
Fig. 25 After clicking the “Explore Color” tab (a logical decision), the Personal Color Viewer is hidden in a
link called “Paint Your Room Online” in the busy left sidebar.
34
Designers should carefully consider the alternate paths users might take to the find the
visualizer applications. Participant 2 used the search feature in the upper-right corner of
Sherwin-Williams and Glidden when navigating the site failed her. Searching “sample
virtual room” in Sherwin-Williams failed to report the visualizer application in the results.
Although that phrase is not an exact description of the app, it is close enough that it
should bring up the application in results. When she searched “visualizer” in Glidden’s
site, she was able to find the application. While Participant 3 was struggling with
Benjamin Moore’s organization, he used the site map to find the visualizer. Search and
site map are both valid ways of finding features or pages within a site, so the designer
must make sure these are fully functional.
Operating the Visualizer
Once the user finds the visualizer, he or she needs to be given information about how to
use it. Some might claim that users should be forced to play with the application and
figure it out on their own. However, a company runs the risk of the user leaving the site
due to frustration and confusion.
Participants had difficulty applying color to walls in all three paint sites. While using
Sherwin-Williams, Participant 1 could not figure out how to apply the paint at first. While
using Glidden, Participant 2 thought the site was easy to use in general, but she did not
realize she had to drag the colors to paint the walls. She admitted she knew there were
instructions but did not read them. Participant 3 had to use trial and error in Benjamin
Moore’s app in order to apply color.
Companies should presents clear instructions to users immediately after the application
launches, perhaps in the form of a popup box that explains how to apply color. There
could even be a brief demonstration (maybe in the form of a short multimedia clip) to
inform the user. As Donald A. Norman writes in The Design of Everyday Things,
technology should “make visible what should be visible” (Norman 192). Instructions on
how to use the app should be visible, if they are needed.
In addition, the app designers should provide better feedback to users to let them know
they applied the color successfully. According to Norman, feedback is “sending back to
the user information about what action has actually been done, what result has been
accomplished” (Norman 27). Maybe after the first time the user drags a color to the wall
(if that is the functionality), a small popup could appear that says, “You have applied
Scarlet Hue.”
Interface
Poor interface design can cause confusion and disorientation to a website user. In the
case of the paint sites and visualizer applications, interface design impacted navigation
and the ease of interpreting application functionalities.
Navigation
Sherwin-Williams demonstrated inadequate navigation on their homepage. Participant 1
noted that it was difficult to get back to the main page after browsing through the
35
different branches of the company (e.g., automotive finishes, aerospace finishes). She
said that within each branch, the sites were easy to navigate, but between different
Sherwin-Williams sites, navigating was difficult. The different branches of the company
should have been connected to each other; a logo labeled “Sherwin-Williams Home”
placed on each division’s site could have solved this issue.
Registering and logging in created navigational difficulties for Sherwin-Williams users.
Both participants who tested Sherwin-Williams experienced delays and frustration as a
result of the site’s cumbersome registration process. One main reason was that when
users tried to share or save scenes, they were prompted to register. Registration took
place in a new tab on the browser, and it was a long, involved procedure that involved
typing in a lot of information (see Fig. 26, page 37), as well as confirming one’s account
by logging into personal email. Sherwin-Williams is sacrificing user satisfaction for the
sake of collecting personal information for marketing purposes. Participant 2
experienced much frustration because she thought she had lost her work and had to
start over. Instead of requiring the user to jump to a completely different browser tab,
registration could take place in a modal panel hovering over the visualizer.
Interpreting Application Interface Elements
The color palette, which displays paint colors offered by the company, is fraught with
interface dangers. Each of the sites had color palette elements that confused the
participants.
While using Sherwin-Williams, Participant 1 misinterpreted a row of paint swatches for
suggested coordinated colors. According to 100 Things Every Designer Needs to Know
About People, when items are grouped together, people often think they belong
together (Weinschenk 21). Other participants reacted to coordinating colors with
ambivalence or confusion. At times, they were apathetic and weren’t sure how to use
them, and sometimes they thought the suggested coordinating colors were “ugly”
(Participant 2).
Participant 3 encountered trouble using the color palette on Glidden’s and Benjamin
Moore’s visualizers. On Glidden, he said at first he didn’t notice the additional color
features located below the main scene of the interface. While using Benjamin Moore’s
visualizer, he became frustrated by the busy interface and the fact that the color
swatches took up so much space on the screen (see Fig. 27, page 38).
36
Fig. 26 Registration is an unnecessarily long process on Sherwin-Williams. Opening in a new tab doesn’t
help the user either.
“Extra” color palette features confused participants who used the Glidden visualizer.
These extra features included “Flip the Chip,” photo swatches, and the carousel located
within the “Top 10” tab at the bottom of the interface. “Flip the chip” (see Fig. 28, page
39) is supposed to help the user choose coordinating colors, but its placement is odd
and its purpose is not obvious. The photos scattered among the color swatches are
confusing as well; they seem to be color “inspiration,” but users think they can drag the
photo to the wall and are perplexed when nothing happens. Finally, the “Top 10”
carousel moves very quickly, which surprised and scared away Participant 2 (see Fig.
29, page 39). Glidden should avoid placing unexpected features within the main
visualizer interface. Users expect to apply colors to the walls, but they may not expect to
use these auxiliary features. And if the features are not self-evident or have an
unnerving quality, they should be eliminated. In Designing Interfaces, Jenifer Tidwell
recommends, “Follow conventions whenever possible” (244). By subverting
expectations in the name of novelty, the designers may be turning off users.
37
Fig. 27 The color swatches (on the right) take up one third of the Benjamin Moore visualizer interface.
Also, the busy-ness of the color arrangements annoyed Participant 3.
38
Fig. 28 Glidden’s oddly placed “Flip the Chip” feature confused Participant 2.
Fig. 29 Glidden’s “Top 10” carousel moves too quickly.
39
Realistic Scenes v. Simple Walls
A question a paint visualizer designer might ask is, “Is it better to have realistic scenes
that contain furniture, or is it better to have a simple layout of paintable walls?” It is
unclear which option was more desirable in this study. Participant 2 liked the SherwinWilliams interface more than Glidden’s interface because the Sherwin-Williams rooms
featured furniture and realistic arrangements. (See Fig. 30.) She said it helped her
visualize the room better. However, Participant 3 said he preferred Glidden because it
had more “depth” than the Benjamin Moore interface. Additionally, Participant 3 had so
much trouble with Benjamin Moore’s interface that the “prettier” appearance lost
functionality. Ideally, the visualizer room should be attractive and realistic (so that the
user can “visualize” the room), but interface problems should not get in the way of the
ultimate function of the application.
Fig. 30 Sherwin-Williams (top) has a realistic interface with furniture, while Glidden (bottom) uses a more
abstract interface.
Interaction
The paint visualizer application is meant to be a fun, “interactive” way to play around
with wall colors in a virtual environment. The problems users encountered while
interacting with the site and the app mostly revolved around “false visualizers,” room
elements, and color application. Registration and logging in was an issue, as discussed
earlier in the report, but those issues are not unique to visualizer apps.
40
False Visualizers
Here “false visualizers” refer to photographs that users believed to be the paint
visualizer application but were really static images that could not be manipulated. One
reason for this problem is that paint companies seem to cram a lot of content into their
sites (e.g., color inspiration photos, décor ideas). While users are looking for the
visualizer application, they may think these photos are editable.
For example, while browsing Glidden, Participant 2 clicked the link “Find Ideas Fast.”
This path ultimately took her to photos of rooms in the “Inspiration” area, and she
clicked around the photo, thinking this might be a visualizer. The red tagging elements
added to her misconception. (See Fig. 16, page 21.)
While browsing Benjamin Moore, Participant 3 also clicked through several areas that
appeared to have elements of a color visualizer (e.g., color swatch gallery, “Rooms by
Color”). Paint companies should limit the inclusion of features that appear to be similar
but have different functionalities (or no functionality at all). See Fig. 31.
Fig. 31 Participant 3 was confused by the Color Gallery and Rooms by Color area; he clicked around as
if he was trying to apply colors.
In this case, Norman might argue that the interface presents the illusion of an
“affordance”—the perceived ability to apply color to a scene by selecting swatches and
dragging colors to areas of the room (Norman 9). Only when nothing happens does the
user realize the photo is devoid of interactive features.
41
Virtual Room Elements
Which elements of the rooms, or scenes, helped the user apply color? Which elements
hindered users?
These questions are difficult to answer in such a small user group. Participant 1 did not
like the wall outlines that appeared in the Sherwin-Williams scenes, even though they
indicate that the area could “receive” a dragged color (see Fig. 32). Conversely,
Participant 2 did not discuss the wall outlines while using the Sherwin-Williams app and
did not have any trouble applying color. Designers should investigate whether users
prefer to see wall outlines while using the visualizer app.
Fig. 32 Participant 2 did not care for the outlined walls (see red arrow), but other participant did not seem
to mind.
The color swatches were another major element of the interface where interaction
problems can arise. The busy arrangement of the Benjamin Moore swatches caused
both participants confusion. Glidden’s “Flip the Chip” confused participants, as well.
Sherwin-Williams seemed to have the least problematic color swatch layout, which
made applying colors less of a complicated task. When designing this type of app,
simpler is better, it seems. “One mistake that designers sometimes make is giving too
much information all at once,” writes Weinschenk in 100 Things Every Designer Needs
to Know About People (62). Designers can fix this mistake by keeping the color swatch
interface simple and placing the more complex functions in the periphery of the interface.
Color Application
The functionality of applying the colors to the walls involves all three I’s, but Interaction
is the most important “I” in this area. As discussed previously, the user needs to know
how to apply the color as well as whether the color was applied correctly.
42
Summary of How Color Can Be Applied to Walls:
Sherwin-Williams: Drag and Drop applies color; clicking swatch does nothing
Glidden: Drag and Drop applies color; clicking swatch activates “Flip the Chip” feature
Benjamin Moore: Drag and Drop applies color; clicking swatch applies color
Norman argues that objects should provide signals to the user about how to use the
product though constraints and affordances (82). One of the challenges the designer
faces is to provide those signals without crowding the interface with too many
instructions and extra “stuff.” Drag and Drop seems to be the most logical way to apply
a color, and all the sites used this functionality. Perhaps this is because Drag and Drop
mimics the action of applying paint to a wall in real life. Clicking the swatch resulted in
three different results on each site, as shown above. It’s not clear which result is the
most useful or easy to operate for the user. This would be research question a designer
could explore by testing user responses to the possible results from clicking a swatch.
Summary
The ultimate goal of a paint visualizer application is to help the user imagine what
different colors look like on walls. This helps the user make decisions about which
paints to buy and use in his or her home. The paint visualizer application should serve
the user by making that goal easy to accomplish. When a paint visualizer app is difficult
to use, the user becomes frustrated and may decide to go to another paint site and
eventually buy paint from that company. So, it is in the best interest of companies to
make sure their apps run smoothly and without much frustration to the user.
This usability test analyzed the different challenges three users faced when using three
different paint visualizer applications. Many more pages could be written about the
various factors that influence ease of use, but this report focuses on the most critical
issues. Information from questionnaires, Think Alouds, recorded testing sessions, and
oral debriefings provided the information needed to answer fulfill the purpose of the
study and answer the research questions.
In general, it is difficult to conclude which site was the easiest to use. The Likert
responses on the Post-Test Questionnaire indicated that Glidden posed the least
serious problems to users. However, participants did like the “realistic” scenes used by
Sherwin-Williams and Benjamin Moore. If a designer could take away a major lesson
from this study, it would be that realistic scenes are appealing to users, but it is also
important to ensure that navigation is clear and easy and that interface features do not
confuse and frustrate users. Also, the frustration of registering before using the “Save”
feature on Sherwin-Williams is an important lesson in considering the needs the user. A
usability study involving more users would yield more conclusive results, but this study
was still able to identify some significant design issues.
43
Based on the data and results of this study, the following recommendations are
suggested, organized by Information, Interface, and Interaction design.
Information






Make the visualizer easy to find by ensuring clear navigation
Give the visualizer an easy-to-understand, obvious name
Organization content in a logical way
Avoid cluttered sidebars containing many links in a small font
Give clear, visible instructions about how to use the visualizer (if it’s not selfevident)
Provide feedback to let the user know that color was successfully applied
Interface





Avoid disruptive registration and logging in processes (avoid taking the user to a
completely new interface)
Organize colors in a simple swatch palette
Avoid placing unusual or “novelty” features in the main area of the interface
Avoid overcrowded or complicated color palettes that take up too much space
Do not sacrifice functionality for appearance
Interaction




44
Differentiate the visualizer application from content that looks too similar to a
visualizer (perhaps by including a link to the visualizer on the page)
Determine which visual elements on the interface help users apply color to the
scene
Simple color palettes are easier to use than busier ones
Drag and Drop is an intuitive functionality in paint visualizer apps; other
functionalities are inconsistently utilized and can cause confusion
References
Nielsen, Jakob, et al. E-Commerce User Experience: Methodology. Nielson Norman
Group, 2000.
Norman, Donald A. The Design of Everyday Things. New York: Basic Books, 2002.
Rubin, Jeffrey, and Dana Chisnell. Handbook of Usability Testing. Indianapolis:
Wiley, 2008.
Stevenson, Megan P., and Min Liu. “Learning a Language with Web 2.0: Exploring the
Use of Social Networking Features of Foreign Language Learning Websites.”
CALICO Journal 27(2) (2010): 1-26.
Tidwell, Jenifer. Designing Interfaces. Sebastopol: O’Reilly Media, 2011.
Weinschenk, Susan M. 100 Things Every Designer Needs to Know About People.
Berkeley: New Riders, 2011.
Website Links
http://finance.yahoo.com/q/pr?s=SHW
excellence.sherwin.com
http://www.glidden.com/media/get-file.do?fileType=article-pdf&id=22
http://phx.corporateir.net/External.File?item=UGFyZW50SUQ9MjA2NDk2fENoaWxkSUQ9LTF8VHlwZT0z&
t=1
http://finance.yahoo.com/news/berkshire-hathaway-inc-news-release-210000163.html
URLs for Sites Evaluated
Sherwin-Williams: http://www.sherwin-williams.com
Benjamin Moore: http://www.benjaminmoore.com/en-us/welcome-to-benjamin-moore
Glidden: http://www.glidden.com/index.do
Appendix A
45
Background Questionnaire
(Administered through Google Forms)
First and Last Name
______________________
How would you rate your computer skill?
☐ Beginner
☐ Intermediate
☐ Advanced
Have you ever used a web-based paint visualizer program?
☐ Yes
☐ No
I am interested in home décor and design.
Select the rating that best describes your interest.
Strongly Disagree 1 2 3 4 5 Strongly Agree
How many hours per day do you use the Internet (excluding work and/or school)?
☐ Less than 1
☐ 1-3
☐ 3-5
☐ More than 5
Select the activities you commonly complete on the Internet.
Select all that apply.
☐ Shopping
☐ Social Networking
☐ Email
☐ Playing games
☐ Image-“pinning” sites
☐ Visiting décor blogs and sites
Age
__________
Gender
☐ Male ☐ Female
46
Appendix B
Post-Test Questionnaire
(Administered by paper/pen)
First and Last Name
____________________
Participant #: _____ Scenario #: _____
Date: __________
Website tested: ____________________
Oral Debriefing 1: First Impressions
Does the site seem easy to navigate?
______________________________________________________________
Oral Debriefing 2: After Scenario Completion
Please select the rating that best describes your opinion.
Site Feedback
The site was easy to navigate.
It was easy to find the paint visualizer
application.
It was easy to launch the paint
visualizer application.
It was easy to use the paint visualizer
application.
I enjoyed using the paint visualizer
application.
It was easy to complete the task I was
given.
Strongly
disagree
Disagree
Neutral/
Undecided
Agree
Strongly
agree
1
1
2
2
3
3
4
4
5
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
Additional Questions
Which specific features of the site were confusing?
________________________________________________________________
Which specific features of the site were difficult to use?
________________________________________________________________
47
(After completing the second scenario):
Which one of the two visualizers did you prefer? Why?
________________________________________________________________
Which visualizer did you dislike? Why?
________________________________________________________________
48
Appendix C
Introduction Script
Thank you for participating in this usability study. My objective today is to observe you
while you use two paint visualizer web applications.
To start out, you will be given 2–3 minutes to explore each site. Then, you will be given
one task for the 1st site and another task for the 2nd site. You can complete both tasks
today, or we can meet another day to complete the 2 nd task.
Your screen will be recorded as you work, but no video of you or your face will be
captured during the test. The screen capture is intended to record your mouse
movements and clicks.
I will also be taking notes as you complete your tasks. I am going to ask you say your
thoughts out loud when you feel comfortable. I will now demonstrate this Think Aloud
technique to you.
I will give you a print out of the task to read. When you are ready to start, let me know.
When I say “Begin,” you may begin working on your task. When you are done, say “I’m
done.”
After you complete each task, I will ask you a few questions. Then, you will complete a
short questionnaire.
Do you have any questions before we begin?
49
About the Author
This paper is written by Adina Efron for the course EDC385G Designs & Strategies for New Media at
the University of Texas at Austin.
http://www.edb.utexas.edu/education/departments/ci/programs/lt/degrees/masters/
50