Assignment 3: Usability Testing
Spring 2015
Angela West
University of Tennessee – School of Information Sciences
Executive Summary
This assignment will analyze the website for the weather website “The Weather Channel” and
explore various problems and design elements that can be made more efficient to users. The visual
aesthetics and overall usefulness of the site will be examined in order to suggest improvements in
overall functionality.
Section 1. Scenario
The website chosen for this survey was the official site of The Weather Channel
(www.weather.com). This website is one of the top 1,000 most visited websites in the world (Alexa,
2015) and is widely considered to be the primary resource on weather-related news and events.
The website’s primary goal is to provide users with weather forecasts, but in recent years has
expanded to include various other services (namely entertainment-oriented services such as video
links). The scenarios demonstrated in the tasks chosen for this assignment were designed to mimic a
typical user’s experiences with the website. These tasks involve searching for specific weather
information and contact information.
Section 2. Methodology
For this usability test, the author of this paper recruited two participants: The first participant
was a man (aged somewhere between 26-35 years) with a Master’s degree. The second participant was
a 60+ year old woman with no post-high school education. Both users were unfamiliar with the website
being tested. Anonymity was assured and the participants were informed that they were not being
tested personally – only the website’s effectiveness was being measured. The test was performed in a
college library setting (the John Cooke Wyllie library at the University of Virginia’s College at Wise).
The tasks and script were prepared beforehand, which each participant being given three tasks
to complete within one hour. As the participants performed the tasks, the author made notes on their
actions, comments, and behavior regarding the website. Users were encouraged to work through the
tasks on their own, with no help from the survey-taker. Tasks were timed, although no time limit was
imposed. The participants were allowed to perform the tasks at whatever speed they were comfortable
with and were encouraged not to rush.
Afterwards, an optional questionnaire was distributed to the participants. Again, anonymity
was assured and both participants filled it out. This questionnaire, along with the tester’s notes, were
compiled and analyzed for this paper.
Section 3: Summary of Findings
Task 1: Look up a detailed, daily forecast for three different towns/cities in the United States.
This task was fairly straightforward, as the main focus of the website involves this particular
action. However, both subjects had a minor amount of difficulty finding the search bar due to the
immense amount of graphics and information on the main page. The older participant was slower to
find the search bar, and was unsure of what to click in order to perform the search. Below is a
screenshot of the main page of the website:
(image: The Weather Channel, 2015. It should be noted that the empty space to the lower right featured
an advertisement that did not show on the screenshot)
Both participants were able to perform this task within five minutes. Once the search bar was
located, it was then easy to type in the address and find the forecast. The search bar was located in the
same spot on every page, which made it easy for the users to search multiple locations. This task was
the quickest and easiest to complete.
Task 2: Access the weather radar and perform a “past” animation.
This is another common task on The Weather Channel’s website. Many users will view the
weather radar maps to see where storms are forming in the area, and the “past” animation is useful in
determining the current path of a storm. As this is a common and useful task, the participants were
asked to both find and access their area’s weather radar and turn on the animation.
Both users were slow to scroll down on the detailed weather page. As the screenshot below
demonstrates, users are forced to scroll through at least one and a half screens before seeing the
weather radar. Above the radar are several unrelated articles, which one participant referred to as
“Clickbait” (a type of headline/link that includes vague, sensationalized wording designed to entice
viewers to click on the article). For example, this page included articles such as “This Photo is Deeply
Troublesome,” “You’ll Feel Bad for Laughing, but it’s OK, His Dad Laughed Too!,” and “Ghost Ship
Carries Unexpected Hitchhikers.” None of these are related to weather and both participants found
them annoying and distracting.
(image: The Weather Channel, 2015)
Once the radar is found, the user must then click either it or “Weather in Motion” to bring them
to an interactive radar map. At this point, the user is presented with even more new options, such as
displaying the satellite imagery, weather stations, cloud cover, and so forth. The participants were
asked to perform a “past” animation on the radar, which is executed by clicking the “play” arrow at the
bottom of the radar screen. It took the participants one to two minutes to realize that the arrow
activates the past animation.
Overall, this task took one user around 1 minute to complete, and the other user around 3
minutes.
Task 3: Find a way to contact customer service, preferably by phone.
This task proved to be the most difficult by far. One participant searched fruitlessly for over ten
minutes before giving up and resorting to a Google search (where they found the phone number within
seconds). The other participant was less successful and did not find the information at all. One user
spent just over 11 minutes searching for this information, and the other user spent a full 20 minutes.
Participant 1, who is younger and more tech savvy, knew to go to the bottom of the main page
and look for contact information. However, the only relevant link was one labeled “Support.” This link
takes a user to a text-heavy site filled with feedback forum links and FAQs (see image below). The
“Contact support” link on the upper right leads to a pop-up window where a user can type in
feedback/questions and submit. However, no contact information is listed there.
(image: The Weather Channel, 2015)
Section 4: Problems & Recommendations
One of the most common problems encountered by the participants was the overwhelming
amount of information and graphics presented on each page of the site. Graphics, text, and animations
are frequently placed together in close proximity, which is good for casual browsing but not for precise
searches (such as the tasks that were presented to the participants). More visual blank space would
help focus the viewer’s attention to certain sections, rather than encourage their gaze to travel around
the entire screen. In an article by Gisele Muller on Treehouse.com, it states:
“We need to remember that going for a simple layout instead of a complex one is the key to
keeping readers interested. A lot of elements, images, colors and different shapes in a page can
make your site look more like an infomercial and cause your readers leave because they’re
uncomfortable.” (Muller, 2012)
As the following screenshot shows, the front page could almost be considered “cluttered” since there is
no blank space for a viewer to rest the eyes.
(image: The Weather Channel, 2015)
The participants also expressed frustration in finding specific links. As mentioned previously,
one user was even forced to resort to Google when searching for contact information. Both participants
had difficulty sorting through the “help” and “FAQ” pages, as they found the poor subject headings and
“wall of text” design to be too visually exhausting. Changing the help section, perhaps to something
similar to Amazon.com (with clearly defined sections and color-coded headings, as well as collapsible
menus) would benefit The Weather Channel’s website as well. Additionally, The Weather Channel’s
help screen is several screens long, whereas the Amazon help page is on one screen. A comparison of
The Weather Channel and Amazon’s help pages can be seen below:
(left image: Amazon.com, 2015, right image: The Weather Channel, 2015)
Section 5: Suggested Improvements
The website, while loaded with valuable information, could certainly stand a more streamlined
design. The following are possible changes that could improve the user experience:
1. Remove “Clickbait” articles. In a recent article by Khalid El-Arini (an engineer for the wildly popular
social media site, Facebook), it is stated that:
“… when we asked people in an initial survey what type of content they preferred to see in their
News Feeds, 80% of the time people preferred headlines that helped them decide if they
wanted to read the full article before they had to click through.” (El-Arini, 2014)
This indicates that the majority of people are not swayed by “clickbait” headlines and would prefer
headlines with more information. These vague headlines serve to visually clutter a page with
unrelated and irrelevant information, therefore making the site harder to read and, to some, less
professional.
2. Create visible, easy-to-find links for all users. Links were often overlooked by the participants in
this study. The overwhelming amount of information frequently meant that the users would miss
relevant links amidst the sea of unrelated graphics and text. Even the help section, which has
subject headings, proved to be time-consuming and not intuitive to the users. Clear subject
headings, more collapsible menus, and white space to draw viewers’ attention would be of great
benefit to the site and make it much easier on the eyes.
3. More visible search bar. While the search bar is visible when first visiting the site, users complained
that it wasn’t readily apparent. Perhaps a more prominent placement or a larger graphic indicating
the search bar would be useful.
4. Radar more prominent on the forecast page. While I understand the idea behind placing links in a
prominent area, the decision to put the radar below the unrelated articles is confusing. One user
assumed that the radar must be on another page before they scrolled down far enough and realized
that it was on the current page.
5. More unified color scheme. When viewing the main page, the user is confronted with a large
amount of graphics, and yet no unified color scheme to unite the elements together. Backgrounds
are white or grey with white or dark blue/black text.
Conclusion:
While The Weather Channel’s website is effective in that it contains all of the information that
the average user might need their services for, there are definitely ways that the overall usability and
functionality could be improved. Although the sample size in this study was very small, it showed that
there are issues with the website that need to be addressed, especially since both participants often
complained about the same issues.
Appendix A
Tasks
1. Look up a detailed, daily forecast for three different towns/cities in the United States.
2. Access the weather radar and perform a “past” animation.
3. Find a way to contact customer service, preferably by phone.
Appendix B
Protocol
Date: 4-05-2015
Location: UVA-Wise John Cooke Wyllie Library
Time: 3:00pm-5:30pm
Timeline:
Welcome the first participant (1 minute)
Introduce the participant to the computer/environment and read script (5 minutes)
Introduce and implement task 1, make notes (5-10 minutes)
Introduce and implement task 2, make notes (5-10 minutes)
Introduce and implement task 3, make notes (15-20 minutes)
Thank the participant and ask for questions/comments (5-10 minutes)
Offer the questionnaire (5 minutes)
Finish writing notes (5-10 minutes)
Repeat for participant 2
Appendix C
Scripts
Usability Test Script
[Welcomes participant and tells him/her to be comfortable, offers drink or restroom break before the
test begins]
Script:
Welcome:
“Hi! Welcome and thank you for participating in our research today. My name is Angela West
and I will be conducting today’s survey. I am a member of Dr. Vandana Singh’s Web Design
course at the University of Tennessee’s SIS program. This survey is designed to test the usability
and productive design of a website, and I will be asking you to perform various tasks on the
website in order to determine its effectiveness.
The survey conducted today will last no longer than an hour. Please feel free to take breaks if
needed. Bear in mind that your identity will remain anonymous and your skills will not be tested
– only the effectiveness of the website’s design. The goal is to find the flaws in the website’s
design and increase its usefulness for all users. If you encounter trouble with a certain section
of the website, it is likely that many other people will also find it ineffective and confusing.
Equipment and Tasks:
The survey will be conducted on a Dell Inspiron laptop with attached USB mouse and Wi-Fi
capabilities. The operating system is Windows 7.
The website that we will be exploring today is The Weather Channel’s website
(http://www.weather.com). There are three tasks that I will be asking you to complete today.
They are as follows:
4. Look up a detailed, daily forecast for three different towns/cities in the United States.
5. Access the weather radar and perform a “past” animation.
6. Find a way to contact customer service, preferably by phone.
These tasks are timed but please do not rush through tasks. Take them at a comfortable pace.
You are not required to perform tasks in a specific way, just whatever way feels most
comfortable for you.
As soon as you are ready, we will begin.
[Introduces the tasks again, one by one]
After the tasks are completed:
Thank you for participating today. If you would, please fill out this questionnaire about your
experience today. Answers will not be distributed outside of this course and your identity will
remain anonymous.
Debrief:
If you have any questions or comments, please let me know.
Thank you once again for coming today and have a great day.
Appendix D
Notes
(including relevant “Think Aloud” comments)
(notes presented in an informal format, just as they were written)
User seems confused by layout.
“Where do I look for..Does support mean contact?” – participant 2
User seems distracted by the amount of graphics on screen
“Too much stuff on here!” – participant 2
“This site is a mess.” – participant 1
Users both called the site “ugly”
“Clickbait” was mentioned many times by participant 1
Participant 2 hates the articles
“What happened to the weather on The Weather Channel?” – participant 1
Help page is too confusing for both participants
Both participants called the site either “busy” or “cluttered”
“Seems kind of sensationalist” – participant 1, on the articles
Both participants made disgruntled noises while trying to search for items
Task 3 is definitely the most difficult
Tasks 1 and 2 are easier but could be made better
The site could use more white space and a better color scheme
Participants are not happy with their experience
“I can’t find anything!” – participant 2, with noted frustration
Participant shows signs of irritation and frustration (particularly with task 3)
Pages are too long, not enough weather-related information per page
Lots of comments for “Where” and “How”
Site is needlessly “busy” with unrelated items
“This makes my eyes tired” – participant 2
Radar is too far down
Radar is below irrelevant articles
White space would improve readability
“Clickbait” articles are listed as main content, not as advertising, looks unprofessional
Tasks should be straightforward but are not due to layout
No obvious color scheme
Task 1 – Success, though improvements could be made
Task 2 – Success, though this process was much more frustrating than it should have been
Task 3 – Failure. The site is confusing and difficult to navigate for this feature.
Appendix E
Questionnaire
The purpose of this questionnaire is to understand the users of The Weather Channel’s website and
their experiences and comments. Your anonymity is assured and the results of this questionnaire will
be used by the tester, a student of the University of Tennessee Knoxville.
Please answer the following:
1. Age:
□ Under 18
□ 18-25
□ 26-35
□ 36-45
□ 46-55
□ 55-60
□ 60+
2. Highest education:
□ Some high school
□ High School
□ College degree
□ Graduate degree
3. How often do you use the internet?
□ Less than once a week
□ Once each week
□ Several times a week
□ Daily
□ Never
4. Have you ever used The Weather Channel’s website before?
□ Yes
□ No
5. Would you recommend The Weather Channel’s website to other users?
□ Yes
□ No
6. Do you have any comments or suggestions?
References
Amazon. (2015). Amazon.com: Online Shopping for Electronics, Apparel, Computers, Books,
DVDs & more. Retrieved from http://amazon.com
El-Arini, K. (2012). News Feed FYI: Click-baiting | Facebook Newsroom. Retrieved from
http://newsroom.fb.com/news/2014/08/news-feed-fyi-click-baiting/
Muller, G. (2012). Whitespace in Web Design: What It Is and Why You Should Use It.
Retrieved from http://blog.teamtreehouse.com/white-space-in-web-design-what-it-isand-why-you-should-use-it
The Weather Channel. (2015). Retrieved from http://www.weather.com
weather.com Site Overview. (2015). Retrieved from
http://www.alexa.com/siteinfo/weather.com